Can a single pipeline entry view the whole pipeline?
I have a function that can generate either tabular output or json ouput. I know that the conventional way to handle this would be to use a parameter like this:
$ func -j | jq .firstField
$ func -t | awk '{print $1}'
I would like to "look ahead" and infer the output type based on the next stage in the pipeline. The ideal usage would be simply:
$ func | jq .firstField
$ func | awk '{print $1}'
I imagine that somewhere within func I would check the 0'th argument to the next stage in the pipeline and see if it contains a j. I'd produce json if so and tables otherwise.
Does bash allow such voodoo? If so, how?
bash
add a comment |
I have a function that can generate either tabular output or json ouput. I know that the conventional way to handle this would be to use a parameter like this:
$ func -j | jq .firstField
$ func -t | awk '{print $1}'
I would like to "look ahead" and infer the output type based on the next stage in the pipeline. The ideal usage would be simply:
$ func | jq .firstField
$ func | awk '{print $1}'
I imagine that somewhere within func I would check the 0'th argument to the next stage in the pipeline and see if it contains a j. I'd produce json if so and tables otherwise.
Does bash allow such voodoo? If so, how?
bash
why not have a flag tofuncsofunc -j | jsonfooorfunc -t | tablefooor ...
– thrig
Nov 30 '18 at 22:55
func is actually a stand in for a command that takes a sql query as a parameter. I want to use zsh-autosuggestions or fzf to recall the query from history and then decide what to do with its output. I keep catching myself recalling the query with the wrong parameter and then forgetting to go modify it before actually running the command. It's silly, but if I can hack around it then I'll never make that kind of mistake again. I'm sure it'll pay off in like, a decade.
– MatrixManAtYrService
Nov 30 '18 at 23:34
Really though, I'm just curious. Using bash pipelines as glue is fun, and if I could make the commands adapt to their neighbors, that would be more fun.
– MatrixManAtYrService
Nov 30 '18 at 23:55
add a comment |
I have a function that can generate either tabular output or json ouput. I know that the conventional way to handle this would be to use a parameter like this:
$ func -j | jq .firstField
$ func -t | awk '{print $1}'
I would like to "look ahead" and infer the output type based on the next stage in the pipeline. The ideal usage would be simply:
$ func | jq .firstField
$ func | awk '{print $1}'
I imagine that somewhere within func I would check the 0'th argument to the next stage in the pipeline and see if it contains a j. I'd produce json if so and tables otherwise.
Does bash allow such voodoo? If so, how?
bash
I have a function that can generate either tabular output or json ouput. I know that the conventional way to handle this would be to use a parameter like this:
$ func -j | jq .firstField
$ func -t | awk '{print $1}'
I would like to "look ahead" and infer the output type based on the next stage in the pipeline. The ideal usage would be simply:
$ func | jq .firstField
$ func | awk '{print $1}'
I imagine that somewhere within func I would check the 0'th argument to the next stage in the pipeline and see if it contains a j. I'd produce json if so and tables otherwise.
Does bash allow such voodoo? If so, how?
bash
bash
asked Nov 30 '18 at 22:37
MatrixManAtYrServiceMatrixManAtYrService
1827
1827
why not have a flag tofuncsofunc -j | jsonfooorfunc -t | tablefooor ...
– thrig
Nov 30 '18 at 22:55
func is actually a stand in for a command that takes a sql query as a parameter. I want to use zsh-autosuggestions or fzf to recall the query from history and then decide what to do with its output. I keep catching myself recalling the query with the wrong parameter and then forgetting to go modify it before actually running the command. It's silly, but if I can hack around it then I'll never make that kind of mistake again. I'm sure it'll pay off in like, a decade.
– MatrixManAtYrService
Nov 30 '18 at 23:34
Really though, I'm just curious. Using bash pipelines as glue is fun, and if I could make the commands adapt to their neighbors, that would be more fun.
– MatrixManAtYrService
Nov 30 '18 at 23:55
add a comment |
why not have a flag tofuncsofunc -j | jsonfooorfunc -t | tablefooor ...
– thrig
Nov 30 '18 at 22:55
func is actually a stand in for a command that takes a sql query as a parameter. I want to use zsh-autosuggestions or fzf to recall the query from history and then decide what to do with its output. I keep catching myself recalling the query with the wrong parameter and then forgetting to go modify it before actually running the command. It's silly, but if I can hack around it then I'll never make that kind of mistake again. I'm sure it'll pay off in like, a decade.
– MatrixManAtYrService
Nov 30 '18 at 23:34
Really though, I'm just curious. Using bash pipelines as glue is fun, and if I could make the commands adapt to their neighbors, that would be more fun.
– MatrixManAtYrService
Nov 30 '18 at 23:55
why not have a flag to
func so func -j | jsonfoo or func -t | tablefoo or ...– thrig
Nov 30 '18 at 22:55
why not have a flag to
func so func -j | jsonfoo or func -t | tablefoo or ...– thrig
Nov 30 '18 at 22:55
func is actually a stand in for a command that takes a sql query as a parameter. I want to use zsh-autosuggestions or fzf to recall the query from history and then decide what to do with its output. I keep catching myself recalling the query with the wrong parameter and then forgetting to go modify it before actually running the command. It's silly, but if I can hack around it then I'll never make that kind of mistake again. I'm sure it'll pay off in like, a decade.
– MatrixManAtYrService
Nov 30 '18 at 23:34
func is actually a stand in for a command that takes a sql query as a parameter. I want to use zsh-autosuggestions or fzf to recall the query from history and then decide what to do with its output. I keep catching myself recalling the query with the wrong parameter and then forgetting to go modify it before actually running the command. It's silly, but if I can hack around it then I'll never make that kind of mistake again. I'm sure it'll pay off in like, a decade.
– MatrixManAtYrService
Nov 30 '18 at 23:34
Really though, I'm just curious. Using bash pipelines as glue is fun, and if I could make the commands adapt to their neighbors, that would be more fun.
– MatrixManAtYrService
Nov 30 '18 at 23:55
Really though, I'm just curious. Using bash pipelines as glue is fun, and if I could make the commands adapt to their neighbors, that would be more fun.
– MatrixManAtYrService
Nov 30 '18 at 23:55
add a comment |
2 Answers
2
active
oldest
votes
There is no formal metadata or query API associated with a pipeline,
beyond what may be salvaged from the process tree via process tools or
from digging around in /proc type filesystems, should those exist. The
parent shell will (probably) have the complete pipeline somewhere in
memory and will know the various child processes involved though again
there is no API by which an arbitrary cat of the (pointless, except as
an example) pipeline cat | cat | cat | ... would know which cat it
is in that pipeline and therefore who its peers are.
% cat | cat -b | cat -e | cat -n
is more useful as with unique flags a human will have an easier
time of telling which is which; pstree(1) in another terminal for
example may show
| -+= 35276 jhqdoe -zsh (zsh)
| |--- 44661 jhqdoe cat -n
| |--- 03968 jhqdoe cat -b
| |--- 96165 jhqdoe cat -e
| --= 26975 jhqdoe cat
but this would not tell us that cat -e pipes to cat -n, only
that the bag of cats all belong to the process group of the parent
shell 35276.
% ps ao ppid,pid,command | grep '[ ]cat'
35276 44661 cat -n
35276 96165 cat -e
35276 3968 cat -b
35276 26975 cat
If the system you are on has /proc or commands to inspect what pipes
or descriptors of a pid are wired up to what you may be able to figure
out what is connected to what in a process group that a process belongs
to. For example over on linux with lsof and a similar cat pipeline
running, the cat -e and cat -n commands can be linked as they both
share the pipe 14301040:
-bash-4.2$ lsof -p 23591 | grep pipe
cat 23591 jhqdoe 0r FIFO 0,9 0t0 14301039 pipe
cat 23591 jhqdoe 1w FIFO 0,9 0t0 14301040 pipe
-bash-4.2$ lsof -p 23592 | grep pipe
cat 23592 jhqdoe 0r FIFO 0,9 0t0 14301040 pipe
so while this information may be available it may take a bunch of
digging around and reconstructing with unportable tools to figure out.
A parent shell could perhaps offer a means to rewrite the pipeline after
it has been input, though the ZSH hook function preexec does not
appear to offer any means of rewriting the command to be run. (Such a
feature may be similar to how LISP macros let a programmer rework the
code.) A parent shell might also offer some sort of API child processes
could use to inspect the pipeline...but these sorts of additions would
need to be written into the shell.
However one could construct a complex pipeline:
func | ( cd ... && ... | ( ... | awk ... ) )
in which case your func would either fail to find awk and react
(maybe) wrongly, or your process pipeline search feature would need to
recurse through all the commands of the next pipeline element and in
that case the awk might be unrelated to func and not need
modification on the fly. Or you could forget that you setup this
behaviour and the awk could be incorrectly modified, which may lead to
hard-to-find bugs...
I was preparing myself to get a "no", but it looks like there's hope. Seems like one could write a utility that would interrogate its context through these means and spit out the pipeline structure. I get what you're saying re:unportable, but I figure you could respond to system differences as they come and have the utility sense enough of its context to adapt. I may try to build such a thing... I've learned a lot from this post, thank you!
– MatrixManAtYrService
Dec 5 '18 at 22:46
add a comment |
I've was able to do this, at least on linux. Here is a script that demonstrates it: https://gist.github.com/MatrixManAtYrService/790a4a058bc841b0ceb2eb0263fb5d88
Example usage:
❯ cat -b | ./luigi | jq .
[
{
"pid": "20832",
"name": "cat -b",
"node": {
"write": "5157339",
"read": null
}
},
{
"pid": "20833",
"name": "bash ./luigi",
"node": {
"write": "5157341",
"read": "5157339"
}
},
{
"pid": "20834",
"name": "jq .",
"node": {
"write": null,
"read": "5157341"
}
}
]
Thanks again @thrig for the tips. I notice that this breaks if I include a builtin likeecho, so I think that the ultimate solution may require shell modifications, which @thrig had mentioned, but it scratches my itch for now.
– MatrixManAtYrService
7 hours ago
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f485271%2fcan-a-single-pipeline-entry-view-the-whole-pipeline%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
There is no formal metadata or query API associated with a pipeline,
beyond what may be salvaged from the process tree via process tools or
from digging around in /proc type filesystems, should those exist. The
parent shell will (probably) have the complete pipeline somewhere in
memory and will know the various child processes involved though again
there is no API by which an arbitrary cat of the (pointless, except as
an example) pipeline cat | cat | cat | ... would know which cat it
is in that pipeline and therefore who its peers are.
% cat | cat -b | cat -e | cat -n
is more useful as with unique flags a human will have an easier
time of telling which is which; pstree(1) in another terminal for
example may show
| -+= 35276 jhqdoe -zsh (zsh)
| |--- 44661 jhqdoe cat -n
| |--- 03968 jhqdoe cat -b
| |--- 96165 jhqdoe cat -e
| --= 26975 jhqdoe cat
but this would not tell us that cat -e pipes to cat -n, only
that the bag of cats all belong to the process group of the parent
shell 35276.
% ps ao ppid,pid,command | grep '[ ]cat'
35276 44661 cat -n
35276 96165 cat -e
35276 3968 cat -b
35276 26975 cat
If the system you are on has /proc or commands to inspect what pipes
or descriptors of a pid are wired up to what you may be able to figure
out what is connected to what in a process group that a process belongs
to. For example over on linux with lsof and a similar cat pipeline
running, the cat -e and cat -n commands can be linked as they both
share the pipe 14301040:
-bash-4.2$ lsof -p 23591 | grep pipe
cat 23591 jhqdoe 0r FIFO 0,9 0t0 14301039 pipe
cat 23591 jhqdoe 1w FIFO 0,9 0t0 14301040 pipe
-bash-4.2$ lsof -p 23592 | grep pipe
cat 23592 jhqdoe 0r FIFO 0,9 0t0 14301040 pipe
so while this information may be available it may take a bunch of
digging around and reconstructing with unportable tools to figure out.
A parent shell could perhaps offer a means to rewrite the pipeline after
it has been input, though the ZSH hook function preexec does not
appear to offer any means of rewriting the command to be run. (Such a
feature may be similar to how LISP macros let a programmer rework the
code.) A parent shell might also offer some sort of API child processes
could use to inspect the pipeline...but these sorts of additions would
need to be written into the shell.
However one could construct a complex pipeline:
func | ( cd ... && ... | ( ... | awk ... ) )
in which case your func would either fail to find awk and react
(maybe) wrongly, or your process pipeline search feature would need to
recurse through all the commands of the next pipeline element and in
that case the awk might be unrelated to func and not need
modification on the fly. Or you could forget that you setup this
behaviour and the awk could be incorrectly modified, which may lead to
hard-to-find bugs...
I was preparing myself to get a "no", but it looks like there's hope. Seems like one could write a utility that would interrogate its context through these means and spit out the pipeline structure. I get what you're saying re:unportable, but I figure you could respond to system differences as they come and have the utility sense enough of its context to adapt. I may try to build such a thing... I've learned a lot from this post, thank you!
– MatrixManAtYrService
Dec 5 '18 at 22:46
add a comment |
There is no formal metadata or query API associated with a pipeline,
beyond what may be salvaged from the process tree via process tools or
from digging around in /proc type filesystems, should those exist. The
parent shell will (probably) have the complete pipeline somewhere in
memory and will know the various child processes involved though again
there is no API by which an arbitrary cat of the (pointless, except as
an example) pipeline cat | cat | cat | ... would know which cat it
is in that pipeline and therefore who its peers are.
% cat | cat -b | cat -e | cat -n
is more useful as with unique flags a human will have an easier
time of telling which is which; pstree(1) in another terminal for
example may show
| -+= 35276 jhqdoe -zsh (zsh)
| |--- 44661 jhqdoe cat -n
| |--- 03968 jhqdoe cat -b
| |--- 96165 jhqdoe cat -e
| --= 26975 jhqdoe cat
but this would not tell us that cat -e pipes to cat -n, only
that the bag of cats all belong to the process group of the parent
shell 35276.
% ps ao ppid,pid,command | grep '[ ]cat'
35276 44661 cat -n
35276 96165 cat -e
35276 3968 cat -b
35276 26975 cat
If the system you are on has /proc or commands to inspect what pipes
or descriptors of a pid are wired up to what you may be able to figure
out what is connected to what in a process group that a process belongs
to. For example over on linux with lsof and a similar cat pipeline
running, the cat -e and cat -n commands can be linked as they both
share the pipe 14301040:
-bash-4.2$ lsof -p 23591 | grep pipe
cat 23591 jhqdoe 0r FIFO 0,9 0t0 14301039 pipe
cat 23591 jhqdoe 1w FIFO 0,9 0t0 14301040 pipe
-bash-4.2$ lsof -p 23592 | grep pipe
cat 23592 jhqdoe 0r FIFO 0,9 0t0 14301040 pipe
so while this information may be available it may take a bunch of
digging around and reconstructing with unportable tools to figure out.
A parent shell could perhaps offer a means to rewrite the pipeline after
it has been input, though the ZSH hook function preexec does not
appear to offer any means of rewriting the command to be run. (Such a
feature may be similar to how LISP macros let a programmer rework the
code.) A parent shell might also offer some sort of API child processes
could use to inspect the pipeline...but these sorts of additions would
need to be written into the shell.
However one could construct a complex pipeline:
func | ( cd ... && ... | ( ... | awk ... ) )
in which case your func would either fail to find awk and react
(maybe) wrongly, or your process pipeline search feature would need to
recurse through all the commands of the next pipeline element and in
that case the awk might be unrelated to func and not need
modification on the fly. Or you could forget that you setup this
behaviour and the awk could be incorrectly modified, which may lead to
hard-to-find bugs...
I was preparing myself to get a "no", but it looks like there's hope. Seems like one could write a utility that would interrogate its context through these means and spit out the pipeline structure. I get what you're saying re:unportable, but I figure you could respond to system differences as they come and have the utility sense enough of its context to adapt. I may try to build such a thing... I've learned a lot from this post, thank you!
– MatrixManAtYrService
Dec 5 '18 at 22:46
add a comment |
There is no formal metadata or query API associated with a pipeline,
beyond what may be salvaged from the process tree via process tools or
from digging around in /proc type filesystems, should those exist. The
parent shell will (probably) have the complete pipeline somewhere in
memory and will know the various child processes involved though again
there is no API by which an arbitrary cat of the (pointless, except as
an example) pipeline cat | cat | cat | ... would know which cat it
is in that pipeline and therefore who its peers are.
% cat | cat -b | cat -e | cat -n
is more useful as with unique flags a human will have an easier
time of telling which is which; pstree(1) in another terminal for
example may show
| -+= 35276 jhqdoe -zsh (zsh)
| |--- 44661 jhqdoe cat -n
| |--- 03968 jhqdoe cat -b
| |--- 96165 jhqdoe cat -e
| --= 26975 jhqdoe cat
but this would not tell us that cat -e pipes to cat -n, only
that the bag of cats all belong to the process group of the parent
shell 35276.
% ps ao ppid,pid,command | grep '[ ]cat'
35276 44661 cat -n
35276 96165 cat -e
35276 3968 cat -b
35276 26975 cat
If the system you are on has /proc or commands to inspect what pipes
or descriptors of a pid are wired up to what you may be able to figure
out what is connected to what in a process group that a process belongs
to. For example over on linux with lsof and a similar cat pipeline
running, the cat -e and cat -n commands can be linked as they both
share the pipe 14301040:
-bash-4.2$ lsof -p 23591 | grep pipe
cat 23591 jhqdoe 0r FIFO 0,9 0t0 14301039 pipe
cat 23591 jhqdoe 1w FIFO 0,9 0t0 14301040 pipe
-bash-4.2$ lsof -p 23592 | grep pipe
cat 23592 jhqdoe 0r FIFO 0,9 0t0 14301040 pipe
so while this information may be available it may take a bunch of
digging around and reconstructing with unportable tools to figure out.
A parent shell could perhaps offer a means to rewrite the pipeline after
it has been input, though the ZSH hook function preexec does not
appear to offer any means of rewriting the command to be run. (Such a
feature may be similar to how LISP macros let a programmer rework the
code.) A parent shell might also offer some sort of API child processes
could use to inspect the pipeline...but these sorts of additions would
need to be written into the shell.
However one could construct a complex pipeline:
func | ( cd ... && ... | ( ... | awk ... ) )
in which case your func would either fail to find awk and react
(maybe) wrongly, or your process pipeline search feature would need to
recurse through all the commands of the next pipeline element and in
that case the awk might be unrelated to func and not need
modification on the fly. Or you could forget that you setup this
behaviour and the awk could be incorrectly modified, which may lead to
hard-to-find bugs...
There is no formal metadata or query API associated with a pipeline,
beyond what may be salvaged from the process tree via process tools or
from digging around in /proc type filesystems, should those exist. The
parent shell will (probably) have the complete pipeline somewhere in
memory and will know the various child processes involved though again
there is no API by which an arbitrary cat of the (pointless, except as
an example) pipeline cat | cat | cat | ... would know which cat it
is in that pipeline and therefore who its peers are.
% cat | cat -b | cat -e | cat -n
is more useful as with unique flags a human will have an easier
time of telling which is which; pstree(1) in another terminal for
example may show
| -+= 35276 jhqdoe -zsh (zsh)
| |--- 44661 jhqdoe cat -n
| |--- 03968 jhqdoe cat -b
| |--- 96165 jhqdoe cat -e
| --= 26975 jhqdoe cat
but this would not tell us that cat -e pipes to cat -n, only
that the bag of cats all belong to the process group of the parent
shell 35276.
% ps ao ppid,pid,command | grep '[ ]cat'
35276 44661 cat -n
35276 96165 cat -e
35276 3968 cat -b
35276 26975 cat
If the system you are on has /proc or commands to inspect what pipes
or descriptors of a pid are wired up to what you may be able to figure
out what is connected to what in a process group that a process belongs
to. For example over on linux with lsof and a similar cat pipeline
running, the cat -e and cat -n commands can be linked as they both
share the pipe 14301040:
-bash-4.2$ lsof -p 23591 | grep pipe
cat 23591 jhqdoe 0r FIFO 0,9 0t0 14301039 pipe
cat 23591 jhqdoe 1w FIFO 0,9 0t0 14301040 pipe
-bash-4.2$ lsof -p 23592 | grep pipe
cat 23592 jhqdoe 0r FIFO 0,9 0t0 14301040 pipe
so while this information may be available it may take a bunch of
digging around and reconstructing with unportable tools to figure out.
A parent shell could perhaps offer a means to rewrite the pipeline after
it has been input, though the ZSH hook function preexec does not
appear to offer any means of rewriting the command to be run. (Such a
feature may be similar to how LISP macros let a programmer rework the
code.) A parent shell might also offer some sort of API child processes
could use to inspect the pipeline...but these sorts of additions would
need to be written into the shell.
However one could construct a complex pipeline:
func | ( cd ... && ... | ( ... | awk ... ) )
in which case your func would either fail to find awk and react
(maybe) wrongly, or your process pipeline search feature would need to
recurse through all the commands of the next pipeline element and in
that case the awk might be unrelated to func and not need
modification on the fly. Or you could forget that you setup this
behaviour and the awk could be incorrectly modified, which may lead to
hard-to-find bugs...
answered Dec 5 '18 at 20:31
thrigthrig
24.5k23056
24.5k23056
I was preparing myself to get a "no", but it looks like there's hope. Seems like one could write a utility that would interrogate its context through these means and spit out the pipeline structure. I get what you're saying re:unportable, but I figure you could respond to system differences as they come and have the utility sense enough of its context to adapt. I may try to build such a thing... I've learned a lot from this post, thank you!
– MatrixManAtYrService
Dec 5 '18 at 22:46
add a comment |
I was preparing myself to get a "no", but it looks like there's hope. Seems like one could write a utility that would interrogate its context through these means and spit out the pipeline structure. I get what you're saying re:unportable, but I figure you could respond to system differences as they come and have the utility sense enough of its context to adapt. I may try to build such a thing... I've learned a lot from this post, thank you!
– MatrixManAtYrService
Dec 5 '18 at 22:46
I was preparing myself to get a "no", but it looks like there's hope. Seems like one could write a utility that would interrogate its context through these means and spit out the pipeline structure. I get what you're saying re:unportable, but I figure you could respond to system differences as they come and have the utility sense enough of its context to adapt. I may try to build such a thing... I've learned a lot from this post, thank you!
– MatrixManAtYrService
Dec 5 '18 at 22:46
I was preparing myself to get a "no", but it looks like there's hope. Seems like one could write a utility that would interrogate its context through these means and spit out the pipeline structure. I get what you're saying re:unportable, but I figure you could respond to system differences as they come and have the utility sense enough of its context to adapt. I may try to build such a thing... I've learned a lot from this post, thank you!
– MatrixManAtYrService
Dec 5 '18 at 22:46
add a comment |
I've was able to do this, at least on linux. Here is a script that demonstrates it: https://gist.github.com/MatrixManAtYrService/790a4a058bc841b0ceb2eb0263fb5d88
Example usage:
❯ cat -b | ./luigi | jq .
[
{
"pid": "20832",
"name": "cat -b",
"node": {
"write": "5157339",
"read": null
}
},
{
"pid": "20833",
"name": "bash ./luigi",
"node": {
"write": "5157341",
"read": "5157339"
}
},
{
"pid": "20834",
"name": "jq .",
"node": {
"write": null,
"read": "5157341"
}
}
]
Thanks again @thrig for the tips. I notice that this breaks if I include a builtin likeecho, so I think that the ultimate solution may require shell modifications, which @thrig had mentioned, but it scratches my itch for now.
– MatrixManAtYrService
7 hours ago
add a comment |
I've was able to do this, at least on linux. Here is a script that demonstrates it: https://gist.github.com/MatrixManAtYrService/790a4a058bc841b0ceb2eb0263fb5d88
Example usage:
❯ cat -b | ./luigi | jq .
[
{
"pid": "20832",
"name": "cat -b",
"node": {
"write": "5157339",
"read": null
}
},
{
"pid": "20833",
"name": "bash ./luigi",
"node": {
"write": "5157341",
"read": "5157339"
}
},
{
"pid": "20834",
"name": "jq .",
"node": {
"write": null,
"read": "5157341"
}
}
]
Thanks again @thrig for the tips. I notice that this breaks if I include a builtin likeecho, so I think that the ultimate solution may require shell modifications, which @thrig had mentioned, but it scratches my itch for now.
– MatrixManAtYrService
7 hours ago
add a comment |
I've was able to do this, at least on linux. Here is a script that demonstrates it: https://gist.github.com/MatrixManAtYrService/790a4a058bc841b0ceb2eb0263fb5d88
Example usage:
❯ cat -b | ./luigi | jq .
[
{
"pid": "20832",
"name": "cat -b",
"node": {
"write": "5157339",
"read": null
}
},
{
"pid": "20833",
"name": "bash ./luigi",
"node": {
"write": "5157341",
"read": "5157339"
}
},
{
"pid": "20834",
"name": "jq .",
"node": {
"write": null,
"read": "5157341"
}
}
]
I've was able to do this, at least on linux. Here is a script that demonstrates it: https://gist.github.com/MatrixManAtYrService/790a4a058bc841b0ceb2eb0263fb5d88
Example usage:
❯ cat -b | ./luigi | jq .
[
{
"pid": "20832",
"name": "cat -b",
"node": {
"write": "5157339",
"read": null
}
},
{
"pid": "20833",
"name": "bash ./luigi",
"node": {
"write": "5157341",
"read": "5157339"
}
},
{
"pid": "20834",
"name": "jq .",
"node": {
"write": null,
"read": "5157341"
}
}
]
answered 7 hours ago
MatrixManAtYrServiceMatrixManAtYrService
1827
1827
Thanks again @thrig for the tips. I notice that this breaks if I include a builtin likeecho, so I think that the ultimate solution may require shell modifications, which @thrig had mentioned, but it scratches my itch for now.
– MatrixManAtYrService
7 hours ago
add a comment |
Thanks again @thrig for the tips. I notice that this breaks if I include a builtin likeecho, so I think that the ultimate solution may require shell modifications, which @thrig had mentioned, but it scratches my itch for now.
– MatrixManAtYrService
7 hours ago
Thanks again @thrig for the tips. I notice that this breaks if I include a builtin like
echo, so I think that the ultimate solution may require shell modifications, which @thrig had mentioned, but it scratches my itch for now.– MatrixManAtYrService
7 hours ago
Thanks again @thrig for the tips. I notice that this breaks if I include a builtin like
echo, so I think that the ultimate solution may require shell modifications, which @thrig had mentioned, but it scratches my itch for now.– MatrixManAtYrService
7 hours ago
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f485271%2fcan-a-single-pipeline-entry-view-the-whole-pipeline%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
why not have a flag to
funcsofunc -j | jsonfooorfunc -t | tablefooor ...– thrig
Nov 30 '18 at 22:55
func is actually a stand in for a command that takes a sql query as a parameter. I want to use zsh-autosuggestions or fzf to recall the query from history and then decide what to do with its output. I keep catching myself recalling the query with the wrong parameter and then forgetting to go modify it before actually running the command. It's silly, but if I can hack around it then I'll never make that kind of mistake again. I'm sure it'll pay off in like, a decade.
– MatrixManAtYrService
Nov 30 '18 at 23:34
Really though, I'm just curious. Using bash pipelines as glue is fun, and if I could make the commands adapt to their neighbors, that would be more fun.
– MatrixManAtYrService
Nov 30 '18 at 23:55