find -exec + vs find | xargs: which one to choose?
I understand that the -exec can take a + option to mimic the behaviour of xargs. Is there any situation where you'd prefer one form over the other?
I personally tend to prefer the first form, if only to avoid using a pipe. I figure surely the developers of find must've done the appropriate optimizations. Am I correct?
bash find pipe xargs
add a comment |
I understand that the -exec can take a + option to mimic the behaviour of xargs. Is there any situation where you'd prefer one form over the other?
I personally tend to prefer the first form, if only to avoid using a pipe. I figure surely the developers of find must've done the appropriate optimizations. Am I correct?
bash find pipe xargs
add a comment |
I understand that the -exec can take a + option to mimic the behaviour of xargs. Is there any situation where you'd prefer one form over the other?
I personally tend to prefer the first form, if only to avoid using a pipe. I figure surely the developers of find must've done the appropriate optimizations. Am I correct?
bash find pipe xargs
I understand that the -exec can take a + option to mimic the behaviour of xargs. Is there any situation where you'd prefer one form over the other?
I personally tend to prefer the first form, if only to avoid using a pipe. I figure surely the developers of find must've done the appropriate optimizations. Am I correct?
bash find pipe xargs
bash find pipe xargs
edited Dec 4 '17 at 1:36
codeforester
370316
370316
asked Jun 27 '12 at 10:45
rahmurahmu
10.4k1970112
10.4k1970112
add a comment |
add a comment |
4 Answers
4
active
oldest
votes
You might want to chain calls to find (once, when you learned, that it is possible, which might be today). This is, of course, only possible as long as you stay in find. Once you pipe to xargs it's out of scope.
Small example, two files a.lst and b.lst:
cat a.lst
fuddel.sh
fiddel.sh
cat b.lst
fuddel.sh
No trick here - simply the fact that both contain "fuddel" but only one contains "fiddel".
Assume we didn't know that. We search a file which matches 2 conditions:
find -exec grep -q fuddel {} ";" -exec grep -q fiddel {} ";" -ls
192097 4 -rw-r--r-- 1 stefan stefan 20 Jun 27 17:05 ./a.lst
Well, maybe you know the syntax for grep or another program to pass both strings as condition, but that's not the point. Every program which can return true or false, given a file as argument, can be used here - grep was just a popular example.
And note, you may follow find -exec with other find commands, like -ls or -delete or something similar. Note, that delete not only does rm (removes files), but rmdir (removes directories) too.
Such a chain is read as an AND combination of commands, as long as not otherwise specified (namely with an -or switch (and parens (which need masking))).
So you aren't leaving the find chain, which is a handy thing. I don't see any advantage in using -xargs, since you have to be careful in passing the files, which is something find doesn't need to do - it automatically handles passing each file as a single argument for you.
If you believe you need some masking for finds {} braces, feel free to visit my question which asks for evidence. My assertion is: You don't.
2
This post has opened my eyes to a new way of usingfind. Thanks a lot!
– rahmu
Feb 7 '13 at 1:49
1
"I don't see any advantage in using -xargs". What's the-execway of doingxargs -P4so that three of four cores don't stay idle?
– Damian Yerrick
Oct 25 '16 at 18:12
1
@DamianYerrick: End the -exec command not in ";" but with a + (/plus sign).
– user unknown
Oct 27 '16 at 5:15
add a comment |
Safely piping file names to xargs requires that your find supports the -print0 option and your xargs has the corresponding option to read it (--null or -0). Otherwise, filenames with unprintable characters or backslashes or quotes or whitespace in the name may cause unexpected behavior. On the other hand, find -exec {} + is in the POSIX find spec, so it is portable, and it is about as safe as find -print0 | xargs -0, and definitely safer than find | xargs. I'd recommend never doing find | xargs without -print0.
6
A notable exception to the portability offind … -exec … {} +is OpenBSD, which only acquired this feature with version 5.1 released in 2012. All BSDs have had-print0for several years, even OpenBSD (though it resisted that feature for a while too). Solaris, on the other hand, sticks to POSIX features, so you get-exec +and no-print0.
– Gilles
Jun 27 '12 at 23:07
-print0is a pain and though you can arguexargs --delimiter "n"isn't equivalent, I've never once used the former after discovering the latter.
– user7000
Jul 28 '15 at 17:27
3
I don't see how-0is more of a pain than--delimiter "n".
– jw013
Jul 28 '15 at 21:54
2
In addition to-0, GNUxargsneeds-rto avoid running the command if there's no input.
– Stéphane Chazelas
Oct 11 '15 at 20:19
2
Another problem of| xargs -r0 cmdis thatcmd's stdin is affected (depending on thexargsimplementation, it's/dev/nullor the pipe.
– Stéphane Chazelas
Oct 11 '15 at 20:20
add a comment |
If you use the -exec ... ; form (remembering to escape the semicolon), you're running the command once per filename. If you use -print0 | xargs -0, you run multiple commands per filename. You should definitely use the -exec + form, which puts multiple files in a single command line and is much faster when a large number of files is involved.
A big plus of using xargs is the ability to run multiple commands in parallel using xargs -P. On multi-core systems, that can provide huge time savings.
6
You meant-Pinstead of-p. Keep in mindxargs -Pis not in the POSIX standard, whereasfind -exec {} +is, which is important if you are going for portability.
– jw013
Jun 27 '12 at 12:54
@Alexios You don't have to escape the plus sign, because it does not have a special meaning for the shell:find /tmp/ -exec ls "{}" +works just fine.
– daniel kullmann
Jun 27 '12 at 13:48
1
Corrent, of course. I've been escaping everything after the-execfor so long (I'm a masochist, I don't even use quotes to escape{}, I always type{}; don't ask), everything looks like it must be escaped now.
– Alexios
Jun 27 '12 at 14:17
1
@Alexios: If you find an example (except from being a masochist) where the masking of the braces is useful, please provide it as an answer to my question here - afaik this hint is outdated and only a relict in the manpage. I've never seen an example wherefind /tmp/ -exec ls {} +wouldn't work.
– user unknown
Jun 27 '12 at 15:00
1
For me, this is muscle memory formed in the days of SunOS 4. Since I've been doing it for years, I completely failed to notice whenbashstarted accepting the braces verbatim. I'm pretty sure at least one of the old shells I've used threw hissy fits if the braces weren't escaped.
– Alexios
Jun 27 '12 at 15:05
add a comment |
Concerning performance I thought that -exec … + would simply be better because it's a single tool doing all the work but a part of GNU findutil's documentation says that -exec … + might be less efficient in some cases:
[find with
-exec … +] can be less efficient than some uses ofxargs; for examplexargsallows new command lines to be built up while the previous command is still executing, and allows you to specify a number of commands to run in parallel. However, thefind ... -exec ... +construct has the advantage of wide portability. GNU findutils did not support ‘-exec ... +’ until version 4.2.12 [January 2005]; one of the reasons for this is that it already had the ‘-print0’ action in any case.
I wasn't exactly sure what that meant so I asked in the chat where derobert explained it as:
findprobably could continue to search for the next batch of files while the-exec … +is running, but it doesn't.find … | xargs …does, because then the find is a different process, and it keeps running until the pipe buffer fills up
(Formatting by me.)
So there is that. But if performance really matters you would have to do realistic benchmarking or rather even ask yourself whether you even would want to use shell for such cases.
Here on this site I think it's better to advise people to use the -exec … + form whenever possible because just because it's simpler and for the reasons mentioned in the other answers here (e.g. handling strange filenames without having to think much).
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f41740%2ffind-exec-vs-find-xargs-which-one-to-choose%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
4 Answers
4
active
oldest
votes
4 Answers
4
active
oldest
votes
active
oldest
votes
active
oldest
votes
You might want to chain calls to find (once, when you learned, that it is possible, which might be today). This is, of course, only possible as long as you stay in find. Once you pipe to xargs it's out of scope.
Small example, two files a.lst and b.lst:
cat a.lst
fuddel.sh
fiddel.sh
cat b.lst
fuddel.sh
No trick here - simply the fact that both contain "fuddel" but only one contains "fiddel".
Assume we didn't know that. We search a file which matches 2 conditions:
find -exec grep -q fuddel {} ";" -exec grep -q fiddel {} ";" -ls
192097 4 -rw-r--r-- 1 stefan stefan 20 Jun 27 17:05 ./a.lst
Well, maybe you know the syntax for grep or another program to pass both strings as condition, but that's not the point. Every program which can return true or false, given a file as argument, can be used here - grep was just a popular example.
And note, you may follow find -exec with other find commands, like -ls or -delete or something similar. Note, that delete not only does rm (removes files), but rmdir (removes directories) too.
Such a chain is read as an AND combination of commands, as long as not otherwise specified (namely with an -or switch (and parens (which need masking))).
So you aren't leaving the find chain, which is a handy thing. I don't see any advantage in using -xargs, since you have to be careful in passing the files, which is something find doesn't need to do - it automatically handles passing each file as a single argument for you.
If you believe you need some masking for finds {} braces, feel free to visit my question which asks for evidence. My assertion is: You don't.
2
This post has opened my eyes to a new way of usingfind. Thanks a lot!
– rahmu
Feb 7 '13 at 1:49
1
"I don't see any advantage in using -xargs". What's the-execway of doingxargs -P4so that three of four cores don't stay idle?
– Damian Yerrick
Oct 25 '16 at 18:12
1
@DamianYerrick: End the -exec command not in ";" but with a + (/plus sign).
– user unknown
Oct 27 '16 at 5:15
add a comment |
You might want to chain calls to find (once, when you learned, that it is possible, which might be today). This is, of course, only possible as long as you stay in find. Once you pipe to xargs it's out of scope.
Small example, two files a.lst and b.lst:
cat a.lst
fuddel.sh
fiddel.sh
cat b.lst
fuddel.sh
No trick here - simply the fact that both contain "fuddel" but only one contains "fiddel".
Assume we didn't know that. We search a file which matches 2 conditions:
find -exec grep -q fuddel {} ";" -exec grep -q fiddel {} ";" -ls
192097 4 -rw-r--r-- 1 stefan stefan 20 Jun 27 17:05 ./a.lst
Well, maybe you know the syntax for grep or another program to pass both strings as condition, but that's not the point. Every program which can return true or false, given a file as argument, can be used here - grep was just a popular example.
And note, you may follow find -exec with other find commands, like -ls or -delete or something similar. Note, that delete not only does rm (removes files), but rmdir (removes directories) too.
Such a chain is read as an AND combination of commands, as long as not otherwise specified (namely with an -or switch (and parens (which need masking))).
So you aren't leaving the find chain, which is a handy thing. I don't see any advantage in using -xargs, since you have to be careful in passing the files, which is something find doesn't need to do - it automatically handles passing each file as a single argument for you.
If you believe you need some masking for finds {} braces, feel free to visit my question which asks for evidence. My assertion is: You don't.
2
This post has opened my eyes to a new way of usingfind. Thanks a lot!
– rahmu
Feb 7 '13 at 1:49
1
"I don't see any advantage in using -xargs". What's the-execway of doingxargs -P4so that three of four cores don't stay idle?
– Damian Yerrick
Oct 25 '16 at 18:12
1
@DamianYerrick: End the -exec command not in ";" but with a + (/plus sign).
– user unknown
Oct 27 '16 at 5:15
add a comment |
You might want to chain calls to find (once, when you learned, that it is possible, which might be today). This is, of course, only possible as long as you stay in find. Once you pipe to xargs it's out of scope.
Small example, two files a.lst and b.lst:
cat a.lst
fuddel.sh
fiddel.sh
cat b.lst
fuddel.sh
No trick here - simply the fact that both contain "fuddel" but only one contains "fiddel".
Assume we didn't know that. We search a file which matches 2 conditions:
find -exec grep -q fuddel {} ";" -exec grep -q fiddel {} ";" -ls
192097 4 -rw-r--r-- 1 stefan stefan 20 Jun 27 17:05 ./a.lst
Well, maybe you know the syntax for grep or another program to pass both strings as condition, but that's not the point. Every program which can return true or false, given a file as argument, can be used here - grep was just a popular example.
And note, you may follow find -exec with other find commands, like -ls or -delete or something similar. Note, that delete not only does rm (removes files), but rmdir (removes directories) too.
Such a chain is read as an AND combination of commands, as long as not otherwise specified (namely with an -or switch (and parens (which need masking))).
So you aren't leaving the find chain, which is a handy thing. I don't see any advantage in using -xargs, since you have to be careful in passing the files, which is something find doesn't need to do - it automatically handles passing each file as a single argument for you.
If you believe you need some masking for finds {} braces, feel free to visit my question which asks for evidence. My assertion is: You don't.
You might want to chain calls to find (once, when you learned, that it is possible, which might be today). This is, of course, only possible as long as you stay in find. Once you pipe to xargs it's out of scope.
Small example, two files a.lst and b.lst:
cat a.lst
fuddel.sh
fiddel.sh
cat b.lst
fuddel.sh
No trick here - simply the fact that both contain "fuddel" but only one contains "fiddel".
Assume we didn't know that. We search a file which matches 2 conditions:
find -exec grep -q fuddel {} ";" -exec grep -q fiddel {} ";" -ls
192097 4 -rw-r--r-- 1 stefan stefan 20 Jun 27 17:05 ./a.lst
Well, maybe you know the syntax for grep or another program to pass both strings as condition, but that's not the point. Every program which can return true or false, given a file as argument, can be used here - grep was just a popular example.
And note, you may follow find -exec with other find commands, like -ls or -delete or something similar. Note, that delete not only does rm (removes files), but rmdir (removes directories) too.
Such a chain is read as an AND combination of commands, as long as not otherwise specified (namely with an -or switch (and parens (which need masking))).
So you aren't leaving the find chain, which is a handy thing. I don't see any advantage in using -xargs, since you have to be careful in passing the files, which is something find doesn't need to do - it automatically handles passing each file as a single argument for you.
If you believe you need some masking for finds {} braces, feel free to visit my question which asks for evidence. My assertion is: You don't.
edited 21 mins ago
malan
655521
655521
answered Jun 27 '12 at 15:23
user unknownuser unknown
7,34312349
7,34312349
2
This post has opened my eyes to a new way of usingfind. Thanks a lot!
– rahmu
Feb 7 '13 at 1:49
1
"I don't see any advantage in using -xargs". What's the-execway of doingxargs -P4so that three of four cores don't stay idle?
– Damian Yerrick
Oct 25 '16 at 18:12
1
@DamianYerrick: End the -exec command not in ";" but with a + (/plus sign).
– user unknown
Oct 27 '16 at 5:15
add a comment |
2
This post has opened my eyes to a new way of usingfind. Thanks a lot!
– rahmu
Feb 7 '13 at 1:49
1
"I don't see any advantage in using -xargs". What's the-execway of doingxargs -P4so that three of four cores don't stay idle?
– Damian Yerrick
Oct 25 '16 at 18:12
1
@DamianYerrick: End the -exec command not in ";" but with a + (/plus sign).
– user unknown
Oct 27 '16 at 5:15
2
2
This post has opened my eyes to a new way of using
find. Thanks a lot!– rahmu
Feb 7 '13 at 1:49
This post has opened my eyes to a new way of using
find. Thanks a lot!– rahmu
Feb 7 '13 at 1:49
1
1
"I don't see any advantage in using -xargs". What's the
-exec way of doing xargs -P4 so that three of four cores don't stay idle?– Damian Yerrick
Oct 25 '16 at 18:12
"I don't see any advantage in using -xargs". What's the
-exec way of doing xargs -P4 so that three of four cores don't stay idle?– Damian Yerrick
Oct 25 '16 at 18:12
1
1
@DamianYerrick: End the -exec command not in ";" but with a + (/plus sign).
– user unknown
Oct 27 '16 at 5:15
@DamianYerrick: End the -exec command not in ";" but with a + (/plus sign).
– user unknown
Oct 27 '16 at 5:15
add a comment |
Safely piping file names to xargs requires that your find supports the -print0 option and your xargs has the corresponding option to read it (--null or -0). Otherwise, filenames with unprintable characters or backslashes or quotes or whitespace in the name may cause unexpected behavior. On the other hand, find -exec {} + is in the POSIX find spec, so it is portable, and it is about as safe as find -print0 | xargs -0, and definitely safer than find | xargs. I'd recommend never doing find | xargs without -print0.
6
A notable exception to the portability offind … -exec … {} +is OpenBSD, which only acquired this feature with version 5.1 released in 2012. All BSDs have had-print0for several years, even OpenBSD (though it resisted that feature for a while too). Solaris, on the other hand, sticks to POSIX features, so you get-exec +and no-print0.
– Gilles
Jun 27 '12 at 23:07
-print0is a pain and though you can arguexargs --delimiter "n"isn't equivalent, I've never once used the former after discovering the latter.
– user7000
Jul 28 '15 at 17:27
3
I don't see how-0is more of a pain than--delimiter "n".
– jw013
Jul 28 '15 at 21:54
2
In addition to-0, GNUxargsneeds-rto avoid running the command if there's no input.
– Stéphane Chazelas
Oct 11 '15 at 20:19
2
Another problem of| xargs -r0 cmdis thatcmd's stdin is affected (depending on thexargsimplementation, it's/dev/nullor the pipe.
– Stéphane Chazelas
Oct 11 '15 at 20:20
add a comment |
Safely piping file names to xargs requires that your find supports the -print0 option and your xargs has the corresponding option to read it (--null or -0). Otherwise, filenames with unprintable characters or backslashes or quotes or whitespace in the name may cause unexpected behavior. On the other hand, find -exec {} + is in the POSIX find spec, so it is portable, and it is about as safe as find -print0 | xargs -0, and definitely safer than find | xargs. I'd recommend never doing find | xargs without -print0.
6
A notable exception to the portability offind … -exec … {} +is OpenBSD, which only acquired this feature with version 5.1 released in 2012. All BSDs have had-print0for several years, even OpenBSD (though it resisted that feature for a while too). Solaris, on the other hand, sticks to POSIX features, so you get-exec +and no-print0.
– Gilles
Jun 27 '12 at 23:07
-print0is a pain and though you can arguexargs --delimiter "n"isn't equivalent, I've never once used the former after discovering the latter.
– user7000
Jul 28 '15 at 17:27
3
I don't see how-0is more of a pain than--delimiter "n".
– jw013
Jul 28 '15 at 21:54
2
In addition to-0, GNUxargsneeds-rto avoid running the command if there's no input.
– Stéphane Chazelas
Oct 11 '15 at 20:19
2
Another problem of| xargs -r0 cmdis thatcmd's stdin is affected (depending on thexargsimplementation, it's/dev/nullor the pipe.
– Stéphane Chazelas
Oct 11 '15 at 20:20
add a comment |
Safely piping file names to xargs requires that your find supports the -print0 option and your xargs has the corresponding option to read it (--null or -0). Otherwise, filenames with unprintable characters or backslashes or quotes or whitespace in the name may cause unexpected behavior. On the other hand, find -exec {} + is in the POSIX find spec, so it is portable, and it is about as safe as find -print0 | xargs -0, and definitely safer than find | xargs. I'd recommend never doing find | xargs without -print0.
Safely piping file names to xargs requires that your find supports the -print0 option and your xargs has the corresponding option to read it (--null or -0). Otherwise, filenames with unprintable characters or backslashes or quotes or whitespace in the name may cause unexpected behavior. On the other hand, find -exec {} + is in the POSIX find spec, so it is portable, and it is about as safe as find -print0 | xargs -0, and definitely safer than find | xargs. I'd recommend never doing find | xargs without -print0.
answered Jun 27 '12 at 12:58
jw013jw013
36.4k7101125
36.4k7101125
6
A notable exception to the portability offind … -exec … {} +is OpenBSD, which only acquired this feature with version 5.1 released in 2012. All BSDs have had-print0for several years, even OpenBSD (though it resisted that feature for a while too). Solaris, on the other hand, sticks to POSIX features, so you get-exec +and no-print0.
– Gilles
Jun 27 '12 at 23:07
-print0is a pain and though you can arguexargs --delimiter "n"isn't equivalent, I've never once used the former after discovering the latter.
– user7000
Jul 28 '15 at 17:27
3
I don't see how-0is more of a pain than--delimiter "n".
– jw013
Jul 28 '15 at 21:54
2
In addition to-0, GNUxargsneeds-rto avoid running the command if there's no input.
– Stéphane Chazelas
Oct 11 '15 at 20:19
2
Another problem of| xargs -r0 cmdis thatcmd's stdin is affected (depending on thexargsimplementation, it's/dev/nullor the pipe.
– Stéphane Chazelas
Oct 11 '15 at 20:20
add a comment |
6
A notable exception to the portability offind … -exec … {} +is OpenBSD, which only acquired this feature with version 5.1 released in 2012. All BSDs have had-print0for several years, even OpenBSD (though it resisted that feature for a while too). Solaris, on the other hand, sticks to POSIX features, so you get-exec +and no-print0.
– Gilles
Jun 27 '12 at 23:07
-print0is a pain and though you can arguexargs --delimiter "n"isn't equivalent, I've never once used the former after discovering the latter.
– user7000
Jul 28 '15 at 17:27
3
I don't see how-0is more of a pain than--delimiter "n".
– jw013
Jul 28 '15 at 21:54
2
In addition to-0, GNUxargsneeds-rto avoid running the command if there's no input.
– Stéphane Chazelas
Oct 11 '15 at 20:19
2
Another problem of| xargs -r0 cmdis thatcmd's stdin is affected (depending on thexargsimplementation, it's/dev/nullor the pipe.
– Stéphane Chazelas
Oct 11 '15 at 20:20
6
6
A notable exception to the portability of
find … -exec … {} + is OpenBSD, which only acquired this feature with version 5.1 released in 2012. All BSDs have had -print0 for several years, even OpenBSD (though it resisted that feature for a while too). Solaris, on the other hand, sticks to POSIX features, so you get -exec + and no -print0.– Gilles
Jun 27 '12 at 23:07
A notable exception to the portability of
find … -exec … {} + is OpenBSD, which only acquired this feature with version 5.1 released in 2012. All BSDs have had -print0 for several years, even OpenBSD (though it resisted that feature for a while too). Solaris, on the other hand, sticks to POSIX features, so you get -exec + and no -print0.– Gilles
Jun 27 '12 at 23:07
-print0 is a pain and though you can argue xargs --delimiter "n" isn't equivalent, I've never once used the former after discovering the latter.– user7000
Jul 28 '15 at 17:27
-print0 is a pain and though you can argue xargs --delimiter "n" isn't equivalent, I've never once used the former after discovering the latter.– user7000
Jul 28 '15 at 17:27
3
3
I don't see how
-0 is more of a pain than --delimiter "n".– jw013
Jul 28 '15 at 21:54
I don't see how
-0 is more of a pain than --delimiter "n".– jw013
Jul 28 '15 at 21:54
2
2
In addition to
-0, GNU xargs needs -r to avoid running the command if there's no input.– Stéphane Chazelas
Oct 11 '15 at 20:19
In addition to
-0, GNU xargs needs -r to avoid running the command if there's no input.– Stéphane Chazelas
Oct 11 '15 at 20:19
2
2
Another problem of
| xargs -r0 cmd is that cmd's stdin is affected (depending on the xargs implementation, it's /dev/null or the pipe.– Stéphane Chazelas
Oct 11 '15 at 20:20
Another problem of
| xargs -r0 cmd is that cmd's stdin is affected (depending on the xargs implementation, it's /dev/null or the pipe.– Stéphane Chazelas
Oct 11 '15 at 20:20
add a comment |
If you use the -exec ... ; form (remembering to escape the semicolon), you're running the command once per filename. If you use -print0 | xargs -0, you run multiple commands per filename. You should definitely use the -exec + form, which puts multiple files in a single command line and is much faster when a large number of files is involved.
A big plus of using xargs is the ability to run multiple commands in parallel using xargs -P. On multi-core systems, that can provide huge time savings.
6
You meant-Pinstead of-p. Keep in mindxargs -Pis not in the POSIX standard, whereasfind -exec {} +is, which is important if you are going for portability.
– jw013
Jun 27 '12 at 12:54
@Alexios You don't have to escape the plus sign, because it does not have a special meaning for the shell:find /tmp/ -exec ls "{}" +works just fine.
– daniel kullmann
Jun 27 '12 at 13:48
1
Corrent, of course. I've been escaping everything after the-execfor so long (I'm a masochist, I don't even use quotes to escape{}, I always type{}; don't ask), everything looks like it must be escaped now.
– Alexios
Jun 27 '12 at 14:17
1
@Alexios: If you find an example (except from being a masochist) where the masking of the braces is useful, please provide it as an answer to my question here - afaik this hint is outdated and only a relict in the manpage. I've never seen an example wherefind /tmp/ -exec ls {} +wouldn't work.
– user unknown
Jun 27 '12 at 15:00
1
For me, this is muscle memory formed in the days of SunOS 4. Since I've been doing it for years, I completely failed to notice whenbashstarted accepting the braces verbatim. I'm pretty sure at least one of the old shells I've used threw hissy fits if the braces weren't escaped.
– Alexios
Jun 27 '12 at 15:05
add a comment |
If you use the -exec ... ; form (remembering to escape the semicolon), you're running the command once per filename. If you use -print0 | xargs -0, you run multiple commands per filename. You should definitely use the -exec + form, which puts multiple files in a single command line and is much faster when a large number of files is involved.
A big plus of using xargs is the ability to run multiple commands in parallel using xargs -P. On multi-core systems, that can provide huge time savings.
6
You meant-Pinstead of-p. Keep in mindxargs -Pis not in the POSIX standard, whereasfind -exec {} +is, which is important if you are going for portability.
– jw013
Jun 27 '12 at 12:54
@Alexios You don't have to escape the plus sign, because it does not have a special meaning for the shell:find /tmp/ -exec ls "{}" +works just fine.
– daniel kullmann
Jun 27 '12 at 13:48
1
Corrent, of course. I've been escaping everything after the-execfor so long (I'm a masochist, I don't even use quotes to escape{}, I always type{}; don't ask), everything looks like it must be escaped now.
– Alexios
Jun 27 '12 at 14:17
1
@Alexios: If you find an example (except from being a masochist) where the masking of the braces is useful, please provide it as an answer to my question here - afaik this hint is outdated and only a relict in the manpage. I've never seen an example wherefind /tmp/ -exec ls {} +wouldn't work.
– user unknown
Jun 27 '12 at 15:00
1
For me, this is muscle memory formed in the days of SunOS 4. Since I've been doing it for years, I completely failed to notice whenbashstarted accepting the braces verbatim. I'm pretty sure at least one of the old shells I've used threw hissy fits if the braces weren't escaped.
– Alexios
Jun 27 '12 at 15:05
add a comment |
If you use the -exec ... ; form (remembering to escape the semicolon), you're running the command once per filename. If you use -print0 | xargs -0, you run multiple commands per filename. You should definitely use the -exec + form, which puts multiple files in a single command line and is much faster when a large number of files is involved.
A big plus of using xargs is the ability to run multiple commands in parallel using xargs -P. On multi-core systems, that can provide huge time savings.
If you use the -exec ... ; form (remembering to escape the semicolon), you're running the command once per filename. If you use -print0 | xargs -0, you run multiple commands per filename. You should definitely use the -exec + form, which puts multiple files in a single command line and is much faster when a large number of files is involved.
A big plus of using xargs is the ability to run multiple commands in parallel using xargs -P. On multi-core systems, that can provide huge time savings.
edited Dec 4 '17 at 1:29
Stephen Rauch
3,348101428
3,348101428
answered Jun 27 '12 at 10:55
AlexiosAlexios
14.5k14966
14.5k14966
6
You meant-Pinstead of-p. Keep in mindxargs -Pis not in the POSIX standard, whereasfind -exec {} +is, which is important if you are going for portability.
– jw013
Jun 27 '12 at 12:54
@Alexios You don't have to escape the plus sign, because it does not have a special meaning for the shell:find /tmp/ -exec ls "{}" +works just fine.
– daniel kullmann
Jun 27 '12 at 13:48
1
Corrent, of course. I've been escaping everything after the-execfor so long (I'm a masochist, I don't even use quotes to escape{}, I always type{}; don't ask), everything looks like it must be escaped now.
– Alexios
Jun 27 '12 at 14:17
1
@Alexios: If you find an example (except from being a masochist) where the masking of the braces is useful, please provide it as an answer to my question here - afaik this hint is outdated and only a relict in the manpage. I've never seen an example wherefind /tmp/ -exec ls {} +wouldn't work.
– user unknown
Jun 27 '12 at 15:00
1
For me, this is muscle memory formed in the days of SunOS 4. Since I've been doing it for years, I completely failed to notice whenbashstarted accepting the braces verbatim. I'm pretty sure at least one of the old shells I've used threw hissy fits if the braces weren't escaped.
– Alexios
Jun 27 '12 at 15:05
add a comment |
6
You meant-Pinstead of-p. Keep in mindxargs -Pis not in the POSIX standard, whereasfind -exec {} +is, which is important if you are going for portability.
– jw013
Jun 27 '12 at 12:54
@Alexios You don't have to escape the plus sign, because it does not have a special meaning for the shell:find /tmp/ -exec ls "{}" +works just fine.
– daniel kullmann
Jun 27 '12 at 13:48
1
Corrent, of course. I've been escaping everything after the-execfor so long (I'm a masochist, I don't even use quotes to escape{}, I always type{}; don't ask), everything looks like it must be escaped now.
– Alexios
Jun 27 '12 at 14:17
1
@Alexios: If you find an example (except from being a masochist) where the masking of the braces is useful, please provide it as an answer to my question here - afaik this hint is outdated and only a relict in the manpage. I've never seen an example wherefind /tmp/ -exec ls {} +wouldn't work.
– user unknown
Jun 27 '12 at 15:00
1
For me, this is muscle memory formed in the days of SunOS 4. Since I've been doing it for years, I completely failed to notice whenbashstarted accepting the braces verbatim. I'm pretty sure at least one of the old shells I've used threw hissy fits if the braces weren't escaped.
– Alexios
Jun 27 '12 at 15:05
6
6
You meant
-P instead of -p. Keep in mind xargs -P is not in the POSIX standard, whereas find -exec {} + is, which is important if you are going for portability.– jw013
Jun 27 '12 at 12:54
You meant
-P instead of -p. Keep in mind xargs -P is not in the POSIX standard, whereas find -exec {} + is, which is important if you are going for portability.– jw013
Jun 27 '12 at 12:54
@Alexios You don't have to escape the plus sign, because it does not have a special meaning for the shell:
find /tmp/ -exec ls "{}" + works just fine.– daniel kullmann
Jun 27 '12 at 13:48
@Alexios You don't have to escape the plus sign, because it does not have a special meaning for the shell:
find /tmp/ -exec ls "{}" + works just fine.– daniel kullmann
Jun 27 '12 at 13:48
1
1
Corrent, of course. I've been escaping everything after the
-exec for so long (I'm a masochist, I don't even use quotes to escape {}, I always type {}; don't ask), everything looks like it must be escaped now.– Alexios
Jun 27 '12 at 14:17
Corrent, of course. I've been escaping everything after the
-exec for so long (I'm a masochist, I don't even use quotes to escape {}, I always type {}; don't ask), everything looks like it must be escaped now.– Alexios
Jun 27 '12 at 14:17
1
1
@Alexios: If you find an example (except from being a masochist) where the masking of the braces is useful, please provide it as an answer to my question here - afaik this hint is outdated and only a relict in the manpage. I've never seen an example where
find /tmp/ -exec ls {} + wouldn't work.– user unknown
Jun 27 '12 at 15:00
@Alexios: If you find an example (except from being a masochist) where the masking of the braces is useful, please provide it as an answer to my question here - afaik this hint is outdated and only a relict in the manpage. I've never seen an example where
find /tmp/ -exec ls {} + wouldn't work.– user unknown
Jun 27 '12 at 15:00
1
1
For me, this is muscle memory formed in the days of SunOS 4. Since I've been doing it for years, I completely failed to notice when
bash started accepting the braces verbatim. I'm pretty sure at least one of the old shells I've used threw hissy fits if the braces weren't escaped.– Alexios
Jun 27 '12 at 15:05
For me, this is muscle memory formed in the days of SunOS 4. Since I've been doing it for years, I completely failed to notice when
bash started accepting the braces verbatim. I'm pretty sure at least one of the old shells I've used threw hissy fits if the braces weren't escaped.– Alexios
Jun 27 '12 at 15:05
add a comment |
Concerning performance I thought that -exec … + would simply be better because it's a single tool doing all the work but a part of GNU findutil's documentation says that -exec … + might be less efficient in some cases:
[find with
-exec … +] can be less efficient than some uses ofxargs; for examplexargsallows new command lines to be built up while the previous command is still executing, and allows you to specify a number of commands to run in parallel. However, thefind ... -exec ... +construct has the advantage of wide portability. GNU findutils did not support ‘-exec ... +’ until version 4.2.12 [January 2005]; one of the reasons for this is that it already had the ‘-print0’ action in any case.
I wasn't exactly sure what that meant so I asked in the chat where derobert explained it as:
findprobably could continue to search for the next batch of files while the-exec … +is running, but it doesn't.find … | xargs …does, because then the find is a different process, and it keeps running until the pipe buffer fills up
(Formatting by me.)
So there is that. But if performance really matters you would have to do realistic benchmarking or rather even ask yourself whether you even would want to use shell for such cases.
Here on this site I think it's better to advise people to use the -exec … + form whenever possible because just because it's simpler and for the reasons mentioned in the other answers here (e.g. handling strange filenames without having to think much).
add a comment |
Concerning performance I thought that -exec … + would simply be better because it's a single tool doing all the work but a part of GNU findutil's documentation says that -exec … + might be less efficient in some cases:
[find with
-exec … +] can be less efficient than some uses ofxargs; for examplexargsallows new command lines to be built up while the previous command is still executing, and allows you to specify a number of commands to run in parallel. However, thefind ... -exec ... +construct has the advantage of wide portability. GNU findutils did not support ‘-exec ... +’ until version 4.2.12 [January 2005]; one of the reasons for this is that it already had the ‘-print0’ action in any case.
I wasn't exactly sure what that meant so I asked in the chat where derobert explained it as:
findprobably could continue to search for the next batch of files while the-exec … +is running, but it doesn't.find … | xargs …does, because then the find is a different process, and it keeps running until the pipe buffer fills up
(Formatting by me.)
So there is that. But if performance really matters you would have to do realistic benchmarking or rather even ask yourself whether you even would want to use shell for such cases.
Here on this site I think it's better to advise people to use the -exec … + form whenever possible because just because it's simpler and for the reasons mentioned in the other answers here (e.g. handling strange filenames without having to think much).
add a comment |
Concerning performance I thought that -exec … + would simply be better because it's a single tool doing all the work but a part of GNU findutil's documentation says that -exec … + might be less efficient in some cases:
[find with
-exec … +] can be less efficient than some uses ofxargs; for examplexargsallows new command lines to be built up while the previous command is still executing, and allows you to specify a number of commands to run in parallel. However, thefind ... -exec ... +construct has the advantage of wide portability. GNU findutils did not support ‘-exec ... +’ until version 4.2.12 [January 2005]; one of the reasons for this is that it already had the ‘-print0’ action in any case.
I wasn't exactly sure what that meant so I asked in the chat where derobert explained it as:
findprobably could continue to search for the next batch of files while the-exec … +is running, but it doesn't.find … | xargs …does, because then the find is a different process, and it keeps running until the pipe buffer fills up
(Formatting by me.)
So there is that. But if performance really matters you would have to do realistic benchmarking or rather even ask yourself whether you even would want to use shell for such cases.
Here on this site I think it's better to advise people to use the -exec … + form whenever possible because just because it's simpler and for the reasons mentioned in the other answers here (e.g. handling strange filenames without having to think much).
Concerning performance I thought that -exec … + would simply be better because it's a single tool doing all the work but a part of GNU findutil's documentation says that -exec … + might be less efficient in some cases:
[find with
-exec … +] can be less efficient than some uses ofxargs; for examplexargsallows new command lines to be built up while the previous command is still executing, and allows you to specify a number of commands to run in parallel. However, thefind ... -exec ... +construct has the advantage of wide portability. GNU findutils did not support ‘-exec ... +’ until version 4.2.12 [January 2005]; one of the reasons for this is that it already had the ‘-print0’ action in any case.
I wasn't exactly sure what that meant so I asked in the chat where derobert explained it as:
findprobably could continue to search for the next batch of files while the-exec … +is running, but it doesn't.find … | xargs …does, because then the find is a different process, and it keeps running until the pipe buffer fills up
(Formatting by me.)
So there is that. But if performance really matters you would have to do realistic benchmarking or rather even ask yourself whether you even would want to use shell for such cases.
Here on this site I think it's better to advise people to use the -exec … + form whenever possible because just because it's simpler and for the reasons mentioned in the other answers here (e.g. handling strange filenames without having to think much).
answered Nov 6 '16 at 17:44
phkphk
4,03252153
4,03252153
add a comment |
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f41740%2ffind-exec-vs-find-xargs-which-one-to-choose%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown