|
|
||||
Making find -exec faster
Here’s a little
find trick that not many people seem to know:Code:
# 13 seconds... $ time find . -type f -exec stat {} \; > /dev/null 13.20s real 3.94s user 9.22s sys # 1.5 seconds! That's almost 10 times faster! $ time find . -type f -exec stat {} + > /dev/null 1.48s real 0.68s user 0.79s sys # Run the first command again, to make sure we’re not being biased by fs # cache or got some fluke [~]% time find . -type f -exec stat {} \; > /dev/null 13.40s real 3.67s user 9.51s sys # FYI... [~]% find . -type f | wc -l 2641 ; for a + .Let’s see what POSIX has to say about it (emphases mine): Quote:
; , find will execute the utility once for every path; if you use + , it will cram as many paths as it can in an invocation.How many? Well, as many as ARG_MAX allows. Quoting from POSIX Again:Quote:
ARG_MAX as 131072 (128k), while FreeBSD (10, i386) gives it as 262144 (256k).Let’s verify this with truss [^1]:Code:
# Amount of files we have $ find . -type f | wc -l 2641 $ truss find . -type f -exec stat {} \; >& truss-slow $ truss find . -type f -exec stat {} + >& truss-fast # Less than ARG_MAX, so we expect one fork() $ find . -type f | xargs | wc -c 119528 # Yup! $ grep fork truss-fast | wc -l 1 # And we fork() once for every file $ grep fork truss-slow | wc -l 2641 There is one small caveat, this won’t work: Code:
# FreeBSD find $ find . -type f -exec cp {} /tmp + find: -exec: no terminating ";" or "+" # GNU find is even more cryptic: $ find: missing argument to `-exec' Quote:
{} + . cp {} /tmp + doesn’t, and thus gives an error.We can work around this by spawning a sh one-liner:Code:
$ find . -type f -exec sh -c 'cp "$@" /tmp' {} +
__________________
UNIX was not designed to stop you from doing stupid things, because that would also stop you from doing clever things. |
|
|||
Ahh! Great, many thanks for the tip Carpetsmoker.
__________________
www.tacoshack.xyz |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Making your own Distro | SL6-A2000 | FreeBSD General | 4 | 22nd August 2011 01:50 PM |
HBGary chief exec resigns over Anon hack | J65nko | News | 0 | 1st March 2011 06:24 PM |
HOWTO: Lightest XFCE - Making XFCE lighter and faster | vermaden | Guides | 27 | 2nd September 2010 12:24 PM |
init: can't exec getty after power failure | mtx | FreeBSD General | 0 | 20th January 2009 10:14 AM |
Exec command in zsh prompt? | bsddaemon | General software and network | 2 | 18th October 2008 09:37 PM |