Filesystem races

[hanzer]

Forked from: Building ports with USE_SYSTRACE=Yes

Curious about how 'mkdir -p' and 'install -d' are touching the filesystem, I quickly ended up in the source code which led me to a [very] brief discussion/mention of the latest revision of mkdir.

From - Fixing mkdir -p vs. systrace

If you spawn something like

mkdir -p a
mkdir -p a/b
mkdir -p a/b/c

in parallel on a multiprocessor box, you will sometimes get an error
like "mkdir: a/b: File exists".

The whole point of mkdir(1)'s -p option is that this shouldn't
happen.

The problem was already fixed in revision 1.20 of src/bin/mkdir/mkdir.c.
Then Todd tweaked it in 1.21 to avoid breaking systrace policies,
but added back a race condition.

The diff below backs out the change from 1.21 and restores the
previous, more robust behavior. Todd is fine with it.

Any comments from the people who actually use systrace?

This seems a bit off [to me]. Abstractly, if multiple agents are acting upon a shared resource without any kind of concurrency control, there are going to be problems (i.e., races). Since FFS isn't a transactional filesystem, [it seems like] both of the latest versions of mkdir are going to have the same potential for race problems (but the previous version of mkdir plays nice with systrace).

Here's a goofy way to watch concurrency collisions occur on filesystem operations.

foo.ksh

#!/bin/ksh
I=0
J=$HOME
while [ $I -lt 80 ]
do
        I=`expr $I + 1`
        J=$J/$I
        mkdir -p $J
done

bar.ksh

#!/bin/ksh
I=0
while [ $I -lt 80 ]
do
        I=`expr $I + 1`
        rm -r $HOME/1
done

baz.ksh

#!/bin/ksh
foo.ksh &
foo.ksh &
bar.ksh &
foo.ksh &
foo.ksh &

Put them all in ${HOME}/bin then:
$ chmod u+x foo.ksh
$ chmod u+x bar.ksh
$ chmod u+x baz.ksh
$ baz.ksh 2> log

Both rm and mkdir should fail in a various ways. If there was some kind of concurrency control on the filesystem such that each rm and mkdir operation were an atomic transaction, this wouldn't happen. But there's not, so it does. I don't think tuning mkdir would solve the fundamental issue in any significant way (a very fast mkdir might have a lower probability of concurrency collisions than a very slow mkdir).

Assuming all of this is correct, I think the systrace friendly version of mkdir is nicer (well, if there were also a systrace friendly version of install).

Is this all crazy (have I missed the boat somewhere)?