DaemonForums  

Go Back   DaemonForums > OpenBSD > OpenBSD General

OpenBSD General Other questions regarding OpenBSD which do not fit in any of the categories below.

Reply
 
Thread Tools Display Modes
  #1   (View Single Post)  
Old 1 Week Ago
hanzer's Avatar
hanzer hanzer is online now
Real Name: Adam Jensen
cybernetic organism
 
Join Date: Oct 2013
Location: Oak Ridge, TN
Posts: 71
Thanked 0 Times in 0 Posts
Default Filesystem races

Forked from: Building ports with USE_SYSTRACE=Yes

Curious about how 'mkdir -p' and 'install -d' are touching the filesystem, I quickly ended up in the source code which led me to a [very] brief discussion/mention of the latest revision of mkdir.

From - Fixing mkdir -p vs. systrace
Quote:
If you spawn something like

mkdir -p a
mkdir -p a/b
mkdir -p a/b/c

in parallel on a multiprocessor box, you will sometimes get an error
like "mkdir: a/b: File exists".

The whole point of mkdir(1)'s -p option is that this shouldn't
happen.

The problem was already fixed in revision 1.20 of src/bin/mkdir/mkdir.c.
Then Todd tweaked it in 1.21 to avoid breaking systrace policies,
but added back a race condition.

The diff below backs out the change from 1.21 and restores the
previous, more robust behavior. Todd is fine with it.

Any comments from the people who actually use systrace?
This seems a bit off [to me]. Abstractly, if multiple agents are acting upon a shared resource without any kind of concurrency control, there are going to be problems (i.e., races). Since FFS isn't a transactional filesystem, [it seems like] both of the latest versions of mkdir are going to have the same potential for race problems (but the previous version of mkdir plays nice with systrace).

Here's a goofy way to watch concurrency collisions occur on filesystem operations.

foo.ksh
Code:
#!/bin/ksh
I=0
J=$HOME
while [ $I -lt 80 ]
do
        I=`expr $I + 1`
        J=$J/$I
        mkdir -p $J
done
bar.ksh
Code:
#!/bin/ksh
I=0
while [ $I -lt 80 ]
do
        I=`expr $I + 1`
        rm -r $HOME/1
done
baz.ksh
Code:
#!/bin/ksh
foo.ksh &
foo.ksh &
bar.ksh &
foo.ksh &
foo.ksh &
Put them all in ${HOME}/bin then:
$ chmod u+x foo.ksh
$ chmod u+x bar.ksh
$ chmod u+x baz.ksh
$ baz.ksh 2> log

Both rm and mkdir should fail in a various ways. If there was some kind of concurrency control on the filesystem such that each rm and mkdir operation were an atomic transaction, this wouldn't happen. But there's not, so it does. I don't think tuning mkdir would solve the fundamental issue in any significant way (a very fast mkdir might have a lower probability of concurrency collisions than a very slow mkdir).

Assuming all of this is correct, I think the systrace friendly version of mkdir is nicer (well, if there were also a systrace friendly version of install).

Is this all crazy (have I missed the boat somewhere)?
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
filesystem full? amandus FreeBSD General 11 11th June 2013 12:10 PM
mounting hfsplus filesystem daemonfowl OpenBSD General 17 7th September 2012 08:24 PM
dvd filesystem type chill FreeBSD General 6 17th June 2008 08:22 AM
Best Filesystem thavinci FreeBSD General 25 29th May 2008 03:32 PM
Filesystem read errors Foon FreeBSD General 0 10th May 2008 07:27 AM


All times are GMT. The time now is 08:27 PM.


Powered by vBulletin® Version 3.8.4
Copyright ©2000 - 2015, Jelsoft Enterprises Ltd.
Content copyright © 2007-2010, the authors
Daemon image copyright ©1988, Marshall Kirk McKusick