Everything you never wanted to know about file locking (2010)
1–10 of 18 posts
Re: Everything you never wanted to know about file locking (2010)
#2So good in depth post. THANK YOU.
Re: Everything you never wanted to know about file locking (2010)
#3Another good read is the SQLite locking module, https://www.sqlite.org/src/artifact/0240c5b547b4cf585c8cac35..., since these guys have to deal with the insanity of locking across different systems in real life.
You know things are bad when the least awful implementation of OS-level locking is the one from Microsoft.
Re: Everything you never wanted to know about file locking (2010)
#4One sure way to get a lock is to make a directory.
#!/bin/sh
if mkdir /your/lockdir
then trap "rmdir /your/lockdir" EXIT INT ABRT TERM
...code goes here...
else echo somebody else has the lock
fi
Re: Everything you never wanted to know about file locking (2010)
#5Usually when I read these writeups, I walk away thinking "Wow, $foo was a more complicated problem than I thought".
With this one, it was "Wow, $foo was a simpler problem than I thought and Unix (and thus Linux and OSX) just totally screwed it up for no reason"
Re: Everything you never wanted to know about file locking (2010)
#6One sure way to get a lock is to make a directory. #!/bin/sh if mkdir /your/lockdir then trap "rmdir /your/lockdir" EXIT INT ABRT TERM ...code goes here... else echo somebody else has the lock fi No matter how many processes attempt to make the directory, only one will succeed. That works for my scripting, but I have never used it in C.
this is great thanks,
was just wondering, could something else remove the dir in between the if and then, before trap?
Just wondering about the atomicity.
Re: Everything you never wanted to know about file locking (2010)
#7One sure way to get a lock is to make a directory. #!/bin/sh if mkdir /your/lockdir then trap "rmdir /your/lockdir" EXIT INT ABRT TERM ...code goes here... else echo somebody else has the lock fi No matter how many processes attempt to make the directory, only one will succeed. That works for my scripting, but I have never used it in C.
this is great thanks, was just wondering, could something else remove the dir in between the if and then, before trap? Just wondering about the atomicity.
Yes, but that is not a weakness in the locking.
Re: Everything you never wanted to know about file locking (2010)
#8One sure way to get a lock is to make a directory. #!/bin/sh if mkdir /your/lockdir then trap "rmdir /your/lockdir" EXIT INT ABRT TERM ...code goes here... else echo somebody else has the lock fi No matter how many processes attempt to make the directory, only one will succeed. That works for my scripting, but I have never used it in C.
this is great thanks, was just wondering, could something else remove the dir in between the if and then, before trap? Just wondering about the atomicity.
The permissions on the parent and lock directory could restrict the access to a specific user and group, but yes, other processes could interfere with this locking if directed to do so.
One condition where this interference is helpful is a crash, where a @reboot entry in the crontab could:
[ -d /your/lockdir ] && rmdir /your/lockdir
My biggest use case for directory locking in scripts is handling inotify events.
Re: Everything you never wanted to know about file locking (2010)
#9As TFA mentions, Unix/POSIX locking is insane.
Note that this page is slightly outdated wrt. flock(). From the manpage (online at https://man7.org/linux/man-pages/man2/flock.2.html):
>
Since Linux 2.0, flock() is implemented as a system call in its
own right rather than being emulated in the GNU C library as a
call to fcntl(2). With this implementation, there is no
interaction between the types of lock placed by flock() and
fcntl(2), and flock() does not detect deadlock. (Note, however,
that on some systems, such as the modern BSDs, flock() and
fcntl(2) locks do interact with one another.)
CIFS details
Up to Linux 5.4, flock() is not propagated over SMB. A file with
such locks will not appear locked for remote clients.
Since Linux 5.5, flock() locks are emulated with SMB byte-range
locks on the entire file. Similarly to NFS, this means that
fcntl(2) and flock() locks interact with one another. Another
important side-effect is that the locks are not advisory anymore:
any IO on a locked file will always fail with EACCES when done
from a separate file descriptor. This difference originates from
the design of locks in the SMB protocol, which provides mandatory
locking semantics.
Remote and mandatory locking semantics may vary with SMB protocol,
mount options and server type. See mount.cifs(8) for additional
information.
NFS details
Up to Linux 2.6.11, flock() does not lock files over NFS (i.e.,
the scope of locks was limited to the local system). Instead, one
could use fcntl(2) byte-range locking, which does work over NFS,
given a sufficiently recent version of Linux and a server which
supports locking.
Since Linux 2.6.12, NFS clients support flock() locks by emulating
them as fcntl(2) byte-range locks on the entire file. This means
that fcntl(2) and flock() locks do interact with one another over
NFS. It also means that in order to place an exclusive lock, the
file must be opened for writing.
Since Linux 2.6.37, the kernel supports a compatibility mode that
allows flock() locks (and also fcntl(2) byte region locks) to be
treated as local; see the discussion of the local_lock option in
nfs(5).Re: Everything you never wanted to know about file locking (2010)
#10Hmm I just ran into an issue with uv where it deadlocks because of something to do with file locking on NFS. This looks informative!