Beware: rename() isn't atomic on Mac OS X: http://www.weirdnet.nl/apple/rename.html
Wow, this breaks a lot of UNIX.
Things UNIX can do atomically
21–30 of 41 posts
Re: Things UNIX can do atomically
#22One more: mkdir(2)
How would mkdir not be atomic? It is a logical operation that only encapsulates a single physical operation.
Rename is an interesting case, because it encapsulates two operations - a copy and a delete. Doing two operations in one step is interesting atomicity. Doing one operation in one step (mkdir) is not as interesting.
Re: Things UNIX can do atomically
#23I just recently discovered fcntl for creating file-based locks (and the python function which calls it, fcntl.flock) and was pretty excited. It seems like a much safer way to do multi-process locking than creating lock files on my own
IIRC opening a file and using link() is about the only portable atomic method of file locking. Works on NFS and pretty much any UNIX. There may have been some other trick to making it work right on NFS but if you're going that far you may just need a lock manager.
Re: Things UNIX can do atomically
#24Beware: rename() isn't atomic on Mac OS X: http://www.weirdnet.nl/apple/rename.html
What's weird is the timeline on that, if you see when it was reported and when it was still present. Surely it can't be that hard to fix this in a proper way ?
Re: Things UNIX can do atomically
#25I just recently discovered fcntl for creating file-based locks (and the python function which calls it, fcntl.flock) and was pretty excited. It seems like a much safer way to do multi-process locking than creating lock files on my own
There is a catch, though; if you have the same file open multiple times, even with different names, then closing one file descriptor will release all locks on that file in the same process, even in different threads. SQLite goes to great lengths to work around this. See: http://www.google.com/search?q=broken+by+design+site:sqlite....
There is a catch, though; if you have the same file open multiple times, even with different names, then closing one file descriptor will release all locks on that file in the same process, even in different threads.
It's even worse than that. A lock is released as soon as any of its owning processes closes any descriptor to that file. As a result, if a child forks and exits, the parent no longer holds any locks...
Re: Things UNIX can do atomically
#26One more: mkdir(2)
How would mkdir not be atomic? It is a logical operation that only encapsulates a single physical operation. Rename is an interesting case, because it encapsulates two operations - a copy and a delete. Doing two operations in one step is interesting atomicity. Doing one operation in one step (mkdir) is not as interesting.
See Phrack 25: http://www.phrack.com/issues.html?issue=25&id=5 for what it was like when mkdir was not atomic.
Re: Things UNIX can do atomically
#27For what it's worth, Windows can do any arbitrary sequence of file operations atomically, using transactional NTFS.
Transactional NTFS only works on Vista, Server 2003, and later. It only works on local drives. There are many other limitations. Because of these limitations, I have not yet been able to use Transacitonal NTFS in any real program. Somebody always wants to store all their stuff--even critical stuff--on some network share.
And on iSCSI LUNs IIRC, which is how most people would use consolidated storage (SAN vs NAS).
Re: Things UNIX can do atomically
#28pipe(&pipefd); assert(len Also, if a file is opened O_APPEND, the seek before every write is atomic, but since writes in general aren't atomic I'm not sure what value that has.
Makes logging things easier, and in correct order!
Re: Things UNIX can do atomically
#29The very first example ("mv -T" to retarget a symlink, something I didn't know about and that the manpage is silent on) is not atomic! The symlink(2) syscall will not overwrite an existing directory entry. You have to call unlink(2) first, which opens a race. That doesn't mean this trick is useless, but calling it "atomic" is wrong, and opens up the possibility of bugs (likely security holes, even) if someone actuall…
> something I didn't know about and that the manpage is silent on It's specific to GNU Coreutils mv(1).
[deleted]