Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Things UNIX can do atomically

rcrowley.org

Post reply on HN
Print view View original

11–20 of 41 posts

Re: Things UNIX can do atomically

#11

Post by bshep »

ajross wrote: post #8
bshep wrote: post #6

Earlier quoted context omitted.

To change a symlink I use: ln -snf NEW_SOURCE TARGET

Which works fine, but is still not atomic. To be clear, saying something is "atomic" means that there is no way for another process to see the process in an intermediate state (in the example in the article: a missing or incorrect symlink to a deployed web application). Underneath the hood, the /bin/ln program still needs to call unlink() on the path first, so there is a short period of time during which the desired…

I never said it was atomic ;-) just that i use that method rather than mv.

I mainly use it when i'm updating apps or SDks to newer versions and want to keep the old ones around just in case.

Re: Things UNIX can do atomically

#12

Post by JeffJenkins »

I just recently discovered fcntl for creating file-based locks (and the python function which calls it, fcntl.flock) and was pretty excited. It seems like a much safer way to do multi-process locking than creating lock files on my own

Re: Things UNIX can do atomically

#13

Post by sophacles »

Skriticos wrote: post #4

I have a copy of "Advanced Programming in the UNIX Environment, Second Edition" (featuring very good description of C/POSIX/SuS/XSI protocols/api's and programming techniques for Linux/BSD/OSX/Solaris/other UNIX systems). Anyone who seriously wants to do thread safe multiprocessing or other complex low level stuff on UNIX platforms should have a copy of it. It's basically "the book" on the topic.

This is one of a very small number of books older than 5 years which I still regularly reference. Amusingly, most of the books on that list were written by W Richcard Stevens as well.

Re: Things UNIX can do atomically

#14

Post by leoc »

Skriticos wrote: post #4

I have a copy of "Advanced Programming in the UNIX Environment, Second Edition" (featuring very good description of C/POSIX/SuS/XSI protocols/api's and programming techniques for Linux/BSD/OSX/Solaris/other UNIX systems). Anyone who seriously wants to do thread safe multiprocessing or other complex low level stuff on UNIX platforms should have a copy of it. It's basically "the book" on the topic.

The Linux-specific /Linux Application Development/ by Johnson and Troan is very good too. http://ladweb.net/

Re: Things UNIX can do atomically

#16

Post by briansmith »

sid0 wrote: post #15

For what it's worth, Windows can do any arbitrary sequence of file operations atomically, using transactional NTFS.

Transactional NTFS only works on Vista, Server 2003, and later. It only works on local drives. There are many other limitations. Because of these limitations, I have not yet been able to use Transacitonal NTFS in any real program. Somebody always wants to store all their stuff--even critical stuff--on some network share.

Re: Things UNIX can do atomically

#17

Post by carey »

JeffJenkins wrote: post #12

I just recently discovered fcntl for creating file-based locks (and the python function which calls it, fcntl.flock) and was pretty excited. It seems like a much safer way to do multi-process locking than creating lock files on my own

There is a catch, though; if you have the same file open multiple times, even with different names, then closing one file descriptor will release all locks on that file in the same process, even in different threads.

SQLite goes to great lengths to work around this. See: http://www.google.com/search?q=broken+by+design+site:sqlite....

Re: Things UNIX can do atomically

#18

Post by geocar »

prodigal_erik wrote: post #9

pipe(&pipefd); assert(len Also, if a file is opened O_APPEND, the seek before every write is atomic, but since writes in general aren't atomic I'm not sure what value that has.

This is a great one: Under POSIX, any write() to a fifo that has at least PIPE_BUF free is atomic.

There's a neat trick I'd like to add that takes advantage of this: Multiple clients can use a message whose size divides evenly into PIPE_BUF without any possibility of interleave, and so they do not need to lock.

    int n=sizeof(struct message);
    struct message m[n/PIPE_BUF];
    assert(sizeof(m)==PIPE_BUF);
    if(pipe(fds)==-1)abort();
    for(i=0;i=0;--i)handle(m+i);}
Each child can write entire messages, without locking and without interleaving, so long as sizeof(struct message)*n is exactly PIPE_BUF.

Btw, your pipe(&pipefd) should've been pipe(pipefd) if pipefd is an array of 2 int

Re: Things UNIX can do atomically

#20

Post by sid0 »

briansmith wrote: post #16
sid0 wrote: post #15

For what it's worth, Windows can do any arbitrary sequence of file operations atomically, using transactional NTFS.

Transactional NTFS only works on Vista, Server 2003, and later. It only works on local drives. There are many other limitations. Because of these limitations, I have not yet been able to use Transacitonal NTFS in any real program. Somebody always wants to store all their stuff--even critical stuff--on some network share.

> Server 2003

Do you mean Server 2008?

I agree that the fact that it's only supported on Vista and later is a problem, but XP will hopefully slide into irrelevancy in the next year or so.

> It only works on local drives.

Are the things described in the blog post atomic over NFS? I remember reading that NFS breaks plenty of the guarantees that a POSIX system provides. Guaranteeing atomicity over a network does seem to be a somewhat harder job than guaranteeing it locally.

Post reply on HN