Live data from Hacker News

Things Unix can do atomically (2010)

rcrowley.org

41–50 of 68 posts

Re: Things Unix can do atomically (2010)

#41

Here's one that's not super well-known: writes to a pipe are atomic as long as the write size is Anyone know if there's a similar guarantee for files?

I think Linux goes out of its ways to make writes <= 512 bytes atomic. Helpful for writing log files. But I don't think this is a true guarantee in any standardised sense.

Re: Things Unix can do atomically (2010)

#42
post #2

In a few simple words, can someone explain what does "atomically" mean? I personally used this term when talking about some Redis operations, but never knew the real gist of the word and concepts behind it. I have a very brief understanding of the term and if I'd have to explain it to a person, I'd say it's "the operation that does not have any side effects when performing its unit of work". Is my understanding even…

> I'd say it's "the operation that does not have any side effects when performing its unit of work". Is my understanding even close to what atomic operation really is?

No. It can has as many side effects as it wants. Atomicity means: when going from state 1 to state 2, no matter how complex the transition, there are no externally observable intermediate states.

Re: Things Unix can do atomically (2010)

#43
post #24

Earlier quoted context omitted.

If something alters an object such that it goes from state A to state B, it might need to do some work along the way (call it "state C"). Atomicity means that if the operation is interrupted or observed while it's going on, the existence of a "state C" never leaks out. It's always in either state A or B; any third state that might exist along the way is never visible. (Hence what people often say: the operation is ei…

edit: I just realized you said "renaming." Original comment left below, but I edited before I get downvoted for a classic reading comprehension fail. Atomicity requires that the leakage mentioned shall not occur from any context aside from its own internal context. That makes your example somewhat of a simplification because these state transitions are visible to other processes. It is a common mistake to try to use…

> It is a common mistake to try to use files for locking, for example, instead of using the more robust flock(1).

Why is this a mistake? It is my understanding that, if all the locking you need is a simple mutex, creating a file with a well-defined name with O_CREAT | O_EXCL is atomic -- the file will either be created or not (in which case the call will fail with EEXIST), and no two processes can possibly both succeed at creating the file. This even works on NFS; it was apparently broken in the NFS client in Linux 2.6.5 and below, but it is supposed to work in NFS, and is generally the only reliable way of getting locks in NFS.

You don't get any better way to wait on the lock than re-trying to create the file, and you don't have any mechanism for dealing with clients that die while holding the lock (i.e., it's an aggressively CP system), but for what it does, it's supposed to work correctly and atomically.

Re: Things Unix can do atomically (2010)

#45

Here's one that's not super well-known: writes to a pipe are atomic as long as the write size is Anyone know if there's a similar guarantee for files?

The PIPE_BUF POSIX requirement is about guaranteeing that when multiple processes write less than PIPE_BUF to the same pipe, the reader will not see its input intermingled from different processes. It is often wrongly interpreted as "a single write() of less than PIPE_BUF will be retrieved via a single read() on the other side".

Re: Things Unix can do atomically (2010)

#46
post #2

In a few simple words, can someone explain what does "atomically" mean? I personally used this term when talking about some Redis operations, but never knew the real gist of the word and concepts behind it. I have a very brief understanding of the term and if I'd have to explain it to a person, I'd say it's "the operation that does not have any side effects when performing its unit of work". Is my understanding even…

Translating in English from Italian Wikipedia: From ancient Greek ἄτομος - àtomos - [indivisible], made of ἄ - a - [Privative alpha] + τέμνειν - témnein - [cut].

Personally, I struggled long time before fully understanding its use in IT, because I learned programming after sub-nuclear physics, thus I had a hard time conciliating the huge atom (a million of billions of times bigger than a nucleus) with the concept of "cannot be split" :-)

Re: Things Unix can do atomically (2010)

#47

Here's one that's not super well-known: writes to a pipe are atomic as long as the write size is Anyone know if there's a similar guarantee for files?

Opening with O_APPEND you get atomic appends if they're And the list already mentions atomic memory operations. Those also apply to memory-mapped files.

Re: Things Unix can do atomically (2010)

#48

It should be noted that those filesystem operations are only atomic with respect to an observer running on the same operating system incarnation. Whether they are also atomic across power loss depends on the filesystem (though with a modern journaling filesystem, that generally should be the case).

That's what people would expect. But there was some drama around ext4, renames and fsync a few years ago.

Re: Things Unix can do atomically (2010)

#49
post #10
post #3

Earlier quoted context omitted.

Not really. It just means that it's indivisible (the original meaning of "atom"). Either it succeeds or fails, you never have to worry about it being half-finished. This includes actions which are so small they are literally indivisible, or actions which roll back to the original state if they fail.

Got it. Now it makes more sense to me. Now I know people tend to talk about atomicity when it comes to low-level-ish things. But say I create some sort of a web service with a bunch of business logic. Does it worth to follow this principle in that case? For instance, client sends an API request (let's say "Add user to friends"), is it even possible to apply atomicity for these type of things? Edit: Thanks everyone fo…

Yes. But in this case, the concept you are probably looking for is "transactional". Transactions are atomic, but the difference is that they can fail (the state is then rolled back), and they can be retried at a later point.

Re: Things Unix can do atomically (2010)

#50
I always thought it would be a good idea for system calls to support transactions. Probably in a limited way because implementing general transactions would require massive changes to the kernel. But it would be nice to be able to do [error checking omitted]:

    begin ();
    fp = fopen ("file", "w");
    fputs (content, fp);
    fclose (fp);
    commit ();
It could solve the whole thing with ending up with zero-length files because you didn't use the right incantation to update a file atomically on ext4 (https://thunk.org/tytso/blog/2009/03/12/delayed-allocation-a...).

In Unix v7 mkdir was not a system call. It was a setuid program implemented using mknod + link. That was racy so the mkdir(2) system call was added. But it could have been solved more generally (and more elegantly) by adding transactions.

Post reply on HN