Here's one that's not super well-known: writes to a pipe are atomic as long as the write size is Anyone know if there's a similar guarantee for files?
Things Unix can do atomically (2010)
21–30 of 68 posts
Re: Things Unix can do atomically (2010)
#22Here's one that's not super well-known: writes to a pipe are atomic as long as the write size is Anyone know if there's a similar guarantee for files?
They won't be split, bit one thing to watch out for is that they can be coalesced.
Re: Things Unix can do atomically (2010)
#23Earlier quoted context omitted.
Got it. Now it makes more sense to me. Now I know people tend to talk about atomicity when it comes to low-level-ish things. But say I create some sort of a web service with a bunch of business logic. Does it worth to follow this principle in that case? For instance, client sends an API request (let's say "Add user to friends"), is it even possible to apply atomicity for these type of things? Edit: Thanks everyone fo…
The business logic of your web application should resolve around database calls. Popular databases should already guarantee these atomicity properties for you through transactions.
> The business logic of your web application should resolve around database calls
Might, not should. All web apps are not just front ends to a single database where transactions are useful and once you leave the realm of a single database into a more distributed type system then transactions are no longer an option.
Re: Things Unix can do atomically (2010)
#24In a few simple words, can someone explain what does "atomically" mean? I personally used this term when talking about some Redis operations, but never knew the real gist of the word and concepts behind it. I have a very brief understanding of the term and if I'd have to explain it to a person, I'd say it's "the operation that does not have any side effects when performing its unit of work". Is my understanding even…
If something alters an object such that it goes from state A to state B, it might need to do some work along the way (call it "state C"). Atomicity means that if the operation is interrupted or observed while it's going on, the existence of a "state C" never leaks out. It's always in either state A or B; any third state that might exist along the way is never visible. (Hence what people often say: the operation is ei…
edit: I just realized you said "renaming." Original comment left below, but I edited before I get downvoted for a classic reading comprehension fail.
Atomicity requires that the leakage mentioned shall not occur from any context aside from its own internal context. That makes your example somewhat of a simplification because these state transitions are visible to other processes. It is a common mistake to try to use files for locking, for example, instead of using the more robust flock(1).
Re: Things Unix can do atomically (2010)
#25The GCC Atomic Builtins mentioned in the article are not specific to Unix. They are compiler constructs, and depend on specific architecture hardware support. All x86 CPUs have such support for some years now. So these atomic operations can also be used in non-Unix software running on x86 CPUs.
The GCC documentation lists other non-intel architectures which also have the features required to support the atomic built-ins.
Re: Things Unix can do atomically (2010)
#26msync() with MS_INVALIDATE doesn't belong on this list. It has nothing to do with atomic memory access. msync() is used when flushing a mapped file to durable storage. I very often see this mistake of conflating flushing caches with atomic access of memory. What's committed to durable storage has nothing to do with what multiple processes will see when mapping a file. All that's needed is the initial mmap to share a…
I haven't tested but I would expect MS_INVALIDATE on a large buffer to be much faster than filling it a word at a time with __sync_val_compare_and_swap (each causing its own bus transaction).
msync() doesn't fill buffers at all. It has no function in the operation you've described.
Even if it did, it's not atomic ...
Re: Things Unix can do atomically (2010)
#27rename(2) was not atomic on OS X for years. It was finally fixed in Lion: http://www.weirdnet.nl/apple/rename.html
It should also be said that `mv`(1) is only atomic if the source and destination are on the same filesystem.
So that implies that realistically there's a scenario where moving files or directories from one filesystem to another and some interruption occurs can lead to lost data?
Re: Things Unix can do atomically (2010)
#28Earlier quoted context omitted.
If something alters an object such that it goes from state A to state B, it might need to do some work along the way (call it "state C"). Atomicity means that if the operation is interrupted or observed while it's going on, the existence of a "state C" never leaks out. It's always in either state A or B; any third state that might exist along the way is never visible. (Hence what people often say: the operation is ei…
edit: I just realized you said "renaming." Original comment left below, but I edited before I get downvoted for a classic reading comprehension fail. Atomicity requires that the leakage mentioned shall not occur from any context aside from its own internal context. That makes your example somewhat of a simplification because these state transitions are visible to other processes. It is a common mistake to try to use…
Nonetheless the reply prompted me to look into the rename case specifically. Apparently on Linux, the replacement of the destination file is atomic (as many of us already knew and take for granted), but there's no guarantee that you won't see both old and new names in flight for a brief moment in time (like the last sentence of my comment).
Would not be surprised if all bets are off once you get an NFS mount involved.
As always it's a tradeoff between useful behaviors and the cost of synchronizing.
Re: Things Unix can do atomically (2010)
#29Earlier quoted context omitted.
It should also be said that `mv`(1) is only atomic if the source and destination are on the same filesystem.
So that implies that realistically there's a scenario where moving files or directories from one filesystem to another and some interruption occurs can lead to lost data?
It's easy to design an algorithm that either succeeds with a move or fails with a copy, never losing data, but someone would have to look deeper into what was actually guaranteed and/or implemented.
Re: Things Unix can do atomically (2010)
#30Earlier quoted context omitted.
Not really. It just means that it's indivisible (the original meaning of "atom"). Either it succeeds or fails, you never have to worry about it being half-finished. This includes actions which are so small they are literally indivisible, or actions which roll back to the original state if they fail.
Got it. Now it makes more sense to me. Now I know people tend to talk about atomicity when it comes to low-level-ish things. But say I create some sort of a web service with a bunch of business logic. Does it worth to follow this principle in that case? For instance, client sends an API request (let's say "Add user to friends"), is it even possible to apply atomicity for these type of things? Edit: Thanks everyone fo…
Here are some real-world examples of what can go wrong if your API requests aren't atomic http://josipfranjkovic.blogspot.jp/2015/04/race-conditions-o...