Live data from Hacker News

Things Unix can do atomically (2010)

rcrowley.org

31–40 of 68 posts

Re: Things Unix can do atomically (2010)

#31
post #9

Earlier quoted context omitted.

It should also be said that `mv`(1) is only atomic if the source and destination are on the same filesystem.

So that implies that realistically there's a scenario where moving files or directories from one filesystem to another and some interruption occurs can lead to lost data?

[deleted]

Re: Things Unix can do atomically (2010)

#32
post #2

In a few simple words, can someone explain what does "atomically" mean? I personally used this term when talking about some Redis operations, but never knew the real gist of the word and concepts behind it. I have a very brief understanding of the term and if I'd have to explain it to a person, I'd say it's "the operation that does not have any side effects when performing its unit of work". Is my understanding even…

[deleted]

Re: Things Unix can do atomically (2010)

#33
post #9

Earlier quoted context omitted.

It should also be said that `mv`(1) is only atomic if the source and destination are on the same filesystem.

So that implies that realistically there's a scenario where moving files or directories from one filesystem to another and some interruption occurs can lead to lost data?

Very implausible. Mv across filesystems is usually implemented as a copy then delete. Worst case, you'll have the data exist in both locations.

Re: Things Unix can do atomically (2010)

#34
Ah the good ol day's, this reminded me of some file transfer problems we used to get.

We had multiple systems that generated usage records, and stored them to flat file (think stuff that would end up on a bill). Because FTP the file was a thing, some other system would come in any copy the file, but every once in awhile there would be a partial file copied that would be missing records. Yep, it was the good ol process was still writing to the file when the collector decided to pick it up.

The first system I had control over, I made sure the vendor wrote to a temporary directory, and then hard linked to the transfer directory when the file operation was complete, knowing it avoided the race condition. I'm pretty sure I had one of the few platforms that handled this correctly, from what I remember we had corrupt files from almost all the platforms we bought.

Anyways, just because the system cost a million dollars doesn't mean it's any good.

Re: Things Unix can do atomically (2010)

#35
post #25

The GCC Atomic Builtins mentioned in the article are not specific to Unix. They are compiler constructs, and depend on specific architecture hardware support. All x86 CPUs have such support for some years now. So these atomic operations can also be used in non-Unix software running on x86 CPUs. The GCC documentation lists other non-intel architectures which also have the features required to support the atomic built-…

Also, if you can depend on recent compilers you should probably be using the standard C or C++ instead.

Re: Things Unix can do atomically (2010)

#36
post #2

In a few simple words, can someone explain what does "atomically" mean? I personally used this term when talking about some Redis operations, but never knew the real gist of the word and concepts behind it. I have a very brief understanding of the term and if I'd have to explain it to a person, I'd say it's "the operation that does not have any side effects when performing its unit of work". Is my understanding even…

Reminds me of graphics double buffering. Back in the days games would write directly in the video buffer, while the graphic chip would scan that same buffer and push the content on screen at the same time.

If your code is too slow (a complex effect, too many character at that point), you might not be done writing a full frame when the graphic chips starts to output the pixels.

This means your TV is now showing partly old and new state. Nothing important most of the time, it's only games, it's only a few ms of absurd information, people's brain can compensate. It is ugly to see though. You have that weird 'line' somewhere below.

Since people changed the structure a bit, with two (or more) buffers, the program computes the new image in one buffer B, while the chip shows another buffer A. When you are done with a picture, the chip will now scan B, while you can write in A. This means the output never shows partial frame anymore.

Re: Things Unix can do atomically (2010)

#37

Earlier quoted context omitted.

So that implies that realistically there's a scenario where moving files or directories from one filesystem to another and some interruption occurs can lead to lost data?

Very implausible. Mv across filesystems is usually implemented as a copy then delete. Worst case, you'll have the data exist in both locations.

It's possible, in case of physical disconnection or power interruption. There is no guarantee that the copied data will be flushed out of buffers into nonvolatile storage before the delete is (unless the program asks for such a flush and the system honors it).

Re: Things Unix can do atomically (2010)

#38
post #2

In a few simple words, can someone explain what does "atomically" mean? I personally used this term when talking about some Redis operations, but never knew the real gist of the word and concepts behind it. I have a very brief understanding of the term and if I'd have to explain it to a person, I'd say it's "the operation that does not have any side effects when performing its unit of work". Is my understanding even…

It's more about multiple operations taking effect "instantaneously" from the perspective of some observer, and sometimes it also implies that either all or none of the operations take effect, but never just some of the operations.

Re: Things Unix can do atomically (2010)

#39

Here's one that's not super well-known: writes to a pipe are atomic as long as the write size is Anyone know if there's a similar guarantee for files?

I read a while back that this PIPE_BUF restriction also applies to multiple writers appending a file.

Re: Things Unix can do atomically (2010)

#40
post #2

In a few simple words, can someone explain what does "atomically" mean? I personally used this term when talking about some Redis operations, but never knew the real gist of the word and concepts behind it. I have a very brief understanding of the term and if I'd have to explain it to a person, I'd say it's "the operation that does not have any side effects when performing its unit of work". Is my understanding even…

I googled for "atomic operation" and this was top result:

https://en.wikipedia.org/wiki/Linearizability

Post reply on HN