Things Unix can do atomically (2010) • Hacker News Board

Re: Things Unix can do atomically (2010)

#11

Post by stormbrew » Mon, May 30, 2016, 11:40 PM UTC

atrust wrote: post #10

sp332 wrote: post #3

Earlier quoted context omitted.

Not really. It just means that it's indivisible (the original meaning of "atom"). Either it succeeds or fails, you never have to worry about it being half-finished. This includes actions which are so small they are literally indivisible, or actions which roll back to the original state if they fail.

Got it. Now it makes more sense to me. Now I know people tend to talk about atomicity when it comes to low-level-ish things. But say I create some sort of a web service with a bunch of business logic. Does it worth to follow this principle in that case? For instance, client sends an API request (let's say "Add user to friends"), is it even possible to apply atomicity for these type of things? Edit: Thanks everyone fo…

Yes. To take a larger example action, creating a user could fail to be atomic if, say, it stored the username in a separate table from the user object, wrote the username first, then referenced it from the user object, but didn't roll back the username insert if the user object insert failed.

Likewise, for the seemingly simpler example if establishing a friend relationship, you may be tracking that relationship in both directions, in which case one could fail and the other succeed.

Re: Things Unix can do atomically (2010)

#12

Post by asveikau » Mon, May 30, 2016, 11:41 PM UTC

atrust wrote: post #2

In a few simple words, can someone explain what does "atomically" mean? I personally used this term when talking about some Redis operations, but never knew the real gist of the word and concepts behind it. I have a very brief understanding of the term and if I'd have to explain it to a person, I'd say it's "the operation that does not have any side effects when performing its unit of work". Is my understanding even…

If something alters an object such that it goes from state A to state B, it might need to do some work along the way (call it "state C"). Atomicity means that if the operation is interrupted or observed while it's going on, the existence of a "state C" never leaks out. It's always in either state A or B; any third state that might exist along the way is never visible. (Hence what people often say: the operation is either completed or didn't happen at all.)

Renaming a file is a good example. Within the internal structure of the filesystem, you have a directory entry in an old location. That must be removed. You may have another file with the same name in the destination directory. That file must be overwritten. Internally, these things happen by a multi-step process, eg: remove entry for old name, remove pre-existing entry for new name, create new entry for new name. But the system creates the appearance of just 1 step. You don't get file not found while it's overwriting the destination file. You don't ever see the file having both old and new names at the same time.

Re: Things Unix can do atomically (2010)

#13

Post by trungaczne » Mon, May 30, 2016, 11:41 PM UTC

atrust wrote: post #10

sp332 wrote: post #3

Earlier quoted context omitted.

Not really. It just means that it's indivisible (the original meaning of "atom"). Either it succeeds or fails, you never have to worry about it being half-finished. This includes actions which are so small they are literally indivisible, or actions which roll back to the original state if they fail.

Got it. Now it makes more sense to me. Now I know people tend to talk about atomicity when it comes to low-level-ish things. But say I create some sort of a web service with a bunch of business logic. Does it worth to follow this principle in that case? For instance, client sends an API request (let's say "Add user to friends"), is it even possible to apply atomicity for these type of things? Edit: Thanks everyone fo…

The business logic of your web application should resolve around database calls. Popular databases should already guarantee these atomicity properties for you through transactions.

Re: Things Unix can do atomically (2010)

#14

Post by toast0 » Mon, May 30, 2016, 11:49 PM UTC

atrust wrote: post #10

sp332 wrote: post #3

Earlier quoted context omitted.

Not really. It just means that it's indivisible (the original meaning of "atom"). Either it succeeds or fails, you never have to worry about it being half-finished. This includes actions which are so small they are literally indivisible, or actions which roll back to the original state if they fail.

Got it. Now it makes more sense to me. Now I know people tend to talk about atomicity when it comes to low-level-ish things. But say I create some sort of a web service with a bunch of business logic. Does it worth to follow this principle in that case? For instance, client sends an API request (let's say "Add user to friends"), is it even possible to apply atomicity for these type of things? Edit: Thanks everyone fo…

Yes, depending on how you store the relationships, its possible that when a adds b to its list of friends, a could be added to b's list of 'friendofs'. If they're stored separately, the edit may not be atomic. If it's stored in a single place (queryable from either direction), it's generally going to be atomic, unless you're doing something way outside the norm.

Re: Things Unix can do atomically (2010)

#15

Post by toast0 » Mon, May 30, 2016, 11:54 PM UTC

trungaczne wrote: post #13

atrust wrote: post #10

Earlier quoted context omitted.

Got it. Now it makes more sense to me. Now I know people tend to talk about atomicity when it comes to low-level-ish things. But say I create some sort of a web service with a bunch of business logic. Does it worth to follow this principle in that case? For instance, client sends an API request (let's say "Add user to friends"), is it even possible to apply atomicity for these type of things? Edit: Thanks everyone fo…

The business logic of your web application should resolve around database calls. Popular databases should already guarantee these atomicity properties for you through transactions.

Database guarantees are great until your data doesn't fit in a single database; it's good to examine what your database is providing for you, and to contemplate the costs for providing it in the database level.

Re: Things Unix can do atomically (2010)

#16

Post by [deleted] » Mon, May 30, 2016, 11:55 PM UTC

atrust wrote: post #10

sp332 wrote: post #3

Earlier quoted context omitted.

Not really. It just means that it's indivisible (the original meaning of "atom"). Either it succeeds or fails, you never have to worry about it being half-finished. This includes actions which are so small they are literally indivisible, or actions which roll back to the original state if they fail.

Got it. Now it makes more sense to me. Now I know people tend to talk about atomicity when it comes to low-level-ish things. But say I create some sort of a web service with a bunch of business logic. Does it worth to follow this principle in that case? For instance, client sends an API request (let's say "Add user to friends"), is it even possible to apply atomicity for these type of things? Edit: Thanks everyone fo…

[deleted]

Re: Things Unix can do atomically (2010)

#17

Post by anon946 » Tue, May 31, 2016, 12:03 AM UTC

atrust wrote: post #10

sp332 wrote: post #3

Earlier quoted context omitted.

Not really. It just means that it's indivisible (the original meaning of "atom"). Either it succeeds or fails, you never have to worry about it being half-finished. This includes actions which are so small they are literally indivisible, or actions which roll back to the original state if they fail.

Got it. Now it makes more sense to me. Now I know people tend to talk about atomicity when it comes to low-level-ish things. But say I create some sort of a web service with a bunch of business logic. Does it worth to follow this principle in that case? For instance, client sends an API request (let's say "Add user to friends"), is it even possible to apply atomicity for these type of things? Edit: Thanks everyone fo…

Atomicity can be important at any level. For example, assume that adding a friend involves two edges in a graph, one edge in each direction. Now assume that there is some other piece of code that does some analysis or processing of friend relationships. This piece of code might rely on there always being two edges, and crash or give erroneous results if not.

Thus, the adding of the two edges must be atomic (when observed from the rest of the system).

(The example is a bit contrived, but hopefully gets the idea across.)

Re: Things Unix can do atomically (2010)

#18

Post by throwaway76543 » Tue, May 31, 2016, 12:16 AM UTC

msync() with MS_INVALIDATE doesn't belong on this list. It has nothing to do with atomic memory access. msync() is used when flushing a mapped file to durable storage. I very often see this mistake of conflating flushing caches with atomic access of memory. What's committed to durable storage has nothing to do with what multiple processes will see when mapping a file.

All that's needed is the initial mmap to share a memory segment, then to use atomic operations like CMPXCHG -- the x86 building block the later mentioned gcc atomic macros leverage.

Re: Things Unix can do atomically (2010)

#19

Post by zAy0LfpBZLC8mAC » Tue, May 31, 2016, 12:38 AM UTC

It should be noted that those filesystem operations are only atomic with respect to an observer running on the same operating system incarnation. Whether they are also atomic across power loss depends on the filesystem (though with a modern journaling filesystem, that generally should be the case).

Re: Things Unix can do atomically (2010)

#20

Post by prodigal_erik » Tue, May 31, 2016, 12:51 AM UTC

throwaway76543 wrote: post #18

msync() with MS_INVALIDATE doesn't belong on this list. It has nothing to do with atomic memory access. msync() is used when flushing a mapped file to durable storage. I very often see this mistake of conflating flushing caches with atomic access of memory. What's committed to durable storage has nothing to do with what multiple processes will see when mapping a file. All that's needed is the initial mmap to share a…

I haven't tested but I would expect MS_INVALIDATE on a large buffer to be much faster than filling it a word at a time with __sync_val_compare_and_swap (each causing its own bus transaction).