Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Diving Deep on S3 Consistency

allthingsdistributed.com

Post reply on HN
Print view View original

31–40 of 55 posts

Re: Diving Deep on S3 Consistency

#31

Post by the_reformation »

skynet-9000 wrote: post #30
ignoramous wrote: post #15

Earlier quoted context omitted.

Vogels spoke briefly about why AWS prefers versioned objects instead here: https://queue.acm.org/detail.cfm?id=3434573 BTW, DynamoDB supports conditional PUTs if your data can fit under 400 KiB.

Why would AWS provide a feature that makes additional transaction and storage charges (as well as subsequent reads to see which is the correct version) irrelevant?

S3 (and most AWS services) are extremely price elastic; i.e., the lower you make them cost, the more people use it (a la electricity.) That's why they've done stuff like drop from 50ms billing to 1ms billing, etc.

Re: Diving Deep on S3 Consistency

#32

Post by skynet-9000 »

juancampa wrote: post #21

Can someone elaborate on this Witness system OP talks about? I'm picturing a replicated, in-memory KV store where the value is some sort of version or timestamp representing the last time the object was modified. Cached reads can verify they are fresh by checking against this version/timestamp, which is acceptable because it's a network+RAM read. Is this somewhat accurate?

I'm picturing the same, but my guess is that it's using a time-synced serializability graph or MVCC in some way.

However, even a "basic" distributed lock system (like a consistently-hashed in-memory DB, sharded across reliable servers) might provide both the scale and single source of truth that's needed. The difficulty arises when one of those servers has a hiccup.

It'd be a delicious irony if it was based on hardware like an old-school mainframe or something like that.

Re: Diving Deep on S3 Consistency

#34

Post by ryeguy »

jjoonathan wrote: post #27
CodesInChaos wrote: post #22

Earlier quoted context omitted.

How do versioned objects make conditional puts unnecessary? I see little relation between them, except that you could use the version identifier in the condition.

Because they let AWS offload the hard part to you, which is what AWS does best :)

This doesn't answer the question being asked.

Re: Diving Deep on S3 Consistency

#35

Post by BrandonY »

deepsun wrote: post #23
wolf550e wrote: post #10

AWS fixed S3 consistency in December 2020: https://aws.amazon.com/about-aws/whats-new/2020/12/amazon-s3...

By the way, Google's GCS had it from the beginning.

Hi, GCS engineer here. GCS offered a lot of consistency from the beginning, but we didn't have strong object listing consistency at the beginning. We got that somewhere around 2017 when we moved object metadata to Spanner. See https://cloud.google.com/blog/products/gcp/how-google-cloud-...

Re: Diving Deep on S3 Consistency

#36

Post by nelsonenzo »

JDTech123 wrote: post #6
whydoineedthis wrote: post #2

I'm confused...did you fix the caching issue in S3 or not? The article seems to explain why there is a caching issue, and that's understandable, but it also reads as if you wanted to fix it. I would think the headliner and bold font if it was actually fixed. For those curious, the problem is that S3 is "eventually consistent", which is normally not a problem. But consider a scenario where you store a config file on S…

In that example, do you not see using S3 for that purpose as trying to use the wrong tool for the task at hand. Using AWS SSM parameter store [0] seems to me that it would be a tool designed to fit that purpose nicely. [0] https://docs.aws.amazon.com/systems-manager/latest/userguide...

Complex config files suck in paramstore. Also, I've used this for mobile app configs that are pulled from s3, so paramstore wouldn't be an option.

Re: Diving Deep on S3 Consistency

#37

Post by nelsonenzo »

jeffbarr wrote: post #7
whydoineedthis wrote: post #2

I'm confused...did you fix the caching issue in S3 or not? The article seems to explain why there is a caching issue, and that's understandable, but it also reads as if you wanted to fix it. I would think the headliner and bold font if it was actually fixed. For those curious, the problem is that S3 is "eventually consistent", which is normally not a problem. But consider a scenario where you store a config file on S…

Yes, see my December 2020 post at https://aws.amazon.com/blogs/aws/amazon-s3-update-strong-rea... : "Effective immediately, all S3 GET, PUT, and LIST operations, as well as operations that change object tags, ACLs, or metadata, are now strongly consistent. What you write is what you will read, and the results of a LIST will be an accurate reflection of what’s in the bucket. This applies to all existing and new S3 obj…

Oh, awesome, I missed that!

Re: Diving Deep on S3 Consistency

#38

Post by nelsonenzo »

jasonpeacock wrote: post #8
whydoineedthis wrote: post #2

I'm confused...did you fix the caching issue in S3 or not? The article seems to explain why there is a caching issue, and that's understandable, but it also reads as if you wanted to fix it. I would think the headliner and bold font if it was actually fixed. For those curious, the problem is that S3 is "eventually consistent", which is normally not a problem. But consider a scenario where you store a config file on S…

This is a general problem in all distributed systems, not just when pulling configuration from S3. Let's assume you had strong consistency in S3. If your app is distributed (tens, hundreds, or thousands of instances running) then all instances are not going to update at the same time, atomically. You still need to design flexibility into your app to handle the case where they are not all running the same config (or s…

What I need is that when I make a call to a service, it gives back consistent results. Ergo, when the app does do a rolling deploy, it will get the right config on startup, not some random version.

It looks like it does exactly that now, it just wasn't clear from the article.

Re: Diving Deep on S3 Consistency

#39

Post by ithkuil »

rossmohax wrote: post #14

Recent S3 consistency improvements are welcome, but S3 still falls behind Google GCS until they support conditional PUTs. GCS allows object to be replaced conditionally with `x-goog-if-generation-match` header, which sometimes can be quite useful.

There is a conditional CopyObject though (x-amz-copy-source-if...)

Can cover some of the use cases

Re: Diving Deep on S3 Consistency

#40

Post by ryanworl »

ithkuil wrote: post #39
rossmohax wrote: post #14

Recent S3 consistency improvements are welcome, but S3 still falls behind Google GCS until they support conditional PUTs. GCS allows object to be replaced conditionally with `x-goog-if-generation-match` header, which sometimes can be quite useful.

There is a conditional CopyObject though (x-amz-copy-source-if...) Can cover some of the use cases

Can you explain how this is useful? It seems like the destination is the important thing here not the source.
Post reply on HN