does it mean s3 could be used efficiently as a file system? Would that compete with FSX?
S3 Strong Consistency
201–210 of 240 posts
Re: S3 Strong Consistency
#202In the early days of S3 this was a real problem. Over the years they've beefed up the backend so objects become became consistent faster, but this is a huge leap.
Re: S3 Strong Consistency
#203Earlier quoted context omitted.
> (of their metadata database) With that you have just moved the question to: how do they ensure that the metadata database is available _and_ strongly consistent at the same time for all the requests? Because the metadata database is certainly also a distributed one, hence you need to query _all_ the nodes or can end up in a split-brain situation and lose consistency (or availability if you choose to down the system…
> hence you need to query _all_ the nodes or can end up in a split-brain situation and lose consistency (or availability if you choose to down the system or a part of the system). No.Depending on the type of consensus algorithm you can get away with reading from a majority or nodes or even less: for example in a Multipaxos system you just read from the master.
Then what happens in case of network partitions?
Re: S3 Strong Consistency
#204This is really cool. We actually got burned on weak S3 consistency a few weeks ago when generating public download links for a customer. Took us a few hours of troubleshooting to realize they had downloaded a cached/older version of the software we had uploaded to the same URL just a few minutes prior. Resolution was to use unique paths per version to guarantee we were talking to the right files each time. One potent…
> they had downloaded a cached/older version of the software we had uploaded to the same URL
This is a great example (to me) of how weak consistency is useful. It exposed a problem you otherwise wouldn't notice.
You were deploying offline artifacts to customers without giving the customer the version of the new artifact. Regardless of backend consistency, the customer will not be able to tell (from the URL, anyway) what version they are downloading. Nor will they be able to, say, download the old version if the new version introduces bugs. By changing your deployment method to use unique URLs per version, your customer gains useful functionality and you avoid having to depend on strong consistency (which actually removes expensive requirements).
If you design for weak consistency, your system ends up more resilient to multiple problems.
Re: S3 Strong Consistency
#205It's interesting to read all these comments here that talk about the eventual consistency like it was some kind of bug.
I'm pulling my hair out because people are using eventual consistency to mean "eventual consistency with a noticeable lag before the consistency is achieved", which bothers the pedant in me.
I mean, technically, strong consistency implies eventual consistency (with lag = 0). But everyone's equating eventual consistency with the noticeable lag itself, implying that EC per se is a bad thing.
For an analogy, it would be like if people were talking about how rectangular cakes suck, because , and thus they use square cakes, but ... square cakes are rectangular too.
(What's going on is that people use to mean . = eventual consistency, rectangle; = strong consistency, square.)
Obviously, it's not a problem for communication because everyone implicitly understands what's going on and uses the words the same way, but I wish people spoke in terms of "has anyone actually seen a consistency lag?" rather than "has anyone actually seen eventual consistency?", since the latter is not the right way to frame it, and is actually a good thing to have, which happened both before and after this development.
Re: S3 Strong Consistency
#206It's interesting to read all these comments here that talk about the eventual consistency like it was some kind of bug.
The previous behavior (new keys weren't strongly consistent) were weird, and made a lot of patterns that AWS itself sort of guides you towards, broken. For example, you can set up an SQS queue to be informed of writes into a bucket. The idea being that some component writes into the bucket, and a consumer then fetches the data & processes it. Except — gotcha! — there wasn't a guarantee of consistency. It would usuall…
So the problem was that users don't know how S3 works?
Re: S3 Strong Consistency
#207Re: S3 Strong Consistency
#208Disclosure: I work on Google Cloud. This is super awesome for customers. I am also beyond excited for all the open-source connectors to finally be simplified so that they don't have to deal with the "oh right, gotta be careful because of list consistency". It was super awesome when we were able to delete a huge chunk of the GCS Connector for Hadoop, and I hope to see the same across the S3-focused connectors. I'm now…
Is your rewrite available?
Re: S3 Strong Consistency
#209I believe this makes Amazon S3 behave more similar to Azure blob storage[1] and Google Cloud Storage[2], which is pretty convenient for folks who are abstracting blob stores across different offerings. For what it’s worth, consistency in S3 was usually pretty good anyways, but I ran into issues where it could vary a bit in the past. If you designed your application with this in mind, of course, it shouldn’t be an iss…
And Wasabi as well: https://wasabi-support.zendesk.com/hc/en-us/articles/1150016...
Re: S3 Strong Consistency
#210Disclosure: I work on Google Cloud. This is super awesome for customers. I am also beyond excited for all the open-source connectors to finally be simplified so that they don't have to deal with the "oh right, gotta be careful because of list consistency". It was super awesome when we were able to delete a huge chunk of the GCS Connector for Hadoop, and I hope to see the same across the S3-focused connectors. I'm now…
Is your rewrite available?
Not yet, it’s too crappy :). I only get to it every once in a while and it shows.
As an example, until last weekend, I would just hardcode an access token rather than doing an oauth dance. Luckily, tame-oauth [1] from the folks at Embark was reasonably easy to integrate with.
I also got a little depressed that zargony/rust-fuse was stuck on an ancient version until I learned that Chris Berner from OpenAI had forked it earlier this year to modernize / update it [2].