Building and operating a pretty big storage system called S3

Post by delecti » Fri, Jul 28, 2023, 2:53 PM UTC

fooker wrote: post #123

rkagerer wrote: post #36

Earlier quoted context omitted.

Personally I'd love working in that kind of environment. That one in a billion hole still itches at me. There's also a slightly-perverse little voice in my head ready with popcorn in case I'm lucky enough to watch the ensuing fallout from the first major crypto hash collision :-).

That probability is significantly lower than one in a billion. One in a billion would be if keys were ~30 bits. Luckily it isn't.

Post by anyoneamous » Fri, Jul 28, 2023, 5:43 PM UTC

danpalmer wrote: post #115

anyoneamous wrote: post #37

Earlier quoted context omitted.

My impression is that the ambiguity gives them freedom to implement in different ways across different regions and over time. The original Glacier was very clearly tape, but given the instant retrieval capabilities the newer S3-Glacier tiers are most likely just low-margin HDDs, maybe with some dynamic powering on and off of drives/servers.

I’m sure it’s a mix. Back when it launched there were a number of rumours about it being Blu-Ray based. They had similar capacity for the space used compared to tapes, were considered very physically stable storage mediums, but had long access time as they would need to be physically moved, like tape, explaining the retrieval times.

Post by Spooky23 » Fri, Jul 28, 2023, 6:21 PM UTC

da39a3ee wrote: post #133

aborsy wrote: post #111

Earlier quoted context omitted.

This phenomenon is just multiplication of the sample size (scale) times a probability (rare).

I agree with what I think is your sentiment -- that people seem to be treating this as if it's some sort of profound insight that you only get if you work at a very senior level in engineering for major US cloud providers, when it's in fact blindingly obvious!

Post by dekhn » Fri, Jul 28, 2023, 7:29 PM UTC

withinboredom wrote: post #95

epistasis wrote: post #71

Earlier quoted context omitted.

Right, most people that try to really optimize these things do not have access to the parallelism tools thay Google has built, and end up doing their own ad-hoc sharding schemes. Things that can be built by 1-3 people over the course of a few weeks tk solve ann immediate scaling problem. And of course BAM itself dates back to before standardized serialization formats were brought out of Google. Even with potential op…

Of course it’s slower. Your using https to do something that’s meant to be raw binary. The overhead is killing you. Something like iscsci, is pretty quick compared to https as a storage protocol.

Post by withinboredom » Fri, Jul 28, 2023, 8:22 PM UTC

dekhn wrote: post #147

withinboredom wrote: post #95

Earlier quoted context omitted.

Of course it’s slower. Your using https to do something that’s meant to be raw binary. The overhead is killing you. Something like iscsci, is pretty quick compared to https as a storage protocol.

I've worked in web tech long enough to recognize that the overhead of HTTP does not explain the difference in performance between "raw binary" protocols and ones that have textual headers. Put another way I've seen extremely low-latency https servers. The latency in S3 doesn't come from using https.