Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

S3 is files, but not a filesystem

calpaterson.com

Post reply on HN
Print view View original

191–200 of 456 posts

Re: S3 is files, but not a filesystem

#191

Post by tjoff »

kbolino wrote: post #174
tjoff wrote: post #140

Earlier quoted context omitted.

But surely you need to track that elsewhere anyway? That some niche edge-case runs efficiently doesn't sound like a defining feature of S3. On the contrary many common operations map terrible to S3, so you kind of need the logic to be elsewhere.

> But surely you need to track that elsewhere anyway? Why? If the S3 structure and listing is sufficient, I don't need to store anything else anywhere else. Many use cases may involve other requirements that S3 can't meet, such as being able to find the same object via different keys, or being able to search through the metadata fields. However, if the requirements match up with S3's structure, then additional servic…

I agree, but something as simple (in functionality) as that ought to be an edge-case. Not a defining feature of S3.

Re: S3 is files, but not a filesystem

#192

Post by breckognize »

> I haven't heard of people having problems [with S3's Durability] but equally: I've never seen these claims tested. I am at least a bit curious about these claims.

Believe the hype. S3's durability is industry leading and traditional file systems don't compare. It's not just the software - it's the physical infrastructure and safety culture.

AWS' availability zone isolation is better than the other cloud providers. When I worked at S3, customers would beat us up over pricing compared to GCP blob storage, but the comparison was unfair because Google would store your data in the same building (or maybe different rooms of the same building) - not with the separation AWS did.

The entire organization was unbelievably paranoid about data integrity (checksum all the things) and bigger events like natural disasters. S3 even operates at a scale where we could detect "bitrot" - random bit flips caused by gamma rays hitting a hard drive platter (roughly one per second across trillions of objects iirc). We even measured failure rates by hard drive vendor/vintage to minimize the chance of data loss if a batch of disks went bad.

I wouldn't store critical data anywhere else.

Source: I wrote the S3 placement system.

Re: S3 is files, but not a filesystem

#193

Post by klodolph »

pepa65 wrote: post #93
hiyer wrote: post #24

Earlier quoted context omitted.

One operation where this difference is significant is renaming a "folder". In UNIX (and even UNIX-y distributed filesystems like HDFS) a rename operation at "folder" level is O(1) as it only involves metadata changes. In S3, renaming a "folder" is O(number of files).

From reading the above, if you have a folder 'dir' and a file 'dir/file', after renaming 'dir' to 'folder', you would just have 'folder' and 'dir/file'.

There is really no such thing as a folder in S3.

If you have something which is dir/file, then NORMALLY “dir” does not exist at all. Only dir/file exists. There is nothing to rename.

If you happen to have something which is named “dir”, then it’s just another file (a.k.a. object). In that scenario, you have two files (objects) named “dir” and “dir/file”. Weird, but nothing stopping you from doing that. You can also have another object named “dir///../file” or something, although that can be inconvenient, for various reasons.

Re: S3 is files, but not a filesystem

#194

Post by tracerbulletx »

breckognize wrote: post #192

> I haven't heard of people having problems [with S3's Durability] but equally: I've never seen these claims tested. I am at least a bit curious about these claims. Believe the hype. S3's durability is industry leading and traditional file systems don't compare. It's not just the software - it's the physical infrastructure and safety culture. AWS' availability zone isolation is better than the other cloud providers.…

My first job was at a startup in 2012 where I was expected to build things at a scale way over what I really had the experience to do. Anyways the best choice I ever made was using RDS and S3 (and django).

Re: S3 is files, but not a filesystem

#195

Post by gamache »

orf wrote: post #115

> And listing files is slow. While the joy of Amazon S3 is that you can read and write at extremely, extremely, high bandwidths, listing out what is there is much much slower. Slower than a slow local filesystem This misses something critical. Yes, s3 has fast reading and writing, but that’s not really what makes it useful . What makes it useful is listing. In an unversioned bucket (or one with no delete markers), li…

> ...listing any given prefix is essentially constant time: I can take any given string, in a bucket with 100 billion objects, and say “give me the next 1000 keys alphabetically that come after this random string”.

I'm not sure we agree on the definition of "constant time" here. Just because you get 1000 keys in one network call doesn't imply anything about the complexity of the backend!

Re: S3 is files, but not a filesystem

#196

Post by supriyo-biswas »

breckognize wrote: post #192

> I haven't heard of people having problems [with S3's Durability] but equally: I've never seen these claims tested. I am at least a bit curious about these claims. Believe the hype. S3's durability is industry leading and traditional file systems don't compare. It's not just the software - it's the physical infrastructure and safety culture. AWS' availability zone isolation is better than the other cloud providers.…

Checksumming the data is not based out of paranoia but simply as a result of having to detect which blocks are unusable in order to run the Reed-Solomon algorithm.

I'd also assume that a sufficient number of these corruption events are used as a signal to "heal" the system by migrating the individual data blocks onto different machines.

Overall, I'd say the things that you mentioned are pretty typical of a storage system, and are not at all specific to S3 :)

Re: S3 is files, but not a filesystem

#197

Post by dekhn »

hnlmorg wrote: post #181
cuno wrote: post #172

Earlier quoted context omitted.

For AWS, we're comparing against filesystems in the datacenter - so EBS, EFS and FSx Lustre. Compared to these, you can see in the graphs where S3 is much faster for workloads with big files and small files: https://cuno.io/technology/ and in even more detail of different types of EBS/EFS/FSx Lustre here: https://cuno.io/blog/making-the-right-choice-comparing-the-c...

EFS is ridiculously slow though. Almost to the point where I fail to see how it’s actually useful for any of the traditional use cases for NFS.

if you turn all the EFS performance knobs up (at a high cost), it's quite fast.

Re: S3 is files, but not a filesystem

#198

Post by svat »

throwaway892238 wrote: post #45

> The "simple" in S3 is a misnomer. S3 is not actually simple. It's deep. Simple doesn't mean "not deep". It means having the fewest parts needed in order to accomplish your requirements. If you require a distributed, centralized, replicated, high-availability, high-durability, high-bandwidth, low-latency, strongly-consistent, synchronous, scalable object store with HTTP REST API, you can't get much simpler than S3.…

> It means having the fewest parts needed in order to accomplish your requirements.

That is exactly what "deep" means, in the terminology of this post (from Ousterhout's book A Philosophy of Software Design). Simple means "not complex" (see also Rich Hickey's talk Simple Made Easy: https://www.infoq.com/presentations/Simple-Made-Easy/), while "deep" means providing/having a lot of internally-complex functionality via a small interface. The latter is a better description of S3 (which is what you seem to be saying too) than "simple" which would mean there isn't much to it.

Re: S3 is files, but not a filesystem

#199

Post by ahepp »

throwaway892238 wrote: post #45

> The "simple" in S3 is a misnomer. S3 is not actually simple. It's deep. Simple doesn't mean "not deep". It means having the fewest parts needed in order to accomplish your requirements. If you require a distributed, centralized, replicated, high-availability, high-durability, high-bandwidth, low-latency, strongly-consistent, synchronous, scalable object store with HTTP REST API, you can't get much simpler than S3.…

In terms the article puts forth, I would almost argue that simple implies deep (and the associated “narrow” interface).

Re: S3 is files, but not a filesystem

#200

Post by orf »

gamache wrote: post #195
orf wrote: post #115

> And listing files is slow. While the joy of Amazon S3 is that you can read and write at extremely, extremely, high bandwidths, listing out what is there is much much slower. Slower than a slow local filesystem This misses something critical. Yes, s3 has fast reading and writing, but that’s not really what makes it useful . What makes it useful is listing. In an unversioned bucket (or one with no delete markers), li…

> ...listing any given prefix is essentially constant time: I can take any given string, in a bucket with 100 billion objects, and say “give me the next 1000 keys alphabetically that come after this random string”. I'm not sure we agree on the definition of "constant time" here. Just because you get 1000 keys in one network call doesn't imply anything about the complexity of the backend!

Constant time irregardless of the number of objects in the bucket and irregardless of the initial starting position of your list request.
Post reply on HN