Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

S3 is files, but not a filesystem

calpaterson.com

Post reply on HN
Print view View original

211–220 of 456 posts

Re: S3 is files, but not a filesystem

#211

Post by belter »

macintux wrote: post #190
belter wrote: post #156

Earlier quoted context omitted.

No. The standard practice is to use a DynamoDB table as the index for your objects in S3. This article misunderstood S3 and could as well have the title: "An Airplane is not a Car" :-)

I don't know that you can characterize that as a "standard practice". Maybe it's widespread, but I've not encountered it.

"Building and Maintaining an Amazon S3 Metadata Index without Servers" - https://aws.amazon.com/pt/blogs/big-data/building-and-mainta...

Here is the architecture of Amazon Drive and the storage of metadata.

"AWS re:Invent 2014 | (ARC309) Building and Scaling Amazon Cloud Drive to Millions of Users" - https://youtu.be/R2pKtmhyNoA

And you can see the use here at correct time: https://youtu.be/R2pKtmhyNoA?t=546

Re: S3 is files, but not a filesystem

#212

Post by medler »

breckognize wrote: post #192

> I haven't heard of people having problems [with S3's Durability] but equally: I've never seen these claims tested. I am at least a bit curious about these claims. Believe the hype. S3's durability is industry leading and traditional file systems don't compare. It's not just the software - it's the physical infrastructure and safety culture. AWS' availability zone isolation is better than the other cloud providers.…

> customers would beat us up over pricing compared to GCP blob storage, but the comparison was unfair because Google would store your data in the same building

I don’t think this is true. Per the Google Cloud Storage docs, data is replicated across multiple zones, and each zone maps to a different cluster. https://cloud.google.com/compute/docs/regions-zones/zone-vir...

Re: S3 is files, but not a filesystem

#213

Post by chubot »

> Filesystem software, especially databases, can't be ported to Amazon S3

This seems mistaken. Porting databases that run on local disk to S3 seems like a good way to get a lashing from https://aphyr.com/

Can any databases do it correctly?

If so, I doubt they work with the model of partial overwrites. They probably have to do something very custom, and either sacrifice a lot of tail latency, or their uptime is capped by the uptime of a single AWS availability zone. Doesn't seem like a great design.

(copy of lobste.rs comment)

Re: S3 is files, but not a filesystem

#215

Post by d0gsg0w00f »

ahepp wrote: post #209

Are filesystems the correct abstraction to build databases on? Isn’t a filesystem a database in a way? Is there a reason to build a database on top of a filesystem abstraction rather than a block abstraction? To say you can’t build an efficient database on top of S3 makes sense to me. S3 is already a certain kind of data-storing abstraction optimized for certain usages. If you try and build another data-storing abstr…

In my $dayjob as cloud architect I sometimes suggest S3 as an alternative to pulling massive JSON blobs from RDS Postgres/Redis etc. As long as their latency minimums are high enough there's no reason you can't.

Re: S3 is files, but not a filesystem

#216

Post by breckognize »

rsync wrote: post #206
breckognize wrote: post #192

> I haven't heard of people having problems [with S3's Durability] but equally: I've never seen these claims tested. I am at least a bit curious about these claims. Believe the hype. S3's durability is industry leading and traditional file systems don't compare. It's not just the software - it's the physical infrastructure and safety culture. AWS' availability zone isolation is better than the other cloud providers.…

"AWS' availability zone isolation is better than the other cloud providers." Not better than all of them. A geo-redundant rsync.net account exists in two different states (or countries) - for instance, primary in Fremont[1] and secondary in Denver. "S3 even operates at a scale where we could detect "bitrot"" That is not a function of scale. My personal server running ZFS detects bitrot just fine - and the scale invol…

Backing up across two different regions is possible for any provider with two "regions" but requires either doubling your storage footprint or accepting a latency hit because you have to make a roundtrip from Fremont to Denver.

The neat thing about AWS' AZ architecture is that it's a sweet spot in the middle. They're far enough apart for good isolation, which provides durability and availability, but close enough that the network round trip time is negligible compared to the disk seek.

Re: bit rot, I mean the frequency of events. If you've got a few disks, you may see one flip every couple years. They happen frequently enough in S3 that you can have expectations about the arrival rate and alarm when that deviates from expectations.

Re: S3 is files, but not a filesystem

#217

Post by hobobaggins »

orf wrote: post #200
gamache wrote: post #195

Earlier quoted context omitted.

> ...listing any given prefix is essentially constant time: I can take any given string, in a bucket with 100 billion objects, and say “give me the next 1000 keys alphabetically that come after this random string”. I'm not sure we agree on the definition of "constant time" here. Just because you get 1000 keys in one network call doesn't imply anything about the complexity of the backend!

Constant time irregardless of the number of objects in the bucket and irregardless of the initial starting position of your list request.

The technical implementation is indeed impressive that it operates more-or-less within constant time, but probably very few use cases actually fit that narrow window, so this technical strength is moot when it comes to actual usage.

Since each request is dependent upon the position received in the last request, 1000 arbitrary keys on your 3rd or 1000th attempt doesn't really help unless you found your needle in the haystack in that request (and in that case the rest of that 1000 key listing was wasted.)

Re: S3 is files, but not a filesystem

#218

Post by mlhpdx »

zX41ZdbW wrote: post #38
YouWhy wrote: post #23

The article is well written, but I am annoyed at the attempt to gatekeep the definition of a filesystem. Like literally any abstraction out there, filesystems are associated with a multitude of possible approaches with conceptually different semantics. It's a bit sophistic to say that Postgres cannot be run on S3 because S3 is not a filesystem; a better choice would have been to explore the underlying assumptions; (I…

ClickHouse can work with S3 as a main storage. This is possible because a table is a set of immutable data parts. Data parts can be written once and deleted, possibly as a result of a background merge operation. S3 API is almost enough, except for cases of concurrent database updates. In this case, it is not possible to rely on S3 only because it does not support an atomic "write if not exists" operation. That's why…

Conditional PUT would be a great addition to S3, indeed.

Re: S3 is files, but not a filesystem

#219

Post by overstay8930 »

cynicalsecurity wrote: post #110

Backblaze B2 is worth mentioning while we are speaking of S3. I'm absolutely in love with their prices (3 times lower than of S3). (I'm not their representative).

We liked B2 but not enough to pay for IPv4 addresses, insane they advertise as a multi-cloud solution but basically kill any chance at adoption when NAT gateways and IPv4 charges are everywhere. We would literally save money paying B2 bandwidth fees (high read low write) but not when being pushed through a NAT64 gateway, or paying an hourly charge just to be able to access B2.

Re: S3 is files, but not a filesystem

#220

Post by returningfory2 »

donatj wrote: post #189

> And listing files is slow. While the joy of Amazon S3 is that you can read and write at extremely, extremely, high bandwidths, listing out what is there is much much slower. Slower than a slow local filesystem. I was taken aback by this recently. At my coworkers request, I was putting some work into a script we have to manage assets in S3. It has a cache for the file listing, and my coworker who wrote it sent me hi…

Are you performing list calls sequentially? If you have O(100k) directories and are doing O(100k) requests sequentially, 15 minutes works out at O(10ms) per request which doesn’t seem that bad? (assuming my math is correct…)
Post reply on HN