Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

S3 is files, but not a filesystem

calpaterson.com

Post reply on HN
Print view View original

421–430 of 456 posts

Re: S3 is files, but not a filesystem

#421

Post by urda »

pclmulqdq wrote: post #253
treflop wrote: post #231

Earlier quoted context omitted.

What’s your experience like at other storage outfits? I only ask because your post is a bit like singing praises for Cinnabon that they make their own dough. The things that you mentioned are standard storage company activities. Checksum-all-the-things is a basic feature of a lot of file systems. If you can already set up your home computer to detect bitrot and alert you, you can bet big storage vendors do it. Keepin…

When I worked at Google in storage, we had our own figures of merit that showed that we were the best and Amazon's durability was trash in comparison to us. As far as I can tell, every cloud provider's object store is too durable to actually measure ("14 9's"), and it's not a problem.

Sorry I am not buying the personal anecdote as the public numbers from both orgs tell a different story. When reliability and long term support come up in conversation, Google is not a name to reach for.

Re: S3 is files, but not a filesystem

#422

Post by pclmulqdq »

urda wrote: post #421
pclmulqdq wrote: post #253

Earlier quoted context omitted.

When I worked at Google in storage, we had our own figures of merit that showed that we were the best and Amazon's durability was trash in comparison to us. As far as I can tell, every cloud provider's object store is too durable to actually measure ("14 9's"), and it's not a problem.

Sorry I am not buying the personal anecdote as the public numbers from both orgs tell a different story. When reliability and long term support come up in conversation, Google is not a name to reach for.

Note that I said "durability," not any other reliability metric. GCP is pretty well-known for its outages and abysmal support. It's a reputation they want to change, but they did earn it.

However, Google is very good at not losing data.

Re: S3 is files, but not a filesystem

#423

Post by zooq_ai »

breckognize wrote: post #192

> I haven't heard of people having problems [with S3's Durability] but equally: I've never seen these claims tested. I am at least a bit curious about these claims. Believe the hype. S3's durability is industry leading and traditional file systems don't compare. It's not just the software - it's the physical infrastructure and safety culture. AWS' availability zone isolation is better than the other cloud providers.…

Google discovered random bit flips caused by gamma rays.

Re: S3 is files, but not a filesystem

#424

Post by Filligree »

EraYaN wrote: post #401
Filligree wrote: post #307

Earlier quoted context omitted.

Bcachefs, and LVM also has a way to do it. Unfortunately I’m not aware of any filesystem that does it while maintaining the full bandwidth of a modern NVMe. Not even with the extra reads factored in; on ZFS I get 800 MB/s max.

ZFS should absolutely be able to go faster even with lz4 compression I get writes above 5 GB/s on a older 32-core EPYC CPU. And that is with mostly random and already compressed data. And that write speed is a limitation of the RAIDz2 on top of not the fastest drives (6 PCIe 3.0 intel U.2 ones from 2 years ago).

Maybe I've configured something wrong, then. I'll do some tests.

Re: S3 is files, but not a filesystem

#425

Post by [deleted] »

pclmulqdq wrote: post #253
treflop wrote: post #231

Earlier quoted context omitted.

What’s your experience like at other storage outfits? I only ask because your post is a bit like singing praises for Cinnabon that they make their own dough. The things that you mentioned are standard storage company activities. Checksum-all-the-things is a basic feature of a lot of file systems. If you can already set up your home computer to detect bitrot and alert you, you can bet big storage vendors do it. Keepin…

When I worked at Google in storage, we had our own figures of merit that showed that we were the best and Amazon's durability was trash in comparison to us. As far as I can tell, every cloud provider's object store is too durable to actually measure ("14 9's"), and it's not a problem.

[deleted]

Re: S3 is files, but not a filesystem

#426

Post by hnlmorg »

harshaw wrote: post #402
hnlmorg wrote: post #181

Earlier quoted context omitted.

EFS is ridiculously slow though. Almost to the point where I fail to see how it’s actually useful for any of the traditional use cases for NFS.

Have you tried it recently? Because we've made it a lot faster over the years.

More recently and for more use cases and varied workflows than most people. But that’s as much as I can say without getting people to sign an NDA.

Our AWS spend is high enough to warrant a very close working relationship with AWS so this is something we have worked with you guys on already.

Re: S3 is files, but not a filesystem

#427

Post by hnlmorg »

geertj wrote: post #412
hnlmorg wrote: post #181

Earlier quoted context omitted.

EFS is ridiculously slow though. Almost to the point where I fail to see how it’s actually useful for any of the traditional use cases for NFS.

> EFS is ridiculously slow though. Almost to the point where I fail to see how it’s actually useful for any of the traditional use cases for NFS. Would you care to elaborate on your experience or use case a bit more? We've made a lot of improvements over the last few years (and are actively working on more), and we have many happy customers. I'd be happy to give a perspective of how well your use case would work with…

Unfortunately I can’t say too much publicly on HN. But one of the big shortcomings is dealing with hundreds of files. It doesn’t even matter if those are big or small files (I’ve had experience with both).

Services like DataSync show that the underlying infra can be performant. But it feels almost impossible to replicate that on EFS via standard POSIX APIs. And unfortunately one of our use cases depend upon that.

If feels, to me at least, like EFS isn’t where AWSs priorities lie. At least if you compare EFS to FSx Lustre and recent developments to S3. Both of which has been the direction our AWS SAs have pushed us.

Re: S3 is files, but not a filesystem

#428

Post by orf »

jameshart wrote: post #399
orf wrote: post #391

Earlier quoted context omitted.

Literally 1 API call.

Two. The one to set up the lifecycle rule. Then the one to delete the bucket, some number of hours later.

Incorrect. One call to trigger a step function that sets up the lifecycle rule, sleeps for 24 hours and then deletes the bucket.

Stop being silly, as if 1 vs 2 API calls matters. You should empty large buckets with lifecycle policies. It's trivial.

Re: S3 is files, but not a filesystem

#429

Post by Kwpolska »

EraYaN wrote: post #406
Kwpolska wrote: post #225

Earlier quoted context omitted.

How could they launch a cloud service like this and not have IPv6 in 2015? What other basic things did they cheap out on?

Most mayor cloud vendors are still not fully dual stack capable so it's not that surprising. And plenty of ISPs have barely started rollout, or even said they just wont.

I understand that AWS has 200 services, some of which are 20 years old, and making them all IPv6-ready would be hard and costly. Backblaze has one cloud service, and the public interface is a boring REST API over boring HTTPS.

Re: S3 is files, but not a filesystem

#430

Post by Veserv »

AdamN wrote: post #389
Veserv wrote: post #377

Earlier quoted context omitted.

I read that page and it does not provide any contractual durability guarantees as far as I can see. It provides "designed for availability" and then contractual availability SLA guarantees. It provides "designed for durability", but presents no contractual durability guarantee as far as I can see. Given that their lawyers clearly indicate that "designed for availability" is not what they are contractually obligated t…

SLAs are more of a financial construct than anything else. Once the payback cost of missing an SLA is built into the contract then it just becomes a conversation about money. I've been at plenty of shops that obviously tried to hit the SLA but if it was missed it just became a financial issue which helped smooth over what otherwise might have been a trust buster. I would never ever think of an SLA as anything more th…

Obviously, as I made no mention of specific performance in the event of breach of contract the remedy for failure to meet contractual obligations would be damages.

The question is: What level of durability does AWS contractually guarantee where a failure to provide that level results in a breach of contract that may incur damages and where, specifically, in the documentation do they specify that?

Post reply on HN