Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

S3 Express Is All You Need

warpstream.com

Post reply on HN
Print view View original

81–87 of 87 posts

Re: S3 Express Is All You Need

#81

Post by halz »

yeeeloit wrote: post #26
throwitaway222 wrote: post #13

I don't understand why EFS never gets major shout outs - it's way better than S3: systems can mount it as a drive, shared across systems, already has had super low latency... Not sure what s3 express is really useful for if EFS already exists.

I wonder if Mountpoint for S3 along with this new Express option makes it a direct competitor to EFS for some use cases. https://docs.aws.amazon.com/AmazonS3/latest/userguide/mountp...

Looks like support for S3 Express was merged in with version 1.30 just a few hours ago https://github.com/awslabs/mountpoint-s3/pull/642

Re: S3 Express Is All You Need

#82

Post by otabdeveloper4 »

promocha wrote: post #55

> “Of course the AWS S3 Express storage costs are still 8x higher than S3 standard, but that’s a non issue for any modern data storage system. Data can be trivially landed into low latency S3 Express buckets, and then compacted out to S3 Standard buckets asynchronously. Most modern data systems already have a form of compaction anyways, so this “storage tiering” is effectively free.” This is key insight. The data sto…

Did you conveniently ignore egress costs?

Re: S3 Express Is All You Need

#83

Post by dpedu »

dekhn wrote: post #67
dpedu wrote: post #31

Earlier quoted context omitted.

My understanding is that EFS is exposed as an NFS share. I haven't used it personally, but NFS mounting is generally fast, nearly instant. What does "load in the entire datastore" mean?

Many servers start up, load a ton of data from storage into RAM, and then happily serve that data for a long time. The latency of the server when starting up before it can service its first request is entirely based on the throughput of the data load. Often these servers will load 128+GB of data into RAM (crazy, huh?) and even if you have 1GB/sec it's still two minutes for the server to start up.

1GB/s is the same speed from EFS or S3.

Re: S3 Express Is All You Need

#84

Post by dekhn »

dpedu wrote: post #83
dekhn wrote: post #67

Earlier quoted context omitted.

Many servers start up, load a ton of data from storage into RAM, and then happily serve that data for a long time. The latency of the server when starting up before it can service its first request is entirely based on the throughput of the data load. Often these servers will load 128+GB of data into RAM (crazy, huh?) and even if you have 1GB/sec it's still two minutes for the server to start up.

1GB/s is the same speed from EFS or S3.

I didn't say otherwise.

Re: S3 Express Is All You Need

#85

Post by warpspin »

kristianp wrote: post #59

I saw "X is all you Need" with the "Attention is all you need" paper [1], which launched the Transformer upon the world. Is it the first instance of that phrase? [1] https://arxiv.org/abs/1706.03762

It starts to get more annoying than "X considered harmful".

Re: S3 Express Is All You Need

#86

Post by Hixon10 »

francoismassot wrote: post #42

We tested S3 Express for our search engine quickwit [0] a couple of weeks ago. While this was really satisfying on the performance side, we were a bit disappointed by the price, and I mostly agree with the article on this matter. I can see some very specific use cases where the pricing should be OK but currently, I would say most of our users will just stay on the classic S3 and add some local SSD caching if they hav…

Did you have access to their private preview version, or it is some another S3 Express, not S3 Express One Zone?
Post reply on HN