Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

S3 isn't getting cheaper

matt-rickard.com

Post reply on HN
Print view View original

61–70 of 215 posts

Re: S3 isn't getting cheaper

#61

Post by jopsen »

judge2020 wrote: post #56
impulser_ wrote: post #50

Earlier quoted context omitted.

Have you looked into other S3 compatible services? We switched to DO Spaces because of the lower bandwidth and storage fees. The savings was actually quite a bit and no noticeable differences for our use case. I know there are others services out there that are also s3 compatible and cheaper.

For my use case, DO Spaces was quite bad in terms of artifacts being served from stale caches after being replaced and the 'purge' API being called on the bucket; this was over 2 years ago so maybe it's better now.

This is the other issue with using another cloud, S3 is extremely well documented.

Some other cloud providers give no statement on reliability/availability/consistency. And worse some providers give statements that violates the CAP theorem.

The big clouds are some what reasonably documented, but many smaller vendors leave you guessing, or promise what I know they can't keep.

Re: S3 isn't getting cheaper

#62

Post by jonatron »

vorpalhex wrote: post #59
jonatron wrote: post #21

Earlier quoted context omitted.

DO's spaces didn't even work for the most basic test of putting a few images on a page when I tried it. I couldn't say it's comparable to S3.

I use DO spaces heavily - for images no less. It works great. What issues did you have?

Uploaded about 100 images. Put those images on a page. 50% of them timed out when downloading. Maybe I was just unlucky, but it was a good way to make me instantly lose all confidence DO Spaces. S3 and Backblaze B2 worked fine for the same thing at the same time as when this happened.

Re: S3 isn't getting cheaper

#63

Post by didgetmaster »

advisedwang wrote: post #34

Hard disks have been getting larger capacity but i/o capability is fairly flat. The result is that price/GB is dropping but price/iops is not. So _cold_ storage is where you would expect to see pricing fall. I don't follows AWS pricing closely but I've seen a lot of news around Glacier over the year so that might reflect this fact.

The capacity and speed curves of drives is quite different. When the first TB drives were released about 15 years ago, the fastest ones were about 100MB/s. Now you can get 20TB drives (20x capacity) and the fastest ones are about 300MB/s (3x speed). It is just far easier and cheaper to make a drive twice as big than it is to make it twice as fast.

SSDs are a different animal, but have some similar characteristics. Within the same generation (e.g. m.2 pcie gen 4), you can get drives that have a lot more capacity but have roughly the same access speeds (i.e. the 2TB version is very similar to the 1TB version). The speed increases between generations is much better than with HDD. Pcie gen 3 drives seemed to max out about 3500MB/s while the gen 4 drives are about double that. I have seen reports that gen 5 drives might double it again

With HAMR technology we might get HDD drives with capacities in the 50TB-100TB range. You can bet that the speed won't be 5x current technology even if they get dual-actuators in them. There will need to be some kind of breakthrough technology to improve it significantly.

This is why we need better data management systems. If the meta-data (e.g. file table) is only 1% of the data that is still a lot of data to read in and store in RAM. We need better systems where the file records are much smaller. https://didgets.substack.com/p/where-did-i-put-that-file

Re: S3 isn't getting cheaper

#64

Post by LunaSea »

tnolet wrote: post #36
YawningAngel wrote: post #30

Earlier quoted context omitted.

This is a very naive question so I might be very wrong, but isn't postgres pretty flexible about objects now?

Yes, but it does blow up TOAST and has a lot of impact on the deletion behavior on busy tables. We removed all larger json blobs from PG. typical settings or config stuff in json in PG are fine. We use that all the time. But larger json blobs of several kilobytes are still an issue for semi timeseries data.

Have you tried using PostgreSQL Large Objects (LOs)?

Re: S3 isn't getting cheaper

#67

Post by 0xbadcafebee »

tnolet wrote: post #24

For my company, neither egress nor storage cost are the big issue. It’s the API call (PUT) cost. We deal with payloads that are just a little too big for a database (we run Postgres and Clickhouse) but just too frequent (~100 per second) and small (think largish json blobs) to be effective on S3. We are write heavy. Reads are probably 1% but need to be instant for a good UI and API experience.

You can host your own S3 API-compatible object storage service on some EC2 instances (exercise left to the reader to figure out how to make that reliable). Zero PUT cost, higher operational overhead.

  Minio: https://github.com/minio/minio
  SeaweedFS: https://github.com/chrislusf/seaweedfs
  Ceph: https://ceph.com/en/discover/technology/

Re: S3 isn't getting cheaper

#68

Post by fleddr »

Storage price reductions slowing down or even coming to a halt makes me wonder what this means for "infinite storage" companies.

I'm talking services like Facebook, Youtube, that are "free" (ad supported) where on a daily basis an absurd amount of new content is added, yet almost nothing ever removed.

If storage needs grow endlessly yet storage costs stopped going down, wouldn't that mean that the model in the long term is unsustainable? Sure you can delay the inevitable (compress content, move old stuff to cold storage) but ultimately storage costs per user goes up whilst income likely does not.

Re: S3 isn't getting cheaper

#69

Post by jopsen »

tnolet wrote: post #24

For my company, neither egress nor storage cost are the big issue. It’s the API call (PUT) cost. We deal with payloads that are just a little too big for a database (we run Postgres and Clickhouse) but just too frequent (~100 per second) and small (think largish json blobs) to be effective on S3. We are write heavy. Reads are probably 1% but need to be instant for a good UI and API experience.

Yeah, S3 is not for tiny blobs..

What I have seen done before is concatenating many small blobs into a single large blob that is stored on S3. This works great for batch processing afterwards.

If you need read access to the objects one option is merge them into a large blob, and then create a small index file that keeps offsets for each of the tiny blobs. Then you fetch the index file, find the offset of the tiny blob you want and, do a range request for this offset into large blob.

This mostly works when you're not read heavy. I recently did an index file for serving HTML files out of a tarball. As an alternative to uploading many small files.

Re: S3 isn't getting cheaper

#70

Post by 0xbadcafebee »

snoopy_telex wrote: post #10
paulpauper wrote: post #2

all amazon services seem so expensive, hundreds of dollars/ month bills easily for running a few large ec2 servers all month.

Why haven't there been a upstart taking market share? Why do only the big players get into the game and hold similar costs?

It's easy to enter the market and offer a cheap product. It's hard to enter the market and offer a very solid product. It's nearly impossible to enter the market and provide the hundreds of services the big players can provide, and operate at the scale they can provide, servicing the number of markets and customers they do, with the level of support they do.

The big players know that what separates the majors from the minors is trust. If you buy from AWS, you know what you get works, and you will pay a premium for that assurance. And also it is really fricking expensive to be AWS.

Post reply on HN