Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

S3 isn't getting cheaper

matt-rickard.com

Post reply on HN
Print view View original

41–50 of 215 posts

Re: S3 isn't getting cheaper

#41

Post by ipaddr »

tpmx wrote: post #33
fxtentacle wrote: post #11

The main issue we have with S3 is the extortionary egress bandwidth fee. Storage pricing seems OK, but what's the point if I can't send those files to users?

I think this would be an excellent area for regulation since it anti-competitive (portability is dramatically reduced since it has been made artifically expensive to move the data to another cloud service) and the cloud services haven't shown any interest in doing something about it on their own.

Regulation for bringing down prices for unnecessary cloud services like hosting a file?

Let's focus on things users cannot change. Using the cloud to host files is an easy and expensive way to store files that should be a luxury or a tax on the foolish.

Re: S3 isn't getting cheaper

#42

Post by wmfiv »

tnolet wrote: post #24

For my company, neither egress nor storage cost are the big issue. It’s the API call (PUT) cost. We deal with payloads that are just a little too big for a database (we run Postgres and Clickhouse) but just too frequent (~100 per second) and small (think largish json blobs) to be effective on S3. We are write heavy. Reads are probably 1% but need to be instant for a good UI and API experience.

Have you looked at Kinesis Firehose? It was pretty much build for this use case although you will still need to see if you can define a partitioning scheme probably in combination with an S3 Select query to meet your query requirements.

https://aws.amazon.com/kinesis/data-firehose/?nc=sn&loc=0

https://aws.amazon.com/blogs/aws/s3-glacier-select/

Re: S3 isn't getting cheaper

#44

Post by waitTho »

snoopy_telex wrote: post #10
paulpauper wrote: post #2

all amazon services seem so expensive, hundreds of dollars/ month bills easily for running a few large ec2 servers all month.

Why haven't there been a upstart taking market share? Why do only the big players get into the game and hold similar costs?

Human biology is not programmed to abide conflict of interest. On paper things can be written just so, but what’s on paper does not stop feelings, friendship, and connection between two people from forming given biology. Pointing at some philosophy to equivocate away physical laws is the simple con leveraged against the people.

The big players leverage their understanding of science and well paid lawyers to play a cognitive game where investment in storage is set aside, as storage is “a solved problem”, they collude to focus government spend on new things they can charge consumers for after charging us via taxes and agency, to build it.

Good luck finding a VC willing to compete against Bezos. They’re not going to target the guy managing the infra risk, providing a cheap platform key to their cheap startup gambling. They’re going to target naive college kids to try and build a rocket for them. Because VCs are smarter than Bezos; do none of the work, own the reward.

Re: S3 isn't getting cheaper

#45

Post by upupandup »

js4ever wrote: post #23
bicijay wrote: post #3

Lately im doing Backblaze B2 + Bunny CDN and im very satisfied with the prices

Blackblaze reliability and performance are below aws, same for bunny cdn. Although I understand it can be interesting for some use cases where perf/reliability is not critical.

I guess you get what you paid for but how much of a difference is there? The cost saving it advertises is quite alluring.

Re: S3 isn't getting cheaper

#46

Post by upupandup »

vorpalhex wrote: post #16
snoopy_telex wrote: post #10

Earlier quoted context omitted.

Why haven't there been a upstart taking market share? Why do only the big players get into the game and hold similar costs?

DigitalOcean, Linode, Hetzner, Vultr...

Which raises my question: Is there some open source github project that lets you create an S3 API compatible with underlying heterogeneous VPS hosts clustered? I'm guessing even with this storage would still be expensive, so how does Backblaze pull this off?

Re: S3 isn't getting cheaper

#47

Post by tpmx »

ipaddr wrote: post #41
tpmx wrote: post #33

Earlier quoted context omitted.

I think this would be an excellent area for regulation since it anti-competitive (portability is dramatically reduced since it has been made artifically expensive to move the data to another cloud service) and the cloud services haven't shown any interest in doing something about it on their own.

Regulation for bringing down prices for unnecessary cloud services like hosting a file? Let's focus on things users cannot change. Using the cloud to host files is an easy and expensive way to store files that should be a luxury or a tax on the foolish.

No, for bringing down the cost of moving that file to a competing cloud service provider.

Re: S3 isn't getting cheaper

#48

Post by gsanderson »

fxtentacle wrote: post #11

The main issue we have with S3 is the extortionary egress bandwidth fee. Storage pricing seems OK, but what's the point if I can't send those files to users?

The margin on bandwidth is enormous. I think Cloudflare did some research into it a while back. But there are rivals providing cheaper S3-compatible storage at scale. Cloudflare's R2 ($15/TB, and no fees for egress). Oracle Cloud Storage ($25/TB, and first 10TB of egress is free). There's Wasabi ($5.99/TB, and no fees for egress or API requests). To name but three. Of course each of those have drawbacks e.g Wasabi is intended for long-term storage, R2 is still in beta etc. But there are options for new data.

Re: S3 isn't getting cheaper

#49

Post by aledalgrande »

advisedwang wrote: post #34

Hard disks have been getting larger capacity but i/o capability is fairly flat. The result is that price/GB is dropping but price/iops is not. So _cold_ storage is where you would expect to see pricing fall. I don't follows AWS pricing closely but I've seen a lot of news around Glacier over the year so that might reflect this fact.

There's now a new Glacier product, called Glacier Instant Retrieval, which supposedly makes it cheaper and faster to retrieve from Glacier, if you do it a limited number of times.

Re: S3 isn't getting cheaper

#50

Post by impulser_ »

fxtentacle wrote: post #11

The main issue we have with S3 is the extortionary egress bandwidth fee. Storage pricing seems OK, but what's the point if I can't send those files to users?

Have you looked into other S3 compatible services?

We switched to DO Spaces because of the lower bandwidth and storage fees. The savings was actually quite a bit and no noticeable differences for our use case.

I know there are others services out there that are also s3 compatible and cheaper.

Post reply on HN