Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building the heap: racking 30 petabytes of hard drives for pretraining

si.inc

Post reply on HN
Print view View original

221–230 of 281 posts

Re: Building the heap: racking 30 petabytes of hard drives for pretraining

#221

Post by computably »

jillesvangurp wrote: post #216

AWS obviously does the same but better. That's why they are so rich. Their cost is a tiny percentage of their revenue. They buy cheap servers, and then run lots of vms on them. Each of which delivers 10s/100s of $ per month. That server pays for itself in revenue within weeks/months. And it will be in service until it stops working which could be over five years. Same with storage, networking, gpus, etc. They've spen…

> AWS at cost price would probably be 60-80x less than what they charge; if not more.

If you look at https://ir.aboutamazon.com/news-release/news-release-details... , it says in 2024,

> AWS segment sales increased 19% year-over-year to $107.6 billion.

> AWS segment operating income was $39.8 billion, compared with operating income of $24.6 billion in 2023.

So about 59% margin, relative to costs. Everybody undercutting AWS is likely doing so at close enough to 20% margins that it makes no sense to fund a startup in the same space.

Re: Building the heap: racking 30 petabytes of hard drives for pretraining

#223

Post by rafaelmn »

computably wrote: post #221
jillesvangurp wrote: post #216

AWS obviously does the same but better. That's why they are so rich. Their cost is a tiny percentage of their revenue. They buy cheap servers, and then run lots of vms on them. Each of which delivers 10s/100s of $ per month. That server pays for itself in revenue within weeks/months. And it will be in service until it stops working which could be over five years. Same with storage, networking, gpus, etc. They've spen…

> AWS at cost price would probably be 60-80x less than what they charge; if not more. If you look at https://ir.aboutamazon.com/news-release/news-release-details... , it says in 2024, > AWS segment sales increased 19% year-over-year to $107.6 billion. > AWS segment operating income was $39.8 billion, compared with operating income of $24.6 billion in 2023. So about 59% margin, relative to costs. Everybody undercuttin…

AWS is everything from GPU farms and AI services to S3.

I would be shocked if S3 had less than a 100% margin at sticker price.

These guys had one of the most important prerequisites to make going on-prem easy - they didn't care about cross-site or reliability.

Re: Building the heap: racking 30 petabytes of hard drives for pretraining

#225

Post by Maxion »

silisili wrote: post #218
jillesvangurp wrote: post #216

AWS obviously does the same but better. That's why they are so rich. Their cost is a tiny percentage of their revenue. They buy cheap servers, and then run lots of vms on them. Each of which delivers 10s/100s of $ per month. That server pays for itself in revenue within weeks/months. And it will be in service until it stops working which could be over five years. Same with storage, networking, gpus, etc. They've spen…

Inertia. It's the new 'nobody gets fired for buying..' A previous project I worked on had relatively little traffic, and AWS costs were rather insane for that. I proposed exploring OVH or DO and probably get costs down to 2 digits per month. Upper management would hear nothing of it - AWS was what they wanted, costs be damned. They were more protecting their own jobs than making a technical decision, I think.

Unless the savings would be more than 100k Eur / 300k++ USD (I.e. total cost of one employee, it's not really worth it. Even then, moving to new infrastructure carries high risk for business disruptions which can cause an even bigger dip in revenue.

The cloud providers have definitely optimized their pricing for maximum profit extraction. Costs are high, and in many cases it's not high enough to actually warrant changing infrstructure to cheaper alternatives.

Sticking with AWS / Azure / GCP carries other benefits, too. You're more likely to find engineers who are experienced with those cloud platforms over, say, OVH.

Re: Building the heap: racking 30 petabytes of hard drives for pretraining

#227

Post by irjustin »

jillesvangurp wrote: post #216

AWS obviously does the same but better. That's why they are so rich. Their cost is a tiny percentage of their revenue. They buy cheap servers, and then run lots of vms on them. Each of which delivers 10s/100s of $ per month. That server pays for itself in revenue within weeks/months. And it will be in service until it stops working which could be over five years. Same with storage, networking, gpus, etc. They've spen…

> This market is ripe for disruption. here should not be a need to shovel hundreds of billions per year into AWS revenue for the industry. The same business operating at 20% margins would be a game changer.

Honestly, I don't understand this line of thinking. You're not alone in it either, but it always reads so naive.

Like you can magically hand wave or it'd be so easy and get a competitor who can do it just as well and/or cheaper than AWS. But somehow these statements seem to ignore that you can't. Otherwise Digital Ocean would've, or cloudflare or .

The industry funnels billions to become vendor locked in because AWS is simply that good - bar none.

Re: Building the heap: racking 30 petabytes of hard drives for pretraining

#228

Post by zer00eyz »

silisili wrote: post #218
jillesvangurp wrote: post #216

AWS obviously does the same but better. That's why they are so rich. Their cost is a tiny percentage of their revenue. They buy cheap servers, and then run lots of vms on them. Each of which delivers 10s/100s of $ per month. That server pays for itself in revenue within weeks/months. And it will be in service until it stops working which could be over five years. Same with storage, networking, gpus, etc. They've spen…

Inertia. It's the new 'nobody gets fired for buying..' A previous project I worked on had relatively little traffic, and AWS costs were rather insane for that. I proposed exploring OVH or DO and probably get costs down to 2 digits per month. Upper management would hear nothing of it - AWS was what they wanted, costs be damned. They were more protecting their own jobs than making a technical decision, I think.

AWS costs are insane for every project.

> Upper management would hear nothing of it

No one knows how to plan ahead any more. It's all "agile" and hardware (and budgeting for it) isnt something most in management are capable of doing any more.

There is also then justifying the CapEx on a 5 year amortization schedule... the thing is even if you borrow that money at current rate (7 percent) you can still come out far ahead of AWS... It's a lot of math, and a lot of accounting (and the accountability that comes with it).

Your average CTO just doesn't have these skills.

Re: Building the heap: racking 30 petabytes of hard drives for pretraining

#229

Post by solatic »

jillesvangurp wrote: post #216

AWS obviously does the same but better. That's why they are so rich. Their cost is a tiny percentage of their revenue. They buy cheap servers, and then run lots of vms on them. Each of which delivers 10s/100s of $ per month. That server pays for itself in revenue within weeks/months. And it will be in service until it stops working which could be over five years. Same with storage, networking, gpus, etc. They've spen…

There's a long tail of smaller clouds: Digital Ocean, Oracle, IBM, Linode/Akamai, not to mention server providers like OVH and Hetzner, and not to mention Chinese clouds like Tencent and Alibaba (which both have US regions), not to mention PaaS providers who run on their own hardware like Fly.io.

I think it's very hard to make a claim that the market is not price-competitive. The problem is that most decision makers don't actually prioritize price, they prioritize support and the larger ecosystem. It's easy to find engineers with experience on the big 3 clouds and they will be able to pick up where the previous engineers left off. No CTO goes to sleep worrying about whether they being vendor-locked into AWS will result in catastrophic business failure tomorrow due to catastrophic hardware failure. There is a larger ecosystem - observability, FinOps tooling, cloud security tooling, managed databases, that are virtually guaranteed to support AWS, sometimes support GCP and Azure, and almost never support any of the other clouds.

It's questionable whether the current situation is really due to companies like Oracle and IBM being unable to compete on price and make strategic partnership deals to build out ecosystem support for their clouds. I think it's more likely that AWS/GCP/Azure "won" the cloud market, and that if regulators were worth a damn, they'd start to address the market concentration instead of ignoring it.

Re: Building the heap: racking 30 petabytes of hard drives for pretraining

#230

Post by vasco »

irjustin wrote: post #227
jillesvangurp wrote: post #216

AWS obviously does the same but better. That's why they are so rich. Their cost is a tiny percentage of their revenue. They buy cheap servers, and then run lots of vms on them. Each of which delivers 10s/100s of $ per month. That server pays for itself in revenue within weeks/months. And it will be in service until it stops working which could be over five years. Same with storage, networking, gpus, etc. They've spen…

> This market is ripe for disruption. here should not be a need to shovel hundreds of billions per year into AWS revenue for the industry. The same business operating at 20% margins would be a game changer. Honestly, I don't understand this line of thinking. You're not alone in it either, but it always reads so naive. Like you can magically hand wave or it'd be so easy and get a competitor who can do it just as well…

Even Google kept considering killing GCP for years. Meta is going to burn a bunch more billions on cartoons that play in your glasses before they get into cloud hosting and the other existing competitors have huge issues crossing the chasm of "everything is an API" that AWS has.
Post reply on HN