Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alternative clouds are booming as companies seek cheaper access to GPUs

techcrunch.com

Post reply on HN
Print view View original

71–80 of 336 posts

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#71

Post by bauruine »

brigadier132 wrote: post #48
rco8786 wrote: post #32

Earlier quoted context omitted.

> AWS' entire business model is making the pricing so confusing My go to line here is that Cloud was a ZIRP. Like the whole entire thing. Took us ~10 years to wind up the cloud, and it will take years to unwind it, but the mass migration away is already happening. To be clear, I don't mean like AWS is going out of business or anything. Just that companies are a) realizing how insanely expensive it is, b) realizing ho…

I don't think people understand how absurd egress costs are. I was talking with some people in game dev because im making a multiplayer game and i wanted to understand what my future costs would be. Bandwidth was #1 over compute.

Yeah it's more than absurd. I have a rented server that does 230TB egress a month. That's 4710$ in AWS egress fees (0.02$ per GB). I pay 40 Euros or about 0.0002 per GB and that includes an AMD Ryzen 5 3600 with 64GB RAM.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#72

Post by rco8786 »

hn_throwaway_99 wrote: post #49
rco8786 wrote: post #32

Earlier quoted context omitted.

> AWS' entire business model is making the pricing so confusing My go to line here is that Cloud was a ZIRP. Like the whole entire thing. Took us ~10 years to wind up the cloud, and it will take years to unwind it, but the mass migration away is already happening. To be clear, I don't mean like AWS is going out of business or anything. Just that companies are a) realizing how insanely expensive it is, b) realizing ho…

> My go to line here is that Cloud was a ZIRP. Like the whole entire thing. Took us ~10 years to wind up the cloud, and it will take years to unwind it, but the mass migration away is already happening. This is Hacker News echo chamber stuff. There is certainly no mass migration away from the cloud. Yes, I personally saw companies in the 2010s say "we're moving everything to the cloud!" without adequate planning or c…

> for many, many companies the cost of hiring people to manage all this infrastructure and services is usually way more than a well-managed cloud project.

Yea, this is the line that everyone uses. I've worked at these companies, and the reality just doesn't line up with that. You end up still needing your whole Ops team, they're just building cloud tooling instead of on-prem tooling.

As to whether or not the migration away is happening...major cloud providers are already seeing people leave, and profits starting to contract. It's very early. Like I said in the original post, it took us a long time to wind up the cloud and it's going to take a long time to come back to reality.

Something roughly cloud-shaped will probably always remain..there are some legit use cases especially for companies that have spiky load profiles. I don't mean we're literally going back to running servers out of our IT closets. I just mean that as a whole we're going to be moving back to simpler deployments, simpler architectures, and most importantly, fixed/predictable costs.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#73

Post by superfrank »

apitman wrote: post #22

Over the weekend a coworker was helping someone in our lab prepare data for an important conference talk. We accidentally ran up a rather large bill because while the EFS storage pricing was simple enough, the usage pricing bit us. It seems like AWS' entire business model is making the pricing so confusing that you don't know what it will cost until after you've used it. It feels weirdly similar to the US healthcare/…

> It seems like AWS' entire business model is making the pricing so confusing that you don't know what it will cost until after you've used it.

Maybe I'm naive, but I don't think that's intentional. I think it's just a byproduct of trying to build something that works for everyone and every use case.

As you make your target market bigger and bigger, you continually hit edge case after edge case that you try and solve with "just one more" rule or option. Eventually the system becomes so complex that no layman can understand it.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#74

Post by programjames »

fancyfredbot wrote: post #64
Gbox4 wrote: post #43

A funny comment from that article: "On CoreWeave, renting an Nvidia A100 40GB — one popular choice for model training and inferencing — costs $2.39 per hour, which works out to $1,200 per month. On Azure, the same GPU costs $3.40 per hour, or $2,482 per month; on Google Cloud, it’s $3.67 per hour, or $2,682 per month." Am I missing something? I am sure I'm a bit rusty in math, but I can still handle a calculator. ~72…

The whole thing is nuts. They have the wrong costs multiplied by the wrong time period to get the wrong answers. https://coreweave.com/gpu-cloud-pricing says an A100 40GB NVLink is $2.06 whereas the article says $2.39. That's $1483.20 a month, whereas the article says $1200 and should say $1720 if they'd got the maths right.

Maybe an LLM helped with the math?

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#75

Post by apitman »

> Given hyperscaler dominance of the overall public cloud market, which demands vast investments in infrastructure and range of services that make little or no revenue, challengers like CoreWeave have an opportunity to succeed with a focus on premium AI services without the burden of hypercaler-level investments overall

Interesting. GPU-only providers targeting the AI market only need to implement a fraction of the services that AWS does. They don't even need to be geographically distributed. What does it matter if your GPU cluster is on the other side of the planet?

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#77

Post by apitman »

bauruine wrote: post #71
brigadier132 wrote: post #48

Earlier quoted context omitted.

I don't think people understand how absurd egress costs are. I was talking with some people in game dev because im making a multiplayer game and i wanted to understand what my future costs would be. Bandwidth was #1 over compute.

Yeah it's more than absurd. I have a rented server that does 230TB egress a month. That's 4710$ in AWS egress fees (0.02$ per GB). I pay 40 Euros or about 0.0002 per GB and that includes an AMD Ryzen 5 3600 with 64GB RAM.

May I ask who your provider is?

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#78

Post by tjoff »

superfrank wrote: post #73
apitman wrote: post #22

Over the weekend a coworker was helping someone in our lab prepare data for an important conference talk. We accidentally ran up a rather large bill because while the EFS storage pricing was simple enough, the usage pricing bit us. It seems like AWS' entire business model is making the pricing so confusing that you don't know what it will cost until after you've used it. It feels weirdly similar to the US healthcare/…

> It seems like AWS' entire business model is making the pricing so confusing that you don't know what it will cost until after you've used it. Maybe I'm naive, but I don't think that's intentional. I think it's just a byproduct of trying to build something that works for everyone and every use case. As you make your target market bigger and bigger, you continually hit edge case after edge case that you try and solve…

That would make some sense if the edge-cases didnt have 10.000% margin.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#79

Post by rootedbox »

@ Core-weave an A100 40GB NVLink is $2.06 ... the only way they are doing this is by burning investor money. At that price it running it 24/7 it will take way over a year to recoup hardware + electricity cost.

So my suggestion.. dump all your work @ Core-weave.. It's cheaper than buying the hardware yourself let alone the cost of managing it.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#80

Post by notarealllama »

dinvlad wrote: post #50
zeroxfe wrote: post #44

Earlier quoted context omitted.

There's no way you're getting a machine for $40/month. :-) Did you mean a dedicated VM or VPS? (I have a bunch of actual dedicated machines with different providers, and this would save me a lot of money.) (Edit: holy moly those prices are fantastic!)

Yeah, Hetzner has 20 vCPU 64GB RAM i5-13500 servers for $40/mo: https://www.hetzner.com/dedicated-rootserver/matrix-ex/ This is ~10x cheaper than the closest AWS option, and without the extra fees. Also of note that Hetzner is profitable, which means AWS has been operating at an insane markup.

Dang, that is a fantastic deal. The €100 / month is even better - DDR5 RAM, 2TB NVME raid 1, and it's all customizable too. Just have to wait for Ubuntu 24.04 to be available and might have to make this switch.
Post reply on HN