Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alternative clouds are booming as companies seek cheaper access to GPUs

techcrunch.com

Post reply on HN
Print view View original

151–160 of 336 posts

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#151

Post by thundergolfer »

Palmik wrote: post #124

I really don't get this. Most of these low cost providers actually re-rent GPUs from Azure, AWS or GCP (*), yet they offer much better on-demand pricing (as low as $3.8/hr for H100 sxm and $2.5/hr for PCIe). And it's a fact that you can get even much better on-demand (not to mention reserved) pricing from the big clouds if you're a decent startup with connections. If one of these clouds offered fair pricing to SMBs,…

It's the opposite of what you say. Slim to none of the alternative cloud providers re-rent their GPUs.

Coreweave, Fluidstack, Lambda Labs, Paperspace, Cudo Compute, Hydra, Datacrunch.io, Vultr, Crusoe Cloud, SF Compute.

As far as I can tell none of these providers give you a GPU originating in Azure, AWS, or GCP.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#152

Post by hn_throwaway_99 »

npalli wrote: post #129
hn_throwaway_99 wrote: post #49

Earlier quoted context omitted.

> My go to line here is that Cloud was a ZIRP. Like the whole entire thing. Took us ~10 years to wind up the cloud, and it will take years to unwind it, but the mass migration away is already happening. This is Hacker News echo chamber stuff. There is certainly no mass migration away from the cloud. Yes, I personally saw companies in the 2010s say "we're moving everything to the cloud!" without adequate planning or c…

No, not HN echo chamber. 83% of CIOs want to move workloads back to on-prem or private cloud. https://x.com/michaeldell/status/1780672823167742135?s=46&t=...

I would take that with a giant tub of salt:

1. As the saying goes, talk is cheap. It's one thing to ask "What do you plan to do?" vs. what you actually do. Look at revenue graphs for AWS, Azure and GCP over the past 5-10 years, right up until the end of 2023. They are definitely not shrinking.

2. I'd be more than a bit skeptical of the messenger, given that Dell obviously has a vested interest in telling people they need to buy more servers.

3. Even if you take what the surveyed CIOs say at face value, asking "Are you planning to move some workloads back to private cloud/on-prem from public cloud" is totally consistent with what I said. There was rush of "just put everything on the cloud" without thinking through it strategically. But just because you're pulling back on a few ill-thought-out cloud projects doesn't mean that overall industry-wide public cloud investment isn't still skyrocketing.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#153

Post by claytonjy »

Palmik wrote: post #124

I really don't get this. Most of these low cost providers actually re-rent GPUs from Azure, AWS or GCP (*), yet they offer much better on-demand pricing (as low as $3.8/hr for H100 sxm and $2.5/hr for PCIe). And it's a fact that you can get even much better on-demand (not to mention reserved) pricing from the big clouds if you're a decent startup with connections. If one of these clouds offered fair pricing to SMBs,…

The middlemen you're talking about do two things: buy lots of reserved compute on the hyperscalers, and then pit the hyperscalers against eachother to get better pricing.

If you're reserving thousands of GPUs from the same hyperscaler, even if they're the only cloud you run on, you're not paying the price shown in the calculator. If you have other suppliers, you'll get an even better deal. Then you resell that reserved compute as on-demand compute, somewhere between your costs and what your customers would pay a hyperscaler directly.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#154

Post by TacticalCoder »

ikiris wrote: post #92
thorncorona wrote: post #65

Earlier quoted context omitted.

> Also of note that Hetzner is profitable, which means AWS has been operating at an insane markup. You can also evidence of this on their 10-Q :^)

If you think offering a cloud service is at all the same as offering someone a box in a rack, I invite you to compete in the space.

It's more than a box in a rack. These providers do actively monitor and fix these boxen. They can all be rebooted remotely as if you physically hit the button, you've got interfaces to access the machine as if you were logging in physically with a DB-9/RS232/ethernet/whatever console, etc.

It's not just "space in a rack and you deal with the servers yourself and you come to fix them if they break".

These companies know what they're doing.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#155

Post by htrp »

la64710 wrote: post #146

How are people using these GPU clouds with their data residing in one of the big cloud providers like AWS or Azure? Are they paying for egress to get data onto the GPU clouds?

Yes.... you eat the egress cost.

Pretty soon, these specialists will build object storage and all of the other "costs" that the legacy hyperscalers already incur.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#156

Post by htrp »

claytonjy wrote: post #131

What are folk's experiences with alternative cloud GPUs for inference ? If you're doing a lot of model training, buying GPUs or long-term reservations of GPUs is a no-brainer. But when it comes to inference, latency matters and it gets trickier talking between e.g. your AWS infra and your GPUs somewhere else. It seems lots of providers can give you enough to get by doing inference in a company's earliest stages. But…

Just go with an inference provider like fireworks/together/modal/baseten

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#158

Post by latchkey »

rootedbox wrote: post #79

@ Core-weave an A100 40GB NVLink is $2.06 ... the only way they are doing this is by burning investor money. At that price it running it 24/7 it will take way over a year to recoup hardware + electricity cost. So my suggestion.. dump all your work @ Core-weave.. It's cheaper than buying the hardware yourself let alone the cost of managing it.

> the only way they are doing this is by burning investor money... it will take way over a year to recoup hardware

They've been running for years already. They can also offer these lower end gpus at that price cause the higher end ones offset things.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#160

Post by Hikikomori »

superfrank wrote: post #115
tjoff wrote: post #78

Earlier quoted context omitted.

That would make some sense if the edge-cases didnt have 10.000% margin.

I've worked at companies that were spending 6 figures a month on AWS and we had dedicated AWS employees who helped us understand our bill and keep pricing in check. We also had GCP and Azure reps constantly reaching out to see if they could win us over by showing us how they could lower our bill. Overcharging enterprise customers for things they're not using is actually a risk to AWS as it gives customers a reason to…

Idk about that, it becomes worse at larger scales. We're at 7 figures weekly with obscene amounts of waste. But at that scale it's like hundreds of Aws bills to understand.
Post reply on HN