Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alternative clouds are booming as companies seek cheaper access to GPUs

techcrunch.com

Post reply on HN
Print view View original

81–90 of 336 posts

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#81

Post by jmgao »

Gbox4 wrote: post #43

A funny comment from that article: "On CoreWeave, renting an Nvidia A100 40GB — one popular choice for model training and inferencing — costs $2.39 per hour, which works out to $1,200 per month. On Azure, the same GPU costs $3.40 per hour, or $2,482 per month; on Google Cloud, it’s $3.67 per hour, or $2,682 per month." Am I missing something? I am sure I'm a bit rusty in math, but I can still handle a calculator. ~72…

The CoreWeave number is completely wrong, but Azure and Google Cloud appear to be exactly correct at 730 hours per month, which happens to be the number of hours in 365 / 12 days.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#82

Post by 015a »

ikiris wrote: post #26
015a wrote: post #16

We're absolutely seeing the same thing even in non-GPU traditional servers. Here is an estimated monthly cost breakdown for a new GCP instance type, the n4-standard-2, in us-central1: 2 vCPU + 8 GB memory: $69.18 10 GB Hyperdisk Balanced: $0.80 3060 provisioned IOPS: $15.30 155 MB/s provisioned throughput: $6.20 Total: $91.48 Like, you can tear apart that $69/mo for 2vCPU + 8gb of memory, no problem. That's utterly i…

if you don't need hyperdisk, why are you on that type of server?

Its the only instance class that's on Emerald Rapids. So if you want the best that Intel has to offer, you need to adopt hyperdisks.

But, to be clear: We're not.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#83

Post by fwip »

cpill wrote: post #69
tempest_ wrote: post #38

Earlier quoted context omitted.

It is really the second one. The minute you want a second site or even HA at a single site the complexity and costs start to explode.

k3s is very simple to setup and and nodes to. if the machines are on the same LAN or the internet then it's not such a complex job albeit you need to know the basics of kubernetes.

Kubernetes is only the software layer, a lot of the cost is in the hardware/infrastructure, and in the salary for those experts.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#84

Post by throp58fg9 »

zeroxfe wrote: post #44
matsz wrote: post #29

Earlier quoted context omitted.

This is precisely why I stick to dedicated servers for my own personal projects. $40/month for a machine, doesn't get any more predictable than this.

There's no way you're getting a machine for $40/month. :-) Did you mean a dedicated VM or VPS? (I have a bunch of actual dedicated machines with different providers, and this would save me a lot of money.) (Edit: holy moly those prices are fantastic!)

[dead]

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#85

Post by coffeebeqn »

Gbox4 wrote: post #43

A funny comment from that article: "On CoreWeave, renting an Nvidia A100 40GB — one popular choice for model training and inferencing — costs $2.39 per hour, which works out to $1,200 per month. On Azure, the same GPU costs $3.40 per hour, or $2,482 per month; on Google Cloud, it’s $3.67 per hour, or $2,682 per month." Am I missing something? I am sure I'm a bit rusty in math, but I can still handle a calculator. ~72…

Have you ever asked a LLM to calculate costs for you? This is it exactly what it looks like

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#86

Post by [deleted] »

Gbox4 wrote: post #43

A funny comment from that article: "On CoreWeave, renting an Nvidia A100 40GB — one popular choice for model training and inferencing — costs $2.39 per hour, which works out to $1,200 per month. On Azure, the same GPU costs $3.40 per hour, or $2,482 per month; on Google Cloud, it’s $3.67 per hour, or $2,682 per month." Am I missing something? I am sure I'm a bit rusty in math, but I can still handle a calculator. ~72…

[deleted]

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#88

Post by nextworddev »

zackangelo wrote: post #46

Would like to take a moment to recommend fly.io for GPU workloads. I've been building a prototype using them for the last couple of weeks and it's been great to use. I didn't have to jump through any hoops or apply for any quota adjustments to get started. And I especially appreciate how easy they make it to automatically scale your GPU instances to zero based on traffic.

Taking a step back, was it necessary to roll your own LLM (or whatever FM) API as opposed to using an off the shelf API

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#89

Post by ikiris »

superfrank wrote: post #73
apitman wrote: post #22

Over the weekend a coworker was helping someone in our lab prepare data for an important conference talk. We accidentally ran up a rather large bill because while the EFS storage pricing was simple enough, the usage pricing bit us. It seems like AWS' entire business model is making the pricing so confusing that you don't know what it will cost until after you've used it. It feels weirdly similar to the US healthcare/…

> It seems like AWS' entire business model is making the pricing so confusing that you don't know what it will cost until after you've used it. Maybe I'm naive, but I don't think that's intentional. I think it's just a byproduct of trying to build something that works for everyone and every use case. As you make your target market bigger and bigger, you continually hit edge case after edge case that you try and solve…

Was it "I didn't understand it and/or put in the effort to understand it first" ? Nah it must be "theres a conspiracy to make it confusing so we pay more"

AWS has one of the nicer to comprehend billing systems.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#90

Post by matsz »

distantsounds wrote: post #45
zeroxfe wrote: post #44

Earlier quoted context omitted.

There's no way you're getting a machine for $40/month. :-) Did you mean a dedicated VM or VPS? (I have a bunch of actual dedicated machines with different providers, and this would save me a lot of money.) (Edit: holy moly those prices are fantastic!)

OVH and Hetzner have offerings at that price range. you can use a tool like serverhunter.com to find all sorts of economically-priced servers.

Yep, I'm renting two servers from Hetzner; have one in Germany and another in Finland. Both cheap (€40/month) and over the few years I've been using their services, I have nothing to complain about.
Post reply on HN