Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alternative clouds are booming as companies seek cheaper access to GPUs

techcrunch.com

Post reply on HN
Print view View original

211–220 of 336 posts

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#211

Post by dangrairo »

Palmik wrote: post #130
indigodaddy wrote: post #95

Earlier quoted context omitted.

Wow that is quite decent pricing actually. This makes me want to try to deploy something like llamafile/ollama or similar to fly.io+gpu for my personal on-demand llama/llm whims. (@simonw I’m looking at you— I feel like if you haven’t already done this on fly.io, that you’re probably thinking about it :) ) Seems private enough.. could throw some basic auth on top of it for me and trusted friends/family so I don’t get…

It's horrible horrible pricing! Their on demand price for A100 is what gets you H100 sxm in other places.

I am curious what "other places" are you comparing it to.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#212

Post by acchow »

015a wrote: post #82
ikiris wrote: post #26

Earlier quoted context omitted.

if you don't need hyperdisk, why are you on that type of server?

Its the only instance class that's on Emerald Rapids. So if you want the best that Intel has to offer, you need to adopt hyperdisks. But, to be clear: We're not.

Curious what the use case is for targeting a specific CPU. These are virtual CPUs anyways, so what benefit does using the latest intel chip offer?

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#213

Post by breakingcups »

edgoode wrote: post #125

This is a trend we noticed early last year, so we started building a single console for all these clouds at https://shadeform.ai . It has been amazing to watch this industry explode, and we believe it is great for consumers. The same instances on Amazon versus these alternative providers are 3x more expensive. NVIDIA and many hardware providers are leaning into this trend. As clouds become more and more vertically in…

How does Shadeform make money?

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#214

Post by silverquiet »

pjlegato wrote: post #202
silverquiet wrote: post #198

Earlier quoted context omitted.

Have you considered perhaps that because I live in Texas, I know a bit more about those programs than you who have just googled and linked the top results? And that perhaps because I live in Texas, I indeed have issues with Texas (still a part of America in spite of the wishes of some on the right wing)? From your own article (I knew this stat would be in there): > Currently, Texas leads the nation in the number of u…

You said giving a fake name at the ER was the only option for poor AMERICANS to obtain healthcare. That is categorically false, and testifes to some sort of deep seated bias against our own country profoundly embedded in your worldview -- one which extends all the way to glib counterfactual promotion of verifiably false information about the country and its supposed moral shortcomings. 2) Texas has a much larger popu…

Pretty sure my buddy is an American - I've known him almost all my life. I'm quite sure he's poor - he asks me for money a bit more than I'd like. The fake name thing was just a strategy he developed after going to the ER due to a blow to the head which caused him to forget his name; I'm sure there are many creative ways to get healthcare, but most of the of poor Texans (whom I'm also pretty sure are Americans considering I'm related to a few of them) I've known just go to the ER and ignore the bills. It's probably not the best system, but it's the one we've got I guess.

As to point 2) I thought about including the info in my previous comment, but it wasn't in your own source material and I'm a bit lazy. So here, I'll do it in this one.

> Texas is still the state with the highest percentage of uninsured residents, at nearly 17 percent, according to the most recent U.S. Census Bureau survey released Thursday. [0]

[0]https://www.texastribune.org/2023/09/14/census-bureau-texas-...

Speaking as a Texan (also pretty sure I'm American) who lives with some disability, I can tell you that I've looked at the systems and it's a real fear of mine that I'll end up as one of these statistics.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#215

Post by rco8786 »

hn_throwaway_99 wrote: post #96
rco8786 wrote: post #72

Earlier quoted context omitted.

> for many, many companies the cost of hiring people to manage all this infrastructure and services is usually way more than a well-managed cloud project. Yea, this is the line that everyone uses. I've worked at these companies, and the reality just doesn't line up with that. You end up still needing your whole Ops team, they're just building cloud tooling instead of on-prem tooling. As to whether or not the migratio…

> I just mean that as a whole we're going to be moving back to simpler deployments, simpler architectures, and most importantly, fixed/predictable costs. That I can definitely agree with, I just believe that's fully possible to do with cloud (for the most part, though there are certainly some head turners like the recent news that AWS was charging for forbidden attempts on private S3 buckets, which is bonkers) and cl…

Yea that’s totally fair. I think we’re using slightly different definitions of “cloud” here. I’m mainly focused on the managed services/abstractions, things that abstract away the server itself and are usage based billed. But firing up a few ec2 instances is still technically “the cloud”, you are right.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#216

Post by tatpacc »

TechDebtDevin wrote: post #196
jameshart wrote: post #189

Earlier quoted context omitted.

Then pay for it with a credit card that gives you a random amount of cash back, and pay your credit card bill using your tax refund that came as a nice surprise at the end of the year, because you over withheld all year so you don’t actually even know what your real take home pay is supposed to be. No wonder Americans have no idea how the economy is doing.

I'm always surprised by people who expect and are excited for their refund. Half the country is pitching about politicians taxation plans when all the while they're volunteering to overpay out of every paycheck.

> people who expect and are excited for their refund.

and then pay % of your refund as a fee to CPA, so they help you maximize your refund.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#217

Post by gadflyinyoureye »

TechDebtDevin wrote: post #196
jameshart wrote: post #189

Earlier quoted context omitted.

Then pay for it with a credit card that gives you a random amount of cash back, and pay your credit card bill using your tax refund that came as a nice surprise at the end of the year, because you over withheld all year so you don’t actually even know what your real take home pay is supposed to be. No wonder Americans have no idea how the economy is doing.

I'm always surprised by people who expect and are excited for their refund. Half the country is pitching about politicians taxation plans when all the while they're volunteering to overpay out of every paycheck.

I know how many guns I have. I know the Feds have more. Therefore give them their pound of flesh before they show up with the myriad of guns.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#218

Post by tpm »

krisoft wrote: post #184
doctorpangloss wrote: post #52

Earlier quoted context omitted.

People complain about this relentlessly, but never change. Additionally, every single business owner I know complains that whatever it is they're selling, (1) it isn't worth the brain damage to sell to customers looking for the lowest price, (2) as long as people comparison shop in a harebrained way, hook pricing (aka up front pricing that looks low and turns out high) is only rational. It's not like they're providin…

> People complain about this relentlessly, but never change What do you mean by this? How would “people change” to get out of resort fees or confusing pricing systems?

Ban them. Here in the EU such behavior is mostly illegal.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#219

Post by williamstein »

carbocation wrote: post #24
015a wrote: post #16

We're absolutely seeing the same thing even in non-GPU traditional servers. Here is an estimated monthly cost breakdown for a new GCP instance type, the n4-standard-2, in us-central1: 2 vCPU + 8 GB memory: $69.18 10 GB Hyperdisk Balanced: $0.80 3060 provisioned IOPS: $15.30 155 MB/s provisioned throughput: $6.20 Total: $91.48 Like, you can tear apart that $69/mo for 2vCPU + 8gb of memory, no problem. That's utterly i…

Also on GCP, the vCPUs are usually hyperthreads (except for t2d- instance types, and perhaps a few others). So that machine you've described has 1 CPU core.

Reference: https://cloud.google.com/compute/docs/machine-resource#recom...

The t2d, t2a and h3 instance types have vCPU = core, and all other instance types have vCPU = thread.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#220

Post by williamstein »

ikiris wrote: post #26
015a wrote: post #16

We're absolutely seeing the same thing even in non-GPU traditional servers. Here is an estimated monthly cost breakdown for a new GCP instance type, the n4-standard-2, in us-central1: 2 vCPU + 8 GB memory: $69.18 10 GB Hyperdisk Balanced: $0.80 3060 provisioned IOPS: $15.30 155 MB/s provisioned throughput: $6.20 Total: $91.48 Like, you can tear apart that $69/mo for 2vCPU + 8gb of memory, no problem. That's utterly i…

if you don't need hyperdisk, why are you on that type of server?

On Google Cloud hyperdisks can be used on h3, c3, c3d, m3, n4 instance types and are required for n4. I.e., you are not allowed to use the n4 instance type without using a hyperdisk.
Post reply on HN