Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alternative clouds are booming as companies seek cheaper access to GPUs

techcrunch.com

Post reply on HN
Print view View original

141–150 of 336 posts

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#141

Post by saltminer »

fffrantz wrote: post #119
fwip wrote: post #83

Earlier quoted context omitted.

Kubernetes is only the software layer, a lot of the cost is in the hardware/infrastructure, and in the salary for those experts.

Agreed 100 percent. Software is the easy part. Getting HVAC, power and network up to the levels of cloud providers is difficult to get right and prohibitively expensive. For instance, the cost for a pair of redundant symmetric gigabit fiber is in the thousands a month and may require tens of thousands of construction costs. These quickly add up, and the upfront costs can quickly reach six figures.

I remember seeing a quote for 500/500 metro E from Comcast several years ago. $12k to install, $1.2k/mo. And that only involved laying a few miles of fiber, no redundancy. Dedicated lines are no joke. If you're AWS or GCP, you can be your own ISP and mitigate this to some extent, but that's just the physical connection they save on.

You can always save by going on-prem, assuming you have no uptime requirements. But the moment you sign an SLA, those savings go out the window.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#143

Post by thefourthchime »

apitman wrote: post #22

Over the weekend a coworker was helping someone in our lab prepare data for an important conference talk. We accidentally ran up a rather large bill because while the EFS storage pricing was simple enough, the usage pricing bit us. It seems like AWS' entire business model is making the pricing so confusing that you don't know what it will cost until after you've used it. It feels weirdly similar to the US healthcare/…

The other way EFS will bite you is if you don't have enough provisioning and you push changes to prod that make it throttle to a point where it doesn't work and then you're making emergency changes from a hotel room on vacation.

I don't think that's AWS explicit business model, but I think they're perfectly fine with it happening.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#144

Post by pjlegato »

silverquiet wrote: post #137
ryandrake wrote: post #41

Earlier quoted context omitted.

> It seems like AWS' entire business model is making the pricing so confusing that you don't know what it will cost until after you've used it. It feels weirdly similar to the US healthcare/insurance situation. Not just the healthcare situation, but everything. There's nothing much more stereotypically American than "not knowing what you're going to pay for something until you're billed." Dozens of little fees on you…

I have to say that my response to "not knowing what you're going to pay for something until you're billed" is basically, "well I guess we'll see if I can pay it then". You sort of start asking yourself, "what if I just don't pay?". I have a (deadbeat) buddy who's entire medical plan is essentially to give a fake name to the ER, and if you're poor in America, what else can you do?

> give a fake name to the ER, and if you're poor in America, what else can you do?

The American government spends something approaching a trillion dollars annually on Medicaid, "a government program that provides health insurance for adults and children with limited income and resources."[1]

Seperately, Obamacare[2] created a private health insurance market where low income people can obtain free or heavily discounted private health insurance coverage, according to their income.

[1] https://en.wikipedia.org/wiki/Medicaid [2] https://en.wikipedia.org/wiki/Affordable_Care_Act

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#145

Post by godelski »

ryandrake wrote: post #41
apitman wrote: post #22

Over the weekend a coworker was helping someone in our lab prepare data for an important conference talk. We accidentally ran up a rather large bill because while the EFS storage pricing was simple enough, the usage pricing bit us. It seems like AWS' entire business model is making the pricing so confusing that you don't know what it will cost until after you've used it. It feels weirdly similar to the US healthcare/…

> It seems like AWS' entire business model is making the pricing so confusing that you don't know what it will cost until after you've used it. It feels weirdly similar to the US healthcare/insurance situation. Not just the healthcare situation, but everything. There's nothing much more stereotypically American than "not knowing what you're going to pay for something until you're billed." Dozens of little fees on you…

Don't get me started on dark patterns! I think the fact that we create them should be something we discuss.

But I think we need to up our game in this cat and mouse game a bit. For example, in aggregators -- like Expedia or Google Flights, etc -- why not try to capture some of these fees in the price? I can search for hotels with parking but what about sorting hotels by price and including the parking price? It's hard to compare when I see a $120 hotel that has a $50 valet vs a $150 hotel that includes parking. But that's the thing I'm actually after a lot of times. Or similar with flights and baggage fees. We should be able to collect a lot of this type of information and properly present it to the users and try to make these types of dark patterns ineffective (still will be cat and mouse and this is only a specific type of pattern, but still, I think there are things we can do)

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#147

Post by TacticalCoder »

zeroxfe wrote: post #44
matsz wrote: post #29

Earlier quoted context omitted.

This is precisely why I stick to dedicated servers for my own personal projects. $40/month for a machine, doesn't get any more predictable than this.

There's no way you're getting a machine for $40/month. :-) Did you mean a dedicated VM or VPS? (I have a bunch of actual dedicated machines with different providers, and this would save me a lot of money.) (Edit: holy moly those prices are fantastic!)

> There's no way you're getting a machine for $40/month. :-)

Oh you can!

I've got several dedicated servers at OVH. My absolute cheapest one is an "ECO" / Kimsufi (Kimsufi is a company which spun out of OVH then, a few years later, back into OVH) which I pay... 5 EUR / month. 6 EUR / month with VAT (so 6.5 USD per month).

Sure, it's not beefy at that price: an Atom N2800 with 4 GB or RAM but it is a dedicated server with its own IPv4 IP (yup, there can be uses for that).

I mostly use it as a jump host / reverse-ssh-with-a-known-fixed-IP thinggy.

They've got great dedicated servers at very good price and they're not the only ones in that space.

These can be rebooted/reinstalled remotely and they're monitored: OVH shall deal with hardware failure, if any, for you (never had any so far).

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#148

Post by jedberg »

Palmik wrote: post #134
jedberg wrote: post #19

Earlier quoted context omitted.

I head up Product at Lambda. We are an NVIDIA preferred partner (and keep winning the preferred partner award). I don't know what the allocation numbers are for other companies, but we get a lot. It's in NVIDIAs best interest to spread the love around and make sure all the GPUs don't go to the hyperscalers.

Are there any plans to support your on demand cloud in the future? In the past few months there are basically no GPUs available. My understanding is that your reserved cloud starts at 64 GPUs (8 pods) and with quite a long reservation time frame.

We have new product coming in just a couple weeks to get clusters as small as 16 GPUs per cluster and only a two week minimum reservation, with very short lead times (the first people to reserve will have almost no lead time!).

Also we expect to have a bunch of capacity come online of the on-demand cloud this year. We're getting the GPUs as quickly as we can and racking them as quickly as possible, but we have to wait just like everyone else for the GPUs to roll off the fab. :)

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#149

Post by 015a »

wmf wrote: post #105
015a wrote: post #16

We're absolutely seeing the same thing even in non-GPU traditional servers. Here is an estimated monthly cost breakdown for a new GCP instance type, the n4-standard-2, in us-central1: 2 vCPU + 8 GB memory: $69.18 10 GB Hyperdisk Balanced: $0.80 3060 provisioned IOPS: $15.30 155 MB/s provisioned throughput: $6.20 Total: $91.48 Like, you can tear apart that $69/mo for 2vCPU + 8gb of memory, no problem. That's utterly i…

Hyperdisk is a SAN; it's not comparable to local storage. Unfortunately Google's local SSDs are also overpriced.

Sure, but you literally cannot use any other kind of SSD with the n4-class instances, and n4 are the only instances they offer on Emerald Rapids; they're advertised as general purpose, flexible, and high performance, basically their workhorses. If you want to use a local SSD you have to use older generation chips.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#150

Post by danielvaughn »

silverquiet wrote: post #137
ryandrake wrote: post #41

Earlier quoted context omitted.

> It seems like AWS' entire business model is making the pricing so confusing that you don't know what it will cost until after you've used it. It feels weirdly similar to the US healthcare/insurance situation. Not just the healthcare situation, but everything. There's nothing much more stereotypically American than "not knowing what you're going to pay for something until you're billed." Dozens of little fees on you…

I have to say that my response to "not knowing what you're going to pay for something until you're billed" is basically, "well I guess we'll see if I can pay it then". You sort of start asking yourself, "what if I just don't pay?". I have a (deadbeat) buddy who's entire medical plan is essentially to give a fake name to the ER, and if you're poor in America, what else can you do?

I worked in a couple of emergency rooms for a total of 6 years, and you aren't wrong. Assuming the care you need can be entirely performed within the ER, then that's a viable strategy so long as you don't intend to re-visit the hospital.

Hospitals are obligated to provide care regardless of your ability to pay in the moment, and of course they often take care of indigent and foreign patients, with the understanding that they won't be able to recoup their costs.

If you don't have your identification on you, then they simply give you a number to call and they usually ask to respond within a few business days because after that, it's harder to get insurance approval.

This all changes if you're admitted to the hospital and need surgery, because I think there's pre-approval required from your insurance carrier. Though since I worked in the ER, my memory is hazy on that.

If they believe you were deceptive and simply refused to be identified, then that is technically illegal, so they will put up your picture and will alert the police if they see you again. But if you never intend to visit the hospital again, it doesn't really matter. No one is going to hunt you down unless you're doing it on such a large scale that it can't be ignored.

This all said, I want to say that I still think it's an unethical thing to do.

Post reply on HN