Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alternative clouds are booming as companies seek cheaper access to GPUs

techcrunch.com

Post reply on HN
Print view View original

31–40 of 336 posts

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#31

Post by drdaeman »

baobabKoodaa wrote: post #30
matsz wrote: post #29

Earlier quoted context omitted.

This is precisely why I stick to dedicated servers for my own personal projects. $40/month for a machine, doesn't get any more predictable than this.

If you rent a dedicated server from AWS, you will be hit for various additional fees which will likely dwarf that $40/month. So the issue here is not really shared vs dedicated instances. The issue here is that a particular cloud provider (namely AWS) has set up an opaque fee structure.

> If you rent a dedicated server from AWS

Who, in their right mind, goes for a bare metal to AWS, when there are so many decent and time-tested options out there?

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#32

Post by rco8786 »

apitman wrote: post #22

Over the weekend a coworker was helping someone in our lab prepare data for an important conference talk. We accidentally ran up a rather large bill because while the EFS storage pricing was simple enough, the usage pricing bit us. It seems like AWS' entire business model is making the pricing so confusing that you don't know what it will cost until after you've used it. It feels weirdly similar to the US healthcare/…

> AWS' entire business model is making the pricing so confusing

My go to line here is that Cloud was a ZIRP. Like the whole entire thing. Took us ~10 years to wind up the cloud, and it will take years to unwind it, but the mass migration away is already happening.

To be clear, I don't mean like AWS is going out of business or anything. Just that companies are a) realizing how insanely expensive it is, b) realizing how wildly volatile the pricing is, and c) starting to reach for services with transparent, fixed pricing

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#33

Post by swores »

lelanthran wrote: post #8
mym1990 wrote: post #6

Isn't the mid/late game of all of these alternative clouds going to be the same as the big cloud players...get customer lock in and then prices slowly creep up until its kind of a wash?

> Isn't the mid/late game of all of these alternative clouds going to be the same as the big cloud players...get customer lock in and then prices slowly creep up until its kind of a wash? That might be the plan, but I doubt it's going to work for GPUs like it did for AWS. There's no moat here, other than the hardware. There's no value-add that the provider can add for free and lock you in.

> There's no value-add that the provider can add for free and lock you in.

Couldn't you have said that about cloud services like AWS before they existed, that they can rent servers to people but there won't be any lock in?

And equally, just because that's the case with GPU providers right now, is there a reason one or more of them wouldn't be able to develop software which runs on the GPUs to do some of the things people are currently renting GPUs for, or middle-ground software which make it easier to do other stuff on them than just renting plain GPU access, and turning it into optional services (with lock in) on top of the raw hardware rental?

As an example idea: it's good that NVIDIA decided to let as many people use CUDA as possible (and I suspect if they hadn't then they wouldn't have seen nearly as much success), but if they or anyone else releases an equivalent to "CUDA v2" tomorrow, but instead of allowing anyone to download it instead put it behind a billing page with AWS-style pricing that covers both software and GPU, would it not succeed if the software did make thinks easier for people just like AWS does (in some ways)?

edit: I just realised I ignored the "for free" bit of your comment - but it wasn't free for Amazon or Google to build their cloud software either.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#34

Post by phh »

015a wrote: post #16

We're absolutely seeing the same thing even in non-GPU traditional servers. Here is an estimated monthly cost breakdown for a new GCP instance type, the n4-standard-2, in us-central1: 2 vCPU + 8 GB memory: $69.18 10 GB Hyperdisk Balanced: $0.80 3060 provisioned IOPS: $15.30 155 MB/s provisioned throughput: $6.20 Total: $91.48 Like, you can tear apart that $69/mo for 2vCPU + 8gb of memory, no problem. That's utterly i…

You feel like it's new, I don't. I've evaluated migrating from full blown servers to cloud at various scales for various usages: OTA server for 1M active devices, stat server for 1M active devices, build server for Android (both pure Android that parallelizes and checks out super nearly and vendor Android where they broke all of that). In all those cases the cost of cloud was like an order of magnitude costlier. You're mentioning SSD, the various clouds I tried (I couldn't remember which I tried sorry...) had bad storage performance (it was especially bad when building Android). Also not all those servers were maintained by my employer, when we pay top end 100Gbps server we can pay someone to maintain it for us as well for much lower money than the "cloud tax".

I have no doubt there are great use case for cloud, and that at the proper scale you can negotiate, and I understand that startups might be faster moving with the cloud. But I feel like the highest value cloud provides is 1. replacing capex with opex, and 2. making scaling easier with the direction: cloud is pay first, get questions from direction later. "on-prem" is negotiate with the direction until the service degrades and then scramble to integrate the new server under the pressure.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#35

Post by ElevenLathe »

baobabKoodaa wrote: post #30
matsz wrote: post #29

Earlier quoted context omitted.

This is precisely why I stick to dedicated servers for my own personal projects. $40/month for a machine, doesn't get any more predictable than this.

If you rent a dedicated server from AWS, you will be hit for various additional fees which will likely dwarf that $40/month. So the issue here is not really shared vs dedicated instances. The issue here is that a particular cloud provider (namely AWS) has set up an opaque fee structure.

In my experience, when people say "dedicated server" they typically mean something like OVH rather than a real "cloud" provider (AWS, GCP, Azure, etc.). In other words, something morally equivalent to colo but without having to ship servers around.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#38

Post by tempest_ »

candiddevmike wrote: post #27
015a wrote: post #16

We're absolutely seeing the same thing even in non-GPU traditional servers. Here is an estimated monthly cost breakdown for a new GCP instance type, the n4-standard-2, in us-central1: 2 vCPU + 8 GB memory: $69.18 10 GB Hyperdisk Balanced: $0.80 3060 provisioned IOPS: $15.30 155 MB/s provisioned throughput: $6.20 Total: $91.48 Like, you can tear apart that $69/mo for 2vCPU + 8gb of memory, no problem. That's utterly i…

IMO, the big thing holding back customers (especially smaller ones) from going on-premise these days is networking. Getting a fat pipe similar to what you get with an instance on a cloud provider can be prohibitively expensive (internet service, gear, staff, etc), especially when you want it to be highly available.

It is really the second one. The minute you want a second site or even HA at a single site the complexity and costs start to explode.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#39

Post by baobabKoodaa »

ElevenLathe wrote: post #35
baobabKoodaa wrote: post #30

Earlier quoted context omitted.

If you rent a dedicated server from AWS, you will be hit for various additional fees which will likely dwarf that $40/month. So the issue here is not really shared vs dedicated instances. The issue here is that a particular cloud provider (namely AWS) has set up an opaque fee structure.

In my experience, when people say "dedicated server" they typically mean something like OVH rather than a real "cloud" provider (AWS, GCP, Azure, etc.). In other words, something morally equivalent to colo but without having to ship servers around.

Yes, that is likely the case here as well. But if someone is not well versed in cloud providers and reads these comments, they might be misinformed without clarifying that this issue is AWS vs other providers, as opposed to dedicated vs shared resources.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#40

Post by leetharris »

jedberg wrote: post #19
freeqaz wrote: post #2

How are these companies even getting these GPUs? I would imagine NVIDIA would give them all to Microsoft and Google if they are in short supply.

I head up Product at Lambda. We are an NVIDIA preferred partner (and keep winning the preferred partner award). I don't know what the allocation numbers are for other companies, but we get a lot. It's in NVIDIAs best interest to spread the love around and make sure all the GPUs don't go to the hyperscalers.

I head engineering at Rev and every time we talk to Nvidia about GPUs they tell us to go to you guys. Every time. They really like Lambda!
Post reply on HN