Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alternative clouds are booming as companies seek cheaper access to GPUs

techcrunch.com

Post reply on HN
Print view View original

311–320 of 336 posts

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#311

Post by benterix »

ikiris wrote: post #92
thorncorona wrote: post #65

Earlier quoted context omitted.

> Also of note that Hetzner is profitable, which means AWS has been operating at an insane markup. You can also evidence of this on their 10-Q :^)

If you think offering a cloud service is at all the same as offering someone a box in a rack, I invite you to compete in the space.

Hetzner is actually offering their Hetzner Cloud and it's a joy to work with because of its simplicity (think about the early days of AWS). You can do everything in Terraform or via their CLI if you prefer. Setting up a full k8s cluster takes maybe 10 minutes including all configuration.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#312

Post by rjst01 »

SllX wrote: post #10
freeqaz wrote: post #2

How are these companies even getting these GPUs? I would imagine NVIDIA would give them all to Microsoft and Google if they are in short supply.

It’s better to have a diversified customer-base rather than being completely beholden to a small number of big customers that might decide they don’t need you down the line.

Indeed, Google and Amazon have already shown an interest in designing their own silicon (TPU and Graviton). Probably Microsoft too, but I'm not aware of an example off the top of my head.

Upstarts are not easily going to be able to pursue that, so NVidia has a strong interest in supporting them.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#313

Post by AnthonyMouse »

withinboredom wrote: post #303
AnthonyMouse wrote: post #296

Earlier quoted context omitted.

> Price tags are pretty much universally handled at the stores. Advertising isn't. If the same ad is viewable by people in different states, what price are they supposed to put on it? Shouldn't this be the same price as what they show in the stores, or else people will have trouble knowing if the advertising was deceptive? > This is a solvable problem. So much so that if you've traveled in most nations you'll see tha…

I have no idea what you are going on about. If you advertise 9.99, it’s 9.99. Sales tax isn’t THAT different between jurisdictions, so we are talking about the seller having to lose a small percentage of margin. Sales are mostly about moving items, so they are already losing huge margins at that point anyway. Anyway, you can always advertise 9.99 + tax and show the actual price with tax in the store tags.

> Sales tax isn’t THAT different between jurisdictions, so we are talking about the seller having to lose a small percentage of margin.

California has a 7.25% sales tax. Oregon has a 0% sales tax. A 7.25% difference is larger than the entire net margin on many products.

> Sales are mostly about moving items, so they are already losing huge margins at that point anyway.

It seems like we've now gone from "there is no reason they can't do this" to "the merchant can just eat the sales tax and go out of business"?

> Anyway, you can always advertise 9.99 + tax and show the actual price with tax in the store tags.

Which is basically what they do now. You see the final price at the register before you pay. You also know that there is going to be sales tax, no one is being misled.

Putting a different price on the tags than there is on the ad would just confuse people and make it harder to tell when you're not getting the advertised price because you have to do fractional arithmetic on every soup can and candy bar to see if it matches.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#314

Post by exceptione »

metadat wrote: post #270
phh wrote: post #34

Earlier quoted context omitted.

You feel like it's new, I don't. I've evaluated migrating from full blown servers to cloud at various scales for various usages: OTA server for 1M active devices, stat server for 1M active devices, build server for Android (both pure Android that parallelizes and checks out super nearly and vendor Android where they broke all of that). In all those cases the cost of cloud was like an order of magnitude costlier. You'…

> In all those cases the cost of cloud was like an order of magnitude costlier. Don't discount the value-add of your skills. Giving any of my family members a stack of bare metal servers would serve no end-purpose or web requests. Approximately zero of said servers would even end up plugged in at all, much less operationalized. My family all works in tech or similarly demanding fields. That is, the cost of "the ops t…

> When it goes south, you're back to being a monkey plugging in cables and scratching your head.

That is if you rent rack space. If you rent bare metal and the bare metal has a failing disk, you file a request to replace said disk, and the data center engineer will be plugging cables, not you.

That means you have to worry about hardware failures, but you don't have the inconvenience of having to be physically present.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#315

Post by Palmik »

thundergolfer wrote: post #151
Palmik wrote: post #124

I really don't get this. Most of these low cost providers actually re-rent GPUs from Azure, AWS or GCP (*), yet they offer much better on-demand pricing (as low as $3.8/hr for H100 sxm and $2.5/hr for PCIe). And it's a fact that you can get even much better on-demand (not to mention reserved) pricing from the big clouds if you're a decent startup with connections. If one of these clouds offered fair pricing to SMBs,…

It's the opposite of what you say. Slim to none of the alternative cloud providers re-rent their GPUs. Coreweave, Fluidstack, Lambda Labs, Paperspace, Cudo Compute, Hydra, Datacrunch.io, Vultr, Crusoe Cloud, SF Compute. As far as I can tell none of these providers give you a GPU originating in Azure, AWS, or GCP.

RunPod is one of the providers that has some of the best on demand pricing (there are places where you can get H100 PCIe for less, but they typically don't have any capacity) without contracts and doesn't own hardware (or didn't as of few months ago).

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#316

Post by withinboredom »

AnthonyMouse wrote: post #313
withinboredom wrote: post #303

Earlier quoted context omitted.

I have no idea what you are going on about. If you advertise 9.99, it’s 9.99. Sales tax isn’t THAT different between jurisdictions, so we are talking about the seller having to lose a small percentage of margin. Sales are mostly about moving items, so they are already losing huge margins at that point anyway. Anyway, you can always advertise 9.99 + tax and show the actual price with tax in the store tags.

> Sales tax isn’t THAT different between jurisdictions, so we are talking about the seller having to lose a small percentage of margin. California has a 7.25% sales tax. Oregon has a 0% sales tax. A 7.25% difference is larger than the entire net margin on many products. > Sales are mostly about moving items, so they are already losing huge margins at that point anyway. It seems like we've now gone from "there is no r…

Don’t advertise California prices in Oregon?

I was referring to city/county taxes, not state taxes. Tor example, a local city to me in the US had a total of 14% sales tax, while outside the city was 10%.

Anyway, a sale’s motivation is to move products off the shelf and make room for more profitable products; they’re already losing money.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#318

Post by balls187 »

ipython wrote: post #281
relativ575 wrote: post #276

Earlier quoted context omitted.

> as for example the anesthesiologist may be out of network and you’ll never know until it’s too late Are you sure you don't make this up? Any insurance that I've ever had always let you search if a particular provider is in their network. I just verified on the website of my current insurance. Dozen of anesthesiologists nearby are listed. In fact they often go out of their way to notify you if your doctors leave the…

So let’s say I want to book an outpatient surgical procedure. I know my surgeon is in network and the facility I visit for the procedure is in network. However there are always other services required that you may not be aware of - anesthesiologists are the most common. This is a very common occurrence which I have encountered (thanks for calling me a liar). See for example https://www.reddit.com/r/personalfinance/co…

I suppose I just got lucky, using my primary doctor to refer me to services, and doing the research to find which referral docs were covered.

Oddly enough, I had the exact opposite experience recently, doc's billing dept said specific medical appliance would not be covered so I had to pay out of pocket, only to find out later it was reimbursed by insurance and I got refunded for my out of pocket cost.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#319

Post by allenho5 »

Palmik wrote: post #315
thundergolfer wrote: post #151

Earlier quoted context omitted.

It's the opposite of what you say. Slim to none of the alternative cloud providers re-rent their GPUs. Coreweave, Fluidstack, Lambda Labs, Paperspace, Cudo Compute, Hydra, Datacrunch.io, Vultr, Crusoe Cloud, SF Compute. As far as I can tell none of these providers give you a GPU originating in Azure, AWS, or GCP.

RunPod is one of the providers that has some of the best on demand pricing (there are places where you can get H100 PCIe for less, but they typically don't have any capacity) without contracts and doesn't own hardware (or didn't as of few months ago).

We have a decent amount of h100 sxm availability on demand on tensordock right now at far less than runpods h100 pcie price

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#320

Post by SllX »

rjst01 wrote: post #312
SllX wrote: post #10

Earlier quoted context omitted.

It’s better to have a diversified customer-base rather than being completely beholden to a small number of big customers that might decide they don’t need you down the line.

Indeed, Google and Amazon have already shown an interest in designing their own silicon (TPU and Graviton). Probably Microsoft too, but I'm not aware of an example off the top of my head. Upstarts are not easily going to be able to pursue that, so NVidia has a strong interest in supporting them.

Microsoft too: https://www.theverge.com/2023/11/15/23960345/microsoft-cpu-g...
Post reply on HN