Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alternative clouds are booming as companies seek cheaper access to GPUs

techcrunch.com

Post reply on HN
Print view View original

231–240 of 336 posts

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#231

Post by _factor »

jameshart wrote: post #189
ryandrake wrote: post #41

Earlier quoted context omitted.

> It seems like AWS' entire business model is making the pricing so confusing that you don't know what it will cost until after you've used it. It feels weirdly similar to the US healthcare/insurance situation. Not just the healthcare situation, but everything. There's nothing much more stereotypically American than "not knowing what you're going to pay for something until you're billed." Dozens of little fees on you…

Then pay for it with a credit card that gives you a random amount of cash back, and pay your credit card bill using your tax refund that came as a nice surprise at the end of the year, because you over withheld all year so you don’t actually even know what your real take home pay is supposed to be. No wonder Americans have no idea how the economy is doing.

Even paying cash. Unless the establishment offers a cash discount, you’re paying inflated credit card induced compensatory pricing.

It’s a real shame banks received 2-5% of most transaction for what costs them pennies. Sure, there are benefits, but their ask isn’t covering it.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#232

Post by balls187 »

dheera wrote: post #225
balls187 wrote: post #178

Earlier quoted context omitted.

You sort of have that already in the US for health, its tied to your insurance and its called max out of pocket. Of course it only applies to covered services, and insurance companies and medical providers don't necessarily make it simple to know what is and is not covered (though it IS improving). If you plan on seeking medical services without going through insurance, the ACA requires providers to provide you with…

> its tied to your insurance and its called max out of pocket F that, if I paid for insurance I shouldn't have to pay anything out of pocket > If you plan on seeking medical services without going through insurance, the ACA requires providers to provide you with a good faith estimate upfront. F that, if there is any out of pocket payments at all, I should be entitled to estimates even WITH insurance and they should b…

> F that, if I paid for insurance I shouldn't have to pay anything out of pocket

That's not how insurance works, bud. For one, you likely don't pay for insurance, its heavily subsidized by your employer, and they will determine which policies to offer you. For two, you share the risk with everyone else who has a policy with your provider.

I could see an argument for insurance companies being legislated to force them to cover more previously uncovered services, as the ACA did.

> F that, if there is any out of pocket payments at all, I should be entitled to estimates even WITH insurance and they should be legally mandated to be within 5% of the actual cost.

Sure, you are welcome to contact your insurance BEFORE you obtain services and find out what they will cover. In fact, its incumbent upon you to do that, and not expect HCP's to do that for you.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#233

Post by bee_rider »

127 wrote: post #209
ryandrake wrote: post #41

Earlier quoted context omitted.

> It seems like AWS' entire business model is making the pricing so confusing that you don't know what it will cost until after you've used it. It feels weirdly similar to the US healthcare/insurance situation. Not just the healthcare situation, but everything. There's nothing much more stereotypically American than "not knowing what you're going to pay for something until you're billed." Dozens of little fees on you…

It's strange from a foreigners perspective because all of this looks exactly like a dark pattern made to distract and confuse gambling addicts. Why does American government allow direct predation on its own citizens?

Did they manage to get AWS to provide some mechanism to get an up-front price in your country? That’s pretty neat if so.

I think lots of people go along with these as-you-go services because they’d rather deal with an unexpected bill, than having their servers shut off.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#234

Post by heavyset_go »

wongarsu wrote: post #113
ryandrake wrote: post #41

Earlier quoted context omitted.

> It seems like AWS' entire business model is making the pricing so confusing that you don't know what it will cost until after you've used it. It feels weirdly similar to the US healthcare/insurance situation. Not just the healthcare situation, but everything. There's nothing much more stereotypically American than "not knowing what you're going to pay for something until you're billed." Dozens of little fees on you…

Which is especially weird for the country that fought the cold war to show the supremacy of capitalism; a system that is based on market actors knowing the price and value of every product and service offered and making rational purchase decisions based on that.

Capitalism can't work without information asymmetry when it comes to pricing.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#235

Post by claytonjy »

htrp wrote: post #156
claytonjy wrote: post #131

What are folk's experiences with alternative cloud GPUs for inference ? If you're doing a lot of model training, buying GPUs or long-term reservations of GPUs is a no-brainer. But when it comes to inference, latency matters and it gets trickier talking between e.g. your AWS infra and your GPUs somewhere else. It seems lots of providers can give you enough to get by doing inference in a company's earliest stages. But…

Just go with an inference provider like fireworks/together/modal/baseten

An issue we've had when looking into some of these is that, they provide a layer of software abstraction we're not looking for. I don't want to use some providers bespoke library to wrap my model code; I just want to use NVIDIA Triton, either by providing an image or by providing a model repo. I only want the inference provider to handle the hardware.

I understand that's exactly what those provider _don't_ want, because it means they can't lock us in. But particularly when comparing an inference provider to GCP, where we already run everything in Triton on GKE, I don't want to rewrite my code just to see what their hardware layer is like.

Another complication is we often run multiple tightly-integrated models for a single application, where having them on the same GPU is critical. This is tricky or impossible in some inference-provider-frameworks.

There's too many options for running the latest LLM, and far fewer for running a bespoke set of fine-tuned models on GPUs.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#236

Post by umeshunni »

Gbox4 wrote: post #43

A funny comment from that article: "On CoreWeave, renting an Nvidia A100 40GB — one popular choice for model training and inferencing — costs $2.39 per hour, which works out to $1,200 per month. On Azure, the same GPU costs $3.40 per hour, or $2,482 per month; on Google Cloud, it’s $3.67 per hour, or $2,682 per month." Am I missing something? I am sure I'm a bit rusty in math, but I can still handle a calculator. ~72…

My favorite quote about journalists goes something like "Never trust a journalist's math. If they could do math, they wouldn't have become journalists."

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#237

Post by silverquiet »

vel0city wrote: post #226
silverquiet wrote: post #198

Earlier quoted context omitted.

Have you considered perhaps that because I live in Texas, I know a bit more about those programs than you who have just googled and linked the top results? And that perhaps because I live in Texas, I indeed have issues with Texas (still a part of America in spite of the wishes of some on the right wing)? From your own article (I knew this stat would be in there): > Currently, Texas leads the nation in the number of u…

> Why do you think so many are uninsured if it is so simple for them to get health coverage? High numbers of people who have illegally immigrated and are worried about getting into programs subsidized by the government which might result in questions about residency status leading to deportation? High rates of misinformation surrounding the costs of insurance and availability of welfare programs in the state? I just…

Yes - my plan if ever separated from employer insurance is to manipulate my income and expenses (I have significant investments I can draw upon and access to a family real estate portfolio that allows me to live rent-free) in order to get the full subsidy. Most people are not like me however; my buddy has several times had his car impounded for not paying car insurance. I don’t think health insurance will make the cut in his budget.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#238

Post by rmah »

pjlegato wrote: post #144
silverquiet wrote: post #137

Earlier quoted context omitted.

I have to say that my response to "not knowing what you're going to pay for something until you're billed" is basically, "well I guess we'll see if I can pay it then". You sort of start asking yourself, "what if I just don't pay?". I have a (deadbeat) buddy who's entire medical plan is essentially to give a fake name to the ER, and if you're poor in America, what else can you do?

> give a fake name to the ER, and if you're poor in America, what else can you do? The American government spends something approaching a trillion dollars annually on Medicaid, "a government program that provides health insurance for adults and children with limited income and resources."[1] Seperately, Obamacare[2] created a private health insurance market where low income people can obtain free or heavily discounte…

Most Americans don't realize just how huge and extensive medicare/medicaid is. They cover 65mil and 88mil people respectively, including 40mil children. That's 153mil people out of 330mil population -- that's 46% of the USA's population. Total spending is $1.9T for both. A cost of $12,400/person covered. $1.9T is 7.5% of the total US GDP of $25.44T. Medicare/medicaid is a massive program. Bigger than social security at $1.4T. It's crazy huge.

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#239

Post by silverquiet »

rmah wrote: post #238
pjlegato wrote: post #144

Earlier quoted context omitted.

> give a fake name to the ER, and if you're poor in America, what else can you do? The American government spends something approaching a trillion dollars annually on Medicaid, "a government program that provides health insurance for adults and children with limited income and resources."[1] Seperately, Obamacare[2] created a private health insurance market where low income people can obtain free or heavily discounte…

Most Americans don't realize just how huge and extensive medicare/medicaid is. They cover 65mil and 88mil people respectively, including 40mil children. That's 153mil people out of 330mil population -- that's 46% of the USA's population. Total spending is $1.9T for both. A cost of $12,400/person covered. $1.9T is 7.5% of the total US GDP of $25.44T. Medicare/medicaid is a massive program. Bigger than social security…

In that case, why not just go all the way to Medicare for All?

Re: Alternative clouds are booming as companies seek cheaper access to GPUs

#240

Post by s1artibartfast »

thaumaturgy wrote: post #108
thfuran wrote: post #67

Earlier quoted context omitted.

>hook pricing (aka up front pricing that looks low and turns out high) is only rational. Yes, many deceptive business practices are rational acts on the part of the businesses. That doesn't mean they should be tolerated.

I have begun to think of this as a "large population error", and I'm noticing it in a lot of different markets. At small scales, annoying your customers is bad business. You only need to lose a few before it begins to hurt. Customer complaints are more likely to be a consideration in business decisions. At larger scales, a business can begin to preferentially adopt practices intended to drive away some customers. Per…

The intuitive answer is competition, where customers move from user-hostile companies to new ones.

The problem in my mind is that the largest companies have too much efficiencies of scale to compete with on price.

When competition can't undercut on price, it is hard to argue that customers aren't being served by monopoly mega corps.

Post reply on HN