Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Azure has run out of compute – anyone else affected?

news.ycombinator.com

Post reply on HN
Print view View original

121–130 of 359 posts

Re: Ask HN: Azure has run out of compute – anyone else affected?

#121

Post by Innominate »

bri3d wrote: post #23

Every cloud provider will have these issues with specific instance types in specific regions, although the Azure Germany situation sounds perhaps a bit more dire. At my past (much larger) employers we’ve always run into hardware capacity issues with AWS too - we’re just able to work around them. Building on cloud requires a lot of trade offs, one being a need for very robust cross-region capability and the ability to…

As much as this happens, I don't feel it's something to be expected or even okay.

The major cloud services are expensive. This extra cost is supposed to provide for cloud services' high level of flexibility. Running out of capacity should be a rare event and treated as a high priority problem to be fixed asap.

Without the ability to rapidly and arbitrarily scale, they're just overpriced server farms.

Re: Ask HN: Azure has run out of compute – anyone else affected?

#122

Post by robertlagrant »

Interesting semi-confirmed anecdote: when lockdown hit, Azure began to refuse to allocate servers. One of the main reasons was they prioritised servers in this way:

1. Government/health/defence cloud customers

2. Teams, which was exploding in use and they wanted to capitalise on it

3. Regular cloud customers

Re: Ask HN: Azure has run out of compute – anyone else affected?

#123

Post by fuzzy2 »

Sort-of. I have a Postgres flexible database in the West Germany Central region that can no longer be scaled. It was only created for testing purposes, so no biggie. The backend is basically a managed Compute resource.

If you need more reliability, I see only one way out: Go multi-region or even multi-cloud.

Re: Ask HN: Azure has run out of compute – anyone else affected?

#124

Post by cyptus »

we have the same issue and escalated it through multiple azure teams.

our quota has been silently set to 0 while there where still instances running. this worked fine until auto-scale scaled the instances down in the night to 1. at the start of the day auto scale was not able to scale back up to the initial amount which did lead into heavy performance issues and outages. we needed to move the instances as azure support did not help us. after many calls with azure and multiple teams involved we finally did not get the quota approved (even if we did have it already and was not asking for „new“ quotas).

also we decided to not be able to host in the German azure region anymore. Even if we could get the quota this is a business risk we don’t want to bear anymore to not be able to scale for unexpected traffic.

this is huge for us as our application requires German servers. We are still in research where to host in future.

Re: Ask HN: Azure has run out of compute – anyone else affected?

#125

Post by cosmic_quanta »

bushbaba wrote: post #92
andrewstuart wrote: post #69

Earlier quoted context omitted.

GPUs are better run in your own office. All cloud providers charge much, much more for GPUs than if you run a local machine. Cloud GPUs are also a lot slower than state of the art consumer GPUs. Cloud GPUs: much slower, less available, much more expensive.

Say you want 100 GPUs all inter connected to your multi petabyte data lake that’s being fed by your production workload. Sure you could buy all that equipment but I’d wager it’s cheaper, more agile, and greater velocity from it being in the cloud

I would argue that the cost profile is different.

Local GPUs are a big up-front cost. But assuming that your workload is stable, in the long run I think local GPUs ends up being cheaper per-hour than cloud.

For startups, it doesn't make sense to make the up-front purchase, fine. But if you're optimizing for long-term (amortized) costs, I'd be curious if cloud is cost-effective.

Re: Ask HN: Azure has run out of compute – anyone else affected?

#126

Post by pclmulqdq »

andrewstuart wrote: post #69
arecurrence wrote: post #58

This is not as rare as public clouds may lead people to believe. I have had to move workloads around since AWS began (even between public clouds on occasion). In particular, GPU availability has been a continuing problem. Unlike interchangeable x64 / arm64 instances with some adjustments based on the new core and ram count... if no GPU instances are available then I simply cannot run the job. AMD's improved support h…

GPUs are better run in your own office. All cloud providers charge much, much more for GPUs than if you run a local machine. Cloud GPUs are also a lot slower than state of the art consumer GPUs. Cloud GPUs: much slower, less available, much more expensive.

This is generally true for all accelerators (I work with cloud and on-prem FPGAs for my startup, Arbitrand).

However, lots of people only need those accelerators once in a while, so time sharing (aka cloud computing) makes a lot of sense and saves a ton of money overall. For FPGAs and some compute GPU applications, not having to handle support for your accelerators is also nice.

Re: Ask HN: Azure has run out of compute – anyone else affected?

#128

Post by layer8 »

bushbaba wrote: post #100
layer8 wrote: post #52

Earlier quoted context omitted.

Why would they make any promises, or be upfront about their resources at the risk of becoming less attractive compared to competitors with more resources? It’s not like many people are shunning the cloud for that reason today (although maybe they should).

Your price point and the clouds margin is tied to not sitting on lots of unused instances. you want there to be adequate capacity not excessive capacity

It goes both ways: cloud providers don’t want to make promises about capacity, and cloud users don’t want to make promises about usage.

I don’t know about price point. Dedicated servers can be cheaper than cloud in many cases, if you have the appropriate know-how, and the cloud business is very profitable for a reason.

Re: Ask HN: Azure has run out of compute – anyone else affected?

#130

Post by cyptus »

dekhn wrote: post #65
janober wrote: post #56

Earlier quoted context omitted.

We already tried to push Microsoft, sadly have they been not very helpful. Still trying to get in contact with somebody that can actually make a difference. After all, are we also not asking for a hundred machines. Can really not imagine that they can not somehow make the resources we require available.

I'll ask again what the person above asked: do you have a TAM. If you don't you're at a big disadvantage.

we tried escalating this through CASM and it did not work. The region is blocked for every quota, even a single instance.
Post reply on HN