Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Azure has run out of compute – anyone else affected?

news.ycombinator.com

Post reply on HN
Print view View original

21–30 of 359 posts

Re: Ask HN: Azure has run out of compute – anyone else affected?

#21

Post by xwowsersx »

xwowsersx wrote: post #10

Oof, that sucks and I feel for you. That said... > setting up in a new region would be complicated for us. Sounds to me like you've got a few weeks to get this working. Deprioritize all other work, get everyone working on this little DevOps/Infra project. You should've been multi-region from the outset, if not multi-cloud. When using the public cloud, we do tend to take it all for granted and don't even think about t…

I'll reply to my own comment in response to a since-deleted reply that went something to the effect of "this is terrible advice for a young startup trying to get to product market for":

I'm totally on board with the idea of being scrappy and taking shortcuts in order to get to PMF as soon as possible. However, it seems the proof is in the pudding here. If you can't service customers due to lack of compute resources, you can't get to PMF.

Also, yes there are certain infrastructure and network topologies that would absolutely be overkill for a young startup. I don't think multi-region is one of those things. I don't have experience with Azure directly, but on every other cloud providers, going multi-region is not something that requires huge amounts of time or resources. You just need to be mindful of it from the outset. And if you decide not to be, then at least be intentional and conscious about the risk and have a plan in place for what happens when you get bit by deciding not to go multi-region.

Re: Ask HN: Azure has run out of compute – anyone else affected?

#22

Post by kulahan »

andrewstuart wrote: post #4

Yes it’s weird that you have to ask them for instances which some actual physical person looks at your request, thinks about it and says yes or no to. Instead of providing you with a list of the resources they do have, you have to play this weird game where you ask for specific instances in specific regions and then within several hours someone emails back to say yes or no. If it’s no, you have to guess again where y…

I briefly worked on an Azure team, and what I remember hearing (a few years ago) was that they were building out data warehouses as fast as they can, but they simply cannot keep up with demand. A good problem to have, I thought, but maybe not in light of this news!

Re: Ask HN: Azure has run out of compute – anyone else affected?

#23

Post by bri3d »

Every cloud provider will have these issues with specific instance types in specific regions, although the Azure Germany situation sounds perhaps a bit more dire. At my past (much larger) employers we’ve always run into hardware capacity issues with AWS too - we’re just able to work around them.

Building on cloud requires a lot of trade offs, one being a need for very robust cross-region capability and the ability to be flexible with what instance types your infrastructure requires.

I’d use this as a driver to either invest in making your software multi regional or cloud agnostic. Multi regional will be easier. If you’re already on k8s you should have a head start here.

Re: Ask HN: Azure has run out of compute – anyone else affected?

#24

Post by websap »

ttrrooppeerr wrote: post #16

> We never thought our startup would be threatened by the unreliability of a company like Microsoft You will be threatened by your own unreliability of building something that's dependant on one region or one cloud.

This is an insidious argument to make. When building a startup you should choose 1 reliable cloud provider and use their best practices to support high availability.

Re: Ask HN: Azure has run out of compute – anyone else affected?

#25

Post by pwarner »

Azure, despite being smaller than AWS, I think has more regions. So each one must be smaller, which likely means less spare capacity.

I also sort of suspect the spot market is less robust there. Lots of Azure is lift and shift on premises workloads, and those aren't using spot. Without people using spot, it's even harder to have spare capacity...

Re: Ask HN: Azure has run out of compute – anyone else affected?

#26

Post by jeffbee »

EC2 us-east-1 is chronically stocked out, too. Black Friday is the worst day of the year for this. At work, we pre-allocated tons of EC2 machines we don't really need, to hedge against EC2 stockout coinciding with some kind of incident. Yes, we are part of the problem.

Re: Ask HN: Azure has run out of compute – anyone else affected?

#28

Post by janober »

ttrrooppeerr wrote: post #16

> We never thought our startup would be threatened by the unreliability of a company like Microsoft You will be threatened by your own unreliability of building something that's dependant on one region or one cloud.

Totally agree, we could for sure have build from the get go multi-region and multi-cloud but we had good reasons not to do it. Depending on the product, technology, ... would actually also strongly recommend almost every startup to do the same.

Re: Ask HN: Azure has run out of compute – anyone else affected?

#29

Post by my123 »

Terretta wrote: post #19
Moissanite wrote: post #9

Azure Germany is a separate partition from the rest of Azure - presumably for compliance reasons. This is distinct from AWS, where Frankfurt is just another region, albeit one with high demand.

> AWS .. Frankfurt is just another region Unlike GCP and Azure, all AWS regions are (were) partitioned by design. This "blast radius" is (was) fantastic for resilience, security, and data sovereignty. It is (was) incredibly easy to be compliant in AWS, not to mention the ruggedness benefits. AWS customers with more money than cloud engineers kept clamoring for cross-region capabilities ("Like GCP has!"), and in last…

Cross-region extensibility points are few and far between. See https://docs.aws.amazon.com/whitepapers/latest/aws-fault-iso... for more details.

Re: Ask HN: Azure has run out of compute – anyone else affected?

#30

Post by websap »

xwowsersx wrote: post #10

Oof, that sucks and I feel for you. That said... > setting up in a new region would be complicated for us. Sounds to me like you've got a few weeks to get this working. Deprioritize all other work, get everyone working on this little DevOps/Infra project. You should've been multi-region from the outset, if not multi-cloud. When using the public cloud, we do tend to take it all for granted and don't even think about t…

Is it that much cheaper for you to build a new region on Azure versus getting setup on AWS?

If you rely on Kubernetes for orchestration and have minimal cloud API dependency, it may be worth that evaluating this option.

Also, do you have a TAM associated with your account? Are you just going through regular support channels? Can they deliver different instance types (not sure what the Azure parallel is), can they deliver short term capacity, etc?

I would try to push Microsoft more here. It's not like they've stopped on-boarding new customers into that region right? What happens if you create a new account in that region?

Post reply on HN