Live data from Hacker News

Oracle Cloud is having a major outage

news.ycombinator.com

51–60 of 62 posts

Re: Oracle Cloud is having a major outage

#51

Earlier quoted context omitted.

It's not really that bad? If you're operating at large scale you should have enough control over your own infrastructure to distribute load to multiple providers. Then if one of them is down, spin up more instances on another one. If you're operating at small scale, store your backups on another provider, and periodically test that you can quickly restore them to that provider. This isn't just about redundancy. Doing…

> If you're operating at small scale, store your backups on another provider, and periodically test that you can quickly restore them to that provider. Sure, but that's not what people mean when they say cross cloud. It usually implies running active workloads. Orchestrate hundreds of workloads that each depend on one another across several clouds and they reorganize upon a failure introduces a lot of new failure mod…

> Sure, but that's not what people mean when they say cross cloud. It usually implies running active workloads.

For workloads large enough to justify load balancing, that isn't that hard. Things should only rarely care which "cloud" they're running on. The amount of the load assigned to each one is a knob you can turn.

For workloads smaller than that, the primary way to achieve redundancy is failover. Whether it's worth the cost and complexity of making that happen automatically instead of manually doesn't have a universal answer.

But "just use a more reliable provider" doesn't work if being down matters. AWS, Azure and Google Cloud have all had major outages. More than that, sometimes a piece of equipment fails in a way that takes adjacent equipment with it, or there is a fire or a burst pipe. They call it "cloud" but somewhere there is physical hardware under your bits and it can fail. Each failure may only affect a limited number of customers, but those customers can include you. If your systems can't be down, you need a plan in place to have them running again somewhere else in short order. And putting the system you use for this on a different provider can save you from a major provider outage.

Re: Oracle Cloud is having a major outage

#52

Wow, there must be literally tens of people who are worried right now! My shitty, snarky comment aside, I am genuinely curious about why someone would choose Oracle as a cloud provider. If you look at their capex spend, it's undeniable they have so vastly underinvested in their cloud compared to AWS, Azure and GCP, that even if you were an "Oracle shop" I'm genuinely curious what benefits their cloud would offer. Edi…

At my last company, the VP got the largest discount from Oracle, so Oracle cloud it was. Engineer attrition rose. Services took longer to build and maintain. Company's stock went down. Layoffs ensued. But the VP got promoted to SVP.

Why would services take longer to build? You mean, compared to going all in on every aws service?

Re: Oracle Cloud is having a major outage

#53

Earlier quoted context omitted.

Unfortunately these can disappear without warning. With AWS this basically never happens.

Can you elaborate on this? These are not preemptible, why would they disappear without warning? I've been running on these for 2 or 3 years without any issues thus far.

When your account changes from the time based free tier to the only forever free tier they often shut things down. Even if you're careful not to use the time based free tier stuff, which I was, it went pop anyway.

If you're below their "idle system" threshold, they'll shut it down.

Get past the 3 month or whatever it is tier. Put a low level load on the system to get it above the idle threshold and you should be good to have a nice personal playground. Probably.

I've had systems go down to both these. They also have rejected every credit card I tried to throw at them. Which is a known problem for them as well as some other cloud providers. I no longer put anything critical on OCI.

Re: Oracle Cloud is having a major outage

#54
post #7

Cross cloud is really complex and error prone. You'll probably cause more outages than you'll prevent by going down that path. Maybe you should consider moving to a major cloud provider that has better services.

It's not that hard. You only need a few rules to make it work.

* Manage your own VPN and don't depend on the vendor's solution. * Only use base level services that are available on every cloud you want to leverage. Which probably means you're on a container based system and not using anything like Lambda.

Then a service being in another cloud just doesn't matter. You're always making requests via your VPN anyway, if it's in a local cloud or remote cloud it doesn't matter.

You'll feel the egress pricing if you integrate cross cloud services that are chatty. So it really depends on your immediate goals.

If you just want redundancy then you need to keep the resources standing in the backup cloud/AZ and just move your entry point via DNS, in which case you don't really need "cross cloud", just the ability to provision else where.

Re: Oracle Cloud is having a major outage

#55

Wow, there must be literally tens of people who are worried right now! My shitty, snarky comment aside, I am genuinely curious about why someone would choose Oracle as a cloud provider. If you look at their capex spend, it's undeniable they have so vastly underinvested in their cloud compared to AWS, Azure and GCP, that even if you were an "Oracle shop" I'm genuinely curious what benefits their cloud would offer. Edi…

I use Oracle for certain workloads because their ARM compute offering is very flexible and cost effective. They let you select CPU count and memory semi-separately, without charging a huge premium for it. You can have 64 CPUs and 64 GB RAM (max CPU, min RAM) or 1 CPU and 64 GB RAM (min CPU, max RAM). I say "semi-separately" because they don't allow less than 1 GB RAM or more than 64 GB per CPU. For the workloads I'm talking about, they need a lot of RAM compared to CPUs. So Oracle's offering is very attractive for it. I've also found that the CPUs significantly outperform even "high performance" dedicated Intel CPUs at other providers for one of my compute heavy workloads (I still don't understand how that could be the case, but I'll take it). It only underperforms my desktop i7-11700k by about 25%.

Re: Oracle Cloud is having a major outage

#56
post #52

Earlier quoted context omitted.

At my last company, the VP got the largest discount from Oracle, so Oracle cloud it was. Engineer attrition rose. Services took longer to build and maintain. Company's stock went down. Layoffs ensued. But the VP got promoted to SVP.

Why would services take longer to build? You mean, compared to going all in on every aws service?

Lack of reliability and lack of documentation about corner cases.

Re: Oracle Cloud is having a major outage

#57
post #47

Earlier quoted context omitted.

Can you elaborate on this? These are not preemptible, why would they disappear without warning? I've been running on these for 2 or 3 years without any issues thus far.

They might be referring to their idle reclamation policy: https://docs.oracle.com/en-us/iaas/Content/FreeTier/freetier...

Thanks, very useful, I was not aware of this. There really should be a warning before this happens. I really need test my incremental backups now.

Re: Oracle Cloud is having a major outage

#58

Earlier quoted context omitted.

They need SVP+ level approval to mark disruption/unavailability.

I love this because I honestly can't tell if its a joke. It's obviously a terrible idea.... but also seems like something Oracle would do.

AWS does the same; it stays green unless the outage is so bad an exec has to approve changing the status page.

Surprisingly azure is very open with outages of all services big and small in my experience, and notifies if any service our tenant is using is impacted.

Re: Oracle Cloud is having a major outage

#59

Wow, there must be literally tens of people who are worried right now! My shitty, snarky comment aside, I am genuinely curious about why someone would choose Oracle as a cloud provider. If you look at their capex spend, it's undeniable they have so vastly underinvested in their cloud compared to AWS, Azure and GCP, that even if you were an "Oracle shop" I'm genuinely curious what benefits their cloud would offer. Edi…

Anyone stupid enough to enter into Oracle licensing for ERP and financials in the first place simply moves to it eventually as Microsofties inevitably move to Azure. In the name of "better" native integration, ie it sucks less as at least Oracle knows how bad Oracle is to deal with.

Doesn't make it better, just makes for vasoline to make the shagging less painful.

Post reply on HN