Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI Status: Multiple engines are down

status.openai.com

Post reply on HN
Print view View original

61–70 of 167 posts

Re: OpenAI Status: Multiple engines are down

#61

Post by verdverm »

danielscrubs wrote: post #47

To put this in perspective: the outage was 52 min. The total number of employees OpenAI have are 375. The launch date was just some months ago. At Google, the Cloud SQL dashboard was unavailable for around 12 hours a couple of weeks ago if I read this correctly: https://status.cloud.google.com/incidents/xg2qrL1UuSJiPDZALJ... The total number of employees of Google are 156 500. Google Cloud was launched 2008. So when…

How Google, and most likely all businesses, should think about outages is more complex than being up or down. Is it just part of the product, like Slack threads yesterday? Is it 1/2 or 1/1M requests or operations?

The Google SRE book is an excellent work: https://sre.google/sre-book/introduction/

Also some perspective, if you look at the OpenAPI outages for Jan 2023, the up time is worse than Google Cloud SQL.

https://status.openai.com/uptime

Re: OpenAI Status: Multiple engines are down

#62

Post by ksubedi »

sva_ wrote: post #50
ksubedi wrote: post #2

I just saw that Azure OpenAI service has a SLA and OpenAI does not. I thought they would have separated the infrastructure for free ChatGPT users and paying API customers.

I think their models all run on Azure infrastructure?!

They do, but Azure also provides a different OpenAI service with different terms and SLA: https://azure.microsoft.com/en-us/products/cognitive-service...

Re: OpenAI Status: Multiple engines are down

#63

Post by kube-system »

la64710 wrote: post #15
TOMDM wrote: post #14

LLM's are incredibly expensive to run. I imagine the huge demand that ChatGPT is seeing would make any cloud vendor sweat if you were to suddenly lump it on top of the usual demand. To me it's entirely unsurprising that OpenAI would have trouble keeping up. Good luck to them.

This is precisely why they should really open source their model so that anyone can download and run it on their own infrastructure. Just like google or others have done and one is free to run it on their own laptop (some even without a GPU) , on premise or on any cloud provider infrastructure. They can continue to provide a hosted service for their model but they should allow it to be downloaded just like BERT.

Their end goal is to be stinkin’ rich, and it looks like that might happen. They’re not going to let the fear of an hour of downtime undermine that.

Re: OpenAI Status: Multiple engines are down

#64

Post by TOMDM »

ronsor wrote: post #57
TOMDM wrote: post #24

Earlier quoted context omitted.

Don't get me wrong, I'd love to be able to run GPT3 and the subsequent finetuned versions thereof myself, but OpenAI has essentially no financial incentive to do so. A few years ago, we could maybe lean on their open aspirations to get that done, but with the "limited profitibility model" they've since instead adopted, I think that dream is mostly gone. At least we still get the occasional treat like Whisper out of t…

It's not as if anyone could afford to self-host the giant GPT-3 model anyway.

I'm rather sure there are a mid sized number of "anyones" who would buy the hardware required to run it given the opportunity.

But I certainly agree that not everyone can, and very few individuals.

The thing is a monster of a model.

Re: OpenAI Status: Multiple engines are down

#66

Post by danielscrubs »

jeffbee wrote: post #54
danielscrubs wrote: post #47

To put this in perspective: the outage was 52 min. The total number of employees OpenAI have are 375. The launch date was just some months ago. At Google, the Cloud SQL dashboard was unavailable for around 12 hours a couple of weeks ago if I read this correctly: https://status.cloud.google.com/incidents/xg2qrL1UuSJiPDZALJ... The total number of employees of Google are 156 500. Google Cloud was launched 2008. So when…

That's a completely bizarre comparison. No number of Android developers will lead to higher availability of Cloud SQL observability. The gross number of employees at an org is meaningless.

Sure. But many Android developers are likely to use Cloud SQL and give quick feedback. How big the benefits are is let as an exercise to the reader.

Re: OpenAI Status: Multiple engines are down

#67

Post by dwaltrip »

osigurdson wrote: post #39
TOMDM wrote: post #14

LLM's are incredibly expensive to run. I imagine the huge demand that ChatGPT is seeing would make any cloud vendor sweat if you were to suddenly lump it on top of the usual demand. To me it's entirely unsurprising that OpenAI would have trouble keeping up. Good luck to them.

This was my assumption but it would be great to see it quantified. Does anyone know the average cost per query?

I saw someone claim it’s about $0.01 per query.

Re: OpenAI Status: Multiple engines are down

#68

Post by DonHopkins »

cellis wrote: post #38
la64710 wrote: post #23

Earlier quoted context omitted.

ChatGPT might but The LLLM model may not require so many nodes specially if someone is running for their own dataset.

GPT NeoX (open source) requires at least 45GB of GPU ram. So no, you're not running it on your laptop.

[flagged]

Re: OpenAI Status: Multiple engines are down

#69

Post by xdavidliu »

jeffbee wrote: post #54
danielscrubs wrote: post #47

To put this in perspective: the outage was 52 min. The total number of employees OpenAI have are 375. The launch date was just some months ago. At Google, the Cloud SQL dashboard was unavailable for around 12 hours a couple of weeks ago if I read this correctly: https://status.cloud.google.com/incidents/xg2qrL1UuSJiPDZALJ... The total number of employees of Google are 156 500. Google Cloud was launched 2008. So when…

That's a completely bizarre comparison. No number of Android developers will lead to higher availability of Cloud SQL observability. The gross number of employees at an org is meaningless.

presumably the data scientists at OpenAI would also not be able to directly lead to higher availability of ChatGPT; it's just an order of magnitude comparison.

Re: OpenAI Status: Multiple engines are down

#70

Post by la64710 »

dontreact wrote: post #59
lmm wrote: post #43

Earlier quoted context omitted.

If they're that well funded, skilled engineers are more likely to be the bottleneck than compute availability.

Serving a request to one of their APIs requires orders of magnitude more compute than your typical web service

Really ? How do you know? Have they shared any credible data around it?

Based on experience of BERT , yes maybe to get the best experience or to serve millions of users you need to run any model on compute intensive infrastructure , BUT if you just want to run for yourself and do some small testing you can very well download it from huggingface and elsewhere and run it on your laptop.

Post reply on HN