Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI Status: Multiple engines are down

status.openai.com

Post reply on HN
Print view View original

91–100 of 167 posts

Re: OpenAI Status: Multiple engines are down

#91

Post by verdverm »

danielscrubs wrote: post #66
jeffbee wrote: post #54

Earlier quoted context omitted.

That's a completely bizarre comparison. No number of Android developers will lead to higher availability of Cloud SQL observability. The gross number of employees at an org is meaningless.

Sure. But many Android developers are likely to use Cloud SQL and give quick feedback. How big the benefits are is let as an exercise to the reader.

They have automated monitoring that would alert SREs before a support ticket likely would. Do you really think someone working on an OS would be using Cloud SQL regularly to notice? Maybe a developer of an Android app is (1) using Cloud SQL (over firestore?) and (2) has monitoring that would alert them effectively and (3) determine the cause is Google and send a support ticket.

As an exercise, which is likely to reach the SRE first?

Re: OpenAI Status: Multiple engines are down

#92

Post by kerpotgh »

danielscrubs wrote: post #47

To put this in perspective: the outage was 52 min. The total number of employees OpenAI have are 375. The launch date was just some months ago. At Google, the Cloud SQL dashboard was unavailable for around 12 hours a couple of weeks ago if I read this correctly: https://status.cloud.google.com/incidents/xg2qrL1UuSJiPDZALJ... The total number of employees of Google are 156 500. Google Cloud was launched 2008. So when…

[dead]

Re: OpenAI Status: Multiple engines are down

#93

Post by verdverm »

xdavidliu wrote: post #69
jeffbee wrote: post #54

Earlier quoted context omitted.

That's a completely bizarre comparison. No number of Android developers will lead to higher availability of Cloud SQL observability. The gross number of employees at an org is meaningless.

presumably the data scientists at OpenAI would also not be able to directly lead to higher availability of ChatGPT; it's just an order of magnitude comparison.

Depends on the reason for the outage. If it is an application error, like OOM or other performance issue, then yes, they might be directly tied to the root cause and solution. If it's an infra issue, then maybe not so much.

I once spent an entire weekend swapping shifts trying to keep the main server of an AI company up because the devs deployed on friday and were unreachable to fix their memory error. The best thing that happened is the boss came in Monday and froze feature development for a sprint, inventing the "tech debt sprint" that became a quarterly activity. Everyone loved it, uptime became better, and even application development got easier.

Size of a company doesn't matter, the magnitude of product offerings aligns with head count. Team size is what matters and is likely a much closer ratio between the companies. Google does have head count and dedicated teams to solve common problems and build solutions at the platform layer. Likely OpenAPI is benefiting from Google's efforts here (from the open source and knowledge sharing which are bar none imho).

From an OpenAPI job opening:

> The Engineering team wraps a massive fleet of GPUs in scalable, robust, infrastructure powered by Kubernetes, Go, Python, Terraform, Kafka, Postgres, and Snowflake. Our APIs are powered by Python Flask and OpenAPI with a React frontend.

Re: OpenAI Status: Multiple engines are down

#95

Post by Waterluvian »

localhost wrote: post #88
la64710 wrote: post #70

Earlier quoted context omitted.

Really ? How do you know? Have they shared any credible data around it? Based on experience of BERT , yes maybe to get the best experience or to serve millions of users you need to run any model on compute intensive infrastructure , BUT if you just want to run for yourself and do some small testing you can very well download it from huggingface and elsewhere and run it on your laptop.

GPT3 is ~175B parameters. At float16 precision, that's 350GB of weights. BLOOM-176B is about the same size. Here's one person's experience; a token is ~0.75 words. "The Python code in this tutorial generates one token every 3 minutes on a computer with an i5 11gen processor, 16GB of RAM, and a Samsung 980 PRO NVME..." [1] https://towardsdatascience.com/run-bloom-the-largest-open-ac...

Goodness... And to think my brain was free!

Re: OpenAI Status: Multiple engines are down

#96

Post by alfor »

la64710 wrote: post #15
TOMDM wrote: post #14

LLM's are incredibly expensive to run. I imagine the huge demand that ChatGPT is seeing would make any cloud vendor sweat if you were to suddenly lump it on top of the usual demand. To me it's entirely unsurprising that OpenAI would have trouble keeping up. Good luck to them.

This is precisely why they should really open source their model so that anyone can download and run it on their own infrastructure. Just like google or others have done and one is free to run it on their own laptop (some even without a GPU) , on premise or on any cloud provider infrastructure. They can continue to provide a hosted service for their model but they should allow it to be downloaded just like BERT.

Don’t worry, we will see plenty of other GPT very soon. The genie is out of the bottle.

Re: OpenAI Status: Multiple engines are down

#97

Post by generalizations »

TOMDM wrote: post #81
scifibestfi wrote: post #55

Earlier quoted context omitted.

Isn't an open source ChatGPT inevitable? There's already open source AI art.

Check out the BLOOM models if you want to see a first stab at that. If you can find an economical way of running it though, let me know.

A cluster of 6 year old 24GB NVIDIA Teslas should do the trick...they run for about $100 apiece. Put 12 or so of them together and you have the VRAM for a GPT3 clone.

Re: OpenAI Status: Multiple engines are down

#98

Post by danielscrubs »

verdverm wrote: post #91
danielscrubs wrote: post #66

Earlier quoted context omitted.

Sure. But many Android developers are likely to use Cloud SQL and give quick feedback. How big the benefits are is let as an exercise to the reader.

They have automated monitoring that would alert SREs before a support ticket likely would. Do you really think someone working on an OS would be using Cloud SQL regularly to notice? Maybe a developer of an Android app is (1) using Cloud SQL (over firestore?) and (2) has monitoring that would alert them effectively and (3) determine the cause is Google and send a support ticket. As an exercise, which is likely to reac…

> They have automated monitoring that would alert SREs before a support ticket likely would.

Yes, but how likely is this unlikely situation?

It took 12 hours to debug and fix after all.

Re: OpenAI Status: Multiple engines are down

#99

Post by lossolo »

danielscrubs wrote: post #47

To put this in perspective: the outage was 52 min. The total number of employees OpenAI have are 375. The launch date was just some months ago. At Google, the Cloud SQL dashboard was unavailable for around 12 hours a couple of weeks ago if I read this correctly: https://status.cloud.google.com/incidents/xg2qrL1UuSJiPDZALJ... The total number of employees of Google are 156 500. Google Cloud was launched 2008. So when…

We don't know anything about these outages and their nature. Maybe OpenAI outage was caused by a typo somewhere in code and Google had rolled out system wide monitoring + SQL change that had cascading issues and required many not obvious steps and processing hundrerds of TBs of data.

I mean I had outages that were 60 seconds long with magnitude less people than OpenAI and serving more req/s than they do. What does it tell you? Nothing at all.

Re: OpenAI Status: Multiple engines are down

#100

Post by somenameforme »

t3rmi wrote: post #30
la64710 wrote: post #15

Earlier quoted context omitted.

This is precisely why they should really open source their model so that anyone can download and run it on their own infrastructure. Just like google or others have done and one is free to run it on their own laptop (some even without a GPU) , on premise or on any cloud provider infrastructure. They can continue to provide a hosted service for their model but they should allow it to be downloaded just like BERT.

Open AI's CEO Sam Altman's take is that they will only ever allow API access to their models to avoid misuse. I don't get HN's take with wanting everything open sourced. Some things are expensive to create and dangerous in the wrong hands. Not everything can and should be open sourced.

Can you think of any non-weapons examples where centralization/gatekeeping of a tech meaningfully and causally benefited society or a technology itself?

Actually, thinking about my own question I'm even inclined to remove the non-weapons qualifier. The most knee jerk response, nuclear weapons, is perhaps the best example of unexpected benefit. The 'decentralization' of nuclear weapons is undoubtedly why the Cold War was the Cold War, and not World War 3. And similarly why we haven't* seen an open war between nations with nuclear weapons. One power to rule over all suddenly turned into "war with this country no longer has a win scenario" effectively ending open warfare between nuclear nations.

There's also the inevitability/optics argument. There are already viable open source alternatives [1], and should this tech ultimately prove viable/useful that will only be the beginning. So there certainly will be "ai" that will be open, it just won't come from OpenAI(tm)(c).

[1] - https://github.com/THUDM/GLM-130B

Post reply on HN