OpenAI Status: Multiple engines are down • Hacker News Board

Re: OpenAI Status: Multiple engines are down

#101

Post by TOMDM » Thu, Jan 26, 2023, 4:17 AM UTC

generalizations wrote: post #97

TOMDM wrote: post #81

Earlier quoted context omitted.

Check out the BLOOM models if you want to see a first stab at that. If you can find an economical way of running it though, let me know.

A cluster of 6 year old 24GB NVIDIA Teslas should do the trick...they run for about $100 apiece. Put 12 or so of them together and you have the VRAM for a GPT3 clone.

Huh. Tempting.

Amazon has them listed at $200, but still, that's only $2,400 for 12 of them.

Still, adds up once you get the hardware you'd need to NVlink 12 of them, and then on top of that, the price of power/perf you get probably isn't great compared to modern compute.

Wonder what your volume would have to be before getting a box with 8 A100's from Lambdalabs would be the better tradeoff.

Re: OpenAI Status: Multiple engines are down

#102

Post by antonvs » Thu, Jan 26, 2023, 4:18 AM UTC

motoboi wrote: post #73

la64710 wrote: post #15

Earlier quoted context omitted.

This is precisely why they should really open source their model so that anyone can download and run it on their own infrastructure. Just like google or others have done and one is free to run it on their own laptop (some even without a GPU) , on premise or on any cloud provider infrastructure. They can continue to provide a hosted service for their model but they should allow it to be downloaded just like BERT.

Something like 1,5Tb memory to run this model in inference mode.

Is that really all? We regularly run multi TB memory clusters for big data processing and ML. I imagined it would be much bigger than that.

To put that in perspective, 24x 64 GB nodes is 1.5 TB.

Re: OpenAI Status: Multiple engines are down

#103

Post by htzhcfzubhfjk » Thu, Jan 26, 2023, 4:21 AM UTC

Google is dead. I already almost stopped using their crappy search engine and use chatgpt for most of my work.

Re: OpenAI Status: Multiple engines are down

#104

Post by verdverm » Thu, Jan 26, 2023, 4:22 AM UTC

danielscrubs wrote: post #98

verdverm wrote: post #91

Earlier quoted context omitted.

They have automated monitoring that would alert SREs before a support ticket likely would. Do you really think someone working on an OS would be using Cloud SQL regularly to notice? Maybe a developer of an Android app is (1) using Cloud SQL (over firestore?) and (2) has monitoring that would alert them effectively and (3) determine the cause is Google and send a support ticket. As an exercise, which is likely to reac…

> They have automated monitoring that would alert SREs before a support ticket likely would. Yes, but how likely is this unlikely situation? It took 12 hours to debug and fix after all.

It was the dashboard, applications depending on the database service were not impacted. You're comparing apples and oranges and ignoring both a much larger context and the nuances

Re: OpenAI Status: Multiple engines are down

#105

Post by lossolo » Thu, Jan 26, 2023, 4:26 AM UTC

generalizations wrote: post #97

TOMDM wrote: post #81

Earlier quoted context omitted.

Check out the BLOOM models if you want to see a first stab at that. If you can find an economical way of running it though, let me know.

A cluster of 6 year old 24GB NVIDIA Teslas should do the trick...they run for about $100 apiece. Put 12 or so of them together and you have the VRAM for a GPT3 clone.

If you have time to wait for results then sure, it could work in theory but in practice they are so slow and power inefficient (compared to newer nodes) that no one uses them for LLMs, that's why they cost ~200$ used on ebay.

Re: OpenAI Status: Multiple engines are down

#106

Post by brookst » Thu, Jan 26, 2023, 4:26 AM UTC

lossolo wrote: post #99

danielscrubs wrote: post #47

To put this in perspective: the outage was 52 min. The total number of employees OpenAI have are 375. The launch date was just some months ago. At Google, the Cloud SQL dashboard was unavailable for around 12 hours a couple of weeks ago if I read this correctly: https://status.cloud.google.com/incidents/xg2qrL1UuSJiPDZALJ... The total number of employees of Google are 156 500. Google Cloud was launched 2008. So when…

We don't know anything about these outages and their nature. Maybe OpenAI outage was caused by a typo somewhere in code and Google had rolled out system wide monitoring + SQL change that had cascading issues and required many not obvious steps and processing hundrerds of TBs of data. I mean I had outages that were 60 seconds long with magnitude less people than OpenAI and serving more req/s than they do. What does it…

Internal people should care about why, but customers/users really don’t. We expect openai / Google to take steps to balance likelihood of outage with impact of outage, and plan accordingly.

So what this tells us, based on very limited data, is that this level out outage happens to even the biggest of companies.

Re: OpenAI Status: Multiple engines are down

#107

Post by riku_iki » Thu, Jan 26, 2023, 4:28 AM UTC

danielscrubs wrote: post #47

To put this in perspective: the outage was 52 min. The total number of employees OpenAI have are 375. The launch date was just some months ago. At Google, the Cloud SQL dashboard was unavailable for around 12 hours a couple of weeks ago if I read this correctly: https://status.cloud.google.com/incidents/xg2qrL1UuSJiPDZALJ... The total number of employees of Google are 156 500. Google Cloud was launched 2008. So when…

> The total number of employees OpenAI have are 375

they outsource infra to MS?

Re: OpenAI Status: Multiple engines are down

#108

Post by riku_iki » Thu, Jan 26, 2023, 4:30 AM UTC

danielscrubs wrote: post #66

jeffbee wrote: post #54

Earlier quoted context omitted.

That's a completely bizarre comparison. No number of Android developers will lead to higher availability of Cloud SQL observability. The gross number of employees at an org is meaningless.

Sure. But many Android developers are likely to use Cloud SQL and give quick feedback. How big the benefits are is let as an exercise to the reader.

> Android developers are likely to use Cloud SQL

"likely" is your hypothesis. They are two different orgs, and internal services can use different infra for analysis.

Re: OpenAI Status: Multiple engines are down

#109

Post by brookst » Thu, Jan 26, 2023, 4:33 AM UTC

Gigachad wrote: post #86

CatWChainsaw wrote: post #46

Earlier quoted context omitted.

I wonder when authoritarian censors will weaponize LLMs.

Seems like the more likely option. They could be used to live scan every post to work out what it's about and it's sentiment. Similar to how ChatGPT can work out if you are asking for something it won't answer, they could be used to work out if you are saying something not allowed.

Why stop there, when many of them could transform it into something that is allowed in-flight?

Re: OpenAI Status: Multiple engines are down

#110

Post by lossolo » Thu, Jan 26, 2023, 4:35 AM UTC

brookst wrote: post #106

lossolo wrote: post #99

Earlier quoted context omitted.

We don't know anything about these outages and their nature. Maybe OpenAI outage was caused by a typo somewhere in code and Google had rolled out system wide monitoring + SQL change that had cascading issues and required many not obvious steps and processing hundrerds of TBs of data. I mean I had outages that were 60 seconds long with magnitude less people than OpenAI and serving more req/s than they do. What does it…

Internal people should care about why , but customers/users really don’t. We expect openai / Google to take steps to balance likelihood of outage with impact of outage, and plan accordingly. So what this tells us, based on very limited data, is that this level out outage happens to even the biggest of companies.

OpenAI whole infra is magnitudes less complex compared to Google cloud and their infra, OpenAI is not creating cloud for thousands/millions of customers with different use cases, they use Azure to host their software, that's why I find that comparison irrelevant because it's comparing apples to oranges.

> plan accordingly

You can't plan for every major issue unless you solved the halting problem and unless you as customer want to pay magnitudes more for services you use. For every major client facing issue you probably have 10s/100s/1000s (depending on the system complexity) incidents that were prevented and you know nothing about.