Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI Status: Multiple engines are down

status.openai.com

Post reply on HN
Print view View original

111–120 of 167 posts

Re: OpenAI Status: Multiple engines are down

#111

Post by danielscrubs »

verdverm wrote: post #104
danielscrubs wrote: post #98

Earlier quoted context omitted.

> They have automated monitoring that would alert SREs before a support ticket likely would. Yes, but how likely is this unlikely situation? It took 12 hours to debug and fix after all.

It was the dashboard, applications depending on the database service were not impacted. You're comparing apples and oranges and ignoring both a much larger context and the nuances

And most of those customers are paying customers?

All I'm saying is that some outages are to be expected from a young company and they are doing excellent work.

Re: OpenAI Status: Multiple engines are down

#112

Post by ygouzerh »

152334H wrote: post #4

With all of these outages, you have to wonder if it's a lack of skilled engineers on their part, or if they simply don't have enough GPUs to keep the lights on all the time.

Knowing all the increase in term of load that they are having these two last months, 99.53 % of uptime on three months is actually quite impressive.

Re: OpenAI Status: Multiple engines are down

#113

Post by somenameforme »

TOMDM wrote: post #80
earthboundkid wrote: post #79

Earlier quoted context omitted.

I think from Apple’s POV, this is great news. Moore’s Law has been dead for years, and there has been no good reason to upgrade your devices until now. AI means it’s 1990 again, and you need to buy a new device every 18 months because the performance leap is so meaningful to the UX.

I'd agree if Apple were in the business of making datacentre infrastructure. Nobody is running these large scale models on their personal devices. Sure, some of the image generation tech is seeing personal use, so you'd have a point there, but these immense language models are something else entirely.

GLM-130B[1] (a 130 billion parameter model vs GPT-3's 175 billion parameter model) is able to run optimally on consumer level high-end hardware, 4xRTX 3090 in particular. That's It's a whole lot cheaper to run neural net style systems than to train them. "Somebody on Twitter"[2] got it setup, and broke down the costs, demonstrated some prompts, and what not. Cliff notes being a fraction of a penny per query, with each taking about 16s to generate. The output's pretty terrible, but it's unclear to me whether that's inherent or a result of priority. I expect OpenAI spent a lot of manpower on supervised training, whereas this system probably had minimal, especially in English (it's from a Chinese university).

[1] - https://github.com/THUDM/GLM-130B

[2] - https://twitter.com/alexjc/status/1617152800571416577

Re: OpenAI Status: Multiple engines are down

#114

Post by albntomat0 »

antonvs wrote: post #102
motoboi wrote: post #73

Earlier quoted context omitted.

Something like 1,5Tb memory to run this model in inference mode.

Is that really all? We regularly run multi TB memory clusters for big data processing and ML. I imagined it would be much bigger than that. To put that in perspective, 24x 64 GB nodes is 1.5 TB.

My understanding is that all the memory has to be GPU memory, with proper interconnects. Still not that crazy, all things considered

Re: OpenAI Status: Multiple engines are down

#116

Post by lossolo »

antonvs wrote: post #102
motoboi wrote: post #73

Earlier quoted context omitted.

Something like 1,5Tb memory to run this model in inference mode.

Is that really all? We regularly run multi TB memory clusters for big data processing and ML. I imagined it would be much bigger than that. To put that in perspective, 24x 64 GB nodes is 1.5 TB.

> 24x 64 GB nodes is 1.5 TB

Looking at your calculations indicates that you mean RAM but it's 1.5 TB GPU VRAM (but this is assuming they use 64 bit precision, which is likely wrong so it's ~750 GB), not RAM.

Re: OpenAI Status: Multiple engines are down

#117

Post by lossolo »

motoboi wrote: post #73
la64710 wrote: post #15

Earlier quoted context omitted.

This is precisely why they should really open source their model so that anyone can download and run it on their own infrastructure. Just like google or others have done and one is free to run it on their own laptop (some even without a GPU) , on premise or on any cloud provider infrastructure. They can continue to provide a hosted service for their model but they should allow it to be downloaded just like BERT.

Something like 1,5Tb memory to run this model in inference mode.

You meant 700 GB? 32 bit precision in 175B model is around ~700GB + overhead or around 350 GB if they use half-precision.

Re: OpenAI Status: Multiple engines are down

#118

Post by albntomat0 »

somenameforme wrote: post #100
t3rmi wrote: post #30

Earlier quoted context omitted.

Open AI's CEO Sam Altman's take is that they will only ever allow API access to their models to avoid misuse. I don't get HN's take with wanting everything open sourced. Some things are expensive to create and dangerous in the wrong hands. Not everything can and should be open sourced.

Can you think of any non-weapons examples where centralization/gatekeeping of a tech meaningfully and causally benefited society or a technology itself? Actually, thinking about my own question I'm even inclined to remove the non-weapons qualifier. The most knee jerk response, nuclear weapons, is perhaps the best example of unexpected benefit. The 'decentralization' of nuclear weapons is undoubtedly why the Cold War…

I agree with your view that nuclear weapons on both sides prevent war. However, they’ve only ever been developed by a small number of capable and motivated nations, with considerable resources involved. The later ones (North Korea, Pakistan) developed them while other nations tried to prevent them from doing so.

If ML models continue their exponential growth in size, a similar outcome is possible.

Re: OpenAI Status: Multiple engines are down

#120

Post by visarga »

aliqot wrote: post #89
Gigachad wrote: post #85

Earlier quoted context omitted.

The more realistic scenario is they charge money to use it. A few cents per query or so and you'll cut out almost all the traffic while still keeping it available to anyone making good use of it.

I'd pay per query, but if it is per-token it's going to have to be free for what the machine says. It often repeats itself, or repeats certain phrases, or will restate an assertion it already agreed was incorrect.

By the way, one of the most popular tasks is information extraction, for example reading an invoice. This is fancy copying.
Post reply on HN