Major outages across ChatGPT and API • Hacker News Board

Re: Major outages across ChatGPT and API

#241

Post by tompetry » Wed, Nov 08, 2023, 5:38 PM UTC

lopatin wrote: post #230

tombert wrote: post #214

I used Google Bard for the first time today specifically because ChatGPT was down. It was honestly perfectly fine, but it has a slightly different tone than ChatGPT that's kind of hard to explain.

I used it for the first time today too, for the same reason. It was slower and much worse at coding. I was just asking it for SQL aggregation queries and it just ignored some of my requirements for the query.

I have seen noticably worse results with Bard, especially with long prompts. Claude (by Anthropic) has been my backup to ChatGPT.

Re: Major outages across ChatGPT and API

#242

Post by yCombLinks » Wed, Nov 08, 2023, 5:41 PM UTC

gumballindie wrote: post #155

nus07 wrote: post #16

All tech folks should just get a PTO today . #ChatGPT_outage Am I supposed to use Google and Stack overflow ? That’s like going back to roll down windows in a car :)

Typicaly those using chatgpt are junior devs, not much lost if they can't access it.

Dumbest take I've seen yet

Re: Major outages across ChatGPT and API

#243

Post by lagniappe » Wed, Nov 08, 2023, 5:44 PM UTC

lgkk wrote: post #218

I'm imagining a bunch of Tech Priests doing rituals and chanting in front of the data center to encourage the Machine Spirit to restart.

Wololooooooo

Re: Major outages across ChatGPT and API

#244

Post by tombert » Wed, Nov 08, 2023, 5:44 PM UTC

lopatin wrote: post #230

tombert wrote: post #214

I used Google Bard for the first time today specifically because ChatGPT was down. It was honestly perfectly fine, but it has a slightly different tone than ChatGPT that's kind of hard to explain.

I used it for the first time today too, for the same reason. It was slower and much worse at coding. I was just asking it for SQL aggregation queries and it just ignored some of my requirements for the query.

In my case, I was just asking it for a cheeky name for a talk I want to give in a few months. The suggestions it gave were of comparable quality to what I think ChatGPT would have given me.

Re: Major outages across ChatGPT and API

#245

Post by pixl97 » Wed, Nov 08, 2023, 5:46 PM UTC

rodnim wrote: post #174

edgyquant wrote: post #130

Earlier quoted context omitted.

Most services have a lot more systems than OpenAI and thus it is degraded performance when a few of them don’t work. Degraded performance isn’t a good thing, I don’t understand the issue with this verbiage.

Having multiple systems down sure sounds like a major outage to me. Not a complete outage, sure, but still a major one.

Black - Entire datacenter is down

Red - Entire services are down

Orange - Partial outage of services, some functionality completely down

Yellow - Functionality performance degraded and may timeout/fail, but may also work

Greed - Situation normal

Re: Major outages across ChatGPT and API

#246

Post by resters » Wed, Nov 08, 2023, 5:46 PM UTC

Aeolos wrote: post #225

xeckr wrote: post #134

Earlier quoted context omitted.

>distributed training networks Now that's an idea. One bottleneck might be a limit on just how much you can parallelize training, though.

There's a ton of work in this area, and the reality is... it doesn't work for LLMs. Moving from 900GB/sec GPU memory bandwidth with infiniband interconnects between nodes to 0.01-0.1GB/sec over the internet is brutal (1000x to 10000x slower...) This works for simple image classifiers, but I've never seen anything like a large language model be trained in a meaningful amount of time this way.

Maybe there is a way to train a neural network in a distributed way by training subsets of it and then connecting the aggregated weight changes to adjacent network segments. It wouldn't recover 1000x interconnect slowdowns, but might still be useful depending on the topology of the network.

Re: Major outages across ChatGPT and API

#247

Post by Zpalmtree » Wed, Nov 08, 2023, 5:46 PM UTC

msp26 wrote: post #26

Might as well have a quick discussion here. How's everyone finding the new models? 4-Turbo is a bit worse than 4 for my NLP work. But it's so much cheaper that I'll probably move every pipeline to using that. Depending on the exact problem it can even be comparable in quality/price to 3.5-turbo. However the fact that output tokens are limited to 4096 is a big asterisk on the 128k context.

4-Turbo is much faster, which for my use case is very important. Wish we could get more than 100 requests per day.. Is the limit higher when you have a higher usage tier?

Re: Major outages across ChatGPT and API

#248

Post by BiteCode_dev » Wed, Nov 08, 2023, 5:47 PM UTC

colinsane wrote: post #172

benterix wrote: post #63

Earlier quoted context omitted.

The point is, OpenAI spent a lot of money on training on all these copyrighted materials ordinary individuals/companies don't have access to, so replicating their effort would mean that you either 1) spend a ridiculous amount of money, 2) use Library Genesis (and still pay millions for GPU usage). So we have very little choice now. Open Source LLMs might be getting close to ChatGPT3 (opinions vary), but OpenAI is sti…

the choice is to live 2 years behind (e.g. integrate the open source stuff and ride that wave of improvement). for businesses in a competitive space, that’s perhaps untenable. but for individuals and anywhere else where this stuff is just a “nice to have”, that’s really just the long-term sustainable approach. it reminds me of a choice like “do i host my website on a Windows Server, or a Linux box” at a time when bot…

2 years behind in terms of timeline, but what factor in terms of productivity and quality of life?

Not to mention openai's lead compounds, so 2 years now and 4 years in 2025 may be 10 times the original prod/qol gain.

Re: Major outages across ChatGPT and API

#249

Post by zarzavat » Wed, Nov 08, 2023, 5:48 PM UTC

benterix wrote: post #63

michaelteter wrote: post #50

Lots of jokes to be made, but we are setting ourselves up for some big rippling negative effects by so quickly building a reliance on providers like OpenAI. It took years before most companies who now use cloud providers to trust and be willing to bet their operations on them. That gave the cloud providers time to make their systems more robust, and to learn how to resolve issues quickly.

The point is, OpenAI spent a lot of money on training on all these copyrighted materials ordinary individuals/companies don't have access to, so replicating their effort would mean that you either 1) spend a ridiculous amount of money, 2) use Library Genesis (and still pay millions for GPU usage). So we have very little choice now. Open Source LLMs might be getting close to ChatGPT3 (opinions vary), but OpenAI is sti…

OpenAI is obviously using libgen. Libgen is necessary but not sufficient for a top AI model. I believe that Google's corporate reluctance to use it is what's holding them back.

Re: Major outages across ChatGPT and API

#250

Post by CamperBob2 » Wed, Nov 08, 2023, 5:48 PM UTC

yCombLinks wrote: post #242

gumballindie wrote: post #155

Earlier quoted context omitted.

Typicaly those using chatgpt are junior devs, not much lost if they can't access it.

Dumbest take I've seen yet

It is certainly a dumb take, but there's a hidden insight buried in there: now anyone can be a "junior dev" at anything. The ability to empower every user, not just the experts, is a big part of the appeal of LLM-based technology.

Can't sell that aspect short; the OpenAI tools have enabled me to do things and understand things that would otherwise have had a much longer learning curve.