Major outages across ChatGPT and API • Hacker News Board

Re: Major outages across ChatGPT and API

#271

Post by hexman » Wed, Nov 08, 2023, 6:08 PM UTC

michaelteter wrote: post #50

Lots of jokes to be made, but we are setting ourselves up for some big rippling negative effects by so quickly building a reliance on providers like OpenAI. It took years before most companies who now use cloud providers to trust and be willing to bet their operations on them. That gave the cloud providers time to make their systems more robust, and to learn how to resolve issues quickly.

Imagine if Apple's or Google's cloud went down and all your apps on iPhone and Android were broken and unavailable. Absolutely all apps on billions of phones.

Cloud =! OpenAI

Clouds store and process shareable information that multiple participants can access. Otherwise AI agents == new applications. OpenAI is the wrong evolution for the future of AI agents

Re: Major outages across ChatGPT and API

#272

Post by dragonwriter » Wed, Nov 08, 2023, 6:09 PM UTC

halfcat wrote: post #123

michaelteter wrote: post #50

Lots of jokes to be made, but we are setting ourselves up for some big rippling negative effects by so quickly building a reliance on providers like OpenAI. It took years before most companies who now use cloud providers to trust and be willing to bet their operations on them. That gave the cloud providers time to make their systems more robust, and to learn how to resolve issues quickly.

The app maker can screw the plug-in author at any moment. For general cloud, avoiding screwing might mean multi cloud. But for LLM, there’s only one option at the highest level of quality for now. People tend to over focus on resilience (minimizing probability of breaking) and neglect the plan for recovery when things do break. I can’t tell you how weirdly foreign this is to many people, how many meetings I’ve been i…

> For general cloud, avoiding screwing might mean multi cloud. But for LLM, there’s only one option at the highest level of quality for now.

There's always only one at the highest level of quality at a fine-grained enough resolution.

Whether there's only one at sufficient quality for use, and if it is possible to switch between them in realtime without problems caused by the switch (e.g., data locked up in the provider that is down) is the relevant question, and whether the cost of building the multi-provider switching capability is worth it given the cost vs. risk of outage. All those are complicated questions that are application specific, not ones that have an easy answer on a global, uniform basis.

Re: Major outages across ChatGPT and API

#273

Post by nonfamous » Wed, Nov 08, 2023, 6:09 PM UTC

sjnair96 wrote: post #154

nonfamous wrote: post #60

Earlier quoted context omitted.

It’s exactly the same models as OpenAI.

But with different, separate content filtering or moderation. I have deployed in prod and managed migration to Azure Openai form Openai, and had to go through content filter issues.

You can request to have content filtering turned off https://learn.microsoft.com/en-us/azure/ai-services/openai/c...

Re: Major outages across ChatGPT and API

#274

Post by kweingar » Wed, Nov 08, 2023, 6:10 PM UTC

ethanbond wrote: post #113

walthamstow wrote: post #13

Earlier quoted context omitted.

I was forced to Google something earlier. It's like when you discover 'craft' coffee/beer/baked goods/whatever and then go back and try the mass market stuff. How did I ever live like this?

Google got worse, try Kagi and you’ll realize the internet isn’t all complete garbage.

Kagi just serves Google results though. But it has some nifty extra features to be sure.

Re: Major outages across ChatGPT and API

#275

Post by rattlesnakedave » Wed, Nov 08, 2023, 6:16 PM UTC

kinonionioa wrote: post #253

nus07 wrote: post #16

All tech folks should just get a PTO today . #ChatGPT_outage Am I supposed to use Google and Stack overflow ? That’s like going back to roll down windows in a car :)

I don't get any time off. My full-time job is cleaning up after people who've been using ChatGPT.

Lucky you. Mine is using ChatGPT to review code written by curmudgeons that don't use it.

Re: Major outages across ChatGPT and API

#276

Post by Smaug123 » Wed, Nov 08, 2023, 6:16 PM UTC

sitzkrieg wrote: post #268

Smaug123 wrote: post #240

Earlier quoted context omitted.

Not really true - Git is distributed, after all. During an outage once I just hosted my copy of a certain Git repo somewhere. You can always push the history back up to the golden copy when GitHub comes back.

i am not talking about git, i am talking about github. lets say i need to merge a PR in GH because use gha pipelines or what have you to deploy a prod fix. this would become severely blocked. where as if openai goes down i can no longer use ai to generate a lame cover letter or whatever i was avoiding actually doing anyway, thats all

This is the realm of standard recovery planning though, isn't it? Like, your processes should be able to handle this, because it's routine: GitHub goes down at least once per month for long enough for them to declare an incident, per https://www.githubstatus.com/history . E.g. one should think carefully before depriving onself of the break-glass ability to do manually what those pipelines do automatically.

Re: Major outages across ChatGPT and API

#277

Post by lannisterstark » Wed, Nov 08, 2023, 6:18 PM UTC

lagniappe wrote: post #243

lgkk wrote: post #218

I'm imagining a bunch of Tech Priests doing rituals and chanting in front of the data center to encourage the Machine Spirit to restart.

Wololooooooo

No wololos in the Imperium, heretic!

Re: Major outages across ChatGPT and API

#278

Post by bertil » Wed, Nov 08, 2023, 6:20 PM UTC

People are learning a lot of important lessons today.

I’ve got friends who have started an incident management company. They are awesome. It feels crass to advertise for them now, but it also feels like the best time to do it.

Re: Major outages across ChatGPT and API

#279

Post by mplanchard » Wed, Nov 08, 2023, 6:27 PM UTC

zeven7 wrote: post #260

colinsane wrote: post #172

Earlier quoted context omitted.

the choice is to live 2 years behind (e.g. integrate the open source stuff and ride that wave of improvement). for businesses in a competitive space, that’s perhaps untenable. but for individuals and anywhere else where this stuff is just a “nice to have”, that’s really just the long-term sustainable approach. it reminds me of a choice like “do i host my website on a Windows Server, or a Linux box” at a time when bot…

> it reminds me of a choice like “do i host my website on a Windows Server, or a Linux box” at a time when both of these things are new. Oof, you reminded me of when I chose to use Flow and then TypeScript won.

Haha this puts me in mind of when I designed a whole deployment strategy for an org based on docker swarm, only to have k8s eat its lunch and swarm to wind up discontinued

Re: Major outages across ChatGPT and API

#280

Post by TheRealNGenius » Wed, Nov 08, 2023, 6:29 PM UTC

rattlesnakedave wrote: post #275

kinonionioa wrote: post #253

Earlier quoted context omitted.

I don't get any time off. My full-time job is cleaning up after people who've been using ChatGPT.

Lucky you. Mine is using ChatGPT to review code written by curmudgeons that don't use it.

[dead]