Earlier quoted context omitted.
No that was after I got downvoted.
Maybe if you post it a few more times your luck will change
Once all the GPTChat hn karma farming bots come back online.
OpenAI Status: Multiple engines are down
71–80 of 167 posts
Re: OpenAI Status: Multiple engines are down
#72Sending major hugs to their team, always hard to keep the lights on when I am sure that they are breaking scale records possibly every day.
This is precisely why they should really open source their model so that anyone can download and run it on their own infrastructure. Just like google or others have done and one is free to run it on their own laptop (some even without a GPU) , on premise or on any cloud provider infrastructure. They can continue to provide a hosted service for their model but they should allow it to be downloaded just like BERT.
Why does using the term "open source" as a verb always sound like fingernails scratching on a chalkboard to me?
Re: OpenAI Status: Multiple engines are down
#73LLM's are incredibly expensive to run. I imagine the huge demand that ChatGPT is seeing would make any cloud vendor sweat if you were to suddenly lump it on top of the usual demand. To me it's entirely unsurprising that OpenAI would have trouble keeping up. Good luck to them.
This is precisely why they should really open source their model so that anyone can download and run it on their own infrastructure. Just like google or others have done and one is free to run it on their own laptop (some even without a GPU) , on premise or on any cloud provider infrastructure. They can continue to provide a hosted service for their model but they should allow it to be downloaded just like BERT.
Something like 1,5Tb memory to run this model in inference mode.
Re: OpenAI Status: Multiple engines are down
#74Earlier quoted context omitted.
Serving a request to one of their APIs requires orders of magnitude more compute than your typical web service
Really ? How do you know? Have they shared any credible data around it? Based on experience of BERT , yes maybe to get the best experience or to serve millions of users you need to run any model on compute intensive infrastructure , BUT if you just want to run for yourself and do some small testing you can very well download it from huggingface and elsewhere and run it on your laptop.
I've tried a few flavors of GPT on my laptop and a single request used orders of magnitude more CPU and RAM than running a "SELECT * FROM WHEREVER" SQL query
Re: OpenAI Status: Multiple engines are down
#75With all of these outages, you have to wonder if it's a lack of skilled engineers on their part, or if they simply don't have enough GPUs to keep the lights on all the time.
It's a reflection of demand, I imagine ChatGPT is still growing with the media attention + word of mouth. I've referred at least 30 friends to try it out, and many are still using it.
ChatGPT is currently the 4th most read page on English Wikipedia right now, earlier I noticed it was 2nd. This is the first time I’ve noticed it on their ranking list.
Re: OpenAI Status: Multiple engines are down
#76Gpt3 will fit on iPhone locally in 3 generations
Re: OpenAI Status: Multiple engines are down
#77To put this in perspective: the outage was 52 min. The total number of employees OpenAI have are 375. The launch date was just some months ago. At Google, the Cloud SQL dashboard was unavailable for around 12 hours a couple of weeks ago if I read this correctly: https://status.cloud.google.com/incidents/xg2qrL1UuSJiPDZALJ... The total number of employees of Google are 156 500. Google Cloud was launched 2008. So when…
How Google, and most likely all businesses, should think about outages is more complex than being up or down. Is it just part of the product, like Slack threads yesterday? Is it 1/2 or 1/1M requests or operations? The Google SRE book is an excellent work: https://sre.google/sre-book/introduction/ Also some perspective, if you look at the OpenAPI outages for Jan 2023, the up time is worse than Google Cloud SQL. https:…
Agreed, Google's public incident reports could be improved with those numbers.
Some chapters in the book include "Being On-Call", "Effective Troubleshooting" & "Postmortem Culture: Learning from Failure".
Which signals to me, a normal work culture and time will improve stability in most companies.
Re: OpenAI Status: Multiple engines are down
#78Earlier quoted context omitted.
Serving a request to one of their APIs requires orders of magnitude more compute than your typical web service
Really ? How do you know? Have they shared any credible data around it? Based on experience of BERT , yes maybe to get the best experience or to serve millions of users you need to run any model on compute intensive infrastructure , BUT if you just want to run for yourself and do some small testing you can very well download it from huggingface and elsewhere and run it on your laptop.
Pretty much any laptop would take hours to get anything significant done with GPT3 as the model would need to be batched in and out of memory from disk.
The amount of memory required to run these models is immense.
If you want a comparison, try running the largest version of the open source BLOOM model yourself.
Re: OpenAI Status: Multiple engines are down
#79LLM's are incredibly expensive to run. I imagine the huge demand that ChatGPT is seeing would make any cloud vendor sweat if you were to suddenly lump it on top of the usual demand. To me it's entirely unsurprising that OpenAI would have trouble keeping up. Good luck to them.
I think from Apple’s POV, this is great news. Moore’s Law has been dead for years, and there has been no good reason to upgrade your devices until now. AI means it’s 1990 again, and you need to buy a new device every 18 months because the performance leap is so meaningful to the UX.
Re: OpenAI Status: Multiple engines are down
#80LLM's are incredibly expensive to run. I imagine the huge demand that ChatGPT is seeing would make any cloud vendor sweat if you were to suddenly lump it on top of the usual demand. To me it's entirely unsurprising that OpenAI would have trouble keeping up. Good luck to them.
I think from Apple’s POV, this is great news. Moore’s Law has been dead for years, and there has been no good reason to upgrade your devices until now. AI means it’s 1990 again, and you need to buy a new device every 18 months because the performance leap is so meaningful to the UX.
I'd agree if Apple were in the business of making datacentre infrastructure.
Nobody is running these large scale models on their personal devices.
Sure, some of the image generation tech is seeing personal use, so you'd have a point there, but these immense language models are something else entirely.