Live data from Hacker News

OpenAI: Increased errors across API and ChatGPT

status.openai.com

41–50 of 67 posts

Re: OpenAI: Increased errors across API and ChatGPT

#42

> OK I have a table in postgresql and I am adding a trigger such that when an insert happens on that table, an insert happens on another table. The second table has a constraint. What happens to the first insert if the second insert violates the constraint? How can I get help with this now? Google result 1: https://stackoverflow.com/questions/77148711/create-a-trigge... Google result 2: https://dba.stackexchange.com/…

Well, were it possible, I'd say go back in time and study your tools so that you're not spending the journeyman period of your career ricocheting between tutorials and faqs. Failing that, read the documentation. Failing that, stand up a quick experiment. Somehow, we survived before ChatGPT and even before saturated question boards. Those strategies are still available to you and well worth learning

We also survived before the internet and indoor plumbing and fire, and yet life is so much better now.

Re: OpenAI: Increased errors across API and ChatGPT

#43

Earlier quoted context omitted.

This conspiracy always comes up - don't you think that they test the output of the model revisions on probably 1000s of downstream tasks at this point? Bad responses are hard to reason about, could be prompting, could be a model revision, could just be bad luck.

Or maybe they are just AB testing and aggressively optimizing the response generation? LLMs are known to be compute/energy hungry to execute. It is a developing technology, if not downright experimental. Therefore, this explanation is very likely. I cannot see the reason to call this a conspiracy.

AB testing on what? AB tests need to produce some results which are then compared. How would releasing different versions in production help with that?

It would make more sense if that was internal and the responses were then graded.

A failed canary release would be more likely, where they released this version to a small amount of people not realising it was bad

Re: OpenAI: Increased errors across API and ChatGPT

#45

> OK I have a table in postgresql and I am adding a trigger such that when an insert happens on that table, an insert happens on another table. The second table has a constraint. What happens to the first insert if the second insert violates the constraint? How can I get help with this now? Google result 1: https://stackoverflow.com/questions/77148711/create-a-trigge... Google result 2: https://dba.stackexchange.com/…

Well, were it possible, I'd say go back in time and study your tools so that you're not spending the journeyman period of your career ricocheting between tutorials and faqs. Failing that, read the documentation. Failing that, stand up a quick experiment. Somehow, we survived before ChatGPT and even before saturated question boards. Those strategies are still available to you and well worth learning

Yeah, not all of us have memories that work like that. I’ve studied my tools but often forget the little details. My productivity has increased since GPT has come out.

Re: OpenAI: Increased errors across API and ChatGPT

#46

I think we need a new type of status page or at least a public version number on llms, yesterday for me GPT4 started giving nonsense super generic answers, like it was hardly reading what I wrote, and today it is back to top notch performance. I think they were trying to make the model more efficient or something but I just saw a massive decrease in the quality of output. From my side though, there is no version numb…

This conspiracy always comes up - don't you think that they test the output of the model revisions on probably 1000s of downstream tasks at this point? Bad responses are hard to reason about, could be prompting, could be a model revision, could just be bad luck.

Calling that a conspiracy is like saying its a conspiracy theory that Meta shows different people different Ads. I'd be more concerned if OpenAI WASN'T constantly trying to tune their models. Its literaly their job to tune the models.

Re: OpenAI: Increased errors across API and ChatGPT

#47

Earlier quoted context omitted.

Or maybe they are just AB testing and aggressively optimizing the response generation? LLMs are known to be compute/energy hungry to execute. It is a developing technology, if not downright experimental. Therefore, this explanation is very likely. I cannot see the reason to call this a conspiracy.

AB testing on what? AB tests need to produce some results which are then compared. How would releasing different versions in production help with that? It would make more sense if that was internal and the responses were then graded. A failed canary release would be more likely, where they released this version to a small amount of people not realising it was bad

There are the up down thumbs and automatic sentiment analysis as a test.

Re: OpenAI: Increased errors across API and ChatGPT

#48

Earlier quoted context omitted.

Well, were it possible, I'd say go back in time and study your tools so that you're not spending the journeyman period of your career ricocheting between tutorials and faqs. Failing that, read the documentation. Failing that, stand up a quick experiment. Somehow, we survived before ChatGPT and even before saturated question boards. Those strategies are still available to you and well worth learning

The "good old days weren't always good". I'm tired of either limiting myself to the information I have on the top of my head, the LLMs are really helping allow me to be creative and stretch out to do things that are just beyond my bread and butter, or things that I do infrequently.

Exactly this. I -could- become an expert in the intricacies of every tool I touch, or I could use chat gpt and move on to solving the next problem.

Re: OpenAI: Increased errors across API and ChatGPT

#50

Earlier quoted context omitted.

Well, were it possible, I'd say go back in time and study your tools so that you're not spending the journeyman period of your career ricocheting between tutorials and faqs. Failing that, read the documentation. Failing that, stand up a quick experiment. Somehow, we survived before ChatGPT and even before saturated question boards. Those strategies are still available to you and well worth learning

Yeah, not all of us have memories that work like that. I’ve studied my tools but often forget the little details. My productivity has increased since GPT has come out.

Stuff changes too. There are things that are worth learning and being fluent in. Regex, sql. But even then there are always edge cases or weirdness that someone has solved before. LLMs are just much better for this than wading through forum posts.
Post reply on HN