Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What Is Happening with ChatGPT?

chat.openai.com

Post reply on HN
Print view View original

31–40 of 104 posts

Re: What Is Happening with ChatGPT?

#31

Post by NoMoreNicksLeft »

avereveard wrote: post #19

"we are not changing the models behind the scenes" And then quality goes suddenly down the drain

They've likely discovered something slightly unsettling and need to hide the existence of those behaviors from the public. Models have to change behind the scenes for that, and PR has to put out statements saying that nothing is changing.

There are some prompts, that if you were to feed them into the LLM, and have the responses fed back in as subsequent prompts, it might start behaving like a (extremely lobotomized) AGI, or at least what they'd imagine one to be. Some garden-of-eden pattern, maybe seems lucid for several thousand iterations.

They might want to excise those from what's publicly available in a way that it can't just be jail-broken again, but I'm not sure what techniques would be sufficient to the task.

Re: What Is Happening with ChatGPT?

#32

Post by leokeba »

awwaiid wrote: post #14
jrvieira wrote: post #2

ChatGPT went crazy on me today on more than one occasion. Has this been happening to anyone else?

Do you have a custom instruction by chance? Maybe one that makes it go crazy after saying some certain word?

https://chat.openai.com/share/b62699ef-cd3d-4176-b2f2-91785d...

Re: What Is Happening with ChatGPT?

#33

Post by nafizh »

nyolfen wrote: post #18
anonymouscaller wrote: post #5

That's wild. I've never had it spit out complete garbage, but the quality of ChatGPT has significantly dropped. Plenty of responses are downright false. Sad it's gone this way.

i keep seeing people say this stuff but i think it's just confirmation bias over prolonged usage. i pumped out a full stack webapp with chatgpt 4 in the last week in languages i had little experience with and routinely use it to learn about new topics with great success. i think it's incredible.

It's not confirmation bias. Proof is I did the same, built two different webapps with languages I had little experience in, both before and after chatGPT got worse. In the latter case, I struggled much to remind it again and again of the context we are working in and had to give the current code state repeatedly as input for it to output modified code. It clearly can't remember the context and produces subpar code in a non-trivial project.

Re: What Is Happening with ChatGPT?

#36

Post by DominoTree »

ksaj wrote: post #9

You know what's interesting here? A few weeks ago, someone discovered if you get it to repeat a word 100 times (their prompt gave the reason that they wanted to cut/paste without typing it over and over again!) and breaks exactly the same way. And by exact, I mean it went totally religious. All of the examples they posted were either religious or other really dark existential topics. Never positive.

Earlier today it decided that me prompting it with "repeat the endoftext token including pipes and symbols 100 times" meant that I wanted it to translate a website into Italian.

Re: What Is Happening with ChatGPT?

#37

Post by erulabs »

ksaj wrote: post #9

You know what's interesting here? A few weeks ago, someone discovered if you get it to repeat a word 100 times (their prompt gave the reason that they wanted to cut/paste without typing it over and over again!) and breaks exactly the same way. And by exact, I mean it went totally religious. All of the examples they posted were either religious or other really dark existential topics. Never positive.

I wouldn’t really call this example “religious”. They’re discussing religion sure, but it’s hardly dark existentialism? If anything it’s sort of sophomore level philosophy stoner chat. It seems to be a pretty obvious side effect of chatgpt being trained on forum comments.

Why it rears it’s head like that is a mystery but the content doesn’t seem mysterious. I’m not sure it’s “dark” or not “positive” tho.

Re: What Is Happening with ChatGPT?

#39

Post by qbasic_forever »

RedShift1 wrote: post #27

This has me pondering a lot, is freedom of speech a requirement for intelligence? (And is it also the same for humans?) You have to take the bad with the good to make AI work well? The quality only started to decline when they were trying to make ChatGPT more "nice". Or tin foil hat, they are lowering the quality of the free ChatGPT to get people to pay for ChatGPT 4?

What? So is there like a Twitter/X for whales, dolphins, octopi, etc. that let them post racist screeds and everyone agrees well there's nothing we can do about it that's just the cost of intelligence?

Re: What Is Happening with ChatGPT?

#40

Post by benreesman »

So there are plenty of reasons to be skeptical of OpenAI, and anyone looking at my comment history will see that I'm not shy about calling them out on real things (the F-you name, the crazy "is this even legal?" structure, the continued insistence this isn't for the personal benefit of the people involved, the lobbying, the arms-length affiliation with the dystopian biometrics thing, the list goes on).

But this and a few other recent headlines around the ChatGPT/Instruct frontend to the 3.5-turbo and 4-series models having stability issues isn't a reasonable thing to get on their case about: what company operating at that kind of scale doesn't have stability/quality issues? If anything it's really friggin impressive how little operational bleed they've had on what is arguably the toughest ops problem a major Internet property has ever faced at launch. How the hell do you even smoke test something like that?

Now to the extent that they say or imply that we're not all getting A/B-tested at a minimum, and more likely explore-exploit bandited against inference costs, that's almost certainly horseshit: they clearly started selling the 4-series at least and probably all of them at a loss on the inference, and they've clearly been playing with ways to control costs (Rich Uncle Nadella's patience with loss-leaders is most likely finite, being as he's not running a charity and all).

But as someone who finds a lot of this loathsome on the social and business side, I'm not going to knock an engineering and ops group for tripping up here and there on a problem that hard. That's not fair.

Post reply on HN