Someone shared this chat with me and it seems like GPT3 has a complete breakdown halfway through?
[deleted]
What happened in this GPT-3 conversation?
11–20 of 306 posts
Re: What happened in this GPT-3 conversation?
#12I think openAI might be experimenting with smaller context lengths to save on costs or something since I've had a few other things break down like this for me today too (even in GPT-4)
That’s what I thought too. I like using local models and those with short contexts will definitely go off into cuckooland if you start scrolling off the end of the buffer.
Re: What happened in this GPT-3 conversation?
#13Speculation: a very low probability token was sampled, shifting the context to unfamiliar area that increased the probability of further unlikely tokens. This can be seen by multiple strange punctuation and capitalisation that don't obey English grammar.
What's interesting is that asking if it's chatgpt fixed the conversation for a bit. This suggests that chatgpt is nearly (or is) hard coded to respond with "No, I am an AI developed by OpenAI known as GPT-3. How may I assist you?".
Another interesting point is the grammar is fixed in its responses after first breaking, suggesting that chatgpt handles previous responses very differently than the current response it's writing. This likely results from instruction tuning for dialog applications, I suspect gpt3 (not chat version) would have not recovered from broken grammar.
Re: What happened in this GPT-3 conversation?
#14Someone shared this chat with me and it seems like GPT3 has a complete breakdown halfway through?
and it recovers too. i wonder if some internals got in a bad state and then it was reset.
Re: What happened in this GPT-3 conversation?
#15This is an interestingly surreal 'error' mode. So surreal its suspicious. It strikes me that there could be keys to how OpenAI is training ChatGPT in failure modes like this.
Re: What happened in this GPT-3 conversation?
#16Devastating effects of seed oils
Re: What happened in this GPT-3 conversation?
#17looks like the temperature got messed up somehow?
Re: What happened in this GPT-3 conversation?
#18This chat is how GPT-3.5 and GPT-4 behave if you crank the temperature up.
Agreed with this. You can create conversations like this quite easily using their API endpoints with the temperature parameter set even just a little bit above the default.
Re: What happened in this GPT-3 conversation?
#19After spurious tokens it failed to stop, then all the high temp context that generated navigated itself into nonsense completion space.
Re: What happened in this GPT-3 conversation?
#20What's the issue here? Not sure I see it.
It seems to be going along quite well until it doesn't. And it's sort of clear to see where it starts going off the rails. If you're going to trust algorithms that are under-trained and over-constrained, you sort of have to expect this kind of thing.
But then again, I always. Always. I am growing.