171–180 of 200 posts
When AI visionaries warned us about machines eventually reaching a point where their capabilities would change exponentially, i didn't realise they meant decay.
Earlier quoted context omitted.
OpenAI released a new gpt-4 model on June 13 https://openai.com/blog/function-calling-and-other-api-updat... , and they update gpt-4 to the latest version every two weeks (aka, gpt-4 switched over to the -0613 on June 27). The -0613 version is really different! It added function calling to the API as a hint to the LLM, and in my experience if you don't use function calling it's significantly worse at code-like tasks,…
Can I ask how you use this function calling in your workflow? Any examples?
Maybe I lack the imagination, but what function should I give to the LLM? "insert(text: string)"?
Earlier quoted context omitted.
I've noticed the same thing. People also like to complain the quality of Google Search has gone down for much of the same reason: if you first do a Google search that returned a good result and then repeat it, you are going to notice the absence. But if you first do a Google search that didn't return the thing you expect you might think such a thing just doesn't exist on the Internet. Ergo, quality decrease is simply…
For Google Search, the sad part, is that the search algorithm hasn't gotten worse. But the web itself has. There is so much more spam, with actual human generated content being siloed more and more in walled gardens, that it starts to become a major issue for Google.
Earlier quoted context omitted.
It's a bit of both. The GPT-4 models have definitely been changing - there's multiple versions right now and you can try them out in the Playground. One of the biggest differences is that the latest model patches all of the GPT-4 jailbreak prompts; quite a big change if you were doing anything remotely spicy. But OA also says that it hasn't been changing the underlying model beyond that (that's probably the tweet you…
It'd be insane if OpenAI wasn't changing GPT-4. That kind of flat footedness would cost them their entire first mover advantage.
I’m convinced this is group hallucination. It must be so interesting to work at OpenAI, knowing you didn’t change a thing, and seeing that because of random chance, some small fraction of 100M users have all tricked each other that suddenly, something is different.
It’s definitely not. Our prompts that were generating JSON output went from around 95% valid JSON to about 10% overnight. The model just started inserting random commentary. We’ve reverted to the 0314 model and it’s working fine again.
Earlier quoted context omitted.
It’s definitely not. Our prompts that were generating JSON output went from around 95% valid JSON to about 10% overnight. The model just started inserting random commentary. We’ve reverted to the 0314 model and it’s working fine again.
I had an API integration written to convert an English language security rule into an XML object designed to instruct a remote machine how to comply with the rule programmatically. April 2023 we had about an 86% accept rate, that number has declined to 31% with no changes to the prompt.
I'm certain it's now doing significantly worse on the same tests, but alas I have lost the historical data to prove it.
Earlier quoted context omitted.
It's a bit of both. The GPT-4 models have definitely been changing - there's multiple versions right now and you can try them out in the Playground. One of the biggest differences is that the latest model patches all of the GPT-4 jailbreak prompts; quite a big change if you were doing anything remotely spicy. But OA also says that it hasn't been changing the underlying model beyond that (that's probably the tweet you…
It'd be insane if OpenAI wasn't changing GPT-4. That kind of flat footedness would cost them their entire first mover advantage.
I’m convinced this is group hallucination. It must be so interesting to work at OpenAI, knowing you didn’t change a thing, and seeing that because of random chance, some small fraction of 100M users have all tricked each other that suddenly, something is different.
Seriously... In that 134 replies thread, 0 transcripts showing actual performance degradation. Just endless "Yes, it seems bla bla." No evidence but just shapes in the clouds.
Here's the best fish I could make today: https://www.svgviewer.dev/s/3IuulHlC
Make of that what you will.
I’m convinced this is group hallucination. It must be so interesting to work at OpenAI, knowing you didn’t change a thing, and seeing that because of random chance, some small fraction of 100M users have all tricked each other that suddenly, something is different.