GPT-4 is not getting worse

Post by visarga » Sat, Sep 16, 2023, 6:34 PM UTC

vharuck wrote: post #112

lairv wrote: post #87

Earlier quoted context omitted.

The main pain point I still have with GPT4 is that it's hard to prevent it from being a yes-man "- I have problem A, what do you think of X ? - Yes X is great ... - But with X I'd encounter problem B, maybe I should go with Y ? - You're right X would cause problem B. With Y ..." One way to limit this issue is to not give ChatGPT any hint on your intent, instead of writing "I have problem A, what do you think of X ?",…

The training data of written content, no matter the sources, will always be biased in one way: it doesn't contain thoughts people kept to themselves. Example 1: Mr. X is very popular author among a small group, who chat daily about it on various websites. Outside of that group, many people know about Mr. X but don't care much about his works. So the majority of content about Mr. X is positive, even though the majorit…

Post by antonvs » Sat, Sep 16, 2023, 6:55 PM UTC

visarga wrote: post #104

jw1224 wrote: post #35

There is an ongoing bug[1] with OpenAI's API, where it stops streaming responses after exactly 5 minutes. When I first came across the issue, I debugged it by writing a prompt along these lines: > Output every number from 1 to 10,000 in written form (e.g. "one", "two", etc.). Respond with one number per line in numeric order. As expected, the API would begin counting every number just as I asked. This would continue…

I have this too, but on information extraction tasks, if the input document is too long or complex it will skip sections saying something along the lines of (instead of the list) (instead of the details) Very annoying when you want to extract everything and is hard to fix. Maybe they had some examples solved by abbreviation in the fine-tuning set and it damaged the model. It doesn't make sense to train this behaviour…

Post by [deleted] » Sat, Sep 16, 2023, 7:51 PM UTC

Wowfunhappy wrote: post #144

jw1224 wrote: post #35

There is an ongoing bug[1] with OpenAI's API, where it stops streaming responses after exactly 5 minutes. When I first came across the issue, I debugged it by writing a prompt along these lines: > Output every number from 1 to 10,000 in written form (e.g. "one", "two", etc.). Respond with one number per line in numeric order. As expected, the API would begin counting every number just as I asked. This would continue…

I can't replicate this. I tested your prompt verbatim in the OpenAI playground. Model set to gpt-4-0314, maximum length set to 6485 (nothing special about this number, I just dragged the slider to something high-ish), all other settings left at their default. I cancelled the request when it reached "one thousand one hundred twenty-one" (which took over five minutes), but up until that point it did not take any shortc…

Post by yieldcrv » Sat, Sep 16, 2023, 9:26 PM UTC

Post by ilaksh » Sat, Sep 16, 2023, 9:56 PM UTC

layer8 wrote: post #122

jw1224 wrote: post #35

There is an ongoing bug[1] with OpenAI's API, where it stops streaming responses after exactly 5 minutes. When I first came across the issue, I debugged it by writing a prompt along these lines: > Output every number from 1 to 10,000 in written form (e.g. "one", "two", etc.). Respond with one number per line in numeric order. As expected, the API would begin counting every number just as I asked. This would continue…

The inability to resolve issues like the mentioned 5-minute timeout with OpenAI support even after months should be a red flag to not build any products based on their services.