Live data from Hacker News

GPT-4 is not getting worse

coagulopath.com

61–70 of 186 posts

Re: GPT-4 is not getting worse

#61

Why you would quiz an LLM on intricacies of pop culture is beyond me. Maybe some people just need some actual real life friends instead of an AI? I almost exclusively use it for coding and technical questions and it's been doing an absolute hell of an amazing job so far!

Why would you quiz your friends about pop culture? That would be treating them as informal research assistants.

Re: GPT-4 is not getting worse

#62
post #35

There is an ongoing bug[1] with OpenAI's API, where it stops streaming responses after exactly 5 minutes. When I first came across the issue, I debugged it by writing a prompt along these lines: > Output every number from 1 to 10,000 in written form (e.g. "one", "two", etc.). Respond with one number per line in numeric order. As expected, the API would begin counting every number just as I asked. This would continue…

It's actually supper annoying. I keep asking it to write out entire pieces of code, instead of only things it changed, can be annoying to merge it otherwise. But nowadays it acknowledges my request "I will write complete code" and proceed to still comment out parts.

I think they probably finetuned that because it used to be the other way around and it was pretty bad UX to wait 1 min to apply a function change to a class or stuff like that.

Re: GPT-4 is not getting worse

#63
post #60

Earlier quoted context omitted.

No? If that's how sentence forming worked it would be impossible to count backwards, or count at all for that matter.

Me: Count backwards from ten GPT-4: 10, 9, 8, 7, 6, 5, 4, 3, 2, 1 Me: Count backwards from a million to 999990 GPT-4: 1,000,000, 999,999, 999,998, 999,997, 999,996, 999,995, 999,994, 999,993, 999,992, 999,991, 999,990 I don't know about that.

I don't know how that connects in any way to what I and the commenter preceding me wrote.

Re: GPT-4 is not getting worse

#64

Earlier quoted context omitted.

Aren't your sentences probabilistically based on the existing corpus of text within your mind?

No? If that's how sentence forming worked it would be impossible to count backwards, or count at all for that matter.

[dead]

Re: GPT-4 is not getting worse

#66

Earlier quoted context omitted.

It's actually supper annoying. I keep asking it to write out entire pieces of code, instead of only things it changed, can be annoying to merge it otherwise. But nowadays it acknowledges my request "I will write complete code" and proceed to still comment out parts.

I think they probably finetuned that because it used to be the other way around and it was pretty bad UX to wait 1 min to apply a function change to a class or stuff like that.

Yeah sometimes you want it but sometimes you dont. I think they are enforcing it to save resources

Re: GPT-4 is not getting worse

#67
The reason for these articles and the frustration people have is that GPT 4 is a black box and is being changed whether we like it or not. Nobody can choose to continue to use a specific snapshot with the alignment that suits them, because a third-party is enforcing their preferred alignment -- and not even specifying what that is!

It's as-if a private company had invented portal technology and used it to establish trade routes with far-off alien civilisations. They even handle the purchase order translation, currency conversion, and so on. It's a miracle! You can order room-temperature superconductors and floaty rocks at a reasonable price.

"Sorry, due to high demand, we had to cut some corners on the translations" comes the announcement.

Now the entire human race sometimes gets copper wire and ordinary rocks.

"You're still trading with the exact same aliens" is the follow-up press release to appease the angry customers.

People point at the clear evidence of almost-but-not-superconductive wire they've received and demand answers.

"No, you do not need to know anything at all about our translation technology. No, you may not visit other planets without our express approval, that might be dangerous to our prof... I mean that might be dangerous to the human race. Yes. Dangerous! Existential, even."

Re: GPT-4 is not getting worse

#68
post #35

There is an ongoing bug[1] with OpenAI's API, where it stops streaming responses after exactly 5 minutes. When I first came across the issue, I debugged it by writing a prompt along these lines: > Output every number from 1 to 10,000 in written form (e.g. "one", "two", etc.). Respond with one number per line in numeric order. As expected, the API would begin counting every number just as I asked. This would continue…

Did you take in to account that the default GPT4 model changed in June? That would explain the response difference. You can still use the original model released in march with the GPT-4-0314 identifier.

I first noticed the counting shortcut back in April. It still persists with both the previous and current models.

Re: GPT-4 is not getting worse

#69
post #35

There is an ongoing bug[1] with OpenAI's API, where it stops streaming responses after exactly 5 minutes. When I first came across the issue, I debugged it by writing a prompt along these lines: > Output every number from 1 to 10,000 in written form (e.g. "one", "two", etc.). Respond with one number per line in numeric order. As expected, the API would begin counting every number just as I asked. This would continue…

Wow, was this with the regular model (gpt-N) or a purported unchanging datestamped model like gpt-N-0314/0613?

This was using gpt-4-0314, fixed and datestamped. The counting shortcut still affects newer models too, but the old version undoubtedly changed behaviour without warning.

Re: GPT-4 is not getting worse

#70
post #60

Earlier quoted context omitted.

Me: Count backwards from ten GPT-4: 10, 9, 8, 7, 6, 5, 4, 3, 2, 1 Me: Count backwards from a million to 999990 GPT-4: 1,000,000, 999,999, 999,998, 999,997, 999,996, 999,995, 999,994, 999,993, 999,992, 999,991, 999,990 I don't know about that.

I don't know how that connects in any way to what I and the commenter preceding me wrote.

You said that sentences aren't formed by probability from an existing corpus, as that method can't count backwards. GPT-4, a system based on such a method, can count backwards, thus disproving your claim.
Post reply on HN