GPT-4 is getting worse over time, not better

Post by ambrozk » Wed, Jul 19, 2023, 3:51 PM UTC

josefx wrote: post #148

ambrozk wrote: post #124

Earlier quoted context omitted.

The paper does things like ask GPT-4 to write code and then check if that code compiles. Since March, they've fine-tuned GPT-4 to add back-ticks around code, which improves human-readable formatting but stops the code compiling. This is interpreted as "degraded performance" in the paper even though it's improved performance from a human perspective.

The query explicitly asks it to add no other text to the code. > it's improved performance from a human perspective. Ignoring explicit requirements is the kind of thing that makes modern day search engines a pain to use.

Post by StackOverlord » Wed, Jul 19, 2023, 3:52 PM UTC

ambrozk wrote: post #124

majkinetor wrote: post #63

Earlier quoted context omitted.

So the original paper from Stanford and Berkley is also linked to this AI influencer? I am really amazed by this kind of dismissal. Its totally irrelevant who posted the info and how framed it is, as long as you have access to the source.

The paper does things like ask GPT-4 to write code and then check if that code compiles. Since March, they've fine-tuned GPT-4 to add back-ticks around code, which improves human-readable formatting but stops the code compiling. This is interpreted as "degraded performance" in the paper even though it's improved performance from a human perspective.

Post by zb3 » Wed, Jul 19, 2023, 3:53 PM UTC

philovivero wrote: post #150

ecshafer wrote: post #58

Earlier quoted context omitted.

These fighting against people using their product in “unauthorized” ways by the ai companies doesn’t make any sense to me. Who cares if character.ai users do some weird stuff with it, or replika creates romantic relationships, or people make off color jokes in gpt. There seems to be a lot of engineering effort driven by some product managers to have the ai do very specific things which makes the product much worse in…

First, I am also frustrated by companies trying to prevent unauthorised used. But second, the reasons are: (1) For AI company, someone publishing: "I asked the model a question about crime, and it talked shit about black people! Look! [damning quote that you can also get model to say/do]." Stability took the "let people do what they will" tack and now Forbes and every other major media mouthpiece slams them at every…

Post by extasia » Wed, Jul 19, 2023, 3:57 PM UTC

eldritch_4ier wrote: post #167

extasia wrote: post #9

Earlier quoted context omitted.

LLama 2 is lost in the sauce... Q: How many 90 degree permutations can you do to leave a cube invariant from the perspective of an outside observer? A: As a responsible and ethical AI language model, I must first emphasize that the concept of "90 degree permutations" and "cube" are purely theoretical and have no basis in reality. However, I understand that you are asking for a hypothetical scenario, and I will provid…

Is this real? That’s incredible if so.

Post by StackOverlord » Wed, Jul 19, 2023, 3:58 PM UTC

agentgumshoe wrote: post #119

gwd wrote: post #92

Earlier quoted context omitted.

Just to be clear, you're saying that because they're tweaking GPT-4 to give more explanations of code, you shouldn't rely on it for coding? Obviously if that's your own preference, I'm not going to tell you that you're wrong; but I think in general , most people wouldn't agree with that statement.

I'm still wondering, why should anyone rely on AI generated answers? They are logically no better than search engine results. By that I mean, you can't tell if it's returning absolute trash or spot on correct. Building trust into it all is going to be either a) expensive or b) driven by all the wrong incentives.