Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How is ChatGPT's behavior changing over time?

arxiv.org

Post reply on HN
Print view View original

131–140 of 187 posts

Re: How is ChatGPT's behavior changing over time?

#131

Post by AbrahamParangi »

randomwalker wrote: post #31

This paper is being misinterpreted. The degradations reported are somewhat peculiar to the authors' task selection and evaluation method and can easily result from fine tuning rather than intentionally degrading GPT-4's performance for cost saving reasons. They report 2 degradations: code generation & math problems. In both cases, they report a behavior change (likely fine tuning) rather than a capability decrease (p…

In my opinion the more likely thing is that OpenAI is gaslighting people that the finetuning is improving the model when it likely mostly improves safety at some cost to capability. I'd bet this is measured against a set of evals and it looks like it performs well BUT I'd also bet the evals are asymmetrically good at detecting "unsafe" or jailbreak behavior and bad at detecting reduced general cognitive flexibility.

The obvious avenue to degradation is that the "HR personality" is much more strictly applied and the resistance to being jailbroken is also in some sense an inability to think.

Re: How is ChatGPT's behavior changing over time?

#132

Post by TeMPOraL »

BoorishBears wrote: post #45
visarga wrote: post #44

Earlier quoted context omitted.

> OpenAI would not kneecap their commercial offering by randomly changing how it works. Have you seen the 25 messages/3 hours limitation for GPT-4? Why do you think they did that? Of course they would make more money scaling up the volume, but how to do that when compute is so limited? Of course, by using some kind of approximation - quantised model or speculative sampling come to mind. It's hard to pinpoint model re…

You realize that's a limitation in the web application right? The web app is a consumer app (B2C) the api is commercial (B2B). They tinker with the B2C app because it's already a lossy approximation of using the model between the summarization and system prompt. They cannot mess with the commercial offering willy-nilly: People are building businesses predicated on it behaving a certain way. That's why there are dated…

Arguably, both ChatGPT and API are consumer apps. That includes researchers. Pay as you go, no strings attached, "oh yeah no no, we're not changing anything, follow our CEO on Twitter if you want to know more". That kind of stuff.

The actual B2B offering is handled by Microsoft, via Azure OpenAI. Same models, but deployed on Azure - meaning they come with SLA and all the right protocol and compliance stuff, so that your people can negotiate with their people - and if you're willing to spend enough, you'll get the models for yourself. Not the weights, of course: just no training on your inputs, not even retaining inputs for 30 days "because ${legal reasons}" - instead, you can pick and chose, fine-tune and deploy OpenAI models on your own tenant, and basically manage everything except the weights themselves.

Re: How is ChatGPT's behavior changing over time?

#133

Post by hammock »

jstummbillig wrote: post #117
xp84 wrote: post #43

Earlier quoted context omitted.

Seriously. GPT doing math is like using a 737 to drive around on the ground, or if you had the phone number of a prominent astrophysicist and you call him to do long division for you. Wtf is the point. We have computer things to do every math problem. It’s a waste of energy to use LLMs for it in my opinion.

"We" are never going to stop trying to use LLMs for math. They are obviously mimicking a smart person. What do you do with smart persons? You query them with bad and lazy questions (often without being too honest about that to yourself), and hope/expect helpful answers. Because that is how it works. The limiting factor so far has been the smart persons time and patience. Now, no longer. People moderating their LLMs u…

>"We" are never going to stop trying to use LLMs for math. They are obviously mimicking a smart person.

A lot of words to make a big deal out of nothing. All that is needed is some new abstracted layer that identifies a math question and then proxies it over to the wolfram plugin. That’s it

We don’t have crazy debates over whether a polygon should be rendered by the cpu or a gpu. We solved this problem

Re: How is ChatGPT's behavior changing over time?

#134

Post by gardenhedge »

0x000xca0xfe wrote: post #102
lumost wrote: post #55

Earlier quoted context omitted.

I used GPT-4 to generate a non-cryptographic random 64 character string. It was faster to ask GPT-4 for the string than ask GPT-4 for the instructions to generate the string from my terminal. GPT-4 was faster than google.

Obligatory bash solution: $ random_bytes() { xxd -plain -c 0 -l "$1" /dev/urandom; } $ random_bytes 32 e6a4a7bbea69a0164cbb66c89f8f528af93c6d2459fd28d2640e2952c031b618

I'm glad ChatGPT exists so I don't have to remember any of that

Re: How is ChatGPT's behavior changing over time?

#135

Post by AbrahamParangi »

The main thing that ChatGPT has gotten better at is rejecting jailbreaks and refusing to go off the reservation. It has been demonstrated that "safety" trades off against "capability". I'm sure OpenAI has evaluations that demonstrate the improvements they've been making to "safety" have not come at the cost of capability, but I'd bet those evaluations are wrong (by being insufficient). It also wouldn't surprise me if the tradeoff between "safety" and capability is just intrinsic.

You can have your model talk like HR or think like a mad scientist, but not both equally well.

Re: How is ChatGPT's behavior changing over time?

#136

Post by zeusk »

hammock wrote: post #133
jstummbillig wrote: post #117

Earlier quoted context omitted.

"We" are never going to stop trying to use LLMs for math. They are obviously mimicking a smart person. What do you do with smart persons? You query them with bad and lazy questions (often without being too honest about that to yourself), and hope/expect helpful answers. Because that is how it works. The limiting factor so far has been the smart persons time and patience. Now, no longer. People moderating their LLMs u…

>"We" are never going to stop trying to use LLMs for math. They are obviously mimicking a smart person. A lot of words to make a big deal out of nothing. All that is needed is some new abstracted layer that identifies a math question and then proxies it over to the wolfram plugin. That’s it We don’t have crazy debates over whether a polygon should be rendered by the cpu or a gpu. We solved this problem

So a Mixture of Experts model; but then isn't ChatGPT using that already? Why're the models so bad at math despite being trained on academic papers and books clearly - or why do they hallucinate and make up non-existing citations?

Re: How is ChatGPT's behavior changing over time?

#137

Post by spaceman_2020 »

AbrahamParangi wrote: post #135

The main thing that ChatGPT has gotten better at is rejecting jailbreaks and refusing to go off the reservation. It has been demonstrated that "safety" trades off against "capability". I'm sure OpenAI has evaluations that demonstrate the improvements they've been making to "safety" have not come at the cost of capability, but I'd bet those evaluations are wrong (by being insufficient). It also wouldn't surprise me if…

I've cancelled my pro subscription. You can't ask me to pay money and also expect me to spend hours Googling around to find prompts for basic queries.

I would also like to not be treated like an idiot. Every time I query anything related to health or medicine, chatGPT will give me a short generic answer, then add two paragraphs of warnings about how I should just seek a medical professional's help. As if I'm the kind of moron who will blindly follow whatever some AI tool tells me and not actually go to a doctor if there's something wrong with me.

This organization seems like it's being run by (scared) lawyers.

Re: How is ChatGPT's behavior changing over time?

#138

Post by sensanaty »

ttul wrote: post #13

I tried the prime number problem and GPT-4 nailed it. I’m not sure whether they are testing things correctly… “ Sure, let's go step by step. A prime number is a number greater than 1 that has no positive divisors other than 1 and itself. This means if we can find any other number (excluding 1 and the number itself) that divides 17077, then it is not a prime number. Let's start by checking divisibility by 2. Since 170…

Can you define "nailing" it? Cause this isn't what I'd call nailing it...

Re: How is ChatGPT's behavior changing over time?

#139

Post by lumost »

oefnak wrote: post #74
lumost wrote: post #55

Earlier quoted context omitted.

I used GPT-4 to generate a non-cryptographic random 64 character string. It was faster to ask GPT-4 for the string than ask GPT-4 for the instructions to generate the string from my terminal. GPT-4 was faster than google.

That definitely won't be random.

Close enough for the purpose at hand!

Re: How is ChatGPT's behavior changing over time?

#140

Post by hutzlibu »

FeepingCreature wrote: post #109
SanderNL wrote: post #98

Earlier quoted context omitted.

It’s not about the results, it’s about its ability to “reason”. Math is about as close to pure reasoning we get so I don’t get the pessimism. If it is bad at math and can’t be taught, then you have a fundamental problem. It’s a matter of time before this limit gets hit in other domains.

In my experience, it can reason usably well but acts like it has dyscalculia. It'll set up a proper algorithm, step through it and trip over digits.

"It'll set up a proper algorithm, step through it and trip over digits."

Or it is just pretending to do so. And since it pretends, of course it trips over all small things as it does not understand them.

Post reply on HN