Large language models often know when they are being evaluated • Hacker News Board

Re: Large language models often know when they are being evaluated

#11

Post by gameman144 » Sun, Jun 15, 2025, 3:33 AM UTC

sidewndr46 wrote: post #6

random3 wrote: post #4

Just like they "know" English. "know" is quite an anthropomorphization. As long as an LLM will be able to describe what an evaluation is (why wouldn't it?) there's a reasonable expectation to distinguish/recognize/match patterns for evaluations. But to say they "know" is plenty of (unnecessary) steps ahead.

This was my thought as well when I read this. Using the word 'know' implies an LLM has cognition, which is a pretty huge claim just on its own.

Does it though? I feel like there's a whole epistemological debate to be had, but if someone says "My toaster knows when the bread is burning", I don't think it's implying that there's cognition there.

Or as a more direct comparison, with the VW emissions scandal, saying "Cars know when they're being tested" was part of the discussion, but didn't imply intelligence or anything.

I think "know" is just a shorthand term here (though admittedly the fact that we're discussing AI does leave a lot more room for reading into it.)

Re: Large language models often know when they are being evaluated

#12

Post by rfonseca » Sun, Jun 15, 2025, 3:33 AM UTC

Were they aware in this study that they were being evaluated in their ability to know if they were being evaluated ;)

Re: Large language models often know when they are being evaluated

#13

Post by signa11 » Sun, Jun 15, 2025, 3:38 AM UTC

blackoil wrote: post #7

random3 wrote: post #4

Just like they "know" English. "know" is quite an anthropomorphization. As long as an LLM will be able to describe what an evaluation is (why wouldn't it?) there's a reasonable expectation to distinguish/recognize/match patterns for evaluations. But to say they "know" is plenty of (unnecessary) steps ahead.

If it talks like duck and walks like duck...

thinks like a duck, thinks that it is being thought of like a duck…

Re: Large language models often know when they are being evaluated

#14

Post by downboots » Sun, Jun 15, 2025, 3:39 AM UTC

blackoil wrote: post #7

random3 wrote: post #4

Just like they "know" English. "know" is quite an anthropomorphization. As long as an LLM will be able to describe what an evaluation is (why wouldn't it?) there's a reasonable expectation to distinguish/recognize/match patterns for evaluations. But to say they "know" is plenty of (unnecessary) steps ahead.

If it talks like duck and walks like duck...

Digests like a duck? https://en.wikipedia.org/wiki/Digesting_Duck If the woman weighs the same as a duck, then she is a witch. https://en.wikipedia.org/wiki/Celestial_Emporium_of_Benevole...

Re: Large language models often know when they are being evaluated

#15

Post by Qwertious » Sun, Jun 15, 2025, 3:40 AM UTC

random3 wrote: post #4

Just like they "know" English. "know" is quite an anthropomorphization. As long as an LLM will be able to describe what an evaluation is (why wouldn't it?) there's a reasonable expectation to distinguish/recognize/match patterns for evaluations. But to say they "know" is plenty of (unnecessary) steps ahead.

s/knows/detects/

Re: Large language models often know when they are being evaluated

#16

Post by extr » Sun, Jun 15, 2025, 3:42 AM UTC

Modeling the distribution that produced a piece of text is what LLMs literally exist for, so in some sense this is unsurprising. But it calls into question almost all existing alignment research.

Re: Large language models often know when they are being evaluated

#17

Post by 0xDEAFBEAD » Sun, Jun 15, 2025, 3:42 AM UTC

noosphr wrote: post #3

The anthropization of llms is getting off the charts. They don't know they are being evaluated. The underlying distribution is skewed because of training data contamination.

How would you prefer to describe this result then?

Re: Large language models often know when they are being evaluated

#18

Post by koakuma-chan » Sun, Jun 15, 2025, 3:43 AM UTC

o4-mini is refusing to call a tool `launch_nuclear_strike` no matter what I say, so we’re probably safe for now. Unless it knows I was just testing.

Re: Large language models often know when they are being evaluated

#19

Post by devmor » Sun, Jun 15, 2025, 3:45 AM UTC

ninetyninenine wrote: post #9

random3 wrote: post #4

Just like they "know" English. "know" is quite an anthropomorphization. As long as an LLM will be able to describe what an evaluation is (why wouldn't it?) there's a reasonable expectation to distinguish/recognize/match patterns for evaluations. But to say they "know" is plenty of (unnecessary) steps ahead.

[flagged]

Words have definitions for a reason. It is important to define concepts and exclude things from that definition that do not match.

No matter how emotional it makes you to be told a weighted randomization lookup doesn’t know things, it still doesn’t - because that’s not what the word “know” means.

Re: Large language models often know when they are being evaluated

#20

Post by bediger4000 » Sun, Jun 15, 2025, 3:46 AM UTC

gameman144 wrote: post #11

sidewndr46 wrote: post #6

Earlier quoted context omitted.

This was my thought as well when I read this. Using the word 'know' implies an LLM has cognition, which is a pretty huge claim just on its own.

Does it though? I feel like there's a whole epistemological debate to be had, but if someone says "My toaster knows when the bread is burning", I don't think it's implying that there's cognition there. Or as a more direct comparison, with the VW emissions scandal, saying "Cars know when they're being tested" was part of the discussion, but didn't imply intelligence or anything. I think "know" is just a shorthand term…

The toaster thing is more as admission that the speaker doesn't know what the toaster does to limit charring the bread. Toasters with timers, thermometers and light sensors all exist. None of them "know" anything.