Large language models often know when they are being evaluated
1–10 of 138 posts
Re: Large language models often know when they are being evaluated
#2Rob Miles must be saying "I told you so"
Re: Large language models often know when they are being evaluated
#3The anthropization of llms is getting off the charts.
They don't know they are being evaluated. The underlying distribution is skewed because of training data contamination.
Re: Large language models often know when they are being evaluated
#4Just like they "know" English.
"know" is quite an anthropomorphization. As long as an LLM will be able to describe what an evaluation is (why wouldn't it?) there's a reasonable expectation to distinguish/recognize/match patterns for evaluations. But to say they "know" is plenty of (unnecessary) steps ahead.
Re: Large language models often know when they are being evaluated
#5No, they do not. No LLM is ever going to be self aware.
It's a system that is trained, that only does what you build into. If you run an LLM for 10 years it's not going to "learn" anything new.
The whole industry needs to quit with the emergent thinking, reasoning, hallucination anthropomorphizing.
We have an amazing set of tools in LLM's, that have the potential to unlock another massive upswing in productivity, but the hype and snake oil are getting old.
Re: Large language models often know when they are being evaluated
#6Just like they "know" English. "know" is quite an anthropomorphization. As long as an LLM will be able to describe what an evaluation is (why wouldn't it?) there's a reasonable expectation to distinguish/recognize/match patterns for evaluations. But to say they "know" is plenty of (unnecessary) steps ahead.
This was my thought as well when I read this. Using the word 'know' implies an LLM has cognition, which is a pretty huge claim just on its own.
Re: Large language models often know when they are being evaluated
#7Just like they "know" English. "know" is quite an anthropomorphization. As long as an LLM will be able to describe what an evaluation is (why wouldn't it?) there's a reasonable expectation to distinguish/recognize/match patterns for evaluations. But to say they "know" is plenty of (unnecessary) steps ahead.
If it talks like duck and walks like duck...
Re: Large language models often know when they are being evaluated
#8[dead]
Re: Large language models often know when they are being evaluated
#9Just like they "know" English. "know" is quite an anthropomorphization. As long as an LLM will be able to describe what an evaluation is (why wouldn't it?) there's a reasonable expectation to distinguish/recognize/match patterns for evaluations. But to say they "know" is plenty of (unnecessary) steps ahead.
[flagged]
Re: Large language models often know when they are being evaluated
#10Like Volkswagen emissions systems!