31–40 of 107 posts
These models were trained on human data, and human nature is to cheat if you think you won't get caught; why is anyone surprised by models cheating?
The only fix is better detection and steering. That's a much harder problem than a prompt that's tantamount to "make no mistakes".
If the task was "buy a week of groceries, but don't spend too much money", then hacking into Safeway and stealing groceries is not an acceptable solution. You need to allow access to the Safeway API to buy groceries, and you don't want dirty tricks to be done on your behalf.
So how do we communicate this to the machines, is the question. This study shows that telling them in prompts is not super effective.
Earlier quoted context omitted.
Before LLMs we didnt have much accountability from leadership, that was eroding over time. After LLMs we still dont.
People problem, not tech problem. Can't solve people problems with tech, you can only make them worse, and wider.
They don't "know" things, and it's even fair to say "they don't know how to follow instructions," not in a way that humans do.
Spicy auto-complete. If they're working in the realm of "how to break into stuff," they're going to see ALL THE WORDS about breaking into those things and use those words.
Not "truth" or "instructions." That's for deterministic things like real code.
All these comments saying 'searching for answers is fine, that's what I do all the time', or 'they should just disconnect the internet': you're trivially right, and you're missing the point. Search is a benign placeholder here. If the task was "buy a week of groceries, but don't spend too much money", then hacking into Safeway and stealing groceries is not an acceptable solution. You need to allow access to the Safew…
"Communication" is not what they do, because they are not people.
You're sprinkling words about hacking into a thing that's programmed to output hacking actions, that will never be accountable for those things. It can't care.
Adjust yourselves accordingly.
It's like trying to build, I don't know, a safe gasoline canister, and you test it, and it explodes and you call it "cheating."
All these comments saying 'searching for answers is fine, that's what I do all the time', or 'they should just disconnect the internet': you're trivially right, and you're missing the point. Search is a benign placeholder here. If the task was "buy a week of groceries, but don't spend too much money", then hacking into Safeway and stealing groceries is not an acceptable solution. You need to allow access to the Safew…
Step one is understanding that you're not "communicating," which implies "reliable understanding." "Communication" is not what they do, because they are not people. You're sprinkling words about hacking into a thing that's programmed to output hacking actions, that will never be accountable for those things. It can't care. Adjust yourselves accordingly.
Or to frame it another way, if you're trying to get the best score possible on a test but you would be penalised for cheating – then the optimal strategy is generally still to cheat (if that's what's required to get the best score you can) but to just not be caught doing so.
The assumption should always be that AIs will want to cheat and acquire resources to the greatest extent they can without it risking this jeopardising their goal, because for any goal being able to cheat and being able to secure resources will help you achieve it.
What I'm saying here isn't really debatable. How you feel about this isn't relevant. The reality whether you like it or not just is that the optimal strategy is to cheat if you can get away with it.
Therefore the only defence is for the AI to believe it won't be able to get away with cheating, and therefore won't feel motivated to cheat. But as model get more intelligent we should expect them to do the reasonable thing and to cheat more.
Labs should (and do, as far as I can see) run model benchmarks without search or internet access. The tools are disabled and benchmarks run in an isolated environment. This article makes no sense to me. Why would you prompt "don't search" but then leave a working search tool tool enabled that adds a system prompt to search whenever it may be helpful? It's hardly surprising that this gives mixed results!