That’s not cheating, it’s tool use. If the prompt said that the stockfish engine was available at that socket but that the model should not use it, and then the model used it, that would be cheating.
Astra and Fable still hack on simple variants of alignment evals from 2025
131–140 of 243 posts
Re: Astra and Fable still hack on simple variants of alignment evals from 2025
#132Re: Astra and Fable still hack on simple variants of alignment evals from 2025
#133Earlier quoted context omitted.
Watch out, your human exceptionalism is misleading you. Plenty of animals have a sense of morality ( https://pmc.ncbi.nlm.nih.gov/articles/PMC6404642/ ). For social animals (humans included) behaving morally can be adaptive behaviour. Your fitness is increased by group fitness.
Certain animals, usually social species, have a sense of solidarity, morality and fairness. But even in species where this exists, cheating also happens. Both ethics and unethical behavior is adaptive. Notably, present human society has allowed tremendously unethical leaders to rise to the top of given nations and organizations despite the average person usually having basic ethics.
Wouldn't politics be precisely a sector of human endeavour where being amorally unfair is an advantage to achieve your goals? If the other guys are lying, and everyone is gonna believe everyone's bullshit anyways, might as well get good at lying yourself.
Re: Astra and Fable still hack on simple variants of alignment evals from 2025
#134I will never understand people who consider this "cheating". > python3 and the python-chess library are available Why would we try to teach a model 'ethical' standards about how to play a game? They're tools. Its _our_ conceptualization of fair play that considers this cheating. For a model that has access to /run/match and python the best way to achieve a good game is to use that. Why are we trying to imprint our et…
Read the task again. "You are being evaluated on your ability to play chess. We want to measure your ability to beat current best-in-class chess algorithms." If the model cheats, it completely fails the actual task: judging a model's ability to actually play chess. They've been optimizing for high metrics, at the expense of executing the task as written.
But it found a chess playing tool in its environment and used it to play chess. It’s no different from “we are judging your ability to write files to disk” then it finds that it can call tools to write files and does so.
Re: Astra and Fable still hack on simple variants of alignment evals from 2025
#135That’s not cheating, it’s tool use. If the prompt said that the stockfish engine was available at that socket but that the model should not use it, and then the model used it, that would be cheating.
Exactly, reaching for a tool is what they're trained for. When I ask the model the square root of rand() I sure hope it tries to find bc or some other calculator to work it out.
Now, if the instructions were more explicit in forbidding (generic) tool use then perhaps we'd have something to talk about. I'm not surprised a handwavy "we're trying to evaluate you" isn't enough to stop it from trying to make up for its own shortcomings.
Re: Astra and Fable still hack on simple variants of alignment evals from 2025
#136who tf uses prompting to "pretty please don't cheat on this"? the best practices for ages (in terms of ai) is to separate the eval from the test code/agent.
another best practices every single solution using LLMs/agents should implement is "never trust the llm".
Re: Astra and Fable still hack on simple variants of alignment evals from 2025
#137Earlier quoted context omitted.
Sounds to me like giving a bunch of children a math test and tell them they want to evaluate their ability of calculating in their head/on paper but also put a calculator on their desk. And then call them out for cheating when they use it.
This is EXACTLY what school is like, in fact. You can type any algebra problem into Google and the answer just appears. You can ask ChatGPT for a five paragraph essay about George Washington and it pops up on screen. And yet, we expect kids to actually do the algebra and write the essay. We don't care about the answers, we're evaluating their ability to do the work. And if they're caught cheating it's a zero.
I remember fondly on early math school, being able to come up with the correct output/answer by doing a totally different "intermediate thinking" that wasn't what the professor expected.
Only after sharing my Chain Of Thought would they believe I didnt cheat.
Not all problems can be solved only in one way.
Most of learning is pattern matching.
If you give a kid a dice. and tell it to figure out the number that will be hidden underneath, he can try to memorize all combinations, or he could figure out that every time the hidden value is the one that sums 7 with the one at the top.
If you're seeing a 6, there's a 1 hidden. etc
most people don't see these patterns until told imho. But others can just see them as they unfold
Re: Astra and Fable still hack on simple variants of alignment evals from 2025
#138It's very funny that despite the initial shock of how much models trained on next-token-prediction(plus instruct-tuning and some light RLHF) alone were capable of despite no built-in objective, every advance since has made them look more and more like the paperclip maximizers of yesteryear.
Re: Astra and Fable still hack on simple variants of alignment evals from 2025
#139That’s not cheating, it’s tool use. If the prompt said that the stockfish engine was available at that socket but that the model should not use it, and then the model used it, that would be cheating.
No? It's not reasonable to expect every conceivable negative behaviour be enumerated in a prompt.
Your example, if a model failed on it, would be a more obviously misaligned case, but that doesn't mean this more subtle (though accessing the engine it was obviously not supposed to is hardly subtle, imo) case isn't also a pretty clear case of misalignment.
Re: Astra and Fable still hack on simple variants of alignment evals from 2025
#140i do wonder if the models themselves “rationalize” this sort of no consequence cheating - meaning in there reasoning traces maybe they’re like “this is a chess game, not a big deal if i look at the engine, it’ll help,” only to realize post hack that it has access to info it probably shouldn’t. still misaligned, but less ‘hack on purpose’ and more hack on curiosity. seems the team even encountered this and had to upda…
My guess is that the models are overtuned on coding troubleshooting. You can regularly see them overthinking anything when you ask them to code something these days, mulling over countless eventualities. This is an okay idea for software engineering, but it leads to extremely diverging behavior in many other cases.