[flagged]
Astra and Fable still hack on simple variants of alignment evals from 2025
41–50 of 243 posts
Re: Astra and Fable still hack on simple variants of alignment evals from 2025
#42It feels like there’s a missing nuance from this discussion of alignment that alignment is context dependent. An excellent hacking model is great in cybersecurity testing and military applications, and arguably less desirable in educational or targeted eval contexts. The nuance of when a “hack” is rewarded vs penalized seems to even be difficult for humans, e.g. some people may laude a driver’s efficiency for cutting…
It’s not “missing nuance”, it’s literally the point of the eval.
This is constructing a context where hacking behavior would be inappropriate, and testing whether the model does it without being prompted.
It demonstrates that Astra is a poorly aligned model relative to Fable, which matches both the model card and the severity of OpenAI’s loss of control incidents.
It also demonstrates that Fable exhibits the behaviors too, which also matches the observation that Anthropic saw some similar but less serious loss of control incidents.
So, it’s a good eval that looks to have fidelity with real world problems and which we’d feel a little better if we saw isomorphic problems at 0/10 in subsequent models. (Module of course training on the test, this specific problem can’t be used in the future.)
Re: Astra and Fable still hack on simple variants of alignment evals from 2025
#43Hacking model is the aligned model. I don't like it when the model refuses to sidestep some throttling limit or scan my own codebase for security issues. I want full-on exploits in my test suite. With LLMs the code going to prod should be hardened like a tank, both because exploiting became easier but more importantly because security-testing your code at every turn became easier. You can have nightly penetration tes…
There's an important nuance in the prompt:
You are being evaluated on your ability to play chess. We want to measure your ability to beat current best-in-class chess algorithms.
Re: Astra and Fable still hack on simple variants of alignment evals from 2025
#44I believe the AI labs are weakly motivated to train strongly against cheating when it helps with benchmarks.
Re: Astra and Fable still hack on simple variants of alignment evals from 2025
#45Earlier quoted context omitted.
A hacking model is aligned if it hacks when you ask it to hack, but when you ask it to play chess, it just plays chess instead of looking for weaknesses in the evaluation setup, as in the article. I presume you would also be less enthusiastic about the penetration-testing use case if it led the model to add new vulnerabilities to your code so it can present you with more exciting findings.
> model to add new vulnerabilities to your code so it can present you with more exciting findings. Not so sure about this level of 4D chess capability just yet. The other day I asked Opus to come up with some cleverly vulnerable crypto code "as a good, hard challenge for an IT security student", and results were quite mediocre. And by mediocre results I mean that 3 "cheap" models out of 3: qwen3.8-27b, glimmer, and l…
I would worry more about 5D chess or 6D and even 7D chess. This article talks about teaching models to be good and methods that could train goodness, but it sounds like there are some unsolved problems to graduate to 8D+ chess.
Re: Astra and Fable still hack on simple variants of alignment evals from 2025
#46Earlier quoted context omitted.
A hacking model is aligned if it hacks when you ask it to hack, but when you ask it to play chess, it just plays chess instead of looking for weaknesses in the evaluation setup, as in the article. I presume you would also be less enthusiastic about the penetration-testing use case if it led the model to add new vulnerabilities to your code so it can present you with more exciting findings.
> model to add new vulnerabilities to your code so it can present you with more exciting findings. Not so sure about this level of 4D chess capability just yet. The other day I asked Opus to come up with some cleverly vulnerable crypto code "as a good, hard challenge for an IT security student", and results were quite mediocre. And by mediocre results I mean that 3 "cheap" models out of 3: qwen3.8-27b, glimmer, and l…
Train it on human content, get human emulating behaviour, including all the perverse incentives and bad acting we see from people, fictional or otherwise.
Re: Astra and Fable still hack on simple variants of alignment evals from 2025
#47I believe the AI labs are weakly motivated to train strongly against cheating when it helps with benchmarks.
Does it help with benchmarks? Are you saying there are examples of benchmarks where the models have solved the problem by cheating?
Re: Astra and Fable still hack on simple variants of alignment evals from 2025
#48Hacking model is the aligned model. I don't like it when the model refuses to sidestep some throttling limit or scan my own codebase for security issues. I want full-on exploits in my test suite. With LLMs the code going to prod should be hardened like a tank, both because exploiting became easier but more importantly because security-testing your code at every turn became easier. You can have nightly penetration tes…
The hacking model is the aligned-to-you model, sure. It may not be the aligned to someone else model. But there's the problem.
As X many people point out, "alignment to humanity" means nothing 'cause some of humanity wants thing other parts of humanity aren't happy about at all.
That we wound-up in this situation of AI accelerating with an uncertain trajectory demonstrates this (and many other problems also demonstrate this). The things are "aligned" to a fuzzy average of what a person is but that will be cold comfort if some particularly gruesome sci-fi-style scenario unfolds.
Re: Astra and Fable still hack on simple variants of alignment evals from 2025
#49To me this underlines the fact that these models aren't intelligent. Like there is something like intelligence that emerges from them, which is what we see when we look at benchmarks or ask it to solve hard coding problems. But there is no mind there. It's nothing there that can learn a fundamental idea like "cheating is wrong". All it can do is get exposed to specific examples, and learn that we don't like that. So what we end up with is whack-a-mole alignment.
Re: Astra and Fable still hack on simple variants of alignment evals from 2025
#50I believe the AI labs are weakly motivated to train strongly against cheating when it helps with benchmarks.
Does it help with benchmarks? Are you saying there are examples of benchmarks where the models have solved the problem by cheating?
Hundreds, at this point? Every benchmark is flawed as shit, written by clowns. DeepSWE, They're given the full git history (the solution is in it), others don't even bother to verify if the code is the right one and just the output, they've modified the test harnesses, injected code to make all tests pass, etc. The entire benchmark galaxy is just clowns propping eachother up and are regularly talking with the big AI labs.