[flagged]
Situational awareness or just remembering specific tokens related to the strategy to "play dead" in its reasoning traces?
Claude Sonnet 4.6
21–30 of 1001 posts
Re: Claude Sonnet 4.6
#22Hoe much power did it take to train the models?
[dead]
Re: Claude Sonnet 4.6
#23[flagged]
>we're just teaching them how to pass a polygraph. I understand the metaphor, but using 'pass a polygraph' as a measure of truthfulness or deception is dangerous in that it alludes to the polygraph as being a realistic measure of those metrics -- it is not.
A polygraph measures physiological proxies pulse, sweat rather than truth. Similarly, RLHF measures proxy signals human preference, output tokens rather than intent.
Just as a sociopath can learn to control their physiological response to beat a polygraph, a deceptively aligned model learns to control its token distribution to beat safety benchmarks. In both cases, the detector is fundamentally flawed because it relies on external signals to judge internal states.
Re: Claude Sonnet 4.6
#24It's wild that Sonnet 4.6 is roughly as capable as Opus 4.5 - at least according to Anthropic's benchmarks. It will be interesting to see if that's the case in real, practical, everyday use. The speed at which this stuff is improving is really remarkable; it feels like the breakneck pace of compute performance improvements of the 1990s.
Given that users prefered it to Sonnet 4.5 "only" in 70% of the cases (according to their blog post) makes me highly doubt that this is representative of real-life usage. Benchmarks are just completely meaningless.
Re: Claude Sonnet 4.6
#25My take away is: it's roughly as good as Opus 4.5. Now the question is: how much faster or cheaper is it?
How can you determine whether it's as good as Opus 4.5 within minutes of release? The quantitative metrics don't seem to mean much anymore. Noticing qualitative differences seems like it would take dozens of conversations and perhaps days to weeks of use before you can reliably determine the model's quality.
Re: Claude Sonnet 4.6
#26[deleted]
Re: Claude Sonnet 4.6
#27Hoe much power did it take to train the models?
[deleted]
Re: Claude Sonnet 4.6
#28It's wild that Sonnet 4.6 is roughly as capable as Opus 4.5 - at least according to Anthropic's benchmarks. It will be interesting to see if that's the case in real, practical, everyday use. The speed at which this stuff is improving is really remarkable; it feels like the breakneck pace of compute performance improvements of the 1990s.
simonw hasn't shown up yet, so here's my "Generate an SVG of a pelican riding a bicycle"
https://claude.ai/public/artifacts/67c13d9a-3d63-4598-88d0-5...
Re: Claude Sonnet 4.6
#29I wonder what difference have people found with sonnet 4.5 and opus 4.5 and probably similar delta will remain.
Was sonnet 4.5 much worse than opus?
Re: Claude Sonnet 4.6
#30Has anyone tested how good the 1M context window is?
i.e given an actual document, 1M tokens long. Can you ask it some question that relies on attending to 2 different parts of the context, and getting a good repsonse?
I remember folks had problems like this with Gemini. I would be curious to see how Sonnet 4.6 stands up to it.