Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic's Safety Superpower

stratechery.com

Post reply on HN
Print view View original

151–160 of 206 posts

Re: Anthropic's Safety Superpower

#151

Post by almostdeadguy »

botw44 wrote: post #9

The whole thesis falls apart though. You can't be on your way to "power over everything" and get distilled into free Chinese models within months. Pick one. The bottleneck is compute and data, not the model. That's why they could only gate it for a bit. The ITAR thing proves it: no nationality controls in place, so the only option was killing the whole thing. Not exactly what an all-powerful gatekeeper does.

To this point, I've never understood the supposed "alignment" between the EA/AI Safety crowd and Anthropic's mission that the author comments on. Be the stewards of the Machine God, but responsibly? I think the Manhattan project, which AI development is commonly analogized to, had a lot more intrinsic properties to gate against uncontrolled proliferation (which still happened to some extent). Also this is a company that is expected to go public this year, at which point there will be a slew of new voices pushing the company to increase its value, mission be damned.

People like Yud at least have a clear consistency in their advocacy that we shouldn't be developing this at all. Anyone who thinks they can reconcile Anthropic's work with the AI safety mission is in total fantasyland, if it's not just a public persona they've adopted strategically.

Re: Anthropic's Safety Superpower

#152

Post by airstrike »

Supermancho wrote: post #114
cube2222 wrote: post #4

Relatedly, I think it's worth noting that Anthropic models have consistently been top-scoring in BullshitBench[0], in a league of their own, really. Not affiliated with the bench in any way, but I think it surfaces important differences between the behavior of the models from different labs. TLDR: The benchmark is measuring pushback in response to nonsensical requests and questions, as opposed to going with it and ha…

> I found my interactions with Fable to be extremely impressive; it made other models, including GPT 5.5 and Opus 4.8, feel small and dumb. > Anthropic models have consistently been top-scoring in BullshitBench[0] eyeroll I find that Anthropic models feel big and dumber. https://www.endorlabs.com/research/ai-code-security-benchmar... puts Fable 5th, which seems about right to me. I'm interested in code utility and co…

I think this just proves anyone can pick a benchmark that supports their point so maybe we shouldn't use treat them as evidence at all.

Re: Anthropic's Safety Superpower

#153

Post by everforward »

trollbridge wrote: post #147
_the_inflator wrote: post #58

Earlier quoted context omitted.

I disagree. It is not the model alone. It needs a system which capitalizes on it. And this is very complex. Hardware, software, architecture - it takes a lot to get it right. Try running the latest OS models on a normal Mac or PC. Claude Fable and Mythos are systems not just pure models. And of course marketing. Don't believe the hype. I think Claude is often times underwhelming. Security concerns are also a concern…

An LLM which provides an OpenAI or Anthropic API-compatible interface + a coding harness like OpenCode or oh-my-pi is a pretty easy "ecosystem" to replicate. Exactly what makes you say Fable or Mythos are "systems, not just pure models"?

Fable can delegate tasks to Opus or Sonnet, so it has some agentic properties and I believe it does them in parallel.

The parallelism is where this starts to fall apart on a local PC. Like I can run some Qwen quants, but I can’t run a decent Qwen model while also running another model smart enough to actually implement it. I’d have to do them in series, and given how long Fable seems to take even with parallelism, I’d probably be waiting days for an answer.

Re: Anthropic's Safety Superpower

#154

Post by trollbridge »

Aperocky wrote: post #101
dofm wrote: post #29

Earlier quoted context omitted.

Regulatory capture is the OpenAI and Anthropic end goal, for certain. But I also think they exist in a sort of un-designed corporate narcissism, which is a common trait in bubble economies — I am not judging them particularly severely. Netscape under Clark and Andreessen and Sun under McNealy both fell into corporate narcissism: the belief that only they really mattered, that they were chosen, and that the world need…

Spot on. There's a certain level of drinking the kool-aid or getting high on their own supply. Anthropic is a lot worse than OpenAI but OpenAI had to go through rounds of shedding.

A\ and OpenAI each have their own unique kind of nonsense. I think OpenAI has just been less successful with persuading the rest of the world that they should have all the money in the world.

Anthropic has been surprisingly successful at convincing them that they should control frontier models because they're so dangerous that... only Anthropic can be trusted with them.

(If they're really so dangerous, the right way to deal with them is through a democratic process and taking them out of the hands of a for-profit private entity.)

Re: Anthropic's Safety Superpower

#155

Post by kranke155 »

0x3f wrote: post #133
mft_ wrote: post #99

Earlier quoted context omitted.

Where would they go? 1) It’s safe to assume the US would do its best to prevent it, and even if Anthropic was successful in exfiltrating their data, code, models, and people, I’d imagine the US would immediately block all US companies from working with them. So they’d be blocked from their own US-based compute, plus Google, Amazon, Microsoft, xAI, Meta, etc. 2) Where would they go? China maybe, but as far as we can t…

> the EU is hardly buzzing with AI innovation Depends what you mean. The academic work seems largely... fine? Plenty of good work came out of Europe or European researchers. It seems the problem is more "trying to build a trillion-dollar company of any kind". It's an interesting question: does the EU seek only to regulate successful modern American companies to death, or home grown ones too? Probably not a gamble wor…

The issue with EU is its not one market, for funding or deploying a company of any kind. There is both no funding scale and no easy distribution scale. Same thing with any kind of lobbying you need to do to move laws to be more amenable to tech, you have to do it 20+ times. Nobody does this, which is why smart EU and UK talent migrates to the US, scales, then just uses the weight of their US business to change reality around them in the EU anwyay. For the most part.

Re: Anthropic's Safety Superpower

#156

Post by everforward »

ACCount37 wrote: post #109
christkv wrote: post #70

Earlier quoted context omitted.

For now I suspect however that the gigantic models are not needed and you will be able to do pretty much what you need in a specific domain with 120b or lower. There is so much trash in the frontier models. I don't need all the world's slam poetry for my coding tasks for example.

Wrong, mostly. Model capability is a function of model size. Raising the bar raises model performance in every domain. An "idiot savant" model that's overtrained for a specific domain would beat a generalist model of the same size. But scale the generalist up enough, and it'll trounce the specialist. Removing poetry data from a model training mix doesn't give you much - it might even cost you some performance - and "…

While I disagree with OP about removing stuff from the model, there’s a valid question about tradeoffs between intelligence and price.

Deepseek Flash is almost certainly wrong more often than Opus or Fable. It also costs like 5% as much.

The question becomes if I run Deepseek in a loop to fix the mistakes it made that Opus/Fable didn’t, can it fix its own bugs in few enough tokens that it’s still cheaper?

So far, the answer seems to be “yes, by a significant margin”. A lot of tasks are simple enough that both Deepseek and Opus or Sonnet can one-shot it, which is a huge cost win for Deepseek. Even on the long tail, it’s usually like 4x the tokens on Deepseek which is still way cheaper than Opus.

There are things that Opus can do that Deepseek just won’t ever really nail, but it happens so infrequently that I just don’t worry. Like most people, most of what I do is the same sort of “3 tier app with a React frontend” that doesn’t take a rocket scientist to work out.

Re: Anthropic's Safety Superpower

#157

Post by trollbridge »

everforward wrote: post #153
trollbridge wrote: post #147

Earlier quoted context omitted.

An LLM which provides an OpenAI or Anthropic API-compatible interface + a coding harness like OpenCode or oh-my-pi is a pretty easy "ecosystem" to replicate. Exactly what makes you say Fable or Mythos are "systems, not just pure models"?

Fable can delegate tasks to Opus or Sonnet, so it has some agentic properties and I believe it does them in parallel. The parallelism is where this starts to fall apart on a local PC. Like I can run some Qwen quants, but I can’t run a decent Qwen model while also running another model smart enough to actually implement it. I’d have to do them in series, and given how long Fable seems to take even with parallelism, I’…

oh-my-pi can delegate tasks to other models too. I usually use DS4 Flash for low priority subagent tasks.

If Fable is "delegating" tasks, then there's actually an agent front end of whatever you think the API is.

We have a local instance of Qwen-3.6 which is more than adequate for running agents. You can mix and match local and cloud-hosted models. (My biggest use case for local models right now is vision models because they're quite small and I can avoid some data-locality issues my customers wouldn't be comfortable with if I sen them to a Chinese model.)

Re: Anthropic's Safety Superpower

#158

Post by horsawlarway »

LoganDark wrote: post #30

> The entire Anthropic origin story is rooted in the founders’ belief that OpenAI wasn’t taking safety seriously enough; the company believes that only they can control AI, and that because they uniquely care about safety, they are justified in trying to control everyone else, up to and including the U.S. government. Anthropic believes they have the responsibility to guard their tools from mis-use. That is all. They…

I'm going to challenge this thought.

I think assuming you have the ability to guard a tool (that you're "selling" for profit) from mis-use is the definition of "controlling behavior".

It's the kind of ethically myopic take that can only really exist in this new digital age - where tools aren't actually sold, they're just digitally rented.

The most telling part of the "control" narrative is that they happily classify "competition" as mis-use. We're headed back to serfdom on a speedrun.

Re: Anthropic's Safety Superpower

#159

Post by overfeed »

ACCount37 wrote: post #130
overfeed wrote: post #123

Earlier quoted context omitted.

> Wrong, mostly. > Model capability is a function of model size Model effectiveness has improved across model sizes. You really should try the latest flash variants more. They have become my default for most tasks except for gnarly high-level planning.

"Capability per parameter" is rising, but parameter count remains an advantage. And small models remain bad, because "good" is a rapidly moving target. A 2026 4B beats 2024 4B, but both are far behind the contemporary frontier. Which makes them bad. There is no such thing as "too much capability" - a "good" model is whatever the current frontier is. In 2024, a "good" model is one that can be trusted to write a 800 li…

> A 2026 4B beats 2024 4B, but both are far behind the contemporary frontier.

The thing about engineering is you don't just use the biggest bolt on the market on every bridge.

> In 2024, a "good" model is one that can be trusted to write a 800 line script. In 2026, it's a model that can be trusted to do gnarly high-level planning and execution both

This sounds a lot like having a single diamond-head hammer as the only tool in your toolbox. As suggested by the name, flash models are fast - sometimes I want to write the equivalent of fifty 800-line scripts. There is such a thing as good enough.

Re: Anthropic's Safety Superpower

#160

Post by thewebguyd »

intended wrote: post #60
blitzar wrote: post #43

Earlier quoted context omitted.

the inevitable trend is that numbers will be free and nobody will control the whole thing ai-celebrities are just clinging to relevance like all the other celebrities out there

HN is the builder side of the conversation, and in my experience, few safety people congregate here. The safety side of tech is a PTSD inducing shit show. Governments are more than happy to champion age verification laws, because parents, around the world, are clamoring for anything to pump the breaks on the social media experiment. Society outside of HN is quite tired of Tech, and I despair of figuring out a way to…

> Society outside of HN is quite tired of Tech, and I despair of figuring out a way to make this clear to the commentariat.

s/Tech/Tech Companies

Tech did it to themselves. People like and want technology. What they don't like and don't want more of is enshittified, user hostile technology. The answer is out there, but our collective school systems failed to teach computing irt free software/open source and instead schools themselves all bought in on enshittified, proprietary tech, or even just dumped trying to teach computing at all outside of "how to login to google classroom and google docs"

I grew up lucky, in that my dad was a dev, my first PC as a kid ran red hat, my high school had an intro to programming class (in BAISC lol). It shaped how I approached computing growing up, and my values. It makes me look at the things we have now and think "No, you're just repackaging community free software and selling it back to me, I'll pass on that."

That experience isn't available to anyone born after that specific era, instead their tech experience is shaped by walled gardens, vendor lock-in, and straight up hostile and manipulative software, so its no wonder they are tired of it. They don't even know a different world (of software) exists.

Post reply on HN