Live data from Hacker News

“Too dangerous to release” or just too expensive?

kingy.ai

161–170 of 189 posts

Re: “Too dangerous to release” or just too expensive?

#161

Earlier quoted context omitted.

How does delaying the release not solve anything? It puts everyone on a notice to fix all security vulnerabilities now

Because the only thing keeping those vulnerabilities in existence was laziness.

"laziness" is an interesting reframing of "rational cost-benefit analysis and the limits of the human mind".

Re: “Too dangerous to release” or just too expensive?

#162
post #71
post #53

(I work at Anthropic) We have publicly stated[1] that our goal is to deploy Mythos-class models at scale when we have the requisite safeguards for offensive cyber risks in place. Mythos is a general frontier model, not a cyber-specific model so there are many reasons why we think our users will benefit from access (with the aforementioned safeguards in place) in due course. Compute has also not factored into our deci…

Multiple people who have already used Mythos or been given its reports on their software have publicly stated that it's all hype, and that it is not really finding any new critical bugs which other models cant.

Of course if it really is overhyped, then it becomes much more difficult to release it publicly. Better to retain the mystique and release the next thing. But we'll see eventually.

Re: “Too dangerous to release” or just too expensive?

#163
post #111

Earlier quoted context omitted.

He's made so many statements that fall under the "boy who cried wolf" category that even if he _does_ believe these statements he needs to be managed better. I'll never forget Anthropic's huge "Oh my God, the AI blackmailed a researcher to save itself!" and the prompt effectively told the AI to do that and gave it forged emails with easy blackmail targets, as if this isn't a common trope in mystery or suspense books/…

It's a common trope, all through the training data, and all the modern AIs have read it, and would probably act similarly? Is that what we should take away from your comment? so we have nothing to worry about. Makes sense. Really, it's just a common trope.

Oh of course wolves have sharp teeth, they're predators. Anyone know knows this can never be bitten.

Re: “Too dangerous to release” or just too expensive?

#164
post #71
post #53

(I work at Anthropic) We have publicly stated[1] that our goal is to deploy Mythos-class models at scale when we have the requisite safeguards for offensive cyber risks in place. Mythos is a general frontier model, not a cyber-specific model so there are many reasons why we think our users will benefit from access (with the aforementioned safeguards in place) in due course. Compute has also not factored into our deci…

Multiple people who have already used Mythos or been given its reports on their software have publicly stated that it's all hype, and that it is not really finding any new critical bugs which other models cant.

In this very thread we have a counter-example. What to think? https://news.ycombinator.com/item?id=48149519

Re: “Too dangerous to release” or just too expensive?

#165

It's pretty clear at this point that Mythos' capability to discover and exploit zero-day vulnerabilities at scale is but an incremental improvement over existing models like the ones available to OpenAI's Plus/Pro subscribers. Anthropic tries to create marketing hype around Mythos using two psychological tricks. 1. Put large numbers in the headlines. "Mythos discovered 271 vulnerabilities in Firefox" makes the model…

> an incremental improvement

I've had to reboot my systems quite a bit more than an incremental improvement would suggest this week

Re: “Too dangerous to release” or just too expensive?

#166
post #93

Earlier quoted context omitted.

I'm fairly certain Amodei believes the "too dangerous to release" hype himself. Even if it's just an incremental improvement, better than getting frog-boiled by repeated 20% improvements until someone builds bioweapons in their backyard.

* sigh * Three things: * Delaying the release accomplishes nothing. * The barrier to someone building/not-building a bioweapon in their backyard is not access to an LLM. * Remember when GPT 3.5 was going to destroy the world? And how it was conscious? And how it was "trying to escape"? Lmao.

You're right, it's silly for me to worry. We've never had a technology that initially appeared benign but turned into a big problem. In fact, no tech company has ever released technologies that cause problems for the rest of society AT ALL. /s

What are the other barriers? Last I checked access to CRISPR is not especially tightly regulated. Even if it is, defense in depth is a thing.

Re: “Too dangerous to release” or just too expensive?

#167
post #115

Earlier quoted context omitted.

Do you have any good sources on that? I have seen things to suggest that not all of the hype is true, but so far I have not encountered anyone claiming all of the hype is untrue. Which is what I interpret "its all hype" (sic) to mean.

CURL has been scanned with multiple LLMs. Mythos was last and as a result found only 1 issue. If Myhos was really much better I'd expect it to find a lot more issues despite the others already there. Also, the competing models are getting better. Opus 4.5 was better than everyone else when it was new, but only a few months later and there are a lot of models that are better (not just the newer Opus models)

Curl had a prominent bug bounty programme, has 180k lines of prod code, and is mainly a client app/lib. I would look at other projects before making judgements about mythos on this one.

Re: “Too dangerous to release” or just too expensive?

#168
post #124

It's pretty clear at this point that Mythos' capability to discover and exploit zero-day vulnerabilities at scale is but an incremental improvement over existing models like the ones available to OpenAI's Plus/Pro subscribers. Anthropic tries to create marketing hype around Mythos using two psychological tricks. 1. Put large numbers in the headlines. "Mythos discovered 271 vulnerabilities in Firefox" makes the model…

>Do the whole "too dangerous to release" shtick. One aspect that isn't really discussed much in this context is how to wrap one's head around the corporate risk with models of ever increasing capability. It might not be too dangerous to society, but it could be too dangerous to Anthropic.

[deleted]

Re: “Too dangerous to release” or just too expensive?

#169
post #166

Earlier quoted context omitted.

* sigh * Three things: * Delaying the release accomplishes nothing. * The barrier to someone building/not-building a bioweapon in their backyard is not access to an LLM. * Remember when GPT 3.5 was going to destroy the world? And how it was conscious? And how it was "trying to escape"? Lmao.

You're right, it's silly for me to worry. We've never had a technology that initially appeared benign but turned into a big problem. In fact, no tech company has ever released technologies that cause problems for the rest of society AT ALL. /s What are the other barriers? Last I checked access to CRISPR is not especially tightly regulated. Even if it is, defense in depth is a thing.

If it was as easy as "knowing how to" someone would've already done it or at least attempted to.*

Plenty of people know how to, 10,000s of researchers, perhaps you know someone who does.

Did you know that your local veterinary shop has enough drugs to kill 100s of people?

Why doesn't it happen?

* It's not that easy.

* There's a ton of regulation that is hard to circumvent, on purpose.

* There's a gigantic deterrent called "spend the rest of your life behind bars" that people tend to avoid.

An LLM, even the most advanced one, does not make any material change in any of these. You cannot bullshit your way into "uhh, I need Ebola samples for ... reasons".

Unironically, your Sunday movie portraying a super-villain jeopardizing a city with his "home lab" full of flasks with colored liquids and BioHazard signs push way more people into becoming interested on this than having access to an LLM.

*: Okay, like 5 people, and way before LLMs were a thing. This has been a thing for decades, we're fine.

Re: “Too dangerous to release” or just too expensive?

#170
post #121
post #115

Earlier quoted context omitted.

Do you have any good sources on that? I have seen things to suggest that not all of the hype is true, but so far I have not encountered anyone claiming all of the hype is untrue. Which is what I interpret "its all hype" (sic) to mean.

For example, It was recently let loose on cURL and its maintainer is less than impressed: https://www.theregister.com/security/2026/05/11/anthropics-b...

If you remove the fluff that the register added and stick with https://daniel.haxx.se/blog/2026/05/11/mythos-finds-a-curl-v... it seems like a claim that's a claim fairly distant to "its all hype". Less than expected perhaps? Maybe the code really is unexpectedly robust? I guess time will tell on that point.
Post reply on HN