Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Detecting and countering misuse of AI: September 2026

anthropic.com

Post reply on HN
Print view View original

171–180 of 262 posts

Re: Detecting and countering misuse of AI: September 2026

#171

Post by thephyber »

iAMkenough wrote: post #143
tedsanders wrote: post #17

Meta: The potential proliferation of biological weapons is serious. Millions could die. It's easy to joke about before it happens, but try to imagine how this thread might look after the successful deployment of a biological weapon by a rogue state or non-state actor. I encourage you to take this topic seriously and contribute posts that add new information or perspectives to the discussion. (I myself think the odds…

Meta: These unregulated companies can't/won't properly prevent their technology from committing felonies against other companies. Why should we trust them to control bioweapon development without regulations? They themselves are non-state actors.

The question isn't only "should we" regulate the companies, but also "how"?

The current Trump Admin can't even decide on the first question, let alone come up for a plan on the second.

The Trump Admin's EO was to ask nicely all of the AI companies to give them 30 days to voluntarily review each model before wide release, but they have also failed to do that for the Mythos/Fable release, only to get a call from Amazon's CEO to David Sachs to convince them to disable Fable (to all non-US citizens).

We elected the party of "minimum regulations" into all 3 branches of government and we will reap what we sewed.

Re: Detecting and countering misuse of AI: September 2026

#172

Post by AustinDev »

thephyber wrote: post #168
AustinDev wrote: post #92

Earlier quoted context omitted.

If the five eyes intelligence services aren't monitoring all biodefense researchers and bio chemists around the world all of the time they've probably failed in their jobs.

The CIA, FBI, and DoD are being ideologically purged. I would assume we are already in the "we are probably increasingly failing at our jobs" phase. It will only get worse as sovereign debt service takes a larger piece of the budget pie.

The five eyes have four other eyes. But, from where I'm sitting the primary eye has only gotten stronger in the last 2 years. See some work we did in late 00's to early 2010's called Nexus 7 and extrapolate to modern capabilities.

Re: Detecting and countering misuse of AI: September 2026

#173

Post by Ancapistani »

Scea91 wrote: post #85
BobbyJo wrote: post #69

Earlier quoted context omitted.

Is your argument that the resulting birth defects were intentional? If not, I'm not sure where you're going with your response.

The civilian cost was intentionally not part of the equation. The toxic effects were known and the numbers show that it was not a few isolated accidents but a intentional disregard for the human cost.

The toxic effects also impacted the area where I grew up here in the Southern US - there’s a road that’s known locally as “Chemical Ranch Rd” where there’s a ~100 acre property that was used for testing Agent Orange.

Re: Detecting and countering misuse of AI: September 2026

#175

Post by nullbio »

m-hodges wrote: post #88

> We discovered that Moonshot AI, the company that produces the Kimi family of models, silently forwarded customer requests to Claude, instead of processing them using Kimi. Moonshot then displayed Claude’s responses to users. These users thought they were using a Kimi model, but received responses from Claude instead. > DeepSeek also silently relayed exchanges to Claude without informing DeepSeek customers. > MiniMa…

Consider me incredibly skeptical of any of these claims.

Re: Detecting and countering misuse of AI: September 2026

#176

Post by nullbio »

tedsanders wrote: post #17

Meta: The potential proliferation of biological weapons is serious. Millions could die. It's easy to joke about before it happens, but try to imagine how this thread might look after the successful deployment of a biological weapon by a rogue state or non-state actor. I encourage you to take this topic seriously and contribute posts that add new information or perspectives to the discussion. (I myself think the odds…

Bioweapons are the most difficult thing to pull off out of all possible attack vectors. A rogue nation state could obviously do it, but they could also do any other number of equally harmful attacks, and none of it requires AI.

A non-state actor is another thing entirely. It requires the right lab equipment, the right lab know-how, the ability to develop the weapon without killing yourself, the ability to not get caught, the ability to buy the right regulated materials, and the list goes on.

Yeah, the world should tighten the security of that industry, but let's not make this an AI fearmonger talking point.

Re: Detecting and countering misuse of AI: September 2026

#177

Post by nullbio »

thephyber wrote: post #169
mekael wrote: post #163

Earlier quoted context omitted.

There are already scores of people capable of building biological weapons or doing GOF work which ends up being essentially the same thing. There’s no need for an LLM, just a phd in virology, a decent lab, and a handful of grad students. [0] All of which, rogue actors have easy access to and could have accomplished for a few millions dollars anytime in the last decade. And yet, we aren’t drowning in our own blood whi…

> There’s no need for an LLM, just a phd in virology, a decent lab, and a handful of grad students. You are missing the forest for the trees. The existing virologist PhDs and the GoF labs are a scarce resource. The model makes the same scarce knowledge accessible to many more malicious actors.

There's > 6 thousand of these labs. I wouldn't really consider that scarce considering the implication.

Re: Detecting and countering misuse of AI: September 2026

#178

Post by nullbio »

hnburnsy wrote: post #102

Quite the double standard here... Conventional Weapons -We identified a cell of threat actors based in northern Yemen -We identified a China-based threat actor who used Claude -We identified likely freelance Russia-based threat actors -We identified a China-based actor who used Claude’s chat -In this case, a Russia-based actor used Claude -We identified a China-based threat actor who used Claude Biological misuse We…

Anthropic:

- We identified someone building a death star with Claude

- We identified someone building a wormhole with Claude

- We identified someone building a blackhole with Claude

- We identified someone building a quantum drive with Claude

We are withholding all evidence though, sorry. Just trust us, it's really bad out there and Claude is really powerful.

Re: Detecting and countering misuse of AI: September 2026

#179

Post by nullbio »

Stevvo wrote: post #76

I don't get it. Surely if you were developing novel biological weapons, you would not use a hosted AI service where Anthropic can read what you are doing. And, why would you need to? Any chemistry graduate could make you dozens of highly effective, proven chemical weapons and explosives.

Step 1: Work for Anthropic

Step 2: Send "how do I make a nuke and a killer virus" to Claude through a Chinese proxy to Claude

Step 3: Send screenshots to congress and ask them to regulate open-weight models into the ground

Re: Detecting and countering misuse of AI: September 2026

#180

Post by nullbio »

sakopov wrote: post #37

I have a conspiracy theory that Anthropic is very busy pumping their moat prior to IPO. There are absolutely wild rumors swirling on X including one about Anthropic AI research solving cancer treatments for all types of cancer.

This is common knowledge, far from conspiracy.
Post reply on HN