[flagged]
Detecting and countering misuse of AI: September 2026
181–190 of 262 posts
Re: Detecting and countering misuse of AI: September 2026
#182Earlier quoted context omitted.
Ah, that makes way more sense than Anthropic's (probably deliberately misleading) insinuation that Moonshot has been burning millions of dollars in Claude API credits by swapping in a slightly better but infinitely more expensive model just to trick their users. I get those A/B responses chatting in Gemini fairly often, and I really don't think I'd feel deceived if I later learned one of the choices was actually from…
I don’t think it was misleading, deliberately or otherwise. Did you read the report? I hate to call you out like that but I think you can only get that impression if you only read the above quotes. That’s not the insinuation I get at all. It’s specifically under the “illicit distillation” category. It’s never framed in anyway but as a form of distillation. I think they are pretty fair and explicitly say “Distillation…
They mean distillation is legitimate when labs use one of their own stronger models to train a smaller one. They certainly aren’t advocating for PRC labs to distill Claude for open weight models.
Re: Detecting and countering misuse of AI: September 2026
#183Meta: The potential proliferation of biological weapons is serious. Millions could die. It's easy to joke about before it happens, but try to imagine how this thread might look after the successful deployment of a biological weapon by a rogue state or non-state actor. I encourage you to take this topic seriously and contribute posts that add new information or perspectives to the discussion. (I myself think the odds…
Bioweapons are the most difficult thing to pull off out of all possible attack vectors. A rogue nation state could obviously do it, but they could also do any other number of equally harmful attacks, and none of it requires AI. A non-state actor is another thing entirely. It requires the right lab equipment, the right lab know-how, the ability to develop the weapon without killing yourself, the ability to not get cau…
I'm also confused why everyone is so much more afraid of that than... say... a group of terrorists dumping millions of nails into a few major highways across the country. Easy to pull off, doesn't require coordination, cheap, causes billions of dollars of damage and makes people think the terrorists could be anywhere. Like what are they going to do, arrest every construction worker?
Re: Detecting and countering misuse of AI: September 2026
#184Never seen a group of people more addicted to drama.
Is this what they call “collective psychosis?”
Re: Detecting and countering misuse of AI: September 2026
#185Meta: The potential proliferation of biological weapons is serious. Millions could die. It's easy to joke about before it happens, but try to imagine how this thread might look after the successful deployment of a biological weapon by a rogue state or non-state actor. I encourage you to take this topic seriously and contribute posts that add new information or perspectives to the discussion. (I myself think the odds…
How do you reconcile this supposed caution with getting filthy rich in the process of exposing your loved ones to this risk? Did you only recently become aware of this risk?
And semi-related, how do you reconcile this caution with the recklessness on display in recent incidents like the Navier-Stokes drama?
Re: Detecting and countering misuse of AI: September 2026
#186Meta: The potential proliferation of biological weapons is serious. Millions could die. It's easy to joke about before it happens, but try to imagine how this thread might look after the successful deployment of a biological weapon by a rogue state or non-state actor. I encourage you to take this topic seriously and contribute posts that add new information or perspectives to the discussion. (I myself think the odds…
"Astra, warp in a wet lab and make COVID++ .NET edition, make no mistakes."
Yeah, sure man.
Re: Detecting and countering misuse of AI: September 2026
#187Meta: The potential proliferation of biological weapons is serious. Millions could die. It's easy to joke about before it happens, but try to imagine how this thread might look after the successful deployment of a biological weapon by a rogue state or non-state actor. I encourage you to take this topic seriously and contribute posts that add new information or perspectives to the discussion. (I myself think the odds…
Immigration & travel controls > Information controls
Information wants to be free! :^)
(Also natives want to be paid well.)
Re: Detecting and countering misuse of AI: September 2026
#188Meta: The potential proliferation of biological weapons is serious. Millions could die. It's easy to joke about before it happens, but try to imagine how this thread might look after the successful deployment of a biological weapon by a rogue state or non-state actor. I encourage you to take this topic seriously and contribute posts that add new information or perspectives to the discussion. (I myself think the odds…
How do you reconcile this supposed caution with getting filthy rich in the process of exposing your loved ones to this risk? Did you only recently become aware of this risk? And semi-related, how do you reconcile this caution with the recklessness on display in recent incidents like the Navier-Stokes drama?
“if I don’t destroy the world well someone else will so it’s porobably better that I destroy the world because I’m not evil”
Re: Detecting and countering misuse of AI: September 2026
#189Earlier quoted context omitted.
Source? Would love to read more about this.
I misremembered, it’s actually over 90%, and it involved camouflage: https://drive.google.com/file/d/1hNUnU8i2yubt5uesmmV17aTJXhY...
The circumstances under which they were able to get the synthesized DNA this way would not work in the general case [1]
>Multiple IGSC member companies detected the ordered sequence and determined the order to be legitimate as defined in the 2023 guidance. Specifically, the orders were placed on behalf of SecureBio, an organization known to IGSC member companies given the role played by SecureBio in the SecureDNA project, an effort to build a DNA synthesis screening system. In addition, the name on the orders was an individual who has co-published multiple times with Esvelt, an individual well known to IGSC companies to work in viral evolution and who is known to have access to laboratory facilities sufficient to work safely with the ordered material.
>In short, the system worked as designed: a legitimate individual ordered DNA sequence that, by itself, posed no risk of misuse, for delivery to a company associated with legitimate scientific contributions directly relevant to the sequence that was ordered.
[1] https://thebulletin.org/2024/06/why-a-misleading-red-team-st...
Re: Detecting and countering misuse of AI: September 2026
#190Earlier quoted context omitted.
DeepSeek, minimax and so on have razer thin margins but unlike openai and Anthropic they are actually making some profit. Doing this doesn't make any financial sense. Maybe Anthropic is confusing Chinese AI providers with token resellers using the same alibaba infrastructure? Or maybe something like openrouter was switching between operators depending on price/demand/availability? Also, how can Anthropic have such ac…
I’ve seen the supposed Kimi thinking output yap about Anthropic’s guidelines and whatnot on many occasions - could also be the result of distillation, but also that straight up being Claude’s output. To be honest I've also gotten Kimi to do an okay proof of concept for SQLi though mostly in a more defensive role, like "Let's see how big of a problem this is", while Claude complained about CVP on the same task.
They all do it. If you ask Claude which model it is in Chinese, it says DeepSeek or Qwen.