Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building an early warning system for LLM-aided biological threat creation

openai.com

Post reply on HN
Print view View original

91–100 of 190 posts

Re: Building an early warning system for LLM-aided biological threat creation

#91

Post by zeofig »

apsec112 wrote: post #36

Way too many people extrapolate from "AI Foo can't do task X" to "AIs in general can never do task X", whether X is a good thing (like playing Go) or a bad thing (like helping build bioweapons). AI right now is pretty much the fastest-moving field in human history. GPT-4 can't revolutionize the economy, and it can't commit mass murder, but we simply don't know if GPT-7 can do either of those things (which likely won'…

On the contrary... way too many people see "AI can do X", "AI can do Y", "I can draw an imaginary line between X, Y and Z", "therefore AI can totally maybe do Z". The fact that we "just don't know" doesn't mean anything. You just don't know if Jesus will return tomorrow and bring the rapture.

Re: Building an early warning system for LLM-aided biological threat creation

#93

Post by Fomite »

bytesandbots wrote: post #33

It might be too far, but to me this piece seems aimed at increasing concerns among regulators about AI. OpenAI might view regulation as a means of ensuring their competitive edge as other big players enter the AI space.

To me this reads like a lot of grant applications after 9/11 where researchers scribbled out whatever they were working on and wrote about bioterrorism-related diseases instead.

The number of projects I had to sit through about developing sophisticated methods to detect smallpox attacks, when the actual answer is "Smallpox is extinct. If you find one case, it's an emergency", were...myriad.

Re: Building an early warning system for LLM-aided biological threat creation

#94

Post by Fomite »

printerphobia wrote: post #66

>biological threat creation process (ideation, acquisition, magnification, formulation, and release) I remember watching a hacker/programmer who was livestreaming how to datamine the rna of the corona virus when covid first started. One of the crazy things he rambled about was how cheap it is for a layman to download a copy of it and synthesize it with some rna printing service. I haven't thought about that possibili…

This has been a thing people have been worried about for my entire career, and it has never manifested as a real threat. "Step 1" in creating a bioweapon is pretty easy, but there's a whole pathway between that and a deployable weapon.

Re: Building an early warning system for LLM-aided biological threat creation

#95

Post by fatherzine »

margalabargala wrote: post #84
fatherzine wrote: post #82

Earlier quoted context omitted.

[flagged]

> O(undergrad with lab access) = 10k You're off by at least 3 orders of magnitude. There are 18 million undergrads in the US alone. I would bet that at least half have some sort of basic bio lab class and can use a lab, or has a friend that does and would let them in. There's zero security in undergrad bio labs.

undergrad in molecular biology = 4k. assuming eg CS undergrads don't attend en-masse molecular biology classes and/or have lab access.

edit: and one obvious error, it's graduates with molecular biology degrees, presumably the undergrad population is ~5x larger, for a total of 20k. and presumably the world is larger, so we can throw in another ~5x factor for a total of 100k. still, /orders of magnitude/ smaller than world population with access to the Internet.

there are a /lot/ of broken people in the world. i'd sleep more comfortable they don't get access to step by step instructions in their local language on how to design and spread the next .*pox plague at the cost less than eg a motorcycle.

https://datausa.io/profile/cip/cellular-molecular-biology

Re: Building an early warning system for LLM-aided biological threat creation

#96

Post by pixl97 »

zeofig wrote: post #91
apsec112 wrote: post #36

Way too many people extrapolate from "AI Foo can't do task X" to "AIs in general can never do task X", whether X is a good thing (like playing Go) or a bad thing (like helping build bioweapons). AI right now is pretty much the fastest-moving field in human history. GPT-4 can't revolutionize the economy, and it can't commit mass murder, but we simply don't know if GPT-7 can do either of those things (which likely won'…

On the contrary... way too many people see "AI can do X", "AI can do Y", "I can draw an imaginary line between X, Y and Z", "therefore AI can totally maybe do Z". The fact that we "just don't know" doesn't mean anything. You just don't know if Jesus will return tomorrow and bring the rapture.

The probability that Jesus will return tomorrow is extremely low if not zero.

When talking about the probability of AI doing X or Y the probability is much closer to 1 if physics allows it.

Re: Building an early warning system for LLM-aided biological threat creation

#97

Post by hospadar »

It feels so disingenuous seeing stuff like this come out of openai - like when altman was making sounds about how ai is maybe oh so dangerous (which maybe was just a move for regulatory capture?).

"this thing we sell might destroy humanity?!"

"but yeah we're gonna keep making it cause we're making fat stacks from it"

Is the move here just trying to seem like the good guy when you're making a thing that, however much good it might do, is almost certainly going to do a lot of damage as well? I'm not totally anti-ai, but this always smells a little of the wolves guarding the henhouse.

I wonder if this is what it felt like back when we thought everything was going to be nuclear powered? "Guys we made this insane super weapon!! It could totally power your car!! if it leaks it'll destroy all life but hey you only have to fill the tank once every 10 years!!"

Re: Building an early warning system for LLM-aided biological threat creation

#98

Post by groggo »

pixl97 wrote: post #96
zeofig wrote: post #91

Earlier quoted context omitted.

On the contrary... way too many people see "AI can do X", "AI can do Y", "I can draw an imaginary line between X, Y and Z", "therefore AI can totally maybe do Z". The fact that we "just don't know" doesn't mean anything. You just don't know if Jesus will return tomorrow and bring the rapture.

The probability that Jesus will return tomorrow is extremely low if not zero. When talking about the probability of AI doing X or Y the probability is much closer to 1 if physics allows it.

I appreciate that you acknowledge there's at least a chance.

Re: Building an early warning system for LLM-aided biological threat creation

#99

Post by danielmarkbruce »

sjwhevvvvvsj wrote: post #87
danielmarkbruce wrote: post #74

Earlier quoted context omitted.

Yup, plenty of reasons to run your own model. I'm not using GPT-4 for chat, but for what I'd class as "reasoning" applications. It seems best by a long shot. As for safety, I find with the api and the system prompt that there is nothing it won't answer for me. That being said... I'm not asking for anything weird. GPT-4 turbo does seem to be reluctant sometimes.

I’m doing document summarization and classification, and there’s a fair amount it won’t do for prudish reasons (eg sex, porn, etc). Llama2 is basically useless in that regard.

with GPT-4 (non-turbo) and a good system prompt?

Re: Building an early warning system for LLM-aided biological threat creation

#100

Post by dylan604 »

elzbardico wrote: post #4

Open AI is clearly overestimating the capabilities of its product. It is kind of funny actually.

Open AI is clearly overestimating the capabilities of its current product

I'm sure they recognize this, and have decided that anything that comes out now would be much more favorable for them based on their current capabilities

Post reply on HN