Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building an early warning system for LLM-aided biological threat creation

openai.com

Post reply on HN
Print view View original

111–120 of 190 posts

Re: Building an early warning system for LLM-aided biological threat creation

#111

Post by jdewerd »

nextos wrote: post #109
Fomite wrote: post #90

Earlier quoted context omitted.

Pretty much this. Having worked in biopreparedness...the instructions aren't the hard part. Both creating and deploying a biological threat are wildly more difficult.

I don't fully agree. Things are becoming quite simple, which is scary. There's a well-intentioned lobby in the UK advocating for some minimal control, i.e. checking DNA / RNA / protein sequences. For instance, cheap mRNA synthesis as a service has lowered the barrier of entry for any malicious actor. It's frankly ridiculous there are no checks in place right now.

> minimal control, i.e. checking DNA / RNA / protein sequences

They were already doing this 15 years ago.

> It's frankly ridiculous there are no checks in place right now.

It's frankly ridiculous that you think this.

Re: Building an early warning system for LLM-aided biological threat creation

#112

Post by marcellus23 »

zeofig wrote: post #103
pixl97 wrote: post #96

Earlier quoted context omitted.

The probability that Jesus will return tomorrow is extremely low if not zero. When talking about the probability of AI doing X or Y the probability is much closer to 1 if physics allows it.

If physics allows it... a considerable assumption. In my view physics (or mathematics, or whatever) doesn't even allow an LLM to "reason", let alone manufacture novel bioweapons. The probability of Jesus returning is considerable too if we assume the bible is true.

What if you ignore LLMs specifically? I think that's the point that the GP was making. Do you believe it's mathematically impossible for any artificial machine to "reason"? Or just LLMs?

Re: Building an early warning system for LLM-aided biological threat creation

#114

Post by drowsspa »

marcellus23 wrote: post #112
zeofig wrote: post #103

Earlier quoted context omitted.

If physics allows it... a considerable assumption. In my view physics (or mathematics, or whatever) doesn't even allow an LLM to "reason", let alone manufacture novel bioweapons. The probability of Jesus returning is considerable too if we assume the bible is true.

What if you ignore LLMs specifically? I think that's the point that the GP was making. Do you believe it's mathematically impossible for any artificial machine to "reason"? Or just LLMs?

I'm guessing the answer will be soul or something of sorts.

Not even based on the previous comments, even non-religious people seem to have this supernatural view of a soul.

Re: Building an early warning system for LLM-aided biological threat creation

#115

Post by croniev »

apsec112 wrote: post #36

Way too many people extrapolate from "AI Foo can't do task X" to "AIs in general can never do task X", whether X is a good thing (like playing Go) or a bad thing (like helping build bioweapons). AI right now is pretty much the fastest-moving field in human history. GPT-4 can't revolutionize the economy, and it can't commit mass murder, but we simply don't know if GPT-7 can do either of those things (which likely won'…

Would you say that GPT-4 can reason now? I am not convinced this is case, it seems like it has just become more consistent at providing us with an output that we consider reasonable because it was engineered precisely to do that.

Re: Building an early warning system for LLM-aided biological threat creation

#116

Post by nextos »

jdewerd wrote: post #111
nextos wrote: post #109

Earlier quoted context omitted.

I don't fully agree. Things are becoming quite simple, which is scary. There's a well-intentioned lobby in the UK advocating for some minimal control, i.e. checking DNA / RNA / protein sequences. For instance, cheap mRNA synthesis as a service has lowered the barrier of entry for any malicious actor. It's frankly ridiculous there are no checks in place right now.

> minimal control, i.e. checking DNA / RNA / protein sequences They were already doing this 15 years ago. > It's frankly ridiculous there are no checks in place right now. It's frankly ridiculous that you think this.

That's your opinion, but I've seen firsthand that there's a lack of controls in this area.

Re: Building an early warning system for LLM-aided biological threat creation

#117

Post by knightoffaith »

zeofig wrote: post #103
pixl97 wrote: post #96

Earlier quoted context omitted.

The probability that Jesus will return tomorrow is extremely low if not zero. When talking about the probability of AI doing X or Y the probability is much closer to 1 if physics allows it.

If physics allows it... a considerable assumption. In my view physics (or mathematics, or whatever) doesn't even allow an LLM to "reason", let alone manufacture novel bioweapons. The probability of Jesus returning is considerable too if we assume the bible is true.

Why doesn't physics or mathematics allow LLMs to reason?

Re: Building an early warning system for LLM-aided biological threat creation

#118

Post by Nifty3929 »

croniev wrote: post #115
apsec112 wrote: post #36

Way too many people extrapolate from "AI Foo can't do task X" to "AIs in general can never do task X", whether X is a good thing (like playing Go) or a bad thing (like helping build bioweapons). AI right now is pretty much the fastest-moving field in human history. GPT-4 can't revolutionize the economy, and it can't commit mass murder, but we simply don't know if GPT-7 can do either of those things (which likely won'…

Would you say that GPT-4 can reason now? I am not convinced this is case, it seems like it has just become more consistent at providing us with an output that we consider reasonable because it was engineered precisely to do that.

It’s definitely not the case. LLMs of any sort do not in any sense reason or understand anything.

They literally just make stuff up (technically just a continuation of whatever you fed in), which usually sounds good, is often true and sometimes even helpful. Because those are qualities of the training data that was used and is the basis for the stuff it’s making up.

Re: Building an early warning system for LLM-aided biological threat creation

#119

Post by XorNot »

nextos wrote: post #109
Fomite wrote: post #90

Earlier quoted context omitted.

Pretty much this. Having worked in biopreparedness...the instructions aren't the hard part. Both creating and deploying a biological threat are wildly more difficult.

I don't fully agree. Things are becoming quite simple, which is scary. There's a well-intentioned lobby in the UK advocating for some minimal control, i.e. checking DNA / RNA / protein sequences. For instance, cheap mRNA synthesis as a service has lowered the barrier of entry for any malicious actor. It's frankly ridiculous there are no checks in place right now.

...what do you think a vial of mRNA actually does?

I'll give you a hint: if you touched it to your finger, it would be completely destroyed.

Re: Building an early warning system for LLM-aided biological threat creation

#120

Post by XorNot »

printerphobia wrote: post #66

>biological threat creation process (ideation, acquisition, magnification, formulation, and release) I remember watching a hacker/programmer who was livestreaming how to datamine the rna of the corona virus when covid first started. One of the crazy things he rambled about was how cheap it is for a layman to download a copy of it and synthesize it with some rna printing service. I haven't thought about that possibili…

The sad thing about the pandemic is the number of people who still don't know what a virus is, and that it's not "DNA".
Post reply on HN