Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building an early warning system for LLM-aided biological threat creation

openai.com

Post reply on HN
Print view View original

141–150 of 190 posts

Re: Building an early warning system for LLM-aided biological threat creation

#141

Post by Terr_ »

solarpunk wrote: post #39
apsec112 wrote: post #36

Way too many people extrapolate from "AI Foo can't do task X" to "AIs in general can never do task X", whether X is a good thing (like playing Go) or a bad thing (like helping build bioweapons). AI right now is pretty much the fastest-moving field in human history. GPT-4 can't revolutionize the economy, and it can't commit mass murder, but we simply don't know if GPT-7 can do either of those things (which likely won'…

we better make sure microsoft devotes all funding earmarked for openai to ensuring future versions of their product are incapable of creating bad outcomes.

Meh, the humans at Microsoft already create bad outcomes on purpose--for consumers anyway.

Re: Building an early warning system for LLM-aided biological threat creation

#142

Post by foo3a9c4 »

habinero wrote: post #138
foo3a9c4 wrote: post #130

Earlier quoted context omitted.

> It’s definitely not the case. LLMs of any sort do not in any sense reason or understand anything. This seems like a claim about the way that the LLM neural net algorithm works. But AFAIK no one has a good understanding of how the LLM NNs work. Why are you so certain that the LLM NN isn't doing the reasoning-algorithm or the understanding-algorithm?

Neural networks are not new, and they're just mathematical systems. LLMs don't think. At all. They're basically glorified autocorrect. What they're good for is generating a lot of natural-sounding text that fools people into thinking there's more going on than there really is.

> Neural networks are not new

I agree. The McCullough-Pitts paper was published in 1943.

> they're just mathematical systems.

What do you mean by "mathematical system"? AFAIK the GPT4 model is literally a computer program.

> LLMs don't think. At all.

This is the same assertion that OP made and I'm still confused as to how anyone could be certain of its truth given that no one actually knows what is going on inside of the GPT4 program.

> They're basically glorified autocorrect. What they're good for is generating a lot of natural-sounding text that fools people into thinking there's more going on than there really is.

Is that an argument for the claim "LLMs don't think."? It doesn't seem like it to me, but maybe I'm mistaken.

Re: Building an early warning system for LLM-aided biological threat creation

#143

Post by Vecr »

XorNot wrote: post #119
nextos wrote: post #109

Earlier quoted context omitted.

I don't fully agree. Things are becoming quite simple, which is scary. There's a well-intentioned lobby in the UK advocating for some minimal control, i.e. checking DNA / RNA / protein sequences. For instance, cheap mRNA synthesis as a service has lowered the barrier of entry for any malicious actor. It's frankly ridiculous there are no checks in place right now.

...what do you think a vial of mRNA actually does? I'll give you a hint: if you touched it to your finger, it would be completely destroyed.

You don't want RNA anyway, you want to turn the virus's RNA sequence into a DNA sequence (reverse transcription), then do whatever editing you want once that's in your computer, then synthesize the DNA, and grow up the virus in culture by introducing the DNA. The cells in the cell culture will turn the DNA into RNA for you and then assemble the viruses.

Re: Building an early warning system for LLM-aided biological threat creation

#144

Post by mkarrmann »

kmeisthax wrote: post #126

Doesn't this completely deflate the selling point of AI? They force-fed a model the entire Internet and only got a statistically insignificant improvement over human performance.

What makes you think that the "selling point" of AI today is that it is significantly better at everything than humans?

Re: Building an early warning system for LLM-aided biological threat creation

#145

Post by Vecr »

zeofig wrote: post #91
apsec112 wrote: post #36

Way too many people extrapolate from "AI Foo can't do task X" to "AIs in general can never do task X", whether X is a good thing (like playing Go) or a bad thing (like helping build bioweapons). AI right now is pretty much the fastest-moving field in human history. GPT-4 can't revolutionize the economy, and it can't commit mass murder, but we simply don't know if GPT-7 can do either of those things (which likely won'…

On the contrary... way too many people see "AI can do X", "AI can do Y", "I can draw an imaginary line between X, Y and Z", "therefore AI can totally maybe do Z". The fact that we "just don't know" doesn't mean anything. You just don't know if Jesus will return tomorrow and bring the rapture.

https://news.ycombinator.com/item?id=39173034 If this paper replicates, LLM-like large transformer models can approximate Solomonoff induction pretty well, with other neural net types showing clear evidence of generalizing outside the training distribution. If substantial efficiency improvements get implemented and new architectures get developed, that's game over. The AI is smarter than you, and can trivially be made into an agent by having the approximate Solomonoff induction section simulate the behavior of an agent in the real world, and then having another section change the real world to match the simulation by controlling the real-world agent, creating a full cybernetic system, a closed loop.

Re: Building an early warning system for LLM-aided biological threat creation

#146

Post by nradov »

printerphobia wrote: post #66

>biological threat creation process (ideation, acquisition, magnification, formulation, and release) I remember watching a hacker/programmer who was livestreaming how to datamine the rna of the corona virus when covid first started. One of the crazy things he rambled about was how cheap it is for a layman to download a copy of it and synthesize it with some rna printing service. I haven't thought about that possibili…

LLMs change nothing. The sequences for various viruses are going to be published or leaked whether anyone involved uses LLMs or not. It's a total red herring.

Re: Building an early warning system for LLM-aided biological threat creation

#147

Post by Vecr »

bithive123 wrote: post #65
digging wrote: post #48

Earlier quoted context omitted.

It's always really embarrassing to come to these comment sections and see a lot of smart people talk about how they're not being fooled by the "marketing hype" of existential AI risk. Literally the top of the page is saying that they have no conclusive evidence that ChatGPT could actually increase the risk of biological weapons. They are undertaking this effort because the question of how to stop any AI from ending h…

Asking if humanity can invent a machine to protect itself from the existential threats created by the other machines it has invented to me does not sound that intelligent. This is always the pattern; the "greatest minds" invent a machine to kill, then another great technician invents a machine to kill that machine, and so on. Presuming that total security can be achieved mechanically, and in that pursuit bringing abo…

> Asking if humanity can invent a machine to protect itself from the existential threats created by the other machines it has invented to me does not sound that intelligent.

> Humanity can barely manage the existential risks for which it is not responsible

Just skip to carpet bombing datacenters then?

Re: Building an early warning system for LLM-aided biological threat creation

#148

Post by Vecr »

solarpunk wrote: post #60
digging wrote: post #48

Earlier quoted context omitted.

It's always really embarrassing to come to these comment sections and see a lot of smart people talk about how they're not being fooled by the "marketing hype" of existential AI risk. Literally the top of the page is saying that they have no conclusive evidence that ChatGPT could actually increase the risk of biological weapons. They are undertaking this effort because the question of how to stop any AI from ending h…

what's the premise here? this thing will become iteratively better until it could potentially be capable of bad outcomes? if that's the case, how much resources do you think should be dedicated to regulating it? more or less than currently identified existential risks? which entities should be paying for the regulatory controls? what's the proposal here? it's odd because only this one single company that is hedging i…

> why aren't they demanding nvidia start building DRM enabled thermite charges into A100s?

Tweet that at Yudkowsky, he'll probably endorse it.

Re: Building an early warning system for LLM-aided biological threat creation

#149

Post by [deleted] »

pushfoo wrote: post #139
jefftk wrote: post #121

Earlier quoted context omitted.

>> minimal control, i.e. checking DNA / RNA / protein sequences > They were already doing this 15 years ago. I work in this area. Some synthesis providers check, others don't. And the checking isn't great. (Some of my coworkers work on https://securedna.org which is trying to make this screening more robust.)

TL;DR: I don't understand how adding homomorphic encryption makes a cloud virus scanner for physical pathogens a better idea > Only authorized researchers should be able to obtain DNA permitting them to assemble pandemic-capable agents. Imagine this becomes legally mandatory. What happens when they get breached? For example: * Does it enable denial-of-service attacks by returning false positives hospitals legally can…

[deleted]

Re: Building an early warning system for LLM-aided biological threat creation

#150

Post by XorNot »

Vecr wrote: post #143
XorNot wrote: post #119

Earlier quoted context omitted.

...what do you think a vial of mRNA actually does? I'll give you a hint: if you touched it to your finger, it would be completely destroyed.

You don't want RNA anyway, you want to turn the virus's RNA sequence into a DNA sequence (reverse transcription), then do whatever editing you want once that's in your computer, then synthesize the DNA, and grow up the virus in culture by introducing the DNA. The cells in the cell culture will turn the DNA into RNA for you and then assemble the viruses.

Also the whole "testing" part. Unless you're just culturing an already known virus (in which case, why do you need to synthesize anything? You have it in a test tube or something) then there is not in fact any real way to know whether or not some particular genetic modification translates to any type of favorable property. Even gene-linked desirable traits in one species may not correspond to that trait in another, particularly when your metrics are things like "transmissivity" and "lethality" and not "I need slightly more peas per pod".
Post reply on HN