Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building an early warning system for LLM-aided biological threat creation

openai.com

Post reply on HN
Print view View original

121–130 of 190 posts

Re: Building an early warning system for LLM-aided biological threat creation

#121

Post by jefftk »

jdewerd wrote: post #111
nextos wrote: post #109

Earlier quoted context omitted.

I don't fully agree. Things are becoming quite simple, which is scary. There's a well-intentioned lobby in the UK advocating for some minimal control, i.e. checking DNA / RNA / protein sequences. For instance, cheap mRNA synthesis as a service has lowered the barrier of entry for any malicious actor. It's frankly ridiculous there are no checks in place right now.

> minimal control, i.e. checking DNA / RNA / protein sequences They were already doing this 15 years ago. > It's frankly ridiculous there are no checks in place right now. It's frankly ridiculous that you think this.

>> minimal control, i.e. checking DNA / RNA / protein sequences

> They were already doing this 15 years ago.

I work in this area. Some synthesis providers check, others don't. And the checking isn't great.

(Some of my coworkers work on https://securedna.org which is trying to make this screening more robust.)

Re: Building an early warning system for LLM-aided biological threat creation

#122

Post by fragmede »

croniev wrote: post #115
apsec112 wrote: post #36

Way too many people extrapolate from "AI Foo can't do task X" to "AIs in general can never do task X", whether X is a good thing (like playing Go) or a bad thing (like helping build bioweapons). AI right now is pretty much the fastest-moving field in human history. GPT-4 can't revolutionize the economy, and it can't commit mass murder, but we simply don't know if GPT-7 can do either of those things (which likely won'…

Would you say that GPT-4 can reason now? I am not convinced this is case, it seems like it has just become more consistent at providing us with an output that we consider reasonable because it was engineered precisely to do that.

then you have to define reason, and it gets all philosophical. suffice to say, it's able to take a implies b and b implies c to get a implies c, and make up things along the way so calling it glorified autocomplete is a dishonest representation of its abilities. it doesn't love or feel jealous but it's very good at writing essays about whatever I ask it to. it doesn't need to do more than that to be useful to me, today.

the human words we have for consciousness aren't good enough to describe what ChatGPT does. thinking, reasoning, understanding. it processes. it computes. it matrix multiplies. it takes the dot product and the determinant. there are eigenvectors and eigenvalues. it's tensoring and outputting the code and prose I asked it for.

Re: Building an early warning system for LLM-aided biological threat creation

#123

Post by anonymouskimmer »

fatherzine wrote: post #95
margalabargala wrote: post #84

Earlier quoted context omitted.

> O(undergrad with lab access) = 10k You're off by at least 3 orders of magnitude. There are 18 million undergrads in the US alone. I would bet that at least half have some sort of basic bio lab class and can use a lab, or has a friend that does and would let them in. There's zero security in undergrad bio labs.

undergrad in molecular biology = 4k. assuming eg CS undergrads don't attend en-masse molecular biology classes and/or have lab access. edit: and one obvious error, it's graduates with molecular biology degrees, presumably the undergrad population is ~5x larger, for a total of 20k. and presumably the world is larger, so we can throw in another ~5x factor for a total of 100k. still, /orders of magnitude/ smaller than w…

My majors weren't in molecular biology, just Biotech and Biology. I actually learned viral titering and expansion in cell culture on the job, but nearly every bio undergrad will get a basic intro to cell culture techniques. The specific techniques for amplifying a virus are open to anyone with decent search skills, e.g. https://www.addgene.org/protocols/#virus

Re: Building an early warning system for LLM-aided biological threat creation

#124

Post by [deleted] »

jdewerd wrote: post #111
nextos wrote: post #109

Earlier quoted context omitted.

I don't fully agree. Things are becoming quite simple, which is scary. There's a well-intentioned lobby in the UK advocating for some minimal control, i.e. checking DNA / RNA / protein sequences. For instance, cheap mRNA synthesis as a service has lowered the barrier of entry for any malicious actor. It's frankly ridiculous there are no checks in place right now.

> minimal control, i.e. checking DNA / RNA / protein sequences They were already doing this 15 years ago. > It's frankly ridiculous there are no checks in place right now. It's frankly ridiculous that you think this.

[deleted]

Re: Building an early warning system for LLM-aided biological threat creation

#125

Post by sjwhevvvvvsj »

danielmarkbruce wrote: post #99
sjwhevvvvvsj wrote: post #87

Earlier quoted context omitted.

I’m doing document summarization and classification, and there’s a fair amount it won’t do for prudish reasons (eg sex, porn, etc). Llama2 is basically useless in that regard.

with GPT-4 (non-turbo) and a good system prompt?

It’s unpredictable, usually it behaves but on arbitrary web text you’ll eventually trigger a “safety” barrier, and sometimes just trying again will work.

Re: Building an early warning system for LLM-aided biological threat creation

#127

Post by dchichkov »

JieJie wrote: post #64
dchichkov wrote: post #44

Earlier quoted context omitted.

He had explained, it is SVG. You simply remove these masks from the file or change transparency. I've prompted ChatGPT to make a bit more detailed explanation: https://chat.openai.com/share/42e55091-18c2-421e-9452-930114... You can probably prompt it to further to generate python code and unmask the file for you, in the interpreter. Incidentally, this use of GPT4 is somewhat similar to the threat model that they are…

Their reasoning for not using tools or browsing from the "Limitations" section: "No GPT-4 tool usage: Due to our security measures, the GPT-4 models we tested were used without any tools, such as Advanced Data Analysis and Browsing. Enabling the usage of such tools could non-trivially improve the usefulness of our models in this context. We may explore ways to safely incorporate usage of these tools in the future."

Sounds like the Frontier team wasn't able to convince GPTs team to run an extra model.

Re: Building an early warning system for LLM-aided biological threat creation

#128

Post by groby_b »

digging wrote: post #48
elzbardico wrote: post #4

Open AI is clearly overestimating the capabilities of its product. It is kind of funny actually.

It's always really embarrassing to come to these comment sections and see a lot of smart people talk about how they're not being fooled by the "marketing hype" of existential AI risk. Literally the top of the page is saying that they have no conclusive evidence that ChatGPT could actually increase the risk of biological weapons. They are undertaking this effort because the question of how to stop any AI from ending h…

We should also deeply worry about space aliens showing up and blasting us out of the sky. If they're sufficiently powerful, that could absolutely happen! Stop any radio emissions!

xRisk is an absolutely stupid way to reason about AI. It's an unprovable risk that requires "mitigation just in case". All this is is saying "but if it were to happen, the cost is infinity, so any risk is a danger! Infinity times anything is infinity!". It's playground reasoning. (The same playground reasoning the EA community engages in, which is a large vector for the xrisk hype. Just multiply by a large enough number, and you will surely have the biggest number)

To the credit of the authors, they don't engage in that. There is no hand wringing over the absolutely unlikely case of "but what if the AI awakens".

But it's still an extremely weak study - it proves nothing (none of the results are statistically significant), and even if it had shown significant uplift, it's meaningless without a control. Of course people who have access to a knowledge store do slightly better than people who don't. I'm willing to bet that "access to a 10-book research library" produces roughly the same uplift. Without that (trivial) control, it's really bad study design.

And the moment you take this study and its non-results and call it "Building an early warning system for LLM-aided biological threat creation", you've absolutely lost all credibility.

Re: Building an early warning system for LLM-aided biological threat creation

#129

Post by michael_nielsen »

digging wrote: post #48
elzbardico wrote: post #4

Open AI is clearly overestimating the capabilities of its product. It is kind of funny actually.

It's always really embarrassing to come to these comment sections and see a lot of smart people talk about how they're not being fooled by the "marketing hype" of existential AI risk. Literally the top of the page is saying that they have no conclusive evidence that ChatGPT could actually increase the risk of biological weapons. They are undertaking this effort because the question of how to stop any AI from ending h…

I'm not sure why they're so often so bad. I wonder if it's the Upton Sinclair effect; to paraphrase slightly: "It is difficult to get a person to understand something, when their hoped-for future wealth depends on not understanding it."

Re: Building an early warning system for LLM-aided biological threat creation

#130

Post by foo3a9c4 »

Nifty3929 wrote: post #118
croniev wrote: post #115

Earlier quoted context omitted.

Would you say that GPT-4 can reason now? I am not convinced this is case, it seems like it has just become more consistent at providing us with an output that we consider reasonable because it was engineered precisely to do that.

It’s definitely not the case. LLMs of any sort do not in any sense reason or understand anything. They literally just make stuff up (technically just a continuation of whatever you fed in), which usually sounds good, is often true and sometimes even helpful. Because those are qualities of the training data that was used and is the basis for the stuff it’s making up.

> It’s definitely not the case. LLMs of any sort do not in any sense reason or understand anything.

This seems like a claim about the way that the LLM neural net algorithm works. But AFAIK no one has a good understanding of how the LLM NNs work.

Why are you so certain that the LLM NN isn't doing the reasoning-algorithm or the understanding-algorithm?

Post reply on HN