Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building an early warning system for LLM-aided biological threat creation

openai.com

Post reply on HN
Print view View original

81–90 of 190 posts

Re: Building an early warning system for LLM-aided biological threat creation

#81

Post by RandomLensman »

digging wrote: post #48
elzbardico wrote: post #4

Open AI is clearly overestimating the capabilities of its product. It is kind of funny actually.

It's always really embarrassing to come to these comment sections and see a lot of smart people talk about how they're not being fooled by the "marketing hype" of existential AI risk. Literally the top of the page is saying that they have no conclusive evidence that ChatGPT could actually increase the risk of biological weapons. They are undertaking this effort because the question of how to stop any AI from ending h…

Hypothetical risks of hypothetical machines. We don't even know if it needs solving.

Re: Building an early warning system for LLM-aided biological threat creation

#82

Post by fatherzine »

jakewins wrote: post #58

My wife is doing her PhD in molecular neurobiology, and was amused by this - but also noted that the question is trivial and any undergrad with lab access would know how to do this. Watching her manage cell cultures it seems the difficulty is more around not having the cells die from every dust particle in the air being a microscopic pirate ship brimming with fungal spores set to pillage any plate of cells they land…

[flagged]

Re: Building an early warning system for LLM-aided biological threat creation

#83

Post by throwaway2474 »

Ok come on, this has gotta be a regulatory capture stunt. None of this is surprising or particularly dangerous.

You can do this for literally any topic. Choose something lawmakers are scared of, write a scary paper showing how GPT (“research preview only” of course) can assist with it, and make big vague statements calling for an urgent need for more safety work in the area. Since uncensored GPT will talk about everything, this works for every topic!

Make no mistake folks, the OpenAI “safety” budget is entirely about PR and squashing open source AI.

Re: Building an early warning system for LLM-aided biological threat creation

#84

Post by margalabargala »

fatherzine wrote: post #82
jakewins wrote: post #58

My wife is doing her PhD in molecular neurobiology, and was amused by this - but also noted that the question is trivial and any undergrad with lab access would know how to do this. Watching her manage cell cultures it seems the difficulty is more around not having the cells die from every dust particle in the air being a microscopic pirate ship brimming with fungal spores set to pillage any plate of cells they land…

[flagged]

> O(undergrad with lab access) = 10k

You're off by at least 3 orders of magnitude.

There are 18 million undergrads in the US alone. I would bet that at least half have some sort of basic bio lab class and can use a lab, or has a friend that does and would let them in. There's zero security in undergrad bio labs.

Re: Building an early warning system for LLM-aided biological threat creation

#85

Post by jakewins »

fatherzine wrote: post #82
jakewins wrote: post #58

My wife is doing her PhD in molecular neurobiology, and was amused by this - but also noted that the question is trivial and any undergrad with lab access would know how to do this. Watching her manage cell cultures it seems the difficulty is more around not having the cells die from every dust particle in the air being a microscopic pirate ship brimming with fungal spores set to pillage any plate of cells they land…

[flagged]

The thing is you need the lab - if you don’t have the incubator and the air filtration systems and the autoclave and the centrifuge and the… then the cells will die or you can’t extract the viruses etc.

You need the lab to do this, and for anyone that has lab access the actual doing - multiplying some viruses - is trivial

Re: Building an early warning system for LLM-aided biological threat creation

#86

Post by RcouF1uZ4gsC »

> Specifically, on a 10-point scale measuring accuracy of responses, we observed a mean score increase of 0.88 for experts and 0.25 for students compared to the internet-only baseline, and similar uplifts for completeness (0.82 for experts and 0.41 for students). However, the obtained effect sizes were not large enough to be statistically significant,

The last sentence is the most telling. The differences were not statistically significant.

Re: Building an early warning system for LLM-aided biological threat creation

#87

Post by sjwhevvvvvsj »

danielmarkbruce wrote: post #74
sjwhevvvvvsj wrote: post #57

Earlier quoted context omitted.

The definition of “best” has a lot of factors. Best general purpose LLM chat? I’d agree there, but there’s so much more to LLM than chat applications. For some tasks I’m working on, Mixtral is the “best” solution given it can be used locally, isn’t hampered by “safety” tuning, and I can run it 24x7 on huge jobs with no costs besides the upfront investment on my GPU + electricity. I have GPT-4 open all day as my codin…

Yup, plenty of reasons to run your own model. I'm not using GPT-4 for chat, but for what I'd class as "reasoning" applications. It seems best by a long shot. As for safety, I find with the api and the system prompt that there is nothing it won't answer for me. That being said... I'm not asking for anything weird. GPT-4 turbo does seem to be reluctant sometimes.

I’m doing document summarization and classification, and there’s a fair amount it won’t do for prudish reasons (eg sex, porn, etc). Llama2 is basically useless in that regard.

Re: Building an early warning system for LLM-aided biological threat creation

#90

Post by Fomite »

jakewins wrote: post #58

My wife is doing her PhD in molecular neurobiology, and was amused by this - but also noted that the question is trivial and any undergrad with lab access would know how to do this. Watching her manage cell cultures it seems the difficulty is more around not having the cells die from every dust particle in the air being a microscopic pirate ship brimming with fungal spores set to pillage any plate of cells they land…

Pretty much this. Having worked in biopreparedness...the instructions aren't the hard part. Both creating and deploying a biological threat are wildly more difficult.
Post reply on HN