Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building an early warning system for LLM-aided biological threat creation

openai.com

Post reply on HN
Print view View original

101–110 of 190 posts

Re: Building an early warning system for LLM-aided biological threat creation

#101

Post by fatherzine »

jakewins wrote: post #85
fatherzine wrote: post #82

Earlier quoted context omitted.

[flagged]

The thing is you need the lab - if you don’t have the incubator and the air filtration systems and the autoclave and the centrifuge and the… then the cells will die or you can’t extract the viruses etc. You need the lab to do this, and for anyone that has lab access the actual doing - multiplying some viruses - is trivial

if you were to get me in a lab right now, i'd had absolutely zero clue on how to multiply a virus. perhaps i'm old and behind the times.

thing is, i'm not sure about the lab access requirement. i'd expect any number of diy techniques to be functional substitutes. brb, asking chatgpt for more details.

Re: Building an early warning system for LLM-aided biological threat creation

#102

Post by solarpunk »

digging wrote: post #78
solarpunk wrote: post #75

Earlier quoted context omitted.

dude... the concern is /existential/ though, right?! to summarize, i'm not advocating for this, i'm just emphasizing there's a nifty little framework already in place.

Yes, but so is climate change, and we can't even get countries to agree to grow more slowly , let alone shut down everything. > i'm just emphasizing there's a nifty little framework already in place. More than one! Nuclear war, economic isolation, ground invasion. All kinds of nifty things we could do to stop dangerous AI. None of them are likely to happen when the risk is identified. To summarize, any easy solution…

what if the ai superintelligence is banking on us running datacenters non-stop in the hopes of continued return on investment. all while humans are deprived of electricity needed to operate air conditioning in a continuously heating world. and we all die off from what was initially anthtrogenic, but now ai-induced climate change?

i've got a two birds; one stone solution.

Re: Building an early warning system for LLM-aided biological threat creation

#103

Post by zeofig »

pixl97 wrote: post #96
zeofig wrote: post #91

Earlier quoted context omitted.

On the contrary... way too many people see "AI can do X", "AI can do Y", "I can draw an imaginary line between X, Y and Z", "therefore AI can totally maybe do Z". The fact that we "just don't know" doesn't mean anything. You just don't know if Jesus will return tomorrow and bring the rapture.

The probability that Jesus will return tomorrow is extremely low if not zero. When talking about the probability of AI doing X or Y the probability is much closer to 1 if physics allows it.

If physics allows it... a considerable assumption. In my view physics (or mathematics, or whatever) doesn't even allow an LLM to "reason", let alone manufacture novel bioweapons. The probability of Jesus returning is considerable too if we assume the bible is true.

Re: Building an early warning system for LLM-aided biological threat creation

#104

Post by solarpunk »

solarpunk wrote: post #102
digging wrote: post #78

Earlier quoted context omitted.

Yes, but so is climate change, and we can't even get countries to agree to grow more slowly , let alone shut down everything. > i'm just emphasizing there's a nifty little framework already in place. More than one! Nuclear war, economic isolation, ground invasion. All kinds of nifty things we could do to stop dangerous AI. None of them are likely to happen when the risk is identified. To summarize, any easy solution…

what if the ai superintelligence is banking on us running datacenters non-stop in the hopes of continued return on investment. all while humans are deprived of electricity needed to operate air conditioning in a continuously heating world. and we all die off from what was initially anthtrogenic, but now ai-induced climate change? i've got a two birds; one stone solution.

or maybe the ai superintelligence has calculated a trajectory for an asteroid on a collision course for earth guaranteed to eliminate a vast majority of life on the planet, but has a feasible disaster recovery plan for itself. and we would have a fighting chance if we could repurpose those damn nvidia GPUs currently churning through tokens to do more monte carlo experiments specific to calculating how best to deflect the immanent asteroid.

Re: Building an early warning system for LLM-aided biological threat creation

#105

Post by ctrw »

digging wrote: post #49
aftbit wrote: post #45

Earlier quoted context omitted.

Well ironically this study shows that GPT-4 isn't actually very good: >However, the obtained effect sizes were not large enough to be statistically significant, and our study highlighted the need for more research around what performance thresholds indicate a meaningful increase in risk. A "mild uplift" in capabilities that isn't statistically significant doesn't really sound like overestimation.

That's because they're not overestimating their product, they're trying to gauge what risk looks like.

They are going for regulatory capture because they have no moat and open source Ai models are eating their lunch.

Re: Building an early warning system for LLM-aided biological threat creation

#106

Post by pests »

fatherzine wrote: post #101
jakewins wrote: post #85

Earlier quoted context omitted.

The thing is you need the lab - if you don’t have the incubator and the air filtration systems and the autoclave and the centrifuge and the… then the cells will die or you can’t extract the viruses etc. You need the lab to do this, and for anyone that has lab access the actual doing - multiplying some viruses - is trivial

if you were to get me in a lab right now, i'd had absolutely zero clue on how to multiply a virus. perhaps i'm old and behind the times. thing is, i'm not sure about the lab access requirement. i'd expect any number of diy techniques to be functional substitutes. brb, asking chatgpt for more details.

The Thought Emporium[0] on YouTube does homelab DNA editing, grows modified bacteria and viruses, and is currently working on growing human neurons.

I've learned a lot and its very fascinating. I haven't found another maker online doing anything remotely similar. He's like the NileRed for genetic engineering instead of chemistry.

[0] https://www.youtube.com/@thethoughtemporium

Re: Building an early warning system for LLM-aided biological threat creation

#109

Post by nextos »

Fomite wrote: post #90
jakewins wrote: post #58

My wife is doing her PhD in molecular neurobiology, and was amused by this - but also noted that the question is trivial and any undergrad with lab access would know how to do this. Watching her manage cell cultures it seems the difficulty is more around not having the cells die from every dust particle in the air being a microscopic pirate ship brimming with fungal spores set to pillage any plate of cells they land…

Pretty much this. Having worked in biopreparedness...the instructions aren't the hard part. Both creating and deploying a biological threat are wildly more difficult.

I don't fully agree. Things are becoming quite simple, which is scary.

There's a well-intentioned lobby in the UK advocating for some minimal control, i.e. checking DNA / RNA / protein sequences.

For instance, cheap mRNA synthesis as a service has lowered the barrier of entry for any malicious actor. It's frankly ridiculous there are no checks in place right now.

Post reply on HN