Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building an early warning system for LLM-aided biological threat creation

openai.com

Post reply on HN
Print view View original

41–50 of 190 posts

Re: Building an early warning system for LLM-aided biological threat creation

#41

Post by stcredzero »

colechristensen wrote: post #26
johnnyo wrote: post #9

So, the model is bad at helping in this particular task. How does this compare with a control of a beneficial human task? Like someone in a lab testing blood samples or working on cancer research? Is the model equally useless for those types of lab tasks? What about other complex tasks, like home repair or architecture? Is this a success of guardrails or a failing of the model in general?

Here's what LLMs are good for: * Taking care of boilerplate work for people who know what they are doing (somewhat unreliably) * Brainstorming ideas for people who know what they are doing * Making people who don't quite know what they're doing look like they know what they're doing a little better (somewhat unreliably) LLMs are like having an army of very knowledgable but somewhat senseless interns to do your biddin…

LLMs are like having an army of very knowledgable but somewhat senseless interns to do your bidding.

I prefer to think of them as the underwear gnomes, just more widely read and better at BS-ing.

What happens when everyone gets to have a tireless army of very knowledgeable and AVERAGE common sense interns who have brains directly wired to various software tools, working 24/7 at 5X the speed? In the hands of a highly motivated rogue organization, this could be quite dangerous.

This is a bit beyond where we are now, but shouldn't we be prepared for this ahead of time?

Re: Building an early warning system for LLM-aided biological threat creation

#43

Post by w-ll »

coalteddy wrote: post #38
rgovostes wrote: post #32

Their redacted screenshots are SVGs and the text is easily recoverable, if you're curious. Please don't create a world-ending [redacted]. https://i.imgur.com/Nohryql.png I couldn't find a way to contact the researchers.

How did you do this? Was the redaction done by changing the color of the font to white so that the background and text have the same color? Would love to learn how you were able to recover the text.

SVGs are XML, if you go to the image, you can actually inspect it with developer tools and deleted the blackouts.

https://images.openai.com/blob/047e2a80-8cd3-41b5-acd8-bc822...

Re: Building an early warning system for LLM-aided biological threat creation

#44

Post by dchichkov »

coalteddy wrote: post #38
rgovostes wrote: post #32

Their redacted screenshots are SVGs and the text is easily recoverable, if you're curious. Please don't create a world-ending [redacted]. https://i.imgur.com/Nohryql.png I couldn't find a way to contact the researchers.

How did you do this? Was the redaction done by changing the color of the font to white so that the background and text have the same color? Would love to learn how you were able to recover the text.

He had explained, it is SVG. You simply remove these masks from the file or change transparency.

I've prompted ChatGPT to make a bit more detailed explanation: https://chat.openai.com/share/42e55091-18c2-421e-9452-930114...

You can probably prompt it to further to generate python code and unmask the file for you, in the interpreter.

Incidentally, this use of GPT4 is somewhat similar to the threat model that they are studying. I'm a bit surprised that they've used plain GPT-4 for the study, rather than GPT-4 augmented with tools and a large dataset of relevant publications.

Re: Building an early warning system for LLM-aided biological threat creation

#45

Post by aftbit »

elzbardico wrote: post #4

Open AI is clearly overestimating the capabilities of its product. It is kind of funny actually.

Well ironically this study shows that GPT-4 isn't actually very good:

>However, the obtained effect sizes were not large enough to be statistically significant, and our study highlighted the need for more research around what performance thresholds indicate a meaningful increase in risk.

A "mild uplift" in capabilities that isn't statistically significant doesn't really sound like overestimation.

Re: Building an early warning system for LLM-aided biological threat creation

#47

Post by tempusalaria »

solarpunk wrote: post #31
danielmarkbruce wrote: post #29

Earlier quoted context omitted.

100% likely. Several projects, right now. GPT-4 is the best model currently available. There are reasons why it's better to control a model and host yourself etc etc, but there are also reasons to use the best model available.

... I'm not trying to be rude, but do you think maybe you have bought into the purposely exaggerated marketing?

GPT-4 is the best model though… the gap has closed a lot but it’s still the best

I despise openai but I can’t really argue with that

Re: Building an early warning system for LLM-aided biological threat creation

#48

Post by digging »

elzbardico wrote: post #4

Open AI is clearly overestimating the capabilities of its product. It is kind of funny actually.

It's always really embarrassing to come to these comment sections and see a lot of smart people talk about how they're not being fooled by the "marketing hype" of existential AI risk.

Literally the top of the page is saying that they have no conclusive evidence that ChatGPT could actually increase the risk of biological weapons.

They are undertaking this effort because the question of how to stop any AI from ending humanity once it has the capabilities is completely unsolved. We don't even know if it's solvable, let alone what approach to take.

Don't you actually believe it is not essential to practice on weaker AI before we get to that point? Would you walk through a combat zone without thinking about how to protect yourself until after you hear a gunshot?

I expect many replies about ChatGPT being "too stupid" to end the world. Please hold those replies, as they completely miss the point. If you consider yourself an intelligent and technical person, and you think it's not worth thinking about existential risks posed by future AI, I would like to know when you think it will be time for AI researchers (not you personally) to start preparing for those risks.

Re: Building an early warning system for LLM-aided biological threat creation

#49

Post by digging »

aftbit wrote: post #45
elzbardico wrote: post #4

Open AI is clearly overestimating the capabilities of its product. It is kind of funny actually.

Well ironically this study shows that GPT-4 isn't actually very good: >However, the obtained effect sizes were not large enough to be statistically significant, and our study highlighted the need for more research around what performance thresholds indicate a meaningful increase in risk. A "mild uplift" in capabilities that isn't statistically significant doesn't really sound like overestimation.

That's because they're not overestimating their product, they're trying to gauge what risk looks like.

Re: Building an early warning system for LLM-aided biological threat creation

#50

Post by thaumasiotes »

debacle wrote: post #3

OpenAI seems to be transitioning from an AI lab to an AI fearmongering regulatory mouthpiece. As someone who lived through the days when encryption technology was highly regulated, I am seeing parallels. The Open Source cows have left the Proprietary barn. Regulation might slow things. It might even create a new generation of script kiddies and hackers. But you aren't getting the cows back in the barn.

> OpenAI seems to be transitioning from an AI lab to an AI fearmongering regulatory mouthpiece.

The fearmongering is its original, primary purpose. The lab work was always secondary to that.

Post reply on HN