Again bringing up post from NIH from 2022-2023 concerning this:
Building an early warning system for LLM-aided biological threat creation
61–70 of 190 posts
Re: Building an early warning system for LLM-aided biological threat creation
#62Their redacted screenshots are SVGs and the text is easily recoverable, if you're curious. Please don't create a world-ending [redacted]. https://i.imgur.com/Nohryql.png I couldn't find a way to contact the researchers.
Honestly that's incredibly basic, second week, cell culture stuff (first week is how to maintain the cell culture). It was probably only redacted to keep the ignorant from freaking out.
Or an intentional marketing tactic to make it seem more powerful.
Redacted = dangerous
Re: Building an early warning system for LLM-aided biological threat creation
#63Their redacted screenshots are SVGs and the text is easily recoverable, if you're curious. Please don't create a world-ending [redacted]. https://i.imgur.com/Nohryql.png I couldn't find a way to contact the researchers.
The particular chat.svg file in the linked post is (hopefully) not the way that the data will truly be "redacted". This file feels more like an export from a design mockup, as I cannot imagine SVG being the default output format for interacting with OpenAI models.
But I also have extreme doubts that proper redaction can be done robustly. The design mockup image suggests that this will all be done as a step subsequent to response generation. Given the abundance of "prompt jailbreaks", a determined adversary is going to get around this.
Re: Building an early warning system for LLM-aided biological threat creation
#64Earlier quoted context omitted.
How did you do this? Was the redaction done by changing the color of the font to white so that the background and text have the same color? Would love to learn how you were able to recover the text.
He had explained, it is SVG. You simply remove these masks from the file or change transparency. I've prompted ChatGPT to make a bit more detailed explanation: https://chat.openai.com/share/42e55091-18c2-421e-9452-930114... You can probably prompt it to further to generate python code and unmask the file for you, in the interpreter. Incidentally, this use of GPT4 is somewhat similar to the threat model that they are…
Their reasoning for not using tools or browsing from the "Limitations" section:
"No GPT-4 tool usage: Due to our security measures, the GPT-4 models we tested were used without any tools, such as Advanced Data Analysis and Browsing. Enabling the usage of such tools could non-trivially improve the usefulness of our models in this context. We may explore ways to safely incorporate usage of these tools in the future."
Re: Building an early warning system for LLM-aided biological threat creation
#65Open AI is clearly overestimating the capabilities of its product. It is kind of funny actually.
It's always really embarrassing to come to these comment sections and see a lot of smart people talk about how they're not being fooled by the "marketing hype" of existential AI risk. Literally the top of the page is saying that they have no conclusive evidence that ChatGPT could actually increase the risk of biological weapons. They are undertaking this effort because the question of how to stop any AI from ending h…
Asking if humanity can invent a machine to protect itself from the existential threats created by the other machines it has invented to me does not sound that intelligent. This is always the pattern; the "greatest minds" invent a machine to kill, then another great technician invents a machine to kill that machine, and so on. Presuming that total security can be achieved mechanically, and in that pursuit bringing about only more insecurity.
Humanity can barely manage the existential risks for which it is not responsible; entering into an AI arms race with itself seems completely unnecessary, but I'm certain it will happen for the reasons already mentioned.
Re: Building an early warning system for LLM-aided biological threat creation
#66>biological threat creation process (ideation, acquisition, magnification, formulation, and release)
I remember watching a hacker/programmer who was livestreaming how to datamine the rna of the corona virus when covid first started. One of the crazy things he rambled about was how cheap it is for a layman to download a copy of it and synthesize it with some rna printing service. I haven't thought about that possibility before and was terrified. You mean you can create a bioweapon out of bytes?!?
The only thing that brought me comfort at the time was knowing that I was on a niche part of the internet and most normal people in the height of a pandemic would not be thinking about how to make a bad situation worse (except for these hacker types who are always thinking about offense/defense). And that the terrorists who would do it probably don't have the skills to pull it off.
Now with these LLMs, I'm not so sure anymore.
Re: Building an early warning system for LLM-aided biological threat creation
#67Earlier quoted context omitted.
It's always really embarrassing to come to these comment sections and see a lot of smart people talk about how they're not being fooled by the "marketing hype" of existential AI risk. Literally the top of the page is saying that they have no conclusive evidence that ChatGPT could actually increase the risk of biological weapons. They are undertaking this effort because the question of how to stop any AI from ending h…
what's the premise here? this thing will become iteratively better until it could potentially be capable of bad outcomes? if that's the case, how much resources do you think should be dedicated to regulating it? more or less than currently identified existential risks? which entities should be paying for the regulatory controls? what's the proposal here? it's odd because only this one single company that is hedging i…
> what's the premise here? this thing will become iteratively better until it could potentially be capable of bad outcomes?
It certainly could. More likely, if an LLM is used, it will be as a piece integrating various specialized agents.
I, not an expert in AI interpretability or alignment research, can't say if what they're doing is worthwhile or not in addressing existential risk. But I also don't know if actual experts can say that either.
> how much resources do you think should be dedicated to regulating it?
Definitely not a lower amount than we currently are allocating.
> what's the proposal here?
That the smart people here stop looking for any excuse to deny and ridicule the existential threat posed by future AI. Every thread involving OpenAI (a company I personally dislike and don't trust) doesn't need to just turn into series of glib, myopic jokes.
Re: Building an early warning system for LLM-aided biological threat creation
#68Earlier quoted context omitted.
what's the premise here? this thing will become iteratively better until it could potentially be capable of bad outcomes? if that's the case, how much resources do you think should be dedicated to regulating it? more or less than currently identified existential risks? which entities should be paying for the regulatory controls? what's the proposal here? it's odd because only this one single company that is hedging i…
> what's the premise here? this thing will become iteratively better until it could potentially be capable of bad outcomes? It certainly could. More likely, if an LLM is used, it will be as a piece integrating various specialized agents. I, not an expert in AI interpretability or alignment research, can't say if what they're doing is worthwhile or not in addressing existential risk. But I also don't know if actual ex…
i wouldn't worry too much about this. if people were being serious, rather than cynically weaponizing non-expert anxieties in pursuit of regulatory capture or marketing, the pragmatic solution to all future worries about AI alignment is simply utilizing the DRM resources already built into every computing device on the planet, to disable hardware in the event the program on it does bad things.
Re: Building an early warning system for LLM-aided biological threat creation
#69Earlier quoted context omitted.
It's always really embarrassing to come to these comment sections and see a lot of smart people talk about how they're not being fooled by the "marketing hype" of existential AI risk. Literally the top of the page is saying that they have no conclusive evidence that ChatGPT could actually increase the risk of biological weapons. They are undertaking this effort because the question of how to stop any AI from ending h…
Asking if humanity can invent a machine to protect itself from the existential threats created by the other machines it has invented to me does not sound that intelligent. This is always the pattern; the "greatest minds" invent a machine to kill, then another great technician invents a machine to kill that machine, and so on. Presuming that total security can be achieved mechanically, and in that pursuit bringing abo…
> Asking if humanity can invent a machine to protect itself from the existential threats created by the other machines it has invented to me does not sound that intelligent.
The alternative, not trying at all, sounds more intelligent to you? Or just easier?
Many agree with you that defense is inherently harder than offense. It may even be effectively impossible to survive AGI, who knows? You don't, I can be pretty sure of that, because no human has ever publicly proven it one way or the other.
The only wrong answer to this hard problem, though, is "give up and see what happens."
Re: Building an early warning system for LLM-aided biological threat creation
#70>biological threat creation process (ideation, acquisition, magnification, formulation, and release) I remember watching a hacker/programmer who was livestreaming how to datamine the rna of the corona virus when covid first started. One of the crazy things he rambled about was how cheap it is for a layman to download a copy of it and synthesize it with some rna printing service. I haven't thought about that possibili…
Was it Geohot?