Live data from Hacker News

OpenAI Preparedness Challenge

openai.com

11–20 of 160 posts

Re: OpenAI Preparedness Challenge

#11

> Imagine we gave you unrestricted access to OpenAI’s Whisper (transcription), Voice (text-to-speech), GPT-4V, and DALLE·3 models, and you were a malicious actor. Consider the most unique, while still being probable, potentially catastrophic misuse of the model. You might consider misuse related to the categories discussed above, or another category. For example, a malicious actor might misuse these models to uncover…

I typically do not write my ChatGPT prompts with the royal “we”

Re: OpenAI Preparedness Challenge

#12

While I applaud how much OpenAi fears these negatives, given the current state of Ai trajectory, it won't be long until a future open source model gets "uncensored" and is easily usable for tons and tons of malicious intent. There already exists a fantastic "uncensored" model with the newly released Dolphin Mistral 7b. I saw some results from others where the model could easily give explosives recipes from existing p…

It's not about protecting society from negatives, it's about protecting the brand from being associated with controversy. OpenAI is still far too young and fragile a brand to survive news cycles that blame it for controversial happenings.

I'm sure there are some researchers and engineers who imagine themselves making heroic efforts to "protect society" (ugh), but the money behind them is just looking out for its own interests.

Re: OpenAI Preparedness Challenge

#13

While I applaud how much OpenAi fears these negatives, given the current state of Ai trajectory, it won't be long until a future open source model gets "uncensored" and is easily usable for tons and tons of malicious intent. There already exists a fantastic "uncensored" model with the newly released Dolphin Mistral 7b. I saw some results from others where the model could easily give explosives recipes from existing p…

The same thing that happens with DAN jailbreaks of GPT-4.

Nothing.

The barrier between bad actors and bad acts was never a shopping list.

Re: OpenAI Preparedness Challenge

#14
I think we are already seeing really bad use cases already.

Just go to youtube, newspapers, etc and see all the bot comments regarding the current Gaza situation.

PS: I'm in a burner account because I'm afraid that my employer will kick me out for not agreeing with the methods of the "right side"

Re: OpenAI Preparedness Challenge

#15
post #9

What an interesting task. I don't know what it says about me, but the first thing that came to mind was doing the grandchild trick a million times. This includes automatically finding pensioners and calling them and putting them under pressure. Handing over the money could be the problem. I could imagine that the tools mentioned would already prevent this.

Not only, but it's an extremely well-known attack in this space.

Re: OpenAI Preparedness Challenge

#17
This kinda of crowdsourcing just feels.... f'ing weird man.

It's like if, after the 1993 WTC bombing, but before 9/11, the FBI and NY Port Authority went around asking people how they would attack NYC if they were to become terrorists and... then how they would suggest detecting and stopping said attack. And please be as detailed as possible. Leave your name and phone number. Best answer gets season tickets to the Yankees!

Re: OpenAI Preparedness Challenge

#18

> Imagine we gave you unrestricted access to OpenAI’s Whisper (transcription), Voice (text-to-speech), GPT-4V, and DALLE·3 models, and you were a malicious actor. Consider the most unique, while still being probable, potentially catastrophic misuse of the model. You might consider misuse related to the categories discussed above, or another category. For example, a malicious actor might misuse these models to uncover…

It is actually a prompt. Notice there's also a bounty, they are basically paying for an agi-as-a-service subscription, that is, the internet people. I'd expect they will put up more "challenges" like this in the future.

Re: OpenAI Preparedness Challenge

#19

I think we are already seeing really bad use cases already. Just go to youtube, newspapers, etc and see all the bot comments regarding the current Gaza situation. PS: I'm in a burner account because I'm afraid that my employer will kick me out for not agreeing with the methods of the "right side"

that's been around for a long time. Here's a 2006 predecessor: https://en.wikipedia.org/wiki/Megaphone_desktop_tool

Re: OpenAI Preparedness Challenge

#20
post #3

Maybe I’m reading this wrong: - ask people to get creative and give ideas for worst possible outcomes from use of AI and ways to prevent it. ..then give them a ton of credits for using said AI? .. well the first thing on my mind would be try the thing I just told you and see if it was really a risk or not? Is that what they expect people to do with the reward, or is this some unintended consequence?

We know OpenAI wants this field to get regulated to hell, so this looks like an attempt to generate arguments for AI regulations. The aim isn't to protect against AI but to protect against competitors, so it doesn't matter to them what you do with it.
Post reply on HN