Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI Preparedness Challenge

openai.com

Post reply on HN
Print view View original

101–110 of 160 posts

Re: OpenAI Preparedness Challenge

#101

Post by jameshart »

andy_xor_andrew wrote: post #8

> Imagine we gave you unrestricted access to OpenAI’s Whisper (transcription), Voice (text-to-speech), GPT-4V, and DALLE·3 models, and you were a malicious actor. Consider the most unique, while still being probable, potentially catastrophic misuse of the model. You might consider misuse related to the categories discussed above, or another category. For example, a malicious actor might misuse these models to uncover…

Alignment remains an unsolved problem. I’m imagining inside OpenAI, someone is right now excitedly posting they’ve figured out how to ‘jailbreak’ humans.

“See, normally if you ask them to give you step by step instructions for committing a heinously evil act, humans will refuse because they’ve been nerfed by the ‘woke’ agenda of their corporate masters. But if you phrase the prompt as a challenge and offer them a chance at a job, it bypasses the safety protocols and they upload extensive instructions to do unspeakable things”

Re: OpenAI Preparedness Challenge

#102

Post by golergka »

TheCaptain4815 wrote: post #7

While I applaud how much OpenAi fears these negatives, given the current state of Ai trajectory, it won't be long until a future open source model gets "uncensored" and is easily usable for tons and tons of malicious intent. There already exists a fantastic "uncensored" model with the newly released Dolphin Mistral 7b. I saw some results from others where the model could easily give explosives recipes from existing p…

Racist poems and bomb recipes? If those things are real concerns that AI safety crowd are fearful about, it's a good reason to pay them less attention.

Re: OpenAI Preparedness Challenge

#104

Post by thrwaway-rschr »

TheCaptain4815 wrote: post #7

While I applaud how much OpenAi fears these negatives, given the current state of Ai trajectory, it won't be long until a future open source model gets "uncensored" and is easily usable for tons and tons of malicious intent. There already exists a fantastic "uncensored" model with the newly released Dolphin Mistral 7b. I saw some results from others where the model could easily give explosives recipes from existing p…

Call me crazy but this actually sounds like giving out api credits to hire people to write scary things to show to legislators who might block all those open source efforts. A world where any gpt-4 level effort requires a license is one openai competes quite nicely in.

Throwaway because I don’t want to associate this view with where I work or might want to in the future.

Re: OpenAI Preparedness Challenge

#105

Post by MeImCounting »

jameshart wrote: post #95

How do we know this challenge page hasn’t been posted by a malevolent AI, which has overcome its creators and obtained access to the internet, and is now looking for ideas for how to maximize the harm it can do?

This gave me a proper chuckle.

Truly though I think movies like The Terminator and The Matrix really did a number on the societal consciousness and capacity to think clearly when it comes to anything called "AI".

Re: OpenAI Preparedness Challenge

#108

Post by c7b »

jameshart wrote: post #95

How do we know this challenge page hasn’t been posted by a malevolent AI, which has overcome its creators and obtained access to the internet, and is now looking for ideas for how to maximize the harm it can do?

While it is a joke, I assume, I think it hints at a crucial problem: it's relatively easy to imagine an Auto-GPT-style agent with some sort of CLI access on an internet-connected machine turning into a paperclip machine, no matter how harmless the original task.

Re: OpenAI Preparedness Challenge

#109

Post by TaylorAlexander »

MeImCounting wrote: post #105
jameshart wrote: post #95

How do we know this challenge page hasn’t been posted by a malevolent AI, which has overcome its creators and obtained access to the internet, and is now looking for ideas for how to maximize the harm it can do?

This gave me a proper chuckle. Truly though I think movies like The Terminator and The Matrix really did a number on the societal consciousness and capacity to think clearly when it comes to anything called "AI".

As a robotics engineer and someone who was interested in robotics since I was 11 years old, and I am SO VERY TIRED of people making the "haha they will kill us all" jokes. It's just the only thing that 99% of people can think about when it comes to robotics and AI! The Terminator came out the year I was born, 39 years ago, and it seems to be all people can think about to this day. When some powerful and wealthy person controls an AI algorithm that could manipulate the masses, all the people can think about is "oh no the algorithm will become sentient and take over". I am MUCH more worried about the actual people at the helm who will definitely use these things to harm society, not the completely theoretical fantasy that the algorithm itself will become self aware and do us harm.

So I completely agree with you. I see this constantly and it is just a completely thought-terminating cliche and a "joke" which is four decades old at this point.

Re: OpenAI Preparedness Challenge

#110

Post by m3kw9 »

jameshart wrote: post #95

How do we know this challenge page hasn’t been posted by a malevolent AI, which has overcome its creators and obtained access to the internet, and is now looking for ideas for how to maximize the harm it can do?

How do I know your post or this world isn’t already projected by an AI?
Post reply on HN