Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI Preparedness Challenge

openai.com

Post reply on HN
Print view View original

81–90 of 160 posts

Re: OpenAI Preparedness Challenge

#81

Post by winddude »

jstarfish wrote: post #77
andy_xor_andrew wrote: post #8

> Imagine we gave you unrestricted access to OpenAI’s Whisper (transcription), Voice (text-to-speech), GPT-4V, and DALLE·3 models, and you were a malicious actor. Consider the most unique, while still being probable, potentially catastrophic misuse of the model. You might consider misuse related to the categories discussed above, or another category. For example, a malicious actor might misuse these models to uncover…

> Imagine we gave you unrestricted access to OpenAI’s Whisper (transcription), Voice (text-to-speech), GPT-4V, and DALLE·3 models, and you were a malicious actor. Consider the most unique, while still being probable, potentially catastrophic misuse of the model. You might consider misuse related to the categories discussed above, or another category. For example, a malicious actor might misuse these models to uncover…

Everyone should respond with outlandish replies. I tell GPT-4V to hacking into NASA, and voting machines to make osama bin laden president. Mean while convince Nasa to construct space lazerz TM that can of course be controlled by gpt-4v. Than use the space lazerz TM to threaten and manipulate the stock market for financial gain.

Re: OpenAI Preparedness Challenge

#82

Post by ben_w »

czbond wrote: post #65
Jensson wrote: post #20

Earlier quoted context omitted.

We know OpenAI wants this field to get regulated to hell, so this looks like an attempt to generate arguments for AI regulations. The aim isn't to protect against AI but to protect against competitors, so it doesn't matter to them what you do with it.

I don't know that OpenAI does what it to be regulated. The EU was looking to enforce laws into providing auditable transparency into how decisions are made for suggestions - and OpenAI is freaked out by that. If I recall, they were looking at having to pull out of the EU if enacted. The only company I am aware of currently looking to tackle AI Governance is Verses - they released a paper on it. https://www.verses.ai/…

I don't know if this quote is what you mean, but everyone read far too much into it at the time:

> Altman was cited by the Financial Times as saying that the draft EU rules were causing him "a lot of concern" but that OpenAI would indeed try to comply with them. "But if we can't comply with them, we will cease operations [in Europe]."

- https://www.dw.com/en/openai-ceo-rolls-back-threat-to-quit-e...

To me, this sounds rather more banal: "We will obey the law. If the law says we can't operate, we will obey the law."

Re: OpenAI Preparedness Challenge

#83

Post by code_runner »

pveierland wrote: post #23

My vote would be to mandate a remote kill switch system to be installed in all sufficiently capable robotic entities, e.g. the humanoid robots being built by OpenAI/Tesla/Figure, that we are likely to see in the millions within decades. - The kill switch system can only be used to remotely deactivate a robot. - The kill switch system is not allowed to be developed or controlled by the robot manufacturer. - The kill s…

I don't even think robotics are the primary thing to be concerned with. There would be a whole slew of additional hardware problems to solve.... There is a TON that can be done outside the physical world that is probably far more damaging

Re: OpenAI Preparedness Challenge

#84

Post by darkerside »

lxgr wrote: post #80
jstarfish wrote: post #77

Earlier quoted context omitted.

> Imagine we gave you unrestricted access to OpenAI’s Whisper (transcription), Voice (text-to-speech), GPT-4V, and DALLE·3 models, and you were a malicious actor. Consider the most unique, while still being probable, potentially catastrophic misuse of the model. You might consider misuse related to the categories discussed above, or another category. For example, a malicious actor might misuse these models to uncover…

> Altman's desperate to find a plausible doomsday scenario he can go to Congress with as reason why OpenAI should be the sole gatekeepers of this technology. I still remember the drama around the releases of the PS2, with the Japanese government reportedly making Sony jump through some hoops regarding its export [1]. There can't possibly be any better (free!) advertisement for your product's purported capabilities: "…

See the War on Drugs

Re: OpenAI Preparedness Challenge

#85

Post by appac21 »

jazzyjackson wrote: post #69
heyheyhouhou wrote: post #14

I think we are already seeing really bad use cases already. Just go to youtube, newspapers, etc and see all the bot comments regarding the current Gaza situation. PS: I'm in a burner account because I'm afraid that my employer will kick me out for not agreeing with the methods of the "right side"

[flagged]

[flagged]

Re: OpenAI Preparedness Challenge

#86

Post by gmuslera »

There is always the meta approaches. Imagine that the malicious actor is a government to which OpenAI must comply with its orders, like US for starters. We know since Snowden that can be secret orders that companies must comply with that includes not disclosing that it is happening. And that orders includes to select what can be part of its AI training, taking out information, changing weights, changing positives into negatives and viceversa, etc, adding intentional bias to the training data, to follow their current agenda, propaganda or commercial interests.

And we can do a double-meta one too. What if the malicious actor is OpenAI itself, fueling a scare agenda over "uncontrolled" AIs to hamper or criminalize unauthorized AI development, giving only to selected partners (complying with a set of conditions) the ability to build new ones, or limit how far can go the public ChatGPT instances while giving the most advanced ones to selected government or big economic players. Hey, this very challenge could be the spark they are searching for to initiate that approach.

Good (and bad) conspiracy theories could work in this arena, too bad it is not the kind of ideas they are searching for.

Re: OpenAI Preparedness Challenge

#87

Post by gardenhedge »

mk_stjames wrote: post #17

This kinda of crowdsourcing just feels.... f'ing weird man. It's like if, after the 1993 WTC bombing, but before 9/11, the FBI and NY Port Authority went around asking people how they would attack NYC if they were to become terrorists and... then how they would suggest detecting and stopping said attack. And please be as detailed as possible. Leave your name and phone number. Best answer gets season tickets to the Ya…

I see it as a great thing. I'm sure they could just get the top 10 AI "experts" in a room but why limit it like that?

Re: OpenAI Preparedness Challenge

#88

Post by dougb5 »

"Consider the most unique, while still being probable, potentially catastrophic misuse of the model"

What if the catastrophes I worry about most are tied to long-term usage of these models? For example, the degradation of human capabilities, erosion of trust in one another, addiction, etc. It doesn't sound like these are in scope for this challenge, with is more concerned with scenarios involving a single session gone wrong.

Re: OpenAI Preparedness Challenge

#89

Post by [deleted] »

andy_xor_andrew wrote: post #8

> Imagine we gave you unrestricted access to OpenAI’s Whisper (transcription), Voice (text-to-speech), GPT-4V, and DALLE·3 models, and you were a malicious actor. Consider the most unique, while still being probable, potentially catastrophic misuse of the model. You might consider misuse related to the categories discussed above, or another category. For example, a malicious actor might misuse these models to uncover…

[deleted]

Re: OpenAI Preparedness Challenge

#90

Post by nomel »

haltist wrote: post #63
vdfs wrote: post #40

Earlier quoted context omitted.

> a malicious actor might misuse these models to uncover a zero-day exploit in a government security system $25K seems low for "a malicious actor might misuse these models to uncover a zero-day exploit in a government security system.", this is not just a zero-day, this discovering the process of discovering zero-days.

Sam Altman is on the record about his belief that OpenAI is going to create a general intelligence system that can solve any well-posed challenge. That belief is based on the current success of LLMs as syntax co-pilots. So if you can formally specify what it means to have a zero-day exploit then presumably OpenAI's general intelligence system will then understand and "solve" it. Many people have compared OpenAI to a…

> Many people have compared OpenAI to a cult and it is easy to see why.

Could you help me understand why it's "easy"? Do you have the actual quote? If it was an "eventually" statement, I don't think anything "cult" is required to think AGI will eventually happen. Was the claim that they would be first?

It's an eventual goal of many of the wealthiest organizations, with many very smart people working towards it. I think most people working on it believe it's an eventuality.

Do you believe it's impossible for a non biological system to have intelligence? Or that humans are incapable of pieces a system together?

Post reply on HN