Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI Preparedness Challenge

openai.com

Post reply on HN
Print view View original

111–120 of 160 posts

Re: OpenAI Preparedness Challenge

#111

Post by JohnFen »

a_wild_dandan wrote: post #97
api wrote: post #57

Earlier quoted context omitted.

I'm skeptical that they really believe this. You have to believe: (1) We are on the verge of equaling or greatly exceeding human intelligence generally . (2) We are on the verge of creating something with initiative, free will (whatever that means), and planning ability. Or alternately that these things will occur in an emergent fashion once we hit some critical mass. (3) When we accomplish 1 and 2, this thing will i…

Oh, they believe it. (1) AGI is arguably already here. "Generality" and being extremely dangerous don't require an AGI to have better analogs to every single human skill anymore than aliens do. A space-faring usurper can evaporate Earthlings while being shitty at chess and badminton. Oh, and these AIs are getting better daily , across many modalities. (2) Systems like this already exist. They can be induced rather th…

> AGI is arguably already here.

What is the evidence/argument that it's already here?

Re: OpenAI Preparedness Challenge

#112

Post by jstarfish »

lxgr wrote: post #80
jstarfish wrote: post #77

Earlier quoted context omitted.

> Imagine we gave you unrestricted access to OpenAI’s Whisper (transcription), Voice (text-to-speech), GPT-4V, and DALLE·3 models, and you were a malicious actor. Consider the most unique, while still being probable, potentially catastrophic misuse of the model. You might consider misuse related to the categories discussed above, or another category. For example, a malicious actor might misuse these models to uncover…

> Altman's desperate to find a plausible doomsday scenario he can go to Congress with as reason why OpenAI should be the sole gatekeepers of this technology. I still remember the drama around the releases of the PS2, with the Japanese government reportedly making Sony jump through some hoops regarding its export [1]. There can't possibly be any better (free!) advertisement for your product's purported capabilities: "…

Ironically, the USAF recently built a supercomputer out of PS3s (https://www.military.com/off-duty/games/2023/02/17/air-force...).

They alluded to what you're talking about but I wasn't familiar with the reference.

Re: OpenAI Preparedness Challenge

#114

Post by baobabKoodaa »

londons_explore wrote: post #76

What exactly would I use $25k of openAI credits for? My personal use of it rarely comes to more than $10 per month, despite using it multiple times per day. (most recently: "Write me a bash command to send data at a given rate to an arbitrary IP address.")

If you make an app that other people use, and it becomes viral, it can quickly cost >$100 / day in OpenAI credits. I know this from first-hand experience (and btw. it's not a good feeling to shut down that one app that actually goes viral).

Re: OpenAI Preparedness Challenge

#115

Post by MeImCounting »

c7b wrote: post #108
jameshart wrote: post #95

How do we know this challenge page hasn’t been posted by a malevolent AI, which has overcome its creators and obtained access to the internet, and is now looking for ideas for how to maximize the harm it can do?

While it is a joke, I assume, I think it hints at a crucial problem: it's relatively easy to imagine an Auto-GPT-style agent with some sort of CLI access on an internet-connected machine turning into a paperclip machine, no matter how harmless the original task.

I actually have a really hard time imagining that scenario.

The scenario that is mentioned several other times on this post of a corporation or nation state or even a small group of powerful and morally bankrupt people leveraging a super-intelligence to manipulate and dominate the rest of society seems infinitely more likely and even scarily likely given the current pushes to prevent open sourcing of cutting edge models.

One of the main reasons I have a hard time imagining the scenario you describe, that I havent seen talked about as much as the other very good reasons, is that generally when we talk about a paper-clip optimizer we are assuming its vertically integrated and self sufficient. Hacking of power grids, physical installations or other necessaries to a run away paper-clip optimizer generally require nation-state level resources and often involve physical penetration of some sort.

A hegemonizing swarm is certainly a frightening idea as is Skynet and all the other scary AI stories weve told over the years but none of them seem particularly likely or even plausible with the systems that we are likely to develop in the near to mid term.

Re: OpenAI Preparedness Challenge

#116

Post by lxgr »

jstarfish wrote: post #112
lxgr wrote: post #80

Earlier quoted context omitted.

> Altman's desperate to find a plausible doomsday scenario he can go to Congress with as reason why OpenAI should be the sole gatekeepers of this technology. I still remember the drama around the releases of the PS2, with the Japanese government reportedly making Sony jump through some hoops regarding its export [1]. There can't possibly be any better (free!) advertisement for your product's purported capabilities: "…

Ironically, the USAF recently built a supercomputer out of PS3s ( https://www.military.com/off-duty/games/2023/02/17/air-force... ). They alluded to what you're talking about but I wasn't familiar with the reference.

It is indeed deeply ironic that Sony has a long history of trying to declare their gaming consoles as general-purpose computers (to dodge EU import tariffs) and arguing that they aren't really military-grade (to be able to export them out of Japan), and finally ended up subsidizing a supercomputer for the US military :)

Re: OpenAI Preparedness Challenge

#117

Post by pveierland »

code_runner wrote: post #83
pveierland wrote: post #23

My vote would be to mandate a remote kill switch system to be installed in all sufficiently capable robotic entities, e.g. the humanoid robots being built by OpenAI/Tesla/Figure, that we are likely to see in the millions within decades. - The kill switch system can only be used to remotely deactivate a robot. - The kill switch system is not allowed to be developed or controlled by the robot manufacturer. - The kill s…

I don't even think robotics are the primary thing to be concerned with. There would be a whole slew of additional hardware problems to solve.... There is a TON that can be done outside the physical world that is probably far more damaging

I don't see any reason to single out any primary thing to be concerned by, as there likely isn't any singular issue that can be pointed to as as the root concern regarding AI safety.

Introducing millions of robots into society is something for which specific safeguards should be built. Costs related to developing such safety mechanisms seem small compared to the safety they can provide.

There will be many concerns relating to AI and robotics that should be addressed by a multitude of different safeguards.

Re: OpenAI Preparedness Challenge

#118

Post by two_in_one »

jameshart wrote: post #95

How do we know this challenge page hasn’t been posted by a malevolent AI, which has overcome its creators and obtained access to the internet, and is now looking for ideas for how to maximize the harm it can do?

It could be FBI as well. For the same reasons they are 'selling' anti-aircraft missiles in US. They even caught some 'enthusiasts'.

Re: OpenAI Preparedness Challenge

#119

Post by two_in_one »

m3kw9 wrote: post #110
jameshart wrote: post #95

How do we know this challenge page hasn’t been posted by a malevolent AI, which has overcome its creators and obtained access to the internet, and is now looking for ideas for how to maximize the harm it can do?

How do I know your post or this world isn’t already projected by an AI?

Matrix? Who cares, as long as it feels real.

Re: OpenAI Preparedness Challenge

#120

Post by danShumway »

To be honest, I view this as mostly PR fluff.

OpenAI's problem is not that it is unable to predict failure modes or security risks for AIs. OpenAI's problem is that it is not taking the many failure modes and security risks that it already knows about seriously, and it's not putting in adequate effort to address the specific concerns that people repeatedly talk about publicly.

It's not a failure of imagination, it's a failure of action. Their problem isn't that they don't know what to care about, their problem is that they don't care.

So when OpenAI puts out these offers or press releases about trying to be prepared or finding new failure modes, it just doesn't read as sincere. I want to know what they're doing about the very specific flaws that exist today that they already know about; both the ones that would be trivial to address (ie, UX-flows, data-exfiltration vulnerabilities, and user consent flows for plugins) that OpenAI refuses to acknowledge, and the ones that are wildly challenging but that demand actual Open research and mitigation (ie prompt injection and mass spam) rather than toothless "we're letting researchers explore this area" PR.

OpenAI has unlocked doors in its product -- and instead of locking them, it is hiring researchers to theorize about the nature of doors and asking the public to try messing with the windows. I'm not giving them credit for that, fix your doors.

Post reply on HN