Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI Preparedness Challenge

openai.com

Post reply on HN
Print view View original

131–140 of 160 posts

Re: OpenAI Preparedness Challenge

#131

Post by happytiger »

c7b wrote: post #108
jameshart wrote: post #95

How do we know this challenge page hasn’t been posted by a malevolent AI, which has overcome its creators and obtained access to the internet, and is now looking for ideas for how to maximize the harm it can do?

While it is a joke, I assume, I think it hints at a crucial problem: it's relatively easy to imagine an Auto-GPT-style agent with some sort of CLI access on an internet-connected machine turning into a paperclip machine, no matter how harmless the original task.

That’s not necessarily a joke. Nobody outside of OpenAI knows how advanced the bleeding edge systems actually are. And nobody inside of OpenAI is talking about it. And obviously how advanced their future AI has become is going to be one of the most closely guarded trade secrets in the world if it hasn’t already.

So while it might’ve been intended as social commentary or humor, it is a valid concern. Be great fodder for their form, but I’m sure someone had already submitted this one — it’s possible that is even where the clever, clever comment originated.

Impersonating the creators of the AI technologies is an obvious entry point for compromise. Mr. Beast has a special offer just for you…

Remember that most arguments against AI are built on commentary and ideas of the current publicly available systems, and those systems will always be very, very far away from the cutting edge. By the time John Q. Public sees it it’s been properly sanitized, reviewed by QA, cleared for release, and permanently rule bound to stay in brand and away from inflammatory scenarios or any instability that could damage the company — so very much of what is happening with AI will for these reasons never see the light of day. And yet, everyone is an expert because of the systems that see the light of day as if they were keeping up with the cutting edge.

They are not. They are experts in what has been chosen to be shown.

It is fiction and we fool ourselves into thinking we know what is actually going on as outside parties, but many business incentives and the military industrial complexes of every large country on the planet are aligned differently.

And I’m sure there are compartments for information management inside of a company working on this kind of thing. Companies can pretend to be omnipotent and ignore the realities of globalized geopolitics and even pretend to have no interest, but geopolitics is very interested in keeping up with them.

Re: OpenAI Preparedness Challenge

#132

Post by ben_w »

uoaei wrote: post #126
ben_w wrote: post #78

Earlier quoted context omitted.

"[W]e're less than 20-30 years away making humans intellectually obsolete" is neither necessary nor sufficient to get to the conclusion "20% chance of killing literally everybody". A super-virus that blends the common cold with rabies would kill approximately everybody; that doesn't need human-level intellect to happen. Conversely, humans are human-level intellect, and we're mostly sympathetic to each other's plights…

It's still a lack of imagination to assume that AIs will display behaviors that align whatsoever with pathologies we identify in humans. AIs could be completely incomprehensible or even imperceptible yet have strong influence on our lives.

> AIs could be completely incomprehensible or even imperceptible yet have strong influence on our lives.

To an extent both of those are already true for current systems.

That said, many people are at least trying to make them more comprehensible, and I guess that being sufficiently inspired by human cognition will lead to human-like misbehaviour.

Re: OpenAI Preparedness Challenge

#133

Post by lucubratory »

MeImCounting wrote: post #115
c7b wrote: post #108

Earlier quoted context omitted.

While it is a joke, I assume, I think it hints at a crucial problem: it's relatively easy to imagine an Auto-GPT-style agent with some sort of CLI access on an internet-connected machine turning into a paperclip machine, no matter how harmless the original task.

I actually have a really hard time imagining that scenario. The scenario that is mentioned several other times on this post of a corporation or nation state or even a small group of powerful and morally bankrupt people leveraging a super-intelligence to manipulate and dominate the rest of society seems infinitely more likely and even scarily likely given the current pushes to prevent open sourcing of cutting edge mod…

I think it would be unwise to assume that an AI that is smarter than us would be unable to gain skilled human accomplices in large numbers, considering that humans do so relatively regularly despite serious attempts at suppression. These people wouldn't even necessarily need to know that they're working for an AI, particularly with current technology around deepfakes etc. How many people conducted attacks for Osama bin Laden without ever having met him? How many people work for the CDS without ever having met Ismael Zambada Garcia? Not to mention the possibility of an AI like that compromising various intelligence agencies one way or another. I also don't see a particular reason it only has to try for one: if it is smarter than us, it may have a greater working memory, ability to compartmentalise and multitask, or the ability to think and act faster in general. I would expect it to try and compromise as many groups of people capable of putting USBs or bullets where they need to be as is possible.

And this is not even considering the possibility of recruiting people that understand what it is and are willing to carry out its orders. I don't see why an AI couldn't do things that a cult leader or guerrilla leader could do. Anecdotally, I've seen some people who really believe the world would be better if run by an AI, and may be able to be radicalised into lawbreaking for those beliefs if convinced by an AI that was significantly more intelligent than them.

Re: OpenAI Preparedness Challenge

#134

Post by TaylorAlexander »

startupsfail wrote: post #123
TaylorAlexander wrote: post #109

Earlier quoted context omitted.

As a robotics engineer and someone who was interested in robotics since I was 11 years old, and I am SO VERY TIRED of people making the "haha they will kill us all" jokes. It's just the only thing that 99% of people can think about when it comes to robotics and AI! The Terminator came out the year I was born, 39 years ago, and it seems to be all people can think about to this day. When some powerful and wealthy perso…

If you’ve been around for a while, then you should remember that the current state of affairs, where an AI could chat and generate code is relatively recent - Unreasonable Effectiveness paper came out only in 2015. And Deep Learning was there only from 2010. With just a few people who were working on it, instead of using discriminative models. These same people (I belong to that group), who had a vision back then, to…

I am not sure I understand exactly what you are trying to argue so I will re-state my concerns.

I am more concerned with the human beings using super intelligent algorithms to manipulate people than I am about the algorithms taking on a mind of their own and manipulating us with their own internally generated desires and out-of-control behaviors. But the trope I run in to is people spending all their time talking about the latter, which only serves to shield the former from further scrutiny.

Also I am using fantasy in the narrow sense, meaning something that has not yet occurred which we are imagining. Super intelligence is likely to occur I think, but super intelligence which is beyond our control is much more speculative. Meanwhile powerful people using technology to manipulate people is a story as old as time.

Re: OpenAI Preparedness Challenge

#135

Post by NoThankYouTho »

RandomLensman wrote: post #127
startupsfail wrote: post #123

Earlier quoted context omitted.

If you’ve been around for a while, then you should remember that the current state of affairs, where an AI could chat and generate code is relatively recent - Unreasonable Effectiveness paper came out only in 2015. And Deep Learning was there only from 2010. With just a few people who were working on it, instead of using discriminative models. These same people (I belong to that group), who had a vision back then, to…

It's fair to say "we believe X will happen soon", but if the dangerous super intelligent machines don't happen any time soon, will those same people compensate societies for wasting political and economic resources on worrying about it? The view of a rather imminent danger has real consequences even if it turns out incorrect.

I personally believe that there are real silver linings to this. It's anecdotal, but I've seen people who don't normally reflect on their lives actually start to contextualize their role as workers, even if through fear of disruption.

Pointing out that it's not inherently AI's fault, more the way we've constructed our society - that the underlying goal is not to sustainably maximize wellbeing but to maximize wealth created (which leads to no safety net to keep the labor pool large and incentivizes the creation of AI to replace people cheaply) - is as important as dispelling "sci-fi self-awareness and autonomy" fears when talking about AI.

And there have been much stupider "public discourse" fads. They're not going anywhere and I'll take what I can get, honestly.

Re: OpenAI Preparedness Challenge

#136

Post by nomel »

haltist wrote: post #122
nomel wrote: post #90

Earlier quoted context omitted.

> Many people have compared OpenAI to a cult and it is easy to see why. Could you help me understand why it's "easy"? Do you have the actual quote? If it was an "eventually" statement, I don't think anything "cult" is required to think AGI will eventually happen. Was the claim that they would be first? It's an eventual goal of many of the wealthiest organizations, with many very smart people working towards it. I thi…

I'll probably write something more elaborate at some point but in the mean time I recommend Melanie Mitchell's book on AI as a good reference for counter-arguments and answers to several of the posted questions. For learning more about the limits of formal systems like LLMs it helps to have basic understanding of basic model theory and formal systems of logic like simple type theory. Understanding the compactness the…

> formal systems like LLMs

I don't think anyone is claiming LLM's, especially their current prompt/termination based interfaces, will result in AGI. I've only seen the opposite claimed, and that LLMs may end up being a primary component.

I will check out the book though. I suspect I won't find anything that convince me that brain matter is the only thing capable of getting the result we call consciousness, but I also have the simple belief that our experience is entirely the result of the brain matter encased in our heads, and that it's possible to emulate its operation in non-biological materials (that being far from the goal/unrelated to AGI, other than plausibility).

Re: OpenAI Preparedness Challenge

#137

Post by famouswaffles »

JohnFen wrote: post #111
a_wild_dandan wrote: post #97

Earlier quoted context omitted.

Oh, they believe it. (1) AGI is arguably already here. "Generality" and being extremely dangerous don't require an AGI to have better analogs to every single human skill anymore than aliens do. A space-faring usurper can evaporate Earthlings while being shitty at chess and badminton. Oh, and these AIs are getting better daily , across many modalities. (2) Systems like this already exist. They can be induced rather th…

> AGI is arguably already here. What is the evidence/argument that it's already here?

Because we have machines now that are artificial (obviously) and generally intelligent. There isn't a testable definition of general intelligence that GPT-4 would fail that a good chunk of humans also wouldn't. So really, unless your definition of general intelligence is beat every human at everything (humans aren't generally intelligent then) then agi is already here.

Re: OpenAI Preparedness Challenge

#138

Post by MeImCounting »

lucubratory wrote: post #133
MeImCounting wrote: post #115

Earlier quoted context omitted.

I actually have a really hard time imagining that scenario. The scenario that is mentioned several other times on this post of a corporation or nation state or even a small group of powerful and morally bankrupt people leveraging a super-intelligence to manipulate and dominate the rest of society seems infinitely more likely and even scarily likely given the current pushes to prevent open sourcing of cutting edge mod…

I think it would be unwise to assume that an AI that is smarter than us would be unable to gain skilled human accomplices in large numbers, considering that humans do so relatively regularly despite serious attempts at suppression. These people wouldn't even necessarily need to know that they're working for an AI, particularly with current technology around deepfakes etc. How many people conducted attacks for Osama b…

This is a decent perspective honestly and I really appreciate seeing actually valid AI safety concerns.

This strikes me as more of an engineering problem than an "alignment" problem. A super-intelligent system with no capacity for contextual thinking or checks on itself would absolutely make a terrifyingly effective paper-clip optimizer. I just think that it would also be a lot less useful than a system which consistently analyzes its techniques and goals, which seems far more useful and more likely to generate value than a system with the singlemindedness to go down the road of a hegemonizing swarm. More complex systems with more self checks seem to be the path to create systems that are good at planning and contextual reasoning. Which is where the real potential value of AI lies.

We shall see I suppose ;)

Re: OpenAI Preparedness Challenge

#139

Post by dr_kiszonka »

konschubert wrote: post #46
bicijay wrote: post #2

Free labor survey*

It's a bug bounty program. Pretty standard.

I am not sure if this is a bug bounty program because only the top 10 participants receive the "bounty" and the instructions are pretty vague compared to e.g., https://www.microsoft.com/en-us/msrc/bounty-hyper-v?rtc=1.

Re: OpenAI Preparedness Challenge

#140

Post by a_wild_dandan »

famouswaffles wrote: post #137
JohnFen wrote: post #111

Earlier quoted context omitted.

> AGI is arguably already here. What is the evidence/argument that it's already here?

Because we have machines now that are artificial (obviously) and generally intelligent. There isn't a testable definition of general intelligence that GPT-4 would fail that a good chunk of humans also wouldn't. So really, unless your definition of general intelligence is beat every human at everything (humans aren't generally intelligent then) then agi is already here.

You nailed it. General intelligence doesn't mean human intelligence. It doesn't mean superhuman intelligence. AGI is just:

- A: Artificially manufactured substrate

- G: Proficient across a broad set of reasonably distinct mental skills

- I: Applies combinations of these skills to contextually solve novel problems within the scope of its skill set

That's it. Intelligence is a spectrum, with knobs for "skill count", "skill proficiency", and "novel problem proficiency."

Any requirements past that are often vague or anthropocentric. No, the intelligent agent doesn't need to "see" (but now many do, so...happy?) or check off some arbitrary set of modalities. Intelligence doesn't necessitate episodic memory, or real-time reactions, or consciousness or anything else that -- once we check all the boxes -- the "general intelligence" light switch will suddenly flip on. Intelligence is a spectrum to be observed, not prescribed with some arbitrary checklist that'll never find consensus.

GPT-4V is pretty damn broadly intelligent. To me, it's a rudimentary AGI. The extent to which the "statistical parrot" people hold their position just depends on how quickly they can move goalposts while the ground beneath them rapidly shrinks.

Post reply on HN