OpenAI • Hacker News Board

Re: OpenAI

#121

Post by theptip » Sat, Jun 18, 2022, 5:43 PM UTC

ma2rten wrote: post #109

theptip wrote: post #102

Earlier quoted context omitted.

It’s worrying to see very smart guys like LeCun failing to grok the paper clip maximizer issue (or coffee maximizer as Russell phrases it), which is like the one paragraph summary or elevator pitch for AI risk. I think there are plenty of other valid objections to a high E-risk estimate but that one is non-sensical to me. I think Robin Hanson has the most cogent objection to high E-risk estimates, which is basically…

Have you considered that it's not LeCun who is missing something? The AI safety community seems to be unfortunately almost completely separate from the actual AI research community and be making some strong assumptions about how AGI is going to work. Note that LeCun had a reply in the thread and there was a lot more discussion which GP didn't quote.

Fair, perhaps I should retract “fail to grok” and replace it with “fail to focus on”. It does seem that LeCun understands the objections (though he dismisses them out of hand).

Regardless of who is right or wrong, “Don’t fear the terminator” is a weird straw-man to raise in a discussion about AI risk. He’s setting up a weak opponent to argue against, when the AI risk community have a large repertoire of stronger cases. “Don’t fear the paper clip maximizer” would be a stronger case to put forth IMO.

In his response points 2&3 he asserts that alignment is easy; simply train the AI with laws as part of the objective function and it will never break laws. I think there has been a lot of investigation and discussion as to why this is harder than it sounds. For example LeCun is explicitly talking about current models that are statically trained to a fixed objective function, but one can easily imagine a future agentic AI (imagine “personal Siri) that will continue to grow, learn, and update in the world in response to rewards from its owner. Maybe he is right about near-term models but I’m completely unconvinced that his arguments hold generally.

Anyway, maybe the “terminator scenario” is a concern LeCun hears from uninformed reporters/lay people that he felt the need to debunk. It’s a valid point as far as it goes, but it has little to do with the actual state of the cutting edge of AI risk research.

Re: OpenAI

#122

Post by theptip » Sat, Jun 18, 2022, 5:45 PM UTC

nopinsight wrote: post #106

theptip wrote: post #102

Earlier quoted context omitted.

It’s worrying to see very smart guys like LeCun failing to grok the paper clip maximizer issue (or coffee maximizer as Russell phrases it), which is like the one paragraph summary or elevator pitch for AI risk. I think there are plenty of other valid objections to a high E-risk estimate but that one is non-sensical to me. I think Robin Hanson has the most cogent objection to high E-risk estimates, which is basically…

My issue with the Hanson objection as stated above (link to the original would be appreciated) is that it rests on the assumption that the N-1 level AIs still under human control can somehow completely eliminate or suppress the self-modifying AGI long enough until alignment research is complete. Meanwhile, the unaligned AGI could multiply, hide, and accumulate power covertly. Humanity would also need time to align AG…

I should have taken the time to link it above. This is the Hanson article: https://www.overcomingbias.com/2017/08/foom-justifies-ai-ris...

(It links to a previous debate with Eleizer too.)

Re: OpenAI

#123

Post by c1ccccc1 » Sat, Jun 18, 2022, 6:02 PM UTC

astrange wrote: post #17

ductsurprise wrote: post #16

Earlier quoted context omitted.

Agreed. Maybe... Short sighted vs. far sighted vs. ignorance is bliss?

It's a secular/religious divide. (As it says in the post.) Though it's possible the people who think a theoretical future AI will turn the planet into paperclips have merely forgotten that perpetual motion machines aren't possible.

This is your friendly physics reminder that perpetual motion machines have nothing to do with this. It's hard to turn the whole planet into paperclips because paperclips are mostly made of iron, while the planet contains many other elements. Of course, with a high enough level of technology, it might be possible to fuse together the non-iron elements, so that you would end up with just a bunch of iron nuclei. This would even be energetically favourable, since iron is so stable. Then you just have to solve the issue that the paperclips in the center of the planet would be under huge pressure and would be crushed.

Re: OpenAI

#124

Post by nradov » Sat, Jun 18, 2022, 6:21 PM UTC

DalasNoin wrote: post #96

nradov wrote: post #95

Earlier quoted context omitted.

Yes I've seen those things. They are amazing technical achievements, but in the end they're just clever parlor tricks (with perhaps some limited applicability to a few real business problems). They don't look like forward progress towards any sort of true AGI that could ever pass a rigorous Turing test.

Clearly language models can already fool people into thinking they are human, we might be getting quite close to the adversarial turing test already. In the end, a good initial prompt might be the solution to this, something like "pretend to be a human and step by step create a human identity that you then stick to during the conversation". I'm serious

A minority of the population will always be gullible and easily fooled. So what. Some people were already fooled by the original ELIZA program back in 1966. I would only count a Turing test pass if it can convince a jury of multiple educated examiners after a conversation lasting several hours.

Re: OpenAI

#125

Post by tome » Sat, Jun 18, 2022, 6:36 PM UTC

deadpannini wrote: post #117

tome wrote: post #111

Earlier quoted context omitted.

But now you have appealed to anthropomorphism (“intelligence”) to pose a problem yet forbidden anthropomorphism in an attempted counter argument. That doesn’t seem quite fair.

I don't intend to forbid anything - I just think the language of motivation and desire makes it harder to see the risks, because it introduces irrelevant questions into the the conversation like "how can machines want something?" Conversely, at least in this discussion, the term "intelligence" seems pretty neutral.

> I just think the language of motivation and desire makes it harder to see the risks, because it introduces irrelevant questions into the the conversation like "how can machines want something?"

Yet discourse on existential AI risks is predicated on something like a "goal" (e.g. to maximise paperclips). Notions like "goal" also make it harder to see clearly what we are actually discussing.

> the term "intelligence" seems pretty neutral

Hmm, I'm not convinced. It seems like an extremely loaded term to me.

Re: OpenAI

#126

Post by nbardy » Sat, Jun 18, 2022, 7:11 PM UTC

nradov wrote: post #95

DalasNoin wrote: post #92

Earlier quoted context omitted.

Are you aware of some of the recent progress? Did you have a look at the Gato model and Flamingo by DeepMind, or at the chat logs of models like chinchilla and lambda? Or Alphacode? This is all from this year. I think your point is that all these models are still somewhat specialized. At the same time, it appears that the transformer architecture works well with images, short video and text at the same time in the Fl…

Yes I've seen those things. They are amazing technical achievements, but in the end they're just clever parlor tricks (with perhaps some limited applicability to a few real business problems). They don't look like forward progress towards any sort of true AGI that could ever pass a rigorous Turing test.

They’re learning representations of objects. It’s more fundamental than tou seem to realize.

Re: OpenAI

#127

Post by marvin » Sat, Jun 18, 2022, 7:15 PM UTC

tome wrote: post #112

marvin wrote: post #91

Earlier quoted context omitted.

Over the course of years, we figure out how to create AI systems that are more and more useful, to the point where they can be run autonomously and with very little supervision produce economic output that eclipses that of the most capable humans in the world. With generality, this obviously includes the ability to maintain and engineer similar systems, so human supervision of the systems themselves can become redund…

> they can be run autonomously and with very little supervision produce economic output that eclipses that of the most capable humans in the world. What’s the evidence for this? > Perverse instantiation of AI systems was accidentally demonstrated in the lab decades ago What are you referring to?

Historical examples of perverse instantiation are everywhere: Evolutionary agents learning to live off a diet of their own children, machine learning algorithms attempting to learn gripping a ball cheating the system by performing ball-less movements that the camera erroneously classifies as successful, an evolutionary algorithm to optimize the number of circuit elements in a timer creating a timer circuit by picking up an external radio signal unrelated to the task and so on. Some examples are summarized here: https://www.wired.com/story/when-bots-teach-themselves-to-ch...

GP wanted a concrete example of a doomsday scenario of failed AI alignment, so in that context extrapolating to a plausible future of advanced AI agents should suffice. If you need a double-blind peer reviewed study to consider the possibility that intelligent agents more capable than humans could exist in physical reality, I don't think you're in the target audience for the discussion. A little bit of philosophical affinity beyond the status quo is table stakes.

Re: OpenAI

#128

Post by amelius » Sat, Jun 18, 2022, 7:21 PM UTC

afthonos wrote: post #83

pas wrote: post #69

Earlier quoted context omitted.

then it's not really (self)driving.

It’s (not) self-driving in the same way planes do (not) fly. Just because something doesn’t do things the way we (or our or a bird) does doesn’t mean it doesn’t achieve its goal.

Then how about an AI misclassifying a firetruck for a road?

Re: OpenAI

#129

Post by jandrewrogers » Sat, Jun 18, 2022, 8:50 PM UTC

genem9 wrote: post #67

gjvnq wrote: post #50

Earlier quoted context omitted.

There's also the crowed that worries about self-driving cars doing stupid shit like misreading a 20 km/h sign as an 120 km/h.

Pretty sure the self driving vehicles get the speed data from a dataset and not from signs.

No, many ordinary cars contain image classifiers that read speed limit signs dynamically. This is pretty standard. And they sometimes get it wrong e.g. my car reads 80 MPH as 60 MPH about a third of the time, much to my dismay.

The dynamic classification is required because the world isn't static. An increasing number of locales have digital speed limit signs that vary the speed limit dynamically, some times independently per lane. Automation requires cars to respond to the world as it is, not how the world was when it recorded a month ago.

Re: OpenAI

#130

Post by deadpannini » Sat, Jun 18, 2022, 9:22 PM UTC

tome wrote: post #125

deadpannini wrote: post #117

Earlier quoted context omitted.

I don't intend to forbid anything - I just think the language of motivation and desire makes it harder to see the risks, because it introduces irrelevant questions into the the conversation like "how can machines want something?" Conversely, at least in this discussion, the term "intelligence" seems pretty neutral.

> I just think the language of motivation and desire makes it harder to see the risks, because it introduces irrelevant questions into the the conversation like "how can machines want something?" Yet discourse on existential AI risks is predicated on something like a "goal" (e.g. to maximise paperclips). Notions like "goal" also make it harder to see clearly what we are actually discussing. > the term "intelligence"…

AIs absolutely do have goals, determined by their reward functions.

Yes, "intelligence" is a deeply loaded term. It just doesn't matter in the context of the discussion here, so far as I've seen.its ambiguities haven't been relevant.