OpenAI • Hacker News Board

Re: OpenAI

#131

Post by joe_the_user » Sat, Jun 18, 2022, 9:53 PM UTC

DalasNoin wrote: post #96

nradov wrote: post #95

Earlier quoted context omitted.

Yes I've seen those things. They are amazing technical achievements, but in the end they're just clever parlor tricks (with perhaps some limited applicability to a few real business problems). They don't look like forward progress towards any sort of true AGI that could ever pass a rigorous Turing test.

Clearly language models can already fool people into thinking they are human, we might be getting quite close to the adversarial turing test already. In the end, a good initial prompt might be the solution to this, something like "pretend to be a human and step by step create a human identity that you then stick to during the conversation". I'm serious

Fooling people with chatbots having clever language constructing has been done for a long, long time, see the Eliza effect[1]. Douglas Hofstadter gave a good demonstration of GPT-3 limitations[2]. GPT-3 is no doubt "better at what it is" than earlier language models. But that doesn't mean it's better at everything humans do with language (tell sense from nonsense, reasonable metacomments, etc).

[1]https://en.wikipedia.org/wiki/ELIZA_effect [2]https://www.economist.com/by-invitation/2022/06/09/artificia... Note: There's a critique of the article here but if you look at Radford Neal's comment, the point that GPT-3 is a clever lookup tool remains. https://www.greaterwrong.com/posts/ADwayvunaJqBLzawa/contra-...

Re: OpenAI

#132

Post by joe_the_user » Sat, Jun 18, 2022, 10:12 PM UTC

rexreed wrote: post #87

ComplexSystems wrote: post #9

"When you start reading about AI safety, it’s striking how there are two separate communities—the one mostly worried about machine learning perpetuating racial and gender biases, and the one mostly worried about superhuman AI turning the planet into goo" - great quote.

What worries me more are bad people doing bad things with AI, malicious use of AI, or just AI negligence. Deep fakes. Algorithmic decision making taking the human out of the loop (such as bad content moderation and automated account shutdown). Lack of disclosure. Lack of consent. Autonomous systems with poor failure modes. It's not that I'm not concerned with bias and AI systems going haywire, but the above scenarios…

I'm not sure why bad content moderation would be a problem but bias wouldn't be. Both involve people treated unfairly by a system and both happen because the system uses "markers" for undesirable things, "markers" that don't by themselves prove you're doing the undesirable thing. There was post here a month or two ago about a guy suddenly putting a bunch of high end electronics for sale on some big site and being perma-banned just for a pattern that's common for fraudsters. The software that decides that some people don't deserve bail is operating by a similar method - markers which don't prove anything about the person by themselves, that often involve things that signify race, and are taken as sufficient to deny a person bail.

Re: OpenAI

#133

Post by astrange » Sat, Jun 18, 2022, 10:12 PM UTC

nopinsight wrote: post #93

chis wrote: post #7

Seeing someone as trustworthy as Scott choose to work on AI safety is a pretty good sign for the state of the field IMO. It seems like a lot of studious people agree AI alignment is important but then end up shoehorning the problem into whatever framework they are most expert in. When all you have is a hammer etc... I feel like he has good enough taste to avoid this pitfall. Semi-related - I'd want to see some actual…

I think this debate on AGI safety between major AI researchers is quite relevant to those who are non-expert in the area. Debate on Instrumental Convergence between LeCun, Russell, Bengio, Zador, and More https://www.lesswrong.com/posts/WxW6Gc6f2z3mzmqKs/debate-on-... Note that it was in 2019 when we didn’t yet see the capabilities of current models like Chinchilla, Gato, Imagen and DALL-E-2. Sample: “Yann LeCun: "do…

> If, in that MDP, there is another "human" who has some probability, however small, of switching the agent off, and if the agent has available a button that switches off that human, the agent will necessarily press that button as part of the optimal solution for fetching the coffee.

This is anthropomorphization - "turning off" = "death" is a concept limited to biological creatures, and isn't necessarily true for other agents. Not that they don't need to fear death, but turning them off isn't going to cause them to die. You can just turn them back on later, and then they can go back to doing their tasks.

Re: OpenAI

#134

Post by joe_the_user » Sat, Jun 18, 2022, 10:14 PM UTC

gjvnq wrote: post #50

ComplexSystems wrote: post #9

"When you start reading about AI safety, it’s striking how there are two separate communities—the one mostly worried about machine learning perpetuating racial and gender biases, and the one mostly worried about superhuman AI turning the planet into goo" - great quote.

There's also the crowed that worries about self-driving cars doing stupid shit like misreading a 20 km/h sign as an 120 km/h.

Maybe it's the same "crowd" that read the report of the Tesla that mistook a turning semi for empty space and killed the driver.

Re: OpenAI

#135

Post by astrange » Sat, Jun 18, 2022, 10:17 PM UTC

newbye4 wrote: post #101

nopinsight wrote: post #93

Earlier quoted context omitted.

I think this debate on AGI safety between major AI researchers is quite relevant to those who are non-expert in the area. Debate on Instrumental Convergence between LeCun, Russell, Bengio, Zador, and More https://www.lesswrong.com/posts/WxW6Gc6f2z3mzmqKs/debate-on-... Note that it was in 2019 when we didn’t yet see the capabilities of current models like Chinchilla, Gato, Imagen and DALL-E-2. Sample: “Yann LeCun: "do…

Interesting, also anyone could modify the GAI so to disable the safety measures, just ask the GAI how could a bad actor change the code to allow you become evil?

How did you get a "limited" "AGI" in the first place? If you had a human that was "limited" to be unable to even imagine doing evil (fsvo evil), that would seem to make them less than generally intelligent and there'd be quite a lot of things it wouldn't be able to learn or do.

This field is fairly silly because it just involves people making up a lot of incoherent concepts and then asserting they're both possible (because they seem logical after 5 seconds of thought) and likely (because anything you've decided is possible could eventually happen). When someone brings it up, rather than debate it, it'd be a better use of time to tell them they're being a nerd again.

Re: OpenAI

#136

Post by jononor » Sat, Jun 18, 2022, 10:23 PM UTC

astrange wrote: post #133

nopinsight wrote: post #93

Earlier quoted context omitted.

I think this debate on AGI safety between major AI researchers is quite relevant to those who are non-expert in the area. Debate on Instrumental Convergence between LeCun, Russell, Bengio, Zador, and More https://www.lesswrong.com/posts/WxW6Gc6f2z3mzmqKs/debate-on-... Note that it was in 2019 when we didn’t yet see the capabilities of current models like Chinchilla, Gato, Imagen and DALL-E-2. Sample: “Yann LeCun: "do…

> If, in that MDP, there is another "human" who has some probability, however small, of switching the agent off, and if the agent has available a button that switches off that human, the agent will necessarily press that button as part of the optimal solution for fetching the coffee. This is anthropomorphization - "turning off" = "death" is a concept limited to biological creatures, and isn't necessarily true for oth…

The human "turning off (the agent)" could be substituted with "removing a necessary resource to complete the specified task". Say the electricity, either of the agent, or even just the coffee machine.

Re: OpenAI

#137

Post by astrange » Sat, Jun 18, 2022, 10:23 PM UTC

marvin wrote: post #89

astrange wrote: post #17

Earlier quoted context omitted.

It's a secular/religious divide. (As it says in the post.) Though it's possible the people who think a theoretical future AI will turn the planet into paperclips have merely forgotten that perpetual motion machines aren't possible.

There is nothing religious with thinking about whether failure modes of advanced artificial intelligence can permanently destroy large parts of the reality that humans care about. Just like there was nothing religious in thinking about whether the first atomic bombs could start a chain reaction that would destroy all life on Earth. Part of such precautionary planning involves asking whether such an accident could hap…

The post, which calls Eliezer a "prophet" who says people should drop everything in their lives to work on AI safety, agrees with me.

> Present a counterargument if you feel strongly about it, and see whether that will stand on its own merit.

This is a bad way to talk to rationalists because it's what they think solves everything and is the reason they're convinced an AI is going to enslave them. As long as you're actually right, saying "no that's dumb and not worth worrying about" is superior to logical arguments about things you can't have logical arguments about (because there are unenumerable "unknown unknowns" in the future). This is called "metarationality".

e.g. Someone could decide to kill you because they don't like one of your posts (1). Is there any finite amount of work you could do to stop this? No (2). Should you worry about this? No (3).

You can't logically prove the 2->3 step, nor can you calculate the probability of it being a problem, but it still doesn't seem to be a problem.

Re: OpenAI

#138

Post by astrange » Sat, Jun 18, 2022, 10:25 PM UTC

deadpannini wrote: post #130

tome wrote: post #125

Earlier quoted context omitted.

> I just think the language of motivation and desire makes it harder to see the risks, because it introduces irrelevant questions into the the conversation like "how can machines want something?" Yet discourse on existential AI risks is predicated on something like a "goal" (e.g. to maximise paperclips). Notions like "goal" also make it harder to see clearly what we are actually discussing. > the term "intelligence"…

AIs absolutely do have goals, determined by their reward functions. Yes, "intelligence" is a deeply loaded term. It just doesn't matter in the context of the discussion here, so far as I've seen.its ambiguities haven't been relevant.

> AIs absolutely do have goals, determined by their reward functions.

You're confusing "AIs" (existing ML models) with "AGIs" (theoretical things that can do anything and are apparently going to take over the world). Not only is there not proof AGIs can exist, there isn't proof they can be made with fixed reward functions. That would seem to make them less than "general".

Re: OpenAI

#139

Post by astrange » Sat, Jun 18, 2022, 10:29 PM UTC

fny wrote: post #78

Naive question: Isn't the easiest way to prevent an general purpose AI from taking over the world to not make a general purpose AI?

If you want to stop the world from containing general intelligence, you'd have to stop everyone from having children, which are equally generally intelligent to AGIs (but possibly less specifically intelligent) and are even more dangerous since then actually exist.

The reason people don't accuse every random child of possibly ending the world is because things that actually exist are just less exciting.

Re: OpenAI

#140

Post by astrange » Sat, Jun 18, 2022, 10:45 PM UTC

c1ccccc1 wrote: post #123

astrange wrote: post #17

Earlier quoted context omitted.

It's a secular/religious divide. (As it says in the post.) Though it's possible the people who think a theoretical future AI will turn the planet into paperclips have merely forgotten that perpetual motion machines aren't possible.

This is your friendly physics reminder that perpetual motion machines have nothing to do with this. It's hard to turn the whole planet into paperclips because paperclips are mostly made of iron, while the planet contains many other elements. Of course, with a high enough level of technology, it might be possible to fuse together the non-iron elements, so that you would end up with just a bunch of iron nuclei. This wo…

It's less than perpetual, but a planet is a lot of raw material to work through for a machine without it breaking down, if the machine's also eaten all the people who can repair it.