Douglas Hofstadter changes his mind on Deep Learning and AI risk • Hacker News Board

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#371

Post by lacrimacida » Tue, Jul 04, 2023, 3:19 AM UTC

ke88y wrote: post #250

jrumbut wrote: post #244

Earlier quoted context omitted.

> On the contrary, many of these people have PhDs, have been leading research agendas in this field for decades, have been in senior leadership roles for a long time, have launched successful products, etc. etc. Society should definitely hear from those whose careers depend on continued AI research (and I fall into this group myself), but we are a hopelessly biased group. Advancements in AI are going to change everyo…

FWIW, my bias leans the opposite direction. I benefit from AI risk hype. I agree with everything you said and I think exaggerating capabilities and risks distracts society from doing that important work.

Cashing in on hype is short sighted and selfish.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#372

Post by rixed » Tue, Jul 04, 2023, 3:21 AM UTC

ALittleLight wrote: post #358

ke88y wrote: post #355

Earlier quoted context omitted.

> Presumably the experts who feel differently about AI x-risk have similar disregard for your opinion. That you aren't persuaded by experts who disagree doesn't show anything. Of course. I was merely explaining the reason why my response would focus on x-risk rather than better surveys, and why. > but it's not like the oil and gas engineers dismissing climate change risks would describe themselves as self-interested…

The reason you aren't persuasive is because you just ignore arguments without responding to them. At least, you've done so here - perhaps you've responded other places with your decade of experience. When presented with surveys and claims of experts, you just dismiss those out of hand "I'm an expert too, therefore ignore other experts." When presented with an argument you just say "That's religious dogma and I reject…

Could you elaborate on 3? It's easy to envision how AI can be used by $powerful to improve its control of $powerless, even before AGI, but how an AI would grab power from $powerful and then behave even worse is harder to imagine.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#373

Post by lacrimacida » Tue, Jul 04, 2023, 3:24 AM UTC

TedDoesntTalk wrote: post #351

dlkf wrote: post #288

Earlier quoted context omitted.

> You actually need someone with vision and track record of doing right predictions and placing right technology bets. Hofstadter does not exactly fit this description.

Why not? Serious question

I suspect OP that your respnding to thinks something like … Hofstadter is not known to have made himself uber-rich therefore he’s not worth taking advice from or something along those lines

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#374

Post by kamaal » Tue, Jul 04, 2023, 3:31 AM UTC

saulpw wrote: post #321

pbw wrote: post #198

Earlier quoted context omitted.

My point is you cannot teach a cockroach calculus, but if AIs invent a new type of math, they would be able to teach it to us. That's my claim. So the analogy of "we are cockroaches compared to the AI" is wrong, that won't be the case. Once you have "enough" intelligence to have a complex language, like we do, I'm claiming you are in the club of intelligent species, and all species in that club can communicate ideas…

AlphaZero invented new moves in the game of Go, but it can't 'teach' them to us, it can only show us the moves and let us figure it out for ourselves (which we're doing). But note that despite this transfer of knowledge, humans didn't rise up to the level of AlphaZero, and they may never be able to. As a sibling comment points out, some things are computationally bound--and humans have a limit to computational abilit…

>>but it can't 'teach' them to us

I always thought you could ask GPT to illustrate the steps it took to arrive at the answer. I mean it can take your through the process it went through to arrive at the answer. Its as close you get to an explanation.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#375

Post by quesera » Tue, Jul 04, 2023, 4:01 AM UTC

ALittleLight wrote: post #358

ke88y wrote: post #355

Earlier quoted context omitted.

> Presumably the experts who feel differently about AI x-risk have similar disregard for your opinion. That you aren't persuaded by experts who disagree doesn't show anything. Of course. I was merely explaining the reason why my response would focus on x-risk rather than better surveys, and why. > but it's not like the oil and gas engineers dismissing climate change risks would describe themselves as self-interested…

The reason you aren't persuasive is because you just ignore arguments without responding to them. At least, you've done so here - perhaps you've responded other places with your decade of experience. When presented with surveys and claims of experts, you just dismiss those out of hand "I'm an expert too, therefore ignore other experts." When presented with an argument you just say "That's religious dogma and I reject…

What is an AI's motivation to "destroy or dominate humanity"? Why would it prefer any specific outcome at all?

If we ever see tribal identity or a will to live as emergent properties of an AI, then things get more interesting.

That would lead quickly to a whole raft of worldwide legal restrictions around the creation of new consciousnesses. Renegade states would be coerced. Rogue researchers would be arrested, and any who evaded detection would be unable to command the resources necessary to commit anything more than ordinary limited terrorism.

The only plausible new risk, I think, is if a state were to lose control of a military AI. But that's movie plot scenario stuff -- real weapons have safety protocols. An AI could theoretically route around some integrated safety protocols (there would be independent watchdog protocols too), but going back to my first point, why would it?

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#376

Post by flangola7 » Tue, Jul 04, 2023, 4:07 AM UTC

JimtheCoder wrote: post #282

flangola7 wrote: post #269

Earlier quoted context omitted.

With respect: *Have you been living under a rock?*

No, I have not. I do not judge humanity as a whole based on a very vocal minority...

I didn't say anything about judging people. I hold no ill will here. But I've ran trust&safety and now cybersecurity for a decade - humans are not cut out for memetic warfare. Humans cannot handle normal but internet-speed incidental misinformation without society eating itself, much less coordinated disinformation. Democracy is eroding all across the globe and that's without generative systems that will make the problem 1000x worse.

We evolved to pick berries, not discriminate and identify distant manipulative actors with extensive resources. "You are not immune to propaganda."

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#377

Post by ChatGTP » Tue, Jul 04, 2023, 4:13 AM UTC

jimbokun wrote: post #265

ralphc wrote: post #107

When I read about the dangers of AI I'm reminded of the feeling I had after reading Jeff Hawkin's "On Intelligence". He talked about simulating the neocortex to do many of the things that deep learning and LLM's are doing now. His research may or may not be a dead end but his work, and this work, to me seems like we're building the neocortex layer without building the underlying "lizard brain" that higher animals' br…

Or until someone decides to give AI a "lizard brain" and sets it loose.

Presumably this will be easy, once you have a Neocortex with an IQ of 500, creating a lizrd brain is only moments away.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#378

Post by [deleted] » Tue, Jul 04, 2023, 4:16 AM UTC

pbw wrote: post #147

Hofstadter says humans will be like cockroaches compared to AI. This is an oft-repeated line: sometimes we are ants or bacteria. But I think these comparisons might be totally wrong. I think it's very possible there's a Intelligence Completeness theorem that's analogous to Turing Completeness. A theorem that says intelligence is in some ways universal, and that our intelligence will be compatible with all other forms…

[deleted]

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#379

Post by dgreensp » Tue, Jul 04, 2023, 4:25 AM UTC

georgeoliver wrote: post #231

> It's a very traumatic experience when some of your most core beliefs about the world start collapsing. And especially when you think that human beings are soon going to be eclipsed. It felt as if not only are my belief systems collapsing, but it feels as if the entire human race is going to be eclipsed and left in the dust soon. While I unfortunately am expecting some people to do terrible things with LLM, I feel l…

I agree. He talks about LLMs surpassing humans in what they can do, but LLMs can’t do anything, really. And I say this as someone who had a heart-to-heart chat with a ChatGPT persona that made me cry, earlier tonight. Manipulating language in a human-like way is powerful, and maybe Hofstadter as an author who makes a living writing down his deep and whimsical, creative thoughts could be replaced (I grew up reading GEB and mean no disrespect). But being a conscious, physical, active being in the world is different from passing a sort of Turing test. The fact that a computer even winning at Go shook him is revealing.

He has the ingredients of being able to keep LLMs in perspective (is there really an “I” or just the illusion of one?), but he doesn’t understand what the computer is doing; it’s not “mechanical” enough for him.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#380

Post by naasking » Tue, Jul 04, 2023, 4:47 AM UTC

ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

> They almost all roll their eyes aggressively at these sorts of prognostications. I'd say 90+% either laugh or roll their eyes. Why is that?

I literally think they haven't even given it any thought. Who seriously tries to extrapolate beyond 5 years in any field? Even to most doomers, 5 years seems safe.

What about 10? 20? 30? Remember, one of the very first things people tried to do was create a self-directed, self-optimizing AI with AutoGPT. How far forward can you project through how many attempts like this with progressively more sophisticated systems before this might produce an outcome that could be considered catastrophic?

If your answer is anything but an emphatic "infinity", then welcome to the doomers. Now we're just debating when and under circumstances this might take place, and not whether it might happen at all.