Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Douglas Hofstadter changes his mind on Deep Learning and AI risk

lesswrong.com

Post reply on HN
Print view View original

331–340 of 520 posts

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#331

Post by 13years »

ke88y wrote: post #315
13years wrote: post #261

Earlier quoted context omitted.

Those with great visibility often are not the greatest minds. However, in a broader sense we would have been served well if philosophizers would have had more input on the development of social media tech. It is an example of engineers knowing a field narrowly and deeply but not understanding the societal consequence. AI may very well fall into the same pattern and it is something I have written out in some detail of…

I think AI definitely will follow a similar path as social media tech. I don't think that will lead to the extinction of humanity via the development of super human intelligence.

Ironically the destruction to social order by primitive AI may be the very obstacle to achieving AGI.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#332

Post by Sniffnoy »

throwway120385 wrote: post #180
ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

In my opinion the weirdness is that everyone is talking about how AI is so much better than humans and we're all going to get replaced. But almost nobody is talking about how tech companies will use this new technology to abdicate responsibility for the decisions they undertake. If you thought redlining was bad, you should see how mortgage companies treated people when they used an AI to accomplish the same goals. No…

Plenty of people have been talking about that, from my memory?

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#333

Post by ke88y »

ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

Note: I really don't like that my comment is the top comment this story. I think this phenomenon wrt x-risk is real and interesting and worth discussing, but I wish I'd picked a different story to share the thought. (This just happened to be the first related story that popped up since I had the thought.)

There are other more interesting comments that deserve more discussion WRT this article in particular.

If any mods are reading this, please consider this a request to push this comment down so some of the others get more attention. IDK if there's much more to learn from the discussion happening here anyways.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#334

Post by mcint »

ke88y wrote: post #273
kalkin wrote: post #241

Earlier quoted context omitted.

Is there a way that you'd recommend somebody outside the field assess your "90%" claim? Elsewhere in the thread you're dismissive of that one survey - which I agree is weak evidence in itself - and you also deny that the statements of leaders at OpenAI, DeepMind and Anthropic are representative of researchers there, which again may be the case. But how is someone who doesn't know you or your network supposed to asses…

> But how is someone who doesn't know you or your network supposed to assess this? IDK. And FWIW I'm not even sure that the leaders of those organizations all agree on the type and severity of risks, or the actions that should be taken. You could take the survey approach. I think a good survey would need to at least have cross tabs for experience level, experience type, and whether the person directly works on safety…

Someone has been working on survey forecasts, since 2016, https://aiimpacts.org/. As fraught or self-selected as the group might be, someone labors on it.

Unfortunately, fortunately, expectedly, or otherwise, the only people writing about this in a concerted way are the people taking it seriously. And maybe Gary Marcus, whose negative predictions repeatedly became milestones surpassed.

You can read a review of the survey efforts, and complications of the results at https://asteriskmag.com/issues/03/through-a-glass-darkly

  Yogi Berra supposedly said that “prediction is very difficult, especially about the future.”
and the surveys are finding that even achieved milestones are still forecast as a few years out, by most of these carefully sampled experts.

...welcome to the conversation :D.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#335

Post by scrawl »

pbw wrote: post #329
scrawl wrote: post #327

Earlier quoted context omitted.

>smart enough to explain us any idea Are dogs, or pigs, or whales, part of the intelligence club? They are clearly intelligent beings with problem-solving skills. We won't be teaching them basic calculus any time soon.

No non-human animals are in the club that's marked by having a language with an infinitely generative syntax and a large (100,000+ words) and always-growing vocabulary. Intelligence might be a spectrum, but powerful generative language is a step function: you have it or you don't. If you have it, then higher intelligences can communicate complex thoughts to you, if you don't they can't. We have it, so we are in the c…

fair enough. i'm not convinced.

there are many humans who could study mathematics for a lifetime and not be able to comprehend the current best knowledge we possess. i'm one of them. maybe it takes 2 lifetimes. or many more.

a human-level AI operating at machine pace would learn much more than could ever be taught to a human. our powerful generative language capabilities wouldn't matter - it's far beyond our bandwidth. especially so for a superhuman-level AI.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#336

Post by mark_l_watson »

Gern is the best, his writings are usually treasures.

Here, Gern nicely quotes Hoststadter’s concerns about ChatGPT, etc.

Years ago, when I was enjoying Hoststadter’s books, I asked his student Melanie Mitchell for the source code for the CopyCat creativity program they built. She very kindly sent me a copy. I mention this because she recently was one of the four panelists at a Monk Debate on AI safety, and her opinion didn’t jive with what Gern quoted Hoststadter as saying. I agree with Melanie Mitchell’s point of view that we overestimate what LLMs can do. I am super interested in this topic right now because I am 1/5 through writing a new book “Safe For Humans AI.”

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#337

Post by pbw »

fergal_reid wrote: post #317
pbw wrote: post #147

Hofstadter says humans will be like cockroaches compared to AI. This is an oft-repeated line: sometimes we are ants or bacteria. But I think these comparisons might be totally wrong. I think it's very possible there's a Intelligence Completeness theorem that's analogous to Turing Completeness. A theorem that says intelligence is in some ways universal, and that our intelligence will be compatible with all other forms…

Spend some time around a three year old. Human, human intelligence, language skills. Then try explain quicksort to them. Obvious waste of time. They wouldn't be much threat in a zero sum strategic interaction either.

If there were a species whose average adult intelligence was that of a human three-year old, then yes you'd be limited in what you could teach them. But as for what AIs can teach humans, you have to assume we are using competent smart adults. My claim is just what AIs can teach competent smart adults is many many orders of magnitude more than what humans can teach competent smart cockroaches. Thus Hofstadter's analogy is not a good one.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#338

Post by ke88y »

kalkin wrote: post #248
kalkin wrote: post #241

Earlier quoted context omitted.

Is there a way that you'd recommend somebody outside the field assess your "90%" claim? Elsewhere in the thread you're dismissive of that one survey - which I agree is weak evidence in itself - and you also deny that the statements of leaders at OpenAI, DeepMind and Anthropic are representative of researchers there, which again may be the case. But how is someone who doesn't know you or your network supposed to asses…

Relatedly, it'd be helpful if you could point to folks with good ML research credentials making a detailed case against x-risk. The prominent example of whom I'm aware is Yann LeCun, and what I've seen of his arguments take place more in the evopsych field (the stuff about an alleged innate human drive for dominance which AIs will lack) than the field in which he's actually qualified.

TBH, I don't find LeCun particularly convincing. I think he's correct but not persuasive, if that makes sense.

Debunking AI x-risk is a weird thing to spend time on. There's really no up-side, and there aren't a bunch of rich people paying for Institutes and Foundations on Non-Breathtaking-Very-Boring-Safety-Research. Also, most of the arguments in favor of x-risk that lawmakers and laypeople find most convincing are unfalsifiable, so it's a bit like arguing against religion in that respect.

I don't think there are any public intellectuals in this space doing it well. I'm not sure what to make of that. For myself, I make the argument for focusing on concrete safety problems from within the agencies and companies that are allocating resources. I'm not gifted TED talker.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#339

Post by ALittleLight »

ke88y wrote: post #205
ALittleLight wrote: post #156

Earlier quoted context omitted.

On the one hand, we have "the researchers and PhD's" that you know. On the other, we have published surveys of academics and an open letter from noted experts in the field. One side seems to have much better quality of evidence. "All the oil and gas engineers I work with say climate change isn't a thing." Hmm, wow, that's really persuasive! What would you consider evidence of a significant AI risk? From my point of v…

> One side seems to have much better quality of evidence. Since I AM an expert, I care a lot less about what surveys say. I have a lot more experience working on AI Safety than 99% of the ICM/NeurIPS 2021 authors, and probably close to 100% of the respondents. In fact, I think that reviewing community (ICML/NeurIPS c. 2020) is particularly ineffective and inexperienced at selecting and evaluating good safety research…

Presumably the experts who feel differently about AI x-risk have similar disregard for your opinion. That you aren't persuaded by experts who disagree doesn't show anything.

Similarly, you think that x-risk dismissal among the experts you agree with is not self-interested - but it's not like the oil and gas engineers dismissing climate change risks would describe themselves as self-interested liars either. They would likely say, as you realize, "No, my dismissal of the concerns of other experts is legitimate!"

The thing is, it doesn't require a lot of expertise to understand that there is actually an enormous risk and "experts" denying that are simply burying their head in the sand. AI, I use the term expansively, is making huge and rapid progress. We are approaching "human-level" intelligence and there is no guarantee, nor even any indication, that "human-level" is an upper limit. Systems as smart or smarter than we are, that are not controlled for the benefit of humanity, are an existential risk.

There are two basic kinds of risk. First, an AI powered tyranny where a small number of humans dominate everything enabled by AI systems they control - every security camera watched by a tireless intelligence, drones piloted by perfectly loyal intelligences, etc. Second, that the AI systems are not effectively controlled and pursue goals or objectives that are incompatible with human existence. In either case the fact that the AI brings superhuman intelligence to bear means that humanity at large will be overmatched in terms of capabilities.

It is completely possible that neither of these scenarios come to pass, but the fact that one of the scenarios might come to pass is what makes the situation an existential risk - humanity eternally subjected by an irreversible dictatorship, or simply destroyed.

I'm confused about your first item that would convince you of AI risk, but I think the others should all be considered met by ChatGPT. Of course, ChatGPT isn't currently useful in developing new WMDs - but it is absolutely useful in helping to bring someone up to speed on new topics in many different domains. As capabilities advance, why wouldn't ChatGPT using GPT-5, or 6, or 10 be able to helpfully guide the creation of new and better WMDs?

AI isn't currently causing the problems you are worried about but certainly seems on track to do so soon. "Risk" doesn't mean that we are currently in the process of being destroyed by AI, but it does mean that there is a non-negligible possibility that we will soon find ourselves in that process.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#340

Post by ke88y »

kazinator wrote: post #189
ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

> Why is that? Because their work cheerfully presents statements similar to "the middle letter of 'cat' is Z" as the unvarnished truth. (Would be my guess.)

Yeah, working on these systems and seeing how brittle they are and how they're actually built is sobering. I guess from the outside it looks like a few geniuses geniusing with unbounded upside, but from the inside it looks like a hard hacking and hard work that will definitely hit hard limits.
Post reply on HN