Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Douglas Hofstadter changes his mind on Deep Learning and AI risk

lesswrong.com

Post reply on HN
Print view View original

181–190 of 520 posts

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#181

Post by facu17y »

There is no AI risk. There is only the risk of bad or "unfavorable" actors using AI.

AI in a killer drone unleashed on civilians? The bad actor is the one who deployed this weapon.

AI given agency and goal maximization ending up gaining physical form all on its own and killing people? or hacking into bank accounts to enrich its creator?

The latter more likely than the former, but for cyber-offensive AI there is cyber-defensive AI.

Musk lately admitted that he's an AI accelerationist (following lots of the e/acc people and liking their posts) and despite his dystopian view of AI he's pushed it very hard at Tesla. He just wants the US to give him control of it (under the pretext that no one else can manage it safely.)

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#182

Post by wilg »

ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

IMHO, it's simply that predicting the future is difficult, and people disagree on what matters.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#183

Post by ses1984 »

pbw wrote: post #147

Hofstadter says humans will be like cockroaches compared to AI. This is an oft-repeated line: sometimes we are ants or bacteria. But I think these comparisons might be totally wrong. I think it's very possible there's a Intelligence Completeness theorem that's analogous to Turing Completeness. A theorem that says intelligence is in some ways universal, and that our intelligence will be compatible with all other forms…

I think you’re extrapolating from yourself and you think you could learn any field given enough time. What about the type of person with a fixed mindset who thinks they aren’t good at math or chemistry, if ai can’t train them for whatever reason, even if the reason is the person is stubborn and/or willfully ignorant, are they more like a cockroach than a person?

What if someone tries really hard for a long time and can’t learn a field? Do they fail the intelligence test, or does their teacher?

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#184

Post by jrflowers »

ethanbond wrote: post #175
jrflowers wrote: post #172

Earlier quoted context omitted.

Watching the first dot matrix printer output a page of text: “This might print out text that summons Cthulhu”

Is all that money flowing into AI development and deployment in order to produce pieces of paper with text on them? Nope, it's to produce society-scale transformation. So even the optimists wouldn't buy your analogy.

>produce pieces of paper with text on them?

This is a good point — No. LLMs are meant to produce text on a screen, not paper, so I guess it’s more like

Seeing the “it is now safe to shut off your PC” text on a screen for the first time: “I am become Death, destroyer of worlds”

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#185

Post by ke88y »

wilg wrote: post #178
ke88y wrote: post #171

Earlier quoted context omitted.

> Not the guys working at OpenAI or anthropic or deep mind. A lot of them do. A huge percent of people who aren't speaking out are rolling their eyes. But what are you supposed to do? Contradict your boss's boss's boss?

> But what are you supposed to do? Contradict your boss's boss's boss? Yes? Obviously?

> Yes? Obviously?

I mean, I agree, obviously.

My point is that most people don't. And I think for two reasons.

The first and more important reason is that most people aren't involved in The Discourse and don't want to be involved in The Discourse. That's probably 99.9% of the Silent Majority -- they simply don't want to talk about anything on HN or Twitter. They view it as a waste of time or worse. and they aren't wrong. I don't think I am changing any minds here and meanwhile the personal insults kind of suck my energy a bit. So it's mostly a waste of time.

The second reason is that some don't even want to even pseudo-anonymously say something that might get them into deep water at work.

I'm obviously not describing myself, of course. I am here, aren't I :) But I am describing the vast majority of scientists. Keep in mind that most people don't dream of being on the proverbial TED stage and that those who do disproportionately end up on the stage and therefore determine what The Discourse will be.

Big Egos == "my work is existentially important" == all the yelling about x-risk. It's mostly ego.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#186

Post by skepticATX »

ALittleLight wrote: post #156
ke88y wrote: post #134

Earlier quoted context omitted.

> We contacted approximately 4271 researchers who published at the conferences NeurIPS or ICML in 2021... we found email addresses in papers published at those conferences, in other public data, and in records from our previous survey and Zhang et al 2022. We received 738 responses, some partial, for a 17% response rate. Anyone who has reviewed for, or attended, or just read the author lists of ICML/NeurIPS papers is…

On the one hand, we have "the researchers and PhD's" that you know. On the other, we have published surveys of academics and an open letter from noted experts in the field. One side seems to have much better quality of evidence. "All the oil and gas engineers I work with say climate change isn't a thing." Hmm, wow, that's really persuasive! What would you consider evidence of a significant AI risk? From my point of v…

How about someone actually articulate the details of how this supposed super intelligence will be built, what about it’s architecture means that it has no guardrails, how it will bypass numerous real world frictions to achieve its nefarious goals?

Arguments that presuppose a god-like super intelligence are not useful. Sure, if we create a system that is all powerful I’d agree that it can destroy humanity. But specifically how are we going to build this?

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#187

Post by OkayPhysicist »

startupsfail wrote: post #105
ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

Don’t forget that most of these engineers down in the trenches can’t see further than their battlefield. You actually need someone with vision and track record of doing right predictions and placing right technology bets. Ask engineers that had placed their bet on deep learning and generative models back when discriminative models and support vector machines were a rage of dat, a few years before Alexnet (I’m one of…

If you have a PHD, you scouted the battlefield.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#188

Post by mattkevan »

I could be wildly wrong about this (oh hi upcoming robot overlords), but the current AI hype feels similar to something I experienced when Alexa and Google Assistant was the rage.

At the time I worked for a big community site and we often had people pitching ideas for voice assistant apps. However, having actually read the documentation for these things, I knew that they were surprisingly stupid and the grand ideas people had were far closer to sci-fi than something that could actually be built.

I’m not an AI expert, though I have been building and training models for a few years, but despite being good at things that are hard with traditional programs, they’re still surprisingly stupid and most of the discourse seems closer to sci-fi than their actual capabilities.

I’m more worried about these things being implemented badly by people who either bought the sci-fi hype or just don’t care about the drawbacks. E.g. being trained on faulty or biased data, being put in a decision-making role with no supervision or recourse, or even being used in a function that isn’t suitable in the first place.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#189

Post by kazinator »

ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

> Why is that?

Because their work cheerfully presents statements similar to "the middle letter of 'cat' is Z" as the unvarnished truth.

(Would be my guess.)

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#190

Post by pbw »

ses1984 wrote: post #183
pbw wrote: post #147

Hofstadter says humans will be like cockroaches compared to AI. This is an oft-repeated line: sometimes we are ants or bacteria. But I think these comparisons might be totally wrong. I think it's very possible there's a Intelligence Completeness theorem that's analogous to Turing Completeness. A theorem that says intelligence is in some ways universal, and that our intelligence will be compatible with all other forms…

I think you’re extrapolating from yourself and you think you could learn any field given enough time. What about the type of person with a fixed mindset who thinks they aren’t good at math or chemistry, if ai can’t train them for whatever reason, even if the reason is the person is stubborn and/or willfully ignorant, are they more like a cockroach than a person? What if someone tries really hard for a long time and c…

I'm talking about the entire human species, not myself or any one person. I'm saying that humans relating to AIs would not be like cockroaches relating to people. Cockroaches don't have human-level language, but we do, and I'm proposing it is generative and extensible enough to explain any idea. I'm proposing there's non-intelligent species and intelligent species, but there's no intelligent++ species that would look down on us as cockroaches. I'm claiming that won't happen.
Post reply on HN