Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Douglas Hofstadter changes his mind on Deep Learning and AI risk

lesswrong.com

Post reply on HN
Print view View original

191–200 of 520 posts

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#191

Post by ethanbond »

jrflowers wrote: post #184
ethanbond wrote: post #175

Earlier quoted context omitted.

Is all that money flowing into AI development and deployment in order to produce pieces of paper with text on them? Nope, it's to produce society-scale transformation. So even the optimists wouldn't buy your analogy.

>produce pieces of paper with text on them? This is a good point — No. LLMs are meant to produce text on a screen, not paper, so I guess it’s more like Seeing the “it is now safe to shut off your PC” text on a screen for the first time: “I am become Death, destroyer of worlds”

Brilliant proof of exactly my point. When it comes to discussing any possible outcome other than utopia, suddenly the power of these tools drops to zero! Remarkable :)

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#192

Post by pookha »

arisAlexis wrote: post #143
ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

Fake news. 350 people including the top scientists from the top labs signed the petition. Your local univ AI researcher rolls his eyes. Not the guys working at OpenAI or anthropic or deep mind.

This reads like something out of "The Man that Corrupted Hadleyburg"...All of these virtuous experts racing to out-signify the other guy...Makes me want to root for the calculus\linear algebra over the breathless farty expert.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#193

Post by zucked »

mattkevan wrote: post #188

I could be wildly wrong about this (oh hi upcoming robot overlords), but the current AI hype feels similar to something I experienced when Alexa and Google Assistant was the rage. At the time I worked for a big community site and we often had people pitching ideas for voice assistant apps. However, having actually read the documentation for these things, I knew that they were surprisingly stupid and the grand ideas p…

I think there's actually two camps of "AI-fear" right now -- one is that they'll become superhuman smart, gain some form of "sentience", and decide to not play along with our games anymore. These are the loudest voices in the room. I suspect this may happen eventually, but the bigger risk in my book is the second group -- the fear that they'll be given decision-making responsibility for all manner of things, including ones they probably don't have any business being responsible for. I'll go on record that they're going to do a suitable job 90% of the time - but the 10% of time they mess up, it's going to be cataclysmic bad.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#194

Post by mvdtnz »

I was previously undecided on the question of the existential risk of AI. But yesterday I listened to a Munk Debate[0] on the topic and found the detractors of the moot "AI research and development poses an existential threat" so completely unconvincing that if those people are the top minds in the field I am now genuinely concerned. Their arguments that what they are doing is not a risk basically boil down to "nuh uhhh!"

[0] https://pca.st/episode/1fac0e97-1dcc-4b4c-ba50-d2776e6f9d59

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#195

Post by tiffanyg »

ke88y wrote: post #134
zone411 wrote: post #102

Earlier quoted context omitted.

That's not what this survey shows: https://wiki.aiimpacts.org/doku.php?id=ai_timelines:predicti...

> We contacted approximately 4271 researchers who published at the conferences NeurIPS or ICML in 2021... we found email addresses in papers published at those conferences, in other public data, and in records from our previous survey and Zhang et al 2022. We received 738 responses, some partial, for a 17% response rate. Anyone who has reviewed for, or attended, or just read the author lists of ICML/NeurIPS papers is…

You may be far better informed than many you're arguing with, but casual dismissal IS foolish.

The biggest potential / likely issues here aren't mere capabilities of systems and simple replacement of humans here and there. It's acceleration of "truth decay", accelerating and ever-more dramatic economic, social, and political upheaval, etc.

You do not need "Terminator" for there to be dramatic downsides and damage from this technology.

I'm no "doomer", but, looking at the bigger picture and considering upheavals that have occurred in the past, I am more convinced there's danger here than around any other revolutionary technologies I've seen break into the public consciousness and take off.

Arguments about details, what's possible and what's not, limitations of systems, etc. - missing the forest for the trees IMO. I'd personally suggest keeping an eye on white papers from RAND and the like in trying to get some sense of the actual implications in the real world, vs. picking away at the small potatoes details-levels arguments...

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#196

Post by YetAnotherNick »

tyre wrote: post #164
ralphc wrote: post #107

When I read about the dangers of AI I'm reminded of the feeling I had after reading Jeff Hawkin's "On Intelligence". He talked about simulating the neocortex to do many of the things that deep learning and LLM's are doing now. His research may or may not be a dead end but his work, and this work, to me seems like we're building the neocortex layer without building the underlying "lizard brain" that higher animals' br…

This is a great way to put it. The counter to "current generation AI is terrifying" seems to fall along the lines of it not being nearly as close to AGI as the layperson believes. But I don't think that matters. I don't believe that LLMs or image/voice/video generative models need to do much beyond what they can do today in order to wreck civilization level disaster. They don't need to become Skynet, learn to operate…

GPT4 really "doesn't want" be racist against blacks. Like try talking to it, there is(was) no hard filter but I really can't grasp how the bias can go this deep just by finetuning. GPT-4 definitely know all the stereotypes of blacks, but good luck getting it to engage on any.

Suppose we finetune it exactly like that but say opposing democracy or freedom or peace or any other thing we value. And let it create the propoganda or convince people for the same by free posting on the net. "As an AI language model" line could easily be removed.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#197

Post by bobcostas55 »

phillipcarter wrote: post #157
ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

I've come to learn that a lot of the rationalist crowd are really just fanfiction authors. That's fine - people should be able to do that - but I don't like how they're given the limelight on things that they generally have little expertise and hands-on knowledge with. I want to like them and want to have them engaged in discourse, but I find them so insufferable. Not to mention the sub-crowd of rationalists that is…

[flagged]

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#198

Post by pbw »

wslh wrote: post #165
pbw wrote: post #147

Hofstadter says humans will be like cockroaches compared to AI. This is an oft-repeated line: sometimes we are ants or bacteria. But I think these comparisons might be totally wrong. I think it's very possible there's a Intelligence Completeness theorem that's analogous to Turing Completeness. A theorem that says intelligence is in some ways universal, and that our intelligence will be compatible with all other forms…

> A theorem that says intelligence is in some ways universal, and that our intelligence will be compatible with all other forms of intelligence, even if they are much "smarter". I would not be so reductionist. Intelligence doesn't seem to be an universal thing, even IQ (a human invented metric) is measured in terms of some statistics. If you have an IQ of ~60 you have intelligence but a completely different one from…

My point is you cannot teach a cockroach calculus, but if AIs invent a new type of math, they would be able to teach it to us. That's my claim. So the analogy of "we are cockroaches compared to the AI" is wrong, that won't be the case.

Once you have "enough" intelligence to have a complex language, like we do, I'm claiming you are in the club of intelligent species, and all species in that club can communicate ideas with each other. Even if the way they natively think is quite different.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#199

Post by marginalia_nu »

ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

There's this thing where prominent scholars in one field sometimes consider themselves experts in others...

https://en.m.wikipedia.org/wiki/Nobel_disease

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#200

Post by FollowingTheDao »

ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

> But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative models, and previous generations, in some cases for decades? They almost all roll their eyes aggressively at these sorts of prognostications. I'd say 90+% either laugh or roll their eyes.

> Why is that?

It seems pretty obvious that one would likely not criticize something that your are actively profiting from.

And I know a lot of alcoholics who do not criticize drinking as well.

Post reply on HN