Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Douglas Hofstadter changes his mind on Deep Learning and AI risk

lesswrong.com

Post reply on HN
Print view View original

311–320 of 520 posts

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#311

Post by kneebonian »

timmytokyo wrote: post #298
phillipcarter wrote: post #157

Earlier quoted context omitted.

I've come to learn that a lot of the rationalist crowd are really just fanfiction authors. That's fine - people should be able to do that - but I don't like how they're given the limelight on things that they generally have little expertise and hands-on knowledge with. I want to like them and want to have them engaged in discourse, but I find them so insufferable. Not to mention the sub-crowd of rationalists that is…

The "rationalists" are more like a strange AI cult than a scientific or philosophical program. (I always put "rationalist" in quotes when referring to this group of people, because based on the things these people believe, it's a total misnomer.) At this point they're not even really trying to hide it anymore, with their prophet, Eliezer Yudkowsky, wailing in the wilderness about the imminent end of humanity. We've s…

Reminds me of the reasonabilists from Parks and Rec

> The Reasonabilists named themselves because they believe if people criticize them, it'll seem like they are attacking something reasonable.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#312

Post by ke88y »

FollowingTheDao wrote: post #200
ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

> But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative models, and previous generations, in some cases for decades? They almost all roll their eyes aggressively at these sorts of prognostications. I'd say 90+% either laugh or roll their eyes. > Why is that? It seems pretty obvious tha…

> It seems pretty obvious that one would likely not criticize something that your are actively profiting from.

This doesn't track, since the people criticizing are benefiting even more from the same products/companies.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#313

Post by ke88y »

dist-epoch wrote: post #219
petermcneeley wrote: post #169

Earlier quoted context omitted.

Either Nassim is missing the point or you are misquoting him. I think this is more about should experts be in charge of their own domain.

I'm not misquoting him. Nassim is arguing that RISK is a separate discipline, separate from the domain where risk applies. That a person building AI is not the correct choice for estimating AI risk. You don't ask gun making companies to make policies regarding risk of gun owning in society.

> Nassim is arguing that RISK is a separate discipline, separate from the domain where risk applies. That a person building AI is not the correct choice for estimating AI risk.

I think this is often fair. It's actually one of my primary criticisms of the NeurIPS/ICML survey.

FWIW, people working on AI Safety -- like, actually working on it, not philosophizing about it -- are some of the most incredulous and annoyed about the "AGI => extinction" crowd.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#314

Post by sitkack »

YetAnotherNick wrote: post #196
tyre wrote: post #164

Earlier quoted context omitted.

This is a great way to put it. The counter to "current generation AI is terrifying" seems to fall along the lines of it not being nearly as close to AGI as the layperson believes. But I don't think that matters. I don't believe that LLMs or image/voice/video generative models need to do much beyond what they can do today in order to wreck civilization level disaster. They don't need to become Skynet, learn to operate…

GPT4 really "doesn't want" be racist against blacks. Like try talking to it, there is(was) no hard filter but I really can't grasp how the bias can go this deep just by finetuning. GPT-4 definitely know all the stereotypes of blacks, but good luck getting it to engage on any. Suppose we finetune it exactly like that but say opposing democracy or freedom or peace or any other thing we value. And let it create the prop…

Everything is its own mirror. Found the cure to cancer? Maybe also made a cancer gun at the same time.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#315

Post by ke88y »

13years wrote: post #261
ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

Those with great visibility often are not the greatest minds. However, in a broader sense we would have been served well if philosophizers would have had more input on the development of social media tech. It is an example of engineers knowing a field narrowly and deeply but not understanding the societal consequence. AI may very well fall into the same pattern and it is something I have written out in some detail of…

I think AI definitely will follow a similar path as social media tech.

I don't think that will lead to the extinction of humanity via the development of super human intelligence.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#316

Post by YeGoblynQueenne »

kalkin wrote: post #274
YeGoblynQueenne wrote: post #256

Earlier quoted context omitted.

(Not the OP) In this article Le Cun argues in very concrete technical terms about the impossibility of achieving AGI with modern techniques (yes, all of them): Meta's AI guru LeCun: Most of today's AI approaches will never lead to true intelligence https://www.zdnet.com/article/metas-ai-guru-lecun-most-of-to... Edit: on second thought, he gets maybe a bit too technical at times, but I think it should be possible to f…

I wouldn't describe most of what's in that interview as "very concrete technical" terms - at least when it comes to other people's research programs. More importantly, while it's perfectly reasonable for LeCun to believe in his own research program and not others, "this one lab's plan is the one true general-AI research program and most researchers are pursuing dead ends" doesn't seem like a very sturdy foundation on…

You can ignore that, of course he'll push his research. But he never says that what he does will lead to AGI. He's proposing a way forward to overcome some specific limitations he discusses.

Otherwise, he makes some perhaps subtle points about learning hidden variable models that are relevant to modern discussions about necessarily learning world-models in order to best model text.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#317

Post by fergal_reid »

pbw wrote: post #147

Hofstadter says humans will be like cockroaches compared to AI. This is an oft-repeated line: sometimes we are ants or bacteria. But I think these comparisons might be totally wrong. I think it's very possible there's a Intelligence Completeness theorem that's analogous to Turing Completeness. A theorem that says intelligence is in some ways universal, and that our intelligence will be compatible with all other forms…

Spend some time around a three year old. Human, human intelligence, language skills.

Then try explain quicksort to them. Obvious waste of time.

They wouldn't be much threat in a zero sum strategic interaction either.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#319

Post by mcpackieh »

kalkin wrote: post #215
echelon wrote: post #138

Earlier quoted context omitted.

> If you read the survey Not a nice way to engage in debate. I've spent more time listening to and refuting these arguments than most. Including debating Eliezer. > many concerned researchers don't believe we're at a 90% chance of doom, but e.g. 10%. A 10% chance of an asteroid hitting the earth would result in every country in the world diverting all of their budgets into building a means to deflect it. > So, this t…

"If [people who I disagree with] really believed what they said, they'd do [X Y or Z extreme thing]" is almost always a bad argument. Consider the possibility that instead, the people who actually believe the thing you don't have spent more time thinking seriously about their strategy than you have, and you are not in fact better both at being you and at being them than they are. I'm used to this pattern showing up a…

A less-than-optimal response isn't very indicative of anything, but I think you should definitely think twice when the intensity of the protest is grossly out of proportion with the intensity of the rhetoric.

X-risk people are talking about the complete extermination of humanity but all they do is write internet essays about it. They aren't even availing themselves of standard protesting tactics, like standing outside AI businesses with signs or trying to intimidate researchers. Some form of real protest is table stakes for being taken seriously when you're crying about the end of the world.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#320

Post by lukas099 »

georgeoliver wrote: post #231

> It's a very traumatic experience when some of your most core beliefs about the world start collapsing. And especially when you think that human beings are soon going to be eclipsed. It felt as if not only are my belief systems collapsing, but it feels as if the entire human race is going to be eclipsed and left in the dust soon. While I unfortunately am expecting some people to do terrible things with LLM, I feel l…

> That a computer can play chess better than any human doesn't lessen my personal enjoyment of playing chess.

Agreed. I understand what DH is saying, but I fail to see how it translates into this all-consuming terror of his.

Post reply on HN