Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Douglas Hofstadter changes his mind on Deep Learning and AI risk

lesswrong.com

Post reply on HN
Print view View original

101–110 of 520 posts

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#101

Post by scandox »

ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

I think something weird is happening but I think it's what Hofstadter stated in the interview. The ground under his work has shifted massively and he's disturbed by that and that is affecting his judgement.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#102

Post by zone411 »

ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

That's not what this survey shows: https://wiki.aiimpacts.org/doku.php?id=ai_timelines:predicti...

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#103

Post by svieira »

Animats wrote: post #89

"And so it makes me feel diminished. It makes me feel, in some sense, like a very imperfect, flawed structure compared with these computational systems that have, you know, a million times or a billion times more knowledge than I have and are a billion times faster. It makes me feel extremely inferior." Although he did real work in physics, Hofstadter's fame comes from writing popular books about science which explai…

Aristotle was right - or else the electronic calculator, the abacus, and rocks know how to do arithmetic too. (One needs to specify what "knowing" means before smiling at the hairless apes of the past).

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#104

Post by momojo »

JohnFen wrote: post #97
pipo234 wrote: post #31

Earlier quoted context omitted.

Might have completely missed something, but I thought AR/VR was (and still is) a solution looking for a problem. Have they finally stumbled into something people want to do with them, beyond games and porn?

> Have they finally stumbled into something people want to do with them, beyond games and porn? Most people don't want it for games and porn either -- although those two things are the only obvious mass-market applications. There are lots of other real uses, but they're all niche. It's hard to come up with a real, mainstream use that would drive adoption in the general public.

Agreed. During my time at an AR startup, most of our interest was from niche players or one-offs. Current company (biotech, data) is genuinely interested in VR + rendered anatomy data for client reports: https://www.syglass.io/

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#105

Post by startupsfail »

ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

Don’t forget that most of these engineers down in the trenches can’t see further than their battlefield.

You actually need someone with vision and track record of doing right predictions and placing right technology bets.

Ask engineers that had placed their bet on deep learning and generative models back when discriminative models and support vector machines were a rage of dat, a few years before Alexnet (I’m one of such engineers). I’d bet the answer will be different.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#106

Post by echelon »

zone411 wrote: post #102
ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

That's not what this survey shows: https://wiki.aiimpacts.org/doku.php?id=ai_timelines:predicti...

If these people were so concerned, they'd by shouting from the hilltops and throwing their entire life savings into stopping us. They would organize workplace walkouts and strikes. There would be protests and banners. Burning data centers.

Eliezer is one of a handful of people putting their reputation on the line, but that's mostly because that was his schtick in the first place. And even so, his response has been rather muted relative to what I'd expect from someone who thinks the imminent extinction of our species is at hand.

Blake Lemoine's take at Google has been the singular act of protest in line with my expectations. We haven't seen anything else like it, and that speaks volumes.

As it stands, these people are enabling regulatory capture and are doing little to stop The Terminator. Maybe they don't actually feel very threatened.

Look at their actions, not their words.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#107

Post by ralphc »

When I read about the dangers of AI I'm reminded of the feeling I had after reading Jeff Hawkin's "On Intelligence". He talked about simulating the neocortex to do many of the things that deep learning and LLM's are doing now.

His research may or may not be a dead end but his work, and this work, to me seems like we're building the neocortex layer without building the underlying "lizard brain" that higher animals' brains are built upon. The part of the brain that gives us emotions and motivations. The leftover from the reptiles that drive animals to survive, to find pleasure in a full belly, to strive to breed. We use our neocortex and planning facilities but in a lot of ways it's just to satisfy the primitive urges.

My point being, these new AIs are just a higher level "newcortexes" with nothing to motivate them. They can do everything but don't want to do anything. We tell them what to do. The AIs by themselves don't need to be feared, we need to fear what people with lizard brains use them for.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#108

Post by DennisP »

lolinder wrote: post #87
DennisP wrote: post #62

Earlier quoted context omitted.

As long as the AI is less generally intelligent than people, it makes sense to be more afraid of the people. Once the AI is more intelligent than the smartest people, it's more sensible to be most afraid of the AI.

The implication of OP's statement is that they don't believe that AGI is on the horizon, and I'm inclined to agree. This feels a lot like the hype surrounding self-driving cars a few years back, where everyone was convinced fully autonomous vehicles were ~5 years away. It turned out that, while the results we had were impressive, getting the rest of the way to fully replacing humans was much, much harder than was gen…

That may well be the case, but it's still worth thinking about longer-term risks. If it takes, say, forty years to get to AGI, then it's still pretty sobering to consider a serious threat of extinction, just forty years away.

Most of the arguments over what's worth worrying about are people talking past each other, because one side worries about short-term risks and the other side is more focused on the long term.

Another conflict may be between people making linear projections, and those making exponential ones. Whether full self-driving happens next year or in 2050, it will probably still look pretty far away, when it's really just a year or two from exceeding human capabilities. When it's also hard to know exactly how difficult the problem is, there's a good chance that these great leaps will take us by surprise.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#109

Post by gamegoblin »

jameshart wrote: post #75

Interesting to hear him say this: > And I would never have thought that deep thinking could come out of a network that only goes in one direction, out of firing neurons in only one direction. And that doesn't make sense to me, but that just shows that I'm naive. I think people maybe miss that LLM output does involve a ‘loop’ back - maybe even a ‘strange’ loop back, and I’m surprised to see Hofstadter himself fail to…

Just being a part of any auto-regressive system does not contradict his statement.

Go look at the GPT training code, here is the exact line: https://github.com/karpathy/nanoGPT/blob/master/train.py#L12...

The model is only trained to predict the next token. The training regime is purely next-token prediction. There is no loopiness whatsoever here, strange or ordinary.

Just because you take that feedforward neural network and wrap it in a loop to feed it its own output does not change the architecture of the neural net itself. The neural network was trained in one direction and runs in one direction. Hofstadter is surprised that such an architecture yields something that looks like intelligence.

He specifically used the correct term "feedforward" to constrast with recurrent neural networks, which GPT is not: https://en.wikipedia.org/wiki/Feedforward_neural_network

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#110

Post by ke88y »

gamegoblin wrote: post #57
gwright wrote: post #8

Hofstadter makes the claim that "these LLMs and other systems like them are all feed-forward". That doesn't sound right to me, but I'm only a casual observer of LLM tech. Is his assertion accurate? FWIW, ChatGPT doesn't think so. :-)

All the other responses to you at the time of writing this comment are confidently wrong. Definition of Feedforward (from wiki): ``` A feedforward neural network (FNN) is an artificial neural network wherein connections between the nodes do not form a cycle.[1] As such, it is different from its descendant: recurrent neural networks. ``` Hofstadter expected any intelligent neural network would need to be recurrent, ie…

Right. I'm not at all sure what the siblings are talking about. I suspect at least one is confusing linear with feed-forward?

But I'm also surprised that Hofstadter keys in on this so heavily. The fact that he wrote an entire pop-sci book on recursion would, in my mind, make him (1) less surprised that AR and R aren't so dissimilar and (2) more sensitive to the sorts of issues that make R more difficult to get working in practice.

(In my mind, differentiating between auto-regressive and recursive in this case is kind of the same as differentiating between imperative loops and recursion -- there are extremely important differences in practice but being surprised that a program was written using while loops where you imagined left-folds would be absolutely required seems a bit... odd.)

Post reply on HN