Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Douglas Hofstadter changes his mind on Deep Learning and AI risk

lesswrong.com

Post reply on HN
Print view View original

51–60 of 520 posts

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#51

Post by pipo234 »

Kiro wrote: post #26
detourdog wrote: post #15

I have more fear of the people running the system than any AI. I also think that AR/VR is more scary than AI. My fear is how poorly rendered AR/VR can be more positive interaction than interactions with the people that surround the observer.

If it's a positive interaction, then what is your fear?

If @detourdog meant AR/VR in the snow crash / Metaverse sense, I guess the apprehension is similar to the discourse about "Leo Tolstoy on why people drink (2014)" a couple of days ago https://news.ycombinator.com/item?id=36526645

People tend to turn to substances / AR/VR / texting / phone calls / read books / ... to take the edge of confronting the sometimes harsh reality. Of course, there is no way in which AR/VR is likely to intrinsically improve interaction, but is is so much worse that we need to worry?

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#52

Post by gautamcgoel »

gwright wrote: post #8

Hofstadter makes the claim that "these LLMs and other systems like them are all feed-forward". That doesn't sound right to me, but I'm only a casual observer of LLM tech. Is his assertion accurate? FWIW, ChatGPT doesn't think so. :-)

No, it is not correct. Transformers have two components: self-attention layers and multi-layer perceptron layers. The first has an autoregressive/RNN flavor, while the latter is feedforward.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#53

Post by CuriouslyC »

pipo234 wrote: post #31
detourdog wrote: post #15

I have more fear of the people running the system than any AI. I also think that AR/VR is more scary than AI. My fear is how poorly rendered AR/VR can be more positive interaction than interactions with the people that surround the observer.

Might have completely missed something, but I thought AR/VR was (and still is) a solution looking for a problem. Have they finally stumbled into something people want to do with them, beyond games and porn?

AR will take off like crazy when they develop lightweight, clear goggles than can display color with a range of opacities. A million jobs are begging for HUDS that guide the worker, but the current equipment isn't there yet.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#54

Post by irchans »

Hofstadter said, "Well, maybe as important as the wheel."

If AI significantly surpasses humanity in cognitive ability, then I think it will have a much bigger impact than the wheel. (I loved GEB and DH's other writings.)

LLMs have really improved a lot of the last two years and they have shown many unexpected capabilities. I am guessing that they will get some more good input (text mostly), a lot more compute, and algorithmic improvements, so that may very well be enough to become better than 99% of humans at tasks that involve only text. Tasks that require video or image processing may be a little bit more challenging. Having very smart AI's controlling robots may just be five years away. (I recently lost a bet about autonomous driving. Five years ago, I thought that autonomous cars would be better than human drivers by now.)

I'm frightened by what AI will become over the next 10 years.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#55

Post by ImHereToVote »

norir wrote: post #49

> If not, it also just renders humanity a very small phenomenon compared to something else that is far more intelligent and will become incomprehensible to us, as incomprehensible to us as we are to cockroaches. This seems like the reaction of an atheist who already overvalued human intelligence.

So someone who doesn't believe in the things that you believe?

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#56

Post by FrustratedMonky »

hospitalJail wrote: post #3

Lets take someone, who is past their prime and interview them on a topic they have never worked on. Then we can mine it for quotes! >"I never imagined that computers would rival, let alone surpass, human intelligence. And in principle, I thought they could rival human intelligence. I didn't see any reason that they couldn't." Yeah, so he got fooled by LLMs and hasnt been burned by it failing to do the most basic logi…

>"Lets take someone, who is past their prime and interview them on a topic they have never worked on. Then we can mine it for quotes!"

So. Don't interview anybody over 40? Who judges who is past their prime?

Mining for quotes is most interviews. Isn't this why interviews happen?

He's been in the AI field for what? 30 Years? Has multiple books.

I think he has earned enough respect to have an opinion, and it is probably worth more than most people tossing out ad-hoc opinions on AI in the last few months. Better than some 'programmers', who are so in-the-weeds they have lost track of what they are building.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#57

Post by gamegoblin »

gwright wrote: post #8

Hofstadter makes the claim that "these LLMs and other systems like them are all feed-forward". That doesn't sound right to me, but I'm only a casual observer of LLM tech. Is his assertion accurate? FWIW, ChatGPT doesn't think so. :-)

All the other responses to you at the time of writing this comment are confidently wrong.

Definition of Feedforward (from wiki):

``` A feedforward neural network (FNN) is an artificial neural network wherein connections between the nodes do not form a cycle.[1] As such, it is different from its descendant: recurrent neural networks. ```

Hofstadter expected any intelligent neural network would need to be recurrent, ie looping back on itself (in the vein of his book “I am a strange loop”).

GPT is not recurrent. It takes in some text input, does a fixed amount of computation in 1 pass through the network, then outputs the next word. He is surprised it doesn’t need to loop for an arbitrary amount of time to “think about” what to say.

Being put into an auto-regressive system (where the N-th word it generates gets appended to the prompt that gets sent back into the network to generate the N+1th word) doesn’t make the neural network itself not Feedforward.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#58

Post by submeta »

chairhairair wrote: post #30
EA-3167 wrote: post #17

Earlier quoted context omitted.

Well... words and decades of circumstances. If you removed the circumstances (the religion, the conflict, the money, geography, etc) then the words would be absolutely hollow. I think we tend to credit words where often circumstances are doing the heavy lifting. For example try to start a riot with words in Rodeo Drive. Now try to do it in Nanterre. Or better yet, try to start a riot in Nanterre before a 17 year was…

Many people are in desperate situations. Most of them are not convinced to do much about it. Words can aim discontent. Were the economic conditions in Weimar Germany that much worse than many places today?

Reminds me of the idea of a "tipping point." When we hit this point, words can really get people moving. This has been true for big changes like revolutions and movements, like Black Lives Matter, #MeToo, or Fridays for Future.

Words might not do much without the right situation, like the parent mentioned with Rodeo Drive and Nanterre. But they're still important. They can guide people's anger and unhappiness.

In the case of Weimar Germany, the severe economic instability and social discontent following World War I created a fertile ground for radical ideologies to take root. When these conditions coincided with persuasive rhetoric, it catalyzed significant societal change. So, while words can indeed be powerful, they're often most effective when spoken into pre-existing circumstances of tension or dissatisfaction. They can then direct this latent energy towards a specific course of action or change.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#59

Post by mlyle »

jurgenaut23 wrote: post #9

I read Hofstadter's GEB and Tegmark's Our Mathematical Universe, and of course I developed a rather fond admiration of these brilliant minds. For some reason, both of them have developed a profound aversion and fear of what they consider an existential threat. I have a solid theoretical understanding of these systems, and I spent 15 years studying, building, and deploying them at scale and for diverse use cases. The…

I think there's a number of just crappy scenarios that can come with LLMs and other generative AI:

- Further trashing our public discourse: making truth even more uncertain and valuable information even harder to find. We're not doing great with social media, and it's easy to envision that generative AI could make it twice as bad.

- Kneecapping creative work by commoditizing perhaps half of it. There's going to be a lot of bodies fighting over the scraps that remain.

- Fostering learned helplessness. I think you need to be a good writer and thinker to fully use LLMs' capabilities. But a whole lot of kids are looking at machines "writing perfectly" and think they don't need to learn anything.

We don't need any further progress for these things to happen. Further progress may be even scarier, but the above is scary enough.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#60

Post by pipo234 »

CuriouslyC wrote: post #53
pipo234 wrote: post #31

Earlier quoted context omitted.

Might have completely missed something, but I thought AR/VR was (and still is) a solution looking for a problem. Have they finally stumbled into something people want to do with them, beyond games and porn?

AR will take off like crazy when they develop lightweight, clear goggles than can display color with a range of opacities. A million jobs are begging for HUDS that guide the worker, but the current equipment isn't there yet.

> A million jobs are begging for HUDS that guide the worker [..]

I have a pretty dull, comfortable desk job and lack the imagination to come up with any. Can you name some?

(I've been wrong before when I was skeptical when everyone was hyped about iPad, Deep Learning, etc. so please convince me about Apple Vision, Mark's metaverse, or google's glasses and paint what might be in 5 years.)

Post reply on HN