Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Douglas Hofstadter changes his mind on Deep Learning and AI risk

lesswrong.com

Post reply on HN
Print view View original

61–70 of 520 posts

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#61

Post by lolinder »

irchans wrote: post #54

Hofstadter said, "Well, maybe as important as the wheel." If AI significantly surpasses humanity in cognitive ability, then I think it will have a much bigger impact than the wheel. (I loved GEB and DH's other writings.) LLMs have really improved a lot of the last two years and they have shown many unexpected capabilities. I am guessing that they will get some more good input (text mostly), a lot more compute, and al…

> I recently lost a bet about autonomous driving. Five years ago, I thought that autonomous cars would be better than human drivers by now.

I think we're going to see something very similar with LLMs. The autonomous car hype was driven by seeing that they were 80% of the way there and concluding that at the rate they were going they'd make up the remaining 20% quickly. That turned out to be false: the last 20% has been much harder than the first 80%.

LLMs are in a very similar place, even GPT-4. They're good, and they're going to be more and more useful (similar to adaptive cruise control/lane assist). But I predict that they're going to level out and stop improving as rapidly as they have in the past year, and we're going to end up at a new normal that is good but not good enough to cause the crises people are worried about.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#62

Post by DennisP »

detourdog wrote: post #15

I have more fear of the people running the system than any AI. I also think that AR/VR is more scary than AI. My fear is how poorly rendered AR/VR can be more positive interaction than interactions with the people that surround the observer.

As long as the AI is less generally intelligent than people, it makes sense to be more afraid of the people. Once the AI is more intelligent than the smartest people, it's more sensible to be most afraid of the AI.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#63

Post by calf »

mullingitover wrote: post #32

I'll never forget that day when I was a junior at University of Oregon, and had just finished reading GEB. Hofstadter was from UO, and I was taking a data structures class from Andrzej Proskurowski, who knew Hofstadter. Andrzej was a pretty brilliant man who had a reputation for being very blunt. I was in his office hours and I asked him for his take on the book. He said, "Hofstadter is first order...bullshitter."

My theoretical CS professor said GEB was nonsense and we all laughed in class.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#64

Post by StrictDabbler »

iamleppert wrote: post #25

" ...In my book," pretty much sums it up. Literally everyone who seems to pontificate on a Chat bot and new Photoshop features has a book and can't seem to help but mention it. Replace "book" with literally anything else and you can see it's completely about ego and money at the end of the day. He's probably getting ready to announce he accepted a position from one of these recently funded AI companies and whatever h…

Hofstadter won a Pulitzer in 1980. His books are among the most important ever written about the meaning and structure of AI. He's had four decades where he could have milked that reputation for money or influence and he's chosen hard academic research at every branch point.

He sold millions of copies of a densely-written 777 page book about semantic encoding (GEBEGB).

It is insane and/or ignorant to imagine he's jonesing for clout or trying to ride the ML wave.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#65

Post by notpachet »

climatologist wrote: post #21
jurgenaut23 wrote: post #9

I read Hofstadter's GEB and Tegmark's Our Mathematical Universe, and of course I developed a rather fond admiration of these brilliant minds. For some reason, both of them have developed a profound aversion and fear of what they consider an existential threat. I have a solid theoretical understanding of these systems, and I spent 15 years studying, building, and deploying them at scale and for diverse use cases. The…

I think they are not good at expressing what exactly they fear. Both understand that people are embedded in culture and tools which act as cognitive aids and guides. Their fear is that people will start trusting these artificial systems too much and delegating too many important decisions to systems that have no understanding of what it means to be human. This is already happening with children that grew up with iPho…

> Their fear is that people will start trusting these artificial systems too much and delegating too many important decisions to systems that have no understanding of what it means to be human

In the interview, Hofstadter says what he's afraid of in explicit terms:

> It's not clear whether that will mean the end of humanity in the sense of the systems we've created destroying us. It's not clear if that's the case, but it's certainly conceivable. If not, it also just renders humanity a very small phenomenon compared to something else that is far more intelligent and will become incomprehensible to us, as incomprehensible to us as we are to cockroaches.

It's not about whether we'll become dependent on AI. It's that AI will become independent of us. Completely different problem. Not saying I agree with that viewpoint per se, but I don't think you're accurately representing what his fears are.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#66

Post by zerodensity »

irchans wrote: post #54

Hofstadter said, "Well, maybe as important as the wheel." If AI significantly surpasses humanity in cognitive ability, then I think it will have a much bigger impact than the wheel. (I loved GEB and DH's other writings.) LLMs have really improved a lot of the last two years and they have shown many unexpected capabilities. I am guessing that they will get some more good input (text mostly), a lot more compute, and al…

But self driving cars have a much lower accident rate than humans. How did you lose the bet?

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#67

Post by substation13 »

CuriouslyC wrote: post #53
pipo234 wrote: post #31

Earlier quoted context omitted.

Might have completely missed something, but I thought AR/VR was (and still is) a solution looking for a problem. Have they finally stumbled into something people want to do with them, beyond games and porn?

AR will take off like crazy when they develop lightweight, clear goggles than can display color with a range of opacities. A million jobs are begging for HUDS that guide the worker, but the current equipment isn't there yet.

Stacking shelves in a supermarket is a perfect example. 1000s of products means that a bit of experience is required to know where everything goes in order to quickly stack. With a HUD, workers become productive much faster (and are more inter-changeable) - which is something large enterprises love.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#68

Post by hackinthebochs »

gautamcgoel wrote: post #52
gwright wrote: post #8

Hofstadter makes the claim that "these LLMs and other systems like them are all feed-forward". That doesn't sound right to me, but I'm only a casual observer of LLM tech. Is his assertion accurate? FWIW, ChatGPT doesn't think so. :-)

No, it is not correct. Transformers have two components: self-attention layers and multi-layer perceptron layers. The first has an autoregressive/RNN flavor, while the latter is feedforward.

They are definitely feed-forward. Self-attention looks at all pairs of tokens from the context window, but they do not look backwards in time at its own output. The flow of data is layer by layer, each layer gets one shot at influencing the output. That's feed-forward.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#69

Post by proamdev123 »

pipo234 wrote: post #60
CuriouslyC wrote: post #53

Earlier quoted context omitted.

AR will take off like crazy when they develop lightweight, clear goggles than can display color with a range of opacities. A million jobs are begging for HUDS that guide the worker, but the current equipment isn't there yet.

> A million jobs are begging for HUDS that guide the worker [..] I have a pretty dull, comfortable desk job and lack the imagination to come up with any. Can you name some? (I've been wrong before when I was skeptical when everyone was hyped about iPad, Deep Learning, etc. so please convince me about Apple Vision, Mark's metaverse, or google's glasses and paint what might be in 5 years.)

Here’s one: a warehouse worker with HUD that guides picking and/or packing visually.

The HUD could overlay SKU, product description, weight, volume, etc — directly onto the actual item in the storage rack.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#70

Post by mikeg8 »

Loquebantur wrote: post #47
jurgenaut23 wrote: post #9

I read Hofstadter's GEB and Tegmark's Our Mathematical Universe, and of course I developed a rather fond admiration of these brilliant minds. For some reason, both of them have developed a profound aversion and fear of what they consider an existential threat. I have a solid theoretical understanding of these systems, and I spent 15 years studying, building, and deploying them at scale and for diverse use cases. The…

Consider the comparatively simple case of these LLMs providing decent enough chatbots to fool a majority. If you deploy them on social media in a coordinated fashion, you can easily sway public opinion. You only need to train them to adhere to psyops techniques not sufficiently well known nor easily detectable. Of which there are many.

This assumes there will still be centralized social media sites to deploy to which seems less likely based on how Twitter, Reddit, etc are rapidly changing. The effect of propaganda on say FB will be diminished as less and less young users join the platform.
Post reply on HN