Douglas Hofstadter changes his mind on Deep Learning and AI risk • Hacker News Board

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#91

Post by spookie » Mon, Jul 03, 2023, 5:47 PM UTC

pipo234 wrote: post #60

CuriouslyC wrote: post #53

Earlier quoted context omitted.

AR will take off like crazy when they develop lightweight, clear goggles than can display color with a range of opacities. A million jobs are begging for HUDS that guide the worker, but the current equipment isn't there yet.

> A million jobs are begging for HUDS that guide the worker [..] I have a pretty dull, comfortable desk job and lack the imagination to come up with any. Can you name some? (I've been wrong before when I was skeptical when everyone was hyped about iPad, Deep Learning, etc. so please convince me about Apple Vision, Mark's metaverse, or google's glasses and paint what might be in 5 years.)

The US military has researched on using it for vehicle mechanical repairs. This way a soldier without such knowledge would be able to perform on field repairs.

For 3D artists it presents as a more intuitive way to sculpt models. For automotive designers, it allows a cheaper and faster means of iteration, given that such a task requires a much better sense of scale than the one given off of a monitor. Same goes for architecture, which when coupled with a game engine, also allows the customer to preview their future house.

(props for being open to ideas btw)

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#92

Post by twic » Mon, Jul 03, 2023, 5:48 PM UTC

lolinder wrote: post #61

irchans wrote: post #54

Hofstadter said, "Well, maybe as important as the wheel." If AI significantly surpasses humanity in cognitive ability, then I think it will have a much bigger impact than the wheel. (I loved GEB and DH's other writings.) LLMs have really improved a lot of the last two years and they have shown many unexpected capabilities. I am guessing that they will get some more good input (text mostly), a lot more compute, and al…

> I recently lost a bet about autonomous driving. Five years ago, I thought that autonomous cars would be better than human drivers by now. I think we're going to see something very similar with LLMs. The autonomous car hype was driven by seeing that they were 80% of the way there and concluding that at the rate they were going they'd make up the remaining 20% quickly. That turned out to be false: the last 20% has be…

This is my bet too. I think we're going to get some fantastic new tools for doing specific tasks that work like magic, and that we could never have imagined ten years ago. I don't think we're going to get Yud hijacking a B-52.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#93

Post by ke88y » Mon, Jul 03, 2023, 5:48 PM UTC

This is something weird happening around Rationalism/X-Risk/AGI prognostications.

The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing.

But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative models, and previous generations, in some cases for decades? They almost all roll their eyes aggressively at these sorts of prognostications. I'd say 90+% either laugh or roll their eyes.

Why is that?

Personally, I'm much more on the side of the silent majority here. I agree with Altman's criticisms of criticisms about regulatory capture, that they are probably unfair or at least inaccurate.

What I actually think is going on here is something more about Egos than Greatness or Nefarious Agendas.

Ego, not intelligence or experience, is often the largest differentiator between the bench scientist or mid-level manager/professor persona and the CEO/famous professor persona. (The other important thing, of course, is that the former is the group doing the actual work.)

I think that most of our Great Minds and Great Leaders -- in all fields, really -- are not actually our best minds and best leaders. They are, instead, simply our Biggest Egos. And that those people need to puff themselves up by making their areas of ownership/responsibility/expertise sound Existentially Important.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#94

Post by namaria » Mon, Jul 03, 2023, 5:50 PM UTC

mostlysimilar wrote: post #6

jackconsidine wrote: post #2

In GEB Hofstadter dismisses the idea that AI could understand / compose / feel music like a human. I thought about this a lot when I started using GPT, especially early on when it demonstrated an ability to explain why things were funny or sad, intrinsically human qualities hitherto insulated from machine

Repeating the words thousands of humans have written about emotion doesn’t meant it feels them. A sociopath could define empathy and still be missing the deeper experience behind the emotion.

A chatbot nearly capable of passing the Chinese Room thought experiment test is pretty damn impressive. But I think people get too hung up on the one golden moement/tech/product/innovation that changes everything. We've been riding a nearly vertical population, wealth and computer capacity curve for nearly two generations now. We are living through the singularity. Things are already like they have never been before. Billions of people can expect some level of social security, justice and opportunity to pursue life changing income now. This is nothing short of amazing. For most of History most people have been hard working, sickly and oppressed farmers.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#95

Post by reducesuffering » Mon, Jul 03, 2023, 5:50 PM UTC

jurgenaut23 wrote: post #9

I read Hofstadter's GEB and Tegmark's Our Mathematical Universe, and of course I developed a rather fond admiration of these brilliant minds. For some reason, both of them have developed a profound aversion and fear of what they consider an existential threat. I have a solid theoretical understanding of these systems, and I spent 15 years studying, building, and deploying them at scale and for diverse use cases. The…

It’s pretty amazing how so many titans people respect are warning the alarm bells: Hofstadter, Altman, Hassabis, Bengio, Hinton, Tegmark, Sutskever, etc. but the implications are too scary for most people here to accept. So one by one the most common copium response is to downplay each about-facer's achievements. I like the odds betting on the intellectual prowess of that first group. On the other hand, crypto king Marc Andreesen is on your side! But I empathize, because like Hofstadter says:

"And my whole intellectual edifice, my system of beliefs... It's a very traumatic experience when some of your most core beliefs about the world start collapsing. And especially when you think that human beings are soon going to be eclipsed. It felt as if not only are my belief systems collapsing, but it feels as if the entire human race is going to be eclipsed and left in the dust soon."

Don't Look Up https://twitter.com/kristjanmoore/status/1663860424100413440

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#96

Post by FrustratedMonky » Mon, Jul 03, 2023, 5:50 PM UTC

frankfrank13 wrote: post #29

jurgenaut23 wrote: post #9

I read Hofstadter's GEB and Tegmark's Our Mathematical Universe, and of course I developed a rather fond admiration of these brilliant minds. For some reason, both of them have developed a profound aversion and fear of what they consider an existential threat. I have a solid theoretical understanding of these systems, and I spent 15 years studying, building, and deploying them at scale and for diverse use cases. The…

Agreed 100%. He's brilliant, but he's not a programmer. IMO he's been tricked, in a way that Chompsky was not.

"programmers" are also fallible.

I've seen enough programmers get heads down pounding out code, and be completely out of touch on what they are building. If they can lose the big picture on simple apps, then it is not a stretch to think they could lose track on what is consciousness, or what is human.

I am also a programmer. But it does get tiring on HN to give so much credence to 'programmers'. Just because someone can debug some JavaScript doesn't make them an expert. I really doubt that many people here have traced out these algorithms and 'know' what is happening.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#97

Post by JohnFen » Mon, Jul 03, 2023, 5:51 PM UTC

pipo234 wrote: post #31

detourdog wrote: post #15

I have more fear of the people running the system than any AI. I also think that AR/VR is more scary than AI. My fear is how poorly rendered AR/VR can be more positive interaction than interactions with the people that surround the observer.

Might have completely missed something, but I thought AR/VR was (and still is) a solution looking for a problem. Have they finally stumbled into something people want to do with them, beyond games and porn?

> Have they finally stumbled into something people want to do with them, beyond games and porn?

Most people don't want it for games and porn either -- although those two things are the only obvious mass-market applications.

There are lots of other real uses, but they're all niche. It's hard to come up with a real, mainstream use that would drive adoption in the general public.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#98

Post by jebarker » Mon, Jul 03, 2023, 5:51 PM UTC

jameshart wrote: post #75

Interesting to hear him say this: > And I would never have thought that deep thinking could come out of a network that only goes in one direction, out of firing neurons in only one direction. And that doesn't make sense to me, but that just shows that I'm naive. I think people maybe miss that LLM output does involve a ‘loop’ back - maybe even a ‘strange’ loop back, and I’m surprised to see Hofstadter himself fail to…

I was surprised he said this too. Even without the autoregressive part of GPT models you have a deep transformer with attention, so even a single forward pass can modify its own intermediate outputs.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#99

Post by namaria » Mon, Jul 03, 2023, 5:52 PM UTC

calf wrote: post #63

mullingitover wrote: post #32

I'll never forget that day when I was a junior at University of Oregon, and had just finished reading GEB. Hofstadter was from UO, and I was taking a data structures class from Andrzej Proskurowski, who knew Hofstadter. Andrzej was a pretty brilliant man who had a reputation for being very blunt. I was in his office hours and I asked him for his take on the book. He said, "Hofstadter is first order...bullshitter."

My theoretical CS professor said GEB was nonsense and we all laughed in class.

I've heard good things about this book for years and nearly bought it a few times. Could never bring myself to commit the hundreds of hours of reading time. Every time I leaf through it it feels like it's just a collection of anecdotes about how amazing mathematics is. Like I need someone to remind me...

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#100

Post by kilolima » Mon, Jul 03, 2023, 5:53 PM UTC

Animats wrote: post #89

"And so it makes me feel diminished. It makes me feel, in some sense, like a very imperfect, flawed structure compared with these computational systems that have, you know, a million times or a billion times more knowledge than I have and are a billion times faster. It makes me feel extremely inferior." Although he did real work in physics, Hofstadter's fame comes from writing popular books about science which explai…

The parent comment strikes me as an uninformed smear. Have you read any Hofstadter? It's hardly mass market scientific summary as you seem to think. And his career was in philosophy of mind and computer science, not just as a somewhat popular author.

His writing is a bit more complex than the hallucinations of an llm.