Douglas Hofstadter changes his mind on Deep Learning and AI risk • Hacker News Board

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#211

Post by rcxdude » Mon, Jul 03, 2023, 7:21 PM UTC

mecsred wrote: post #84

pipo234 wrote: post #60

Earlier quoted context omitted.

> A million jobs are begging for HUDS that guide the worker [..] I have a pretty dull, comfortable desk job and lack the imagination to come up with any. Can you name some? (I've been wrong before when I was skeptical when everyone was hyped about iPad, Deep Learning, etc. so please convince me about Apple Vision, Mark's metaverse, or google's glasses and paint what might be in 5 years.)

This is something I've worked on so I can comment a bit. The most popular use case I've seen so far is AR manuals, with the sell that you can have technicians up to speed on a piece of equipment without needing human coaching or training. I was shown a demo in the context of robotics recently where you look at a robot and explanations of signals in the wiring harness were shown, hydraulic connections, wear parts, etc…

given the general state of most documentation, I really struggle to imagine who would actually maintain manuals like this.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#212

Post by ymck » Mon, Jul 03, 2023, 7:22 PM UTC

ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

It's simple. All the "Big Brains" missed the real risks of Web 1.0/Web 2.0, focusing only on the positives in a time of hope and economic growth. Now, we have an internet that focuses on how everything is terrible, and a new tech abruptly hits the scene. Of course, the current "Big Brains" meet the clout need to point out how the sky might fall.

AI will be transformative, but it's more likely to follow previous transformations. Unintended consequences, sure, but largely an increase in the standard of living, productivity, and economic opportunity.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#213

Post by kilolima » Mon, Jul 03, 2023, 7:23 PM UTC

frankfrank13 wrote: post #29

jurgenaut23 wrote: post #9

I read Hofstadter's GEB and Tegmark's Our Mathematical Universe, and of course I developed a rather fond admiration of these brilliant minds. For some reason, both of them have developed a profound aversion and fear of what they consider an existential threat. I have a solid theoretical understanding of these systems, and I spent 15 years studying, building, and deploying them at scale and for diverse use cases. The…

Agreed 100%. He's brilliant, but he's not a programmer. IMO he's been tricked, in a way that Chompsky was not.

He's fluent in Lisp, so that would qualify him as a programmer, unless you think that only Silicon Valley wage slaves can be "programmers".

https://en.m.wikipedia.org/wiki/Metamagical_Themas

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#214

Post by ke88y » Mon, Jul 03, 2023, 7:24 PM UTC

skepticATX wrote: post #186

ALittleLight wrote: post #156

Earlier quoted context omitted.

On the one hand, we have "the researchers and PhD's" that you know. On the other, we have published surveys of academics and an open letter from noted experts in the field. One side seems to have much better quality of evidence. "All the oil and gas engineers I work with say climate change isn't a thing." Hmm, wow, that's really persuasive! What would you consider evidence of a significant AI risk? From my point of v…

How about someone actually articulate the details of how this supposed super intelligence will be built, what about it’s architecture means that it has no guardrails, how it will bypass numerous real world frictions to achieve its nefarious goals? Arguments that presuppose a god-like super intelligence are not useful. Sure, if we create a system that is all powerful I’d agree that it can destroy humanity. But specifi…

>Arguments that presuppose a god-like super intelligence are not useful. Sure, if we create a system that is all powerful I’d agree that it can destroy humanity. But specifically how are we going to build this?

Yeah, exactly. THIS is the type of x-risk talk that I find cringe-y and ego-centered.

There are real risks. I've devoted my career to understanding and mitigating them, both big and small. There's also risk in risk mitigation, again, both big and small. Welcome to the world of actual engineering :)

But a super-human god intelligence capable of destroying us all by virtue of that fact that it has an IQ over 9000? I have a book-length criticism of the entire premise and whether we should even waste breath talking about it before we even start the analysis of whether there is even the remotest scrap of evidence that were are anywhere near such a thing being possible.

It's sci-fi. Which is fine. Just don't confuse compelling fiction with sound policy, science, or engineering.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#215

Post by kalkin » Mon, Jul 03, 2023, 7:26 PM UTC

echelon wrote: post #138

zone411 wrote: post #128

Earlier quoted context omitted.

If you read the survey, you'll find that many concerned researchers don't believe we're at a 90% chance of doom, but e.g. 10%. So, this type of response wouldn't be rational if they're thinking that things will go fine most of the time. If these researchers are thinking logically, they would also realize that this kind of reaction has little chance of success, especially if research continues in places like China. It…

> If you read the survey Not a nice way to engage in debate. I've spent more time listening to and refuting these arguments than most. Including debating Eliezer. > many concerned researchers don't believe we're at a 90% chance of doom, but e.g. 10%. A 10% chance of an asteroid hitting the earth would result in every country in the world diverting all of their budgets into building a means to deflect it. > So, this t…

"If [people who I disagree with] really believed what they said, they'd do [X Y or Z extreme thing]" is almost always a bad argument. Consider the possibility that instead, the people who actually believe the thing you don't have spent more time thinking seriously about their strategy than you have, and you are not in fact better both at being you and at being them than they are.

I'm used to this pattern showing up as "lol if you really don't like capitalism how come you use money" but it's just as bad here.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#216

Post by iamleppert » Mon, Jul 03, 2023, 7:29 PM UTC

StrictDabbler wrote: post #64

iamleppert wrote: post #25

" ...In my book," pretty much sums it up. Literally everyone who seems to pontificate on a Chat bot and new Photoshop features has a book and can't seem to help but mention it. Replace "book" with literally anything else and you can see it's completely about ego and money at the end of the day. He's probably getting ready to announce he accepted a position from one of these recently funded AI companies and whatever h…

Hofstadter won a Pulitzer in 1980. His books are among the most important ever written about the meaning and structure of AI. He's had four decades where he could have milked that reputation for money or influence and he's chosen hard academic research at every branch point. He sold millions of copies of a densely-written 777 page book about semantic encoding (GEBEGB). It is insane and/or ignorant to imagine he's jon…

Incredible! Basically proves my point, you came here to tell me to buy his book. Maybe he's really worried that AI is going to put people like him out of a job when it can produce and market content at scale?

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#217

Post by MichaelMoser123 » Mon, Jul 03, 2023, 7:30 PM UTC

> The question is, when will we feel that those things actually deserve to be thought of as being full-fledged, or at least partly fledged, "I"s?

this LLM thing is more like a collective "we", it is making a prediction in the sense of the relevant training data, it probably wouldn't say anything that contradicts the consensus.

Maybe the LLM's are just a mirror of our society. And our society doesn't seem to assign a lot of value to individualism, as such.

i think that might be similar to the movie Solaris by Tarkovsky. The movie is starring an alien ocean, which is some sort of mirror, that is showing us who we are (maybe it has a different meaning, not quite sure about it). You can watch it on youtube: https://www.youtube.com/watch?v=Z8ZhQPaw4rE (i think you also get this theme with Stalker - this zone is also telling us who we are)

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#218

Post by cycomanic » Mon, Jul 03, 2023, 7:31 PM UTC

throwway120385 wrote: post #180

ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

In my opinion the weirdness is that everyone is talking about how AI is so much better than humans and we're all going to get replaced. But almost nobody is talking about how tech companies will use this new technology to abdicate responsibility for the decisions they undertake. If you thought redlining was bad, you should see how mortgage companies treated people when they used an AI to accomplish the same goals. No…

"little Britain" (if you don't know it go look it up you're in for a treat), knew it already: "the computer says no".

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#219

Post by dist-epoch » Mon, Jul 03, 2023, 7:32 PM UTC

petermcneeley wrote: post #169

dist-epoch wrote: post #131

Earlier quoted context omitted.

Nassim Taleb makes a great point: - The best person to judge the risk of playing roulette is not the carpenter who built it. - The best person to judge the risks of a global pandemic is not a virologist working with viruses daily. You can extend that: - The best person to judge the cybersecurity risks of an application is not the programmer implementing it.

Either Nassim is missing the point or you are misquoting him. I think this is more about should experts be in charge of their own domain.

I'm not misquoting him.

Nassim is arguing that RISK is a separate discipline, separate from the domain where risk applies. That a person building AI is not the correct choice for estimating AI risk.

You don't ask gun making companies to make policies regarding risk of gun owning in society.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#220

Post by maizeq » Mon, Jul 03, 2023, 7:32 PM UTC

ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

I work in a AI research lab for a large tech company and my observations are completely opposed to yours.

Almost every researcher I have spoken believes that real risk exists, to some degree or other. Recent surveys of people in industry have largely borne this out - your anecdote sounds more like an anomaly to me.