Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Douglas Hofstadter changes his mind on Deep Learning and AI risk

lesswrong.com

Post reply on HN
Print view View original

301–310 of 520 posts

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#301

Post by hyperthesis »

vemv wrote: post #247

As it might be quite common among the HN crowd, Douglas is my hero - I have read three of his books. First of all, hats off to him for his extraordinary display of humility in this interview. People rarely change their minds publicly, let alone hint that they no longer believe in their own past work. However, I'm genuinely surprised that he, of all people, does sees intelligence in GPT-4 output. I think humans are ju…

He's just saying it's more like an "I" than his pharmacy-bot example. His concern is the future: what seemed "far off" now possibly might be in 5 years.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#302

Post by hackinthebochs »

echelon wrote: post #279
hackinthebochs wrote: post #133

Earlier quoted context omitted.

Oh look, an unfalsifiable claim in service to your predetermined position. How novel. You can believe there's a high chance of what you're working on being dangerous and still be unable to stop working on it. As Oppenheimer put it, "when you see something that is technically sweet, you go ahead and do it".

This whole thing is Pascal's wager. Damned if you do, damned if you don't. Nothing is falsifiable from your side either. The people trying to regulate AI are concentrating economic upside into a handful of companies. I have a real problem with that. It's a lot like the old church shutting down scientific efforts during the time of Copernicus. These systems stand zero chance of jumping from 0 to 100 because complicate…

There's something to be said for not going full steam ahead when we don't have a strong idea of the outcome. This idea that progress is an intrinsic good therefore we must continue the march of technology is a fallacy. It is extreme hubris to think that we will be able to control a potential superintelligence. The cost of being wrong is hard to overstate.

>These systems stand zero chance of jumping from 0 to 100 because complicated systems don't do that.

This doesn't track with the lessons learned from LLMs. The obscene amounts of compute thrown at modern networks changes the calculus completely. ChatGPT essentially existed for years in the form of GPT-3, but no one knew what they had. The lesson to learn is that capabilities can far outpace expectations when obscene amounts of computation are in play.

>The people trying to regulate AI are concentrating economic upside into a handful of companies.

Yes, its clear this is the motivations for much of the anti-doom folks. They don't want to be left out of the fun and profit. Their argument is downstream from this. No, doing things public isn't the answer to safety, just like doing bioengineering research or nuclear research in public isn't the answer to safety.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#303

Post by 93po »

kalkin wrote: post #234
mvdtnz wrote: post #194

I was previously undecided on the question of the existential risk of AI. But yesterday I listened to a Munk Debate[0] on the topic and found the detractors of the moot "AI research and development poses an existential threat" so completely unconvincing that if those people are the top minds in the field I am now genuinely concerned. Their arguments that what they are doing is not a risk basically boil down to "nuh u…

Is there a transcript of this available somewhere?

I hope AI solves this for us universally very soon. So much content is spoken word and it takes 4 times as long to get information from this format.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#304

Post by cubefox »

ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

The fact that such a depressingly ignorant comment gets upvoted makes me reconsider whether HN is really "my tribe". :(

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#305

Post by timmytokyo »

sanderjd wrote: post #290
phillipcarter wrote: post #157

Earlier quoted context omitted.

I've come to learn that a lot of the rationalist crowd are really just fanfiction authors. That's fine - people should be able to do that - but I don't like how they're given the limelight on things that they generally have little expertise and hands-on knowledge with. I want to like them and want to have them engaged in discourse, but I find them so insufferable. Not to mention the sub-crowd of rationalists that is…

I'm not sure if it's coincidental and ironic or entirely intended by you, but I found it funny that my introduction to this crowd was a literal fanfiction written by Eliezer Yudkowsky.

I don't want to speak for GP, but I doubt it's coincidental. Many if not most people's first exposure to "rationalism" was via Yudkowsky's Harry Potter fanfic. There's a pretty decent rundown of rationalism's history in Extropia's Children [1].

[1] https://aiascendant.substack.com/p/extropias-children-chapte...

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#306

Post by StrictDabbler »

iamleppert wrote: post #216
StrictDabbler wrote: post #64

Earlier quoted context omitted.

Hofstadter won a Pulitzer in 1980. His books are among the most important ever written about the meaning and structure of AI. He's had four decades where he could have milked that reputation for money or influence and he's chosen hard academic research at every branch point. He sold millions of copies of a densely-written 777 page book about semantic encoding (GEBEGB). It is insane and/or ignorant to imagine he's jon…

Incredible! Basically proves my point, you came here to tell me to buy his book. Maybe he's really worried that AI is going to put people like him out of a job when it can produce and market content at scale?

Yes. The 78-year-old Pulitzer-prize winner with forty years of tenure, several fellowships, multiple books in every library in the country, who spent his entire life trying to develop AI software, is merely "worried" that AI is going to "put people like him out of a job".

He's explicitly expressing a deep existential sadness at how computationally simple certain artistic endeavors turned out to be and how that's devalued them in his mind but at the end of the day it's really just about his paycheck.

Also I'm totally here trying to sell you his books.

Nice job, Diogenes. Your cynicism cracked the case.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#307

Post by hackinthebochs »

stale2002 wrote: post #291
hackinthebochs wrote: post #133

Earlier quoted context omitted.

Oh look, an unfalsifiable claim in service to your predetermined position. How novel. You can believe there's a high chance of what you're working on being dangerous and still be unable to stop working on it. As Oppenheimer put it, "when you see something that is technically sweet, you go ahead and do it".

Its not an unfalseifiable claim. If 100 Ai "experts" shutdown the OpenAI office for a week due to protests outside their headquarters that would be one way to falseify the claim that "doomers don't actually care". But, as far as I can tell, the doomers aren't doing much of anything besides writing a strongly worded letter here or there.

No, the claim is that "no one can believe that AI leads to doom AND not work tirelessly to tear down the machine building the AI". It's unfalsifiable because there's no way for him to gain knowledge that this belief is false (that they do genuinely belief in doom and do not act in the manner he deems appropriate). It's blatantly self-serving.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#308

Post by petermcneeley »

jll29 wrote: post #271
dist-epoch wrote: post #219

Earlier quoted context omitted.

I'm not misquoting him. Nassim is arguing that RISK is a separate discipline, separate from the domain where risk applies. That a person building AI is not the correct choice for estimating AI risk. You don't ask gun making companies to make policies regarding risk of gun owning in society.

I agree with this, and with Nassim Taleb's point. Having worked in the medical domain in the past, paramedics and medics that should know better were taking extremely high health-related risks (riding a motor bike => crashing and burning to death, smoking => dying from lung canceer, speeding onto a crossing => dying in an ambulance crash before arriving at the 999 call site etc.). So risk is indeed its own discipline…

But then how does that jive with skin in the game? In his Nassim's teaching he brings up the example of the idea that the Roman engineers that built/designed the bridges were forced to have their families live under the bridges. It sounds like those engineers directly involved in the practice do understand the risk of their domains it is merely their incentives that need to be aligned.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#309

Post by ke88y »

lowbloodsugar wrote: post #264
ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

In the "scientists complacent about dangers of the systems they are working on" file, I think we are all familiar with https://en.wikipedia.org/wiki/Demon_core . Now, admittedly, this was scientists being complacent about a thing they knew was dangerous, whereas here we are saying scientists don't think their thing is dangerous. But very clearly, AI could be dangerous, so it's more that these scientists don't think t…

> Presumably the scientists and engineers behind the https://en.wikipedia.org/wiki/Therac-25 didn't think it would kill people.

Therac-25 is an excellent example, but of EXACTLY the opposite point.

On the contrary, abstract AGI safety nonsense taking such a strong grip on academic and industrial AI Safety research is what would most frighten me.

In the intervening decades, people concerned about software safety provided us with the tools needed to prevent disasters like Therac-25, while sci-fi about killer robots was entirely unhelpful in preventing software bugs. People concerned about software safety provided us with robust and secure nuclear (anti-)launch systems, while Wargames didn't do much except excite the public imagination. Etc.

We need scientist's and engineer's efforts and attention focused on real risks and practical solutions, not fanciful philosophizing about sci-fi tropes.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#310

Post by [deleted] »

DennisP wrote: post #278
lolinder wrote: post #275

Earlier quoted context omitted.

Interventions per hour isn't a great metric for deciding if the tech is going to be actually capable of replacing the human driver. The big problem with that number is that the denominator (per hour) only includes times when the human driver has chosen to trust FSD. This means that some improvements will be from the tech getting better, but a good chunk of it will be from drivers becoming better able to identify when…

So add the percentage of driving time using FSD. That's improving too, by quite a bit if you consider that Autopilot only does highways.

[deleted]
Post reply on HN