Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Douglas Hofstadter changes his mind on Deep Learning and AI risk

lesswrong.com

Post reply on HN
Print view View original

321–330 of 520 posts

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#321

Post by saulpw »

pbw wrote: post #198
wslh wrote: post #165

Earlier quoted context omitted.

> A theorem that says intelligence is in some ways universal, and that our intelligence will be compatible with all other forms of intelligence, even if they are much "smarter". I would not be so reductionist. Intelligence doesn't seem to be an universal thing, even IQ (a human invented metric) is measured in terms of some statistics. If you have an IQ of ~60 you have intelligence but a completely different one from…

My point is you cannot teach a cockroach calculus, but if AIs invent a new type of math, they would be able to teach it to us. That's my claim. So the analogy of "we are cockroaches compared to the AI" is wrong, that won't be the case. Once you have "enough" intelligence to have a complex language, like we do, I'm claiming you are in the club of intelligent species, and all species in that club can communicate ideas…

AlphaZero invented new moves in the game of Go, but it can't 'teach' them to us, it can only show us the moves and let us figure it out for ourselves (which we're doing). But note that despite this transfer of knowledge, humans didn't rise up to the level of AlphaZero, and they may never be able to. As a sibling comment points out, some things are computationally bound--and humans have a limit to computational ability (can't give ourselves more neurons/connections), whereas AI does not.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#322

Post by lukas099 »

namaria wrote: post #99
calf wrote: post #63

Earlier quoted context omitted.

My theoretical CS professor said GEB was nonsense and we all laughed in class.

I've heard good things about this book for years and nearly bought it a few times. Could never bring myself to commit the hundreds of hours of reading time. Every time I leaf through it it feels like it's just a collection of anecdotes about how amazing mathematics is. Like I need someone to remind me...

It was a transformative book for me when I first read it, but now when I leaf through it I feel a little underwhelmed; the ideas are just things I've thought about a million times now.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#323

Post by dist-epoch »

petermcneeley wrote: post #308
jll29 wrote: post #271

Earlier quoted context omitted.

I agree with this, and with Nassim Taleb's point. Having worked in the medical domain in the past, paramedics and medics that should know better were taking extremely high health-related risks (riding a motor bike => crashing and burning to death, smoking => dying from lung canceer, speeding onto a crossing => dying in an ambulance crash before arriving at the 999 call site etc.). So risk is indeed its own discipline…

But then how does that jive with skin in the game? In his Nassim's teaching he brings up the example of the idea that the Roman engineers that built/designed the bridges were forced to have their families live under the bridges. It sounds like those engineers directly involved in the practice do understand the risk of their domains it is merely their incentives that need to be aligned.

> But then how does that jive with skin in the game

Simple. Nassim says there are 4 quadrants, one axis Mediocristan-Extremistan, the other Simple-Complex payoff.

Building a bridge is Mediocristan/Simple payoff, a well understood problem with no black swans. So it's easy to compute risk.

Other stuff is Extremistan/Complex payoff - financial trading, pandemics, AI. And he argues that you need RISK professionals for this quadrant, because people working here (traders, virologists, AI builders) do not understand how to compute the risk correctly.

https://www.researchgate.net/profile/Gaute-Bjorklund-Wangen/...

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#324

Post by ethanbond »

jrflowers wrote: post #204
ethanbond wrote: post #191

Earlier quoted context omitted.

Brilliant proof of exactly my point. When it comes to discussing any possible outcome other than utopia, suddenly the power of these tools drops to zero! Remarkable :)

You responded to my previous comment that called utopians as cultish as the fanfic folks. When it comes up discussing any possible outcome that isn’t the opinion of [cult x], the only reason that the other person disagrees is because they are in [cult y] Remarkable :)

What? I never proposed that dichotomy and don't believe in it. You did and you're tripping over it a bit. You can just discard that model and engage with the substance of the topic, you know!

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#325

Post by petermcneeley »

dist-epoch wrote: post #323
petermcneeley wrote: post #308

Earlier quoted context omitted.

But then how does that jive with skin in the game? In his Nassim's teaching he brings up the example of the idea that the Roman engineers that built/designed the bridges were forced to have their families live under the bridges. It sounds like those engineers directly involved in the practice do understand the risk of their domains it is merely their incentives that need to be aligned.

> But then how does that jive with skin in the game Simple. Nassim says there are 4 quadrants, one axis Mediocristan-Extremistan, the other Simple-Complex payoff. Building a bridge is Mediocristan/Simple payoff, a well understood problem with no black swans. So it's easy to compute risk. Other stuff is Extremistan/Complex payoff - financial trading, pandemics, AI. And he argues that you need RISK professionals for th…

You think the virologists do not understand the risks? And what are the biases of the risk professionals? What are their motives?

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#326

Post by casebash »

mvdtnz wrote: post #194

I was previously undecided on the question of the existential risk of AI. But yesterday I listened to a Munk Debate[0] on the topic and found the detractors of the moot "AI research and development poses an existential threat" so completely unconvincing that if those people are the top minds in the field I am now genuinely concerned. Their arguments that what they are doing is not a risk basically boil down to "nuh u…

Are you thinking about trying to learn more about this issue or do something about it?

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#327

Post by scrawl »

pbw wrote: post #147

Hofstadter says humans will be like cockroaches compared to AI. This is an oft-repeated line: sometimes we are ants or bacteria. But I think these comparisons might be totally wrong. I think it's very possible there's a Intelligence Completeness theorem that's analogous to Turing Completeness. A theorem that says intelligence is in some ways universal, and that our intelligence will be compatible with all other forms…

>smart enough to explain us any idea

Are dogs, or pigs, or whales, part of the intelligence club? They are clearly intelligent beings with problem-solving skills. We won't be teaching them basic calculus any time soon.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#328

Post by pbw »

hyperthesis wrote: post #296
pbw wrote: post #147

Hofstadter says humans will be like cockroaches compared to AI. This is an oft-repeated line: sometimes we are ants or bacteria. But I think these comparisons might be totally wrong. I think it's very possible there's a Intelligence Completeness theorem that's analogous to Turing Completeness. A theorem that says intelligence is in some ways universal, and that our intelligence will be compatible with all other forms…

Humans have limited "working memory". We manage to cram more into it via hierarchical decomposition into "chunks", a single concept that is more complex inside. I submit that not everything can be hierarchically decomposed in a way that's useful - i.e. any "abstraction" you try to force on it is more leaky than non-leaky; in that it doesn't simplify its interactions with other chunks. You might say it's the wrong abs…

I suspect the limit of what can be expressed in human language and comprehended by the human mind is vast, but yes, not infinite. I think the AIs will absolutely saturate the connection between them and us, with a non-stop torrent of information which will range from useful to civilization-changing.

And I think this is all very unlike how we are currently impacting the lives of cockroaches with our insights about, well anything. Thus, it's not a good analogy.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#329

Post by pbw »

scrawl wrote: post #327
pbw wrote: post #147

Hofstadter says humans will be like cockroaches compared to AI. This is an oft-repeated line: sometimes we are ants or bacteria. But I think these comparisons might be totally wrong. I think it's very possible there's a Intelligence Completeness theorem that's analogous to Turing Completeness. A theorem that says intelligence is in some ways universal, and that our intelligence will be compatible with all other forms…

>smart enough to explain us any idea Are dogs, or pigs, or whales, part of the intelligence club? They are clearly intelligent beings with problem-solving skills. We won't be teaching them basic calculus any time soon.

No non-human animals are in the club that's marked by having a language with an infinitely generative syntax and a large (100,000+ words) and always-growing vocabulary.

Intelligence might be a spectrum, but powerful generative language is a step function: you have it or you don't. If you have it, then higher intelligences can communicate complex thoughts to you, if you don't they can't. We have it, so we are in the club, we are not cockroaches.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#330

Post by ke88y »

pseudonom- wrote: post #272
ke88y wrote: post #134

Earlier quoted context omitted.

> We contacted approximately 4271 researchers who published at the conferences NeurIPS or ICML in 2021... we found email addresses in papers published at those conferences, in other public data, and in records from our previous survey and Zhang et al 2022. We received 738 responses, some partial, for a 17% response rate. Anyone who has reviewed for, or attended, or just read the author lists of ICML/NeurIPS papers is…

> Ie even lower than my guess of 90% above. These are not comparable numbers. You're comparing "fraction of people" vs "fraction of outcomes". Presumably an eye-roller assigns ~0 probability to "extremely bad" outcomes (or has a shockingly cavalier attitude toward medium-small probabilities of catastrophe).

Yeah, that's correct. Stupid mistake on my part; was writing that comment in a hurry. Not sure why you're grayed out tbh. Thanks for the corrective.

> or has a shockingly cavalier attitude

Meh. Median response time was 40 seconds. The question didn't have a bounded time-frame for the risk. Five is small but non-zero. Also all of the other issues I've already pointed out.

PhD students spending half a minute and writing down a number about risk over an unbounded time-frame is totally uninformative if you want to know how seriously experts take x-risk in time-frames that are relevant to any sort of policy or decision making.

I think you and everyone else making comments about "shockingly cavalier attitude" wildly over-estimate the amount of thought and effort that respondents spend on this question. The "probability times magnitude" framing is not how normal people think about that question. I'd bet they just wrote down a small but not zero number; I'd probably write down 1 or 2 but definitely roll my eyes hard.

Post reply on HN