Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Douglas Hofstadter changes his mind on Deep Learning and AI risk

lesswrong.com

Post reply on HN
Print view View original

291–300 of 520 posts

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#291

Post by stale2002 »

hackinthebochs wrote: post #133
echelon wrote: post #106

Earlier quoted context omitted.

If these people were so concerned, they'd by shouting from the hilltops and throwing their entire life savings into stopping us. They would organize workplace walkouts and strikes. There would be protests and banners. Burning data centers. Eliezer is one of a handful of people putting their reputation on the line, but that's mostly because that was his schtick in the first place. And even so, his response has been ra…

Oh look, an unfalsifiable claim in service to your predetermined position. How novel. You can believe there's a high chance of what you're working on being dangerous and still be unable to stop working on it. As Oppenheimer put it, "when you see something that is technically sweet, you go ahead and do it".

Its not an unfalseifiable claim.

If 100 Ai "experts" shutdown the OpenAI office for a week due to protests outside their headquarters that would be one way to falseify the claim that "doomers don't actually care".

But, as far as I can tell, the doomers aren't doing much of anything besides writing a strongly worded letter here or there.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#292

Post by pbw »

orlp wrote: post #257
pbw wrote: post #198

Earlier quoted context omitted.

My point is you cannot teach a cockroach calculus, but if AIs invent a new type of math, they would be able to teach it to us. That's my claim. So the analogy of "we are cockroaches compared to the AI" is wrong, that won't be the case. Once you have "enough" intelligence to have a complex language, like we do, I'm claiming you are in the club of intelligent species, and all species in that club can communicate ideas…

> but if AIs invent a new type of math, they would be able to teach it to us There are already math proofs made by humans on this very day that are hundreds upon hundreds of pages of lemmas that are highly advanced and building on other advanced results. Understanding such a proof is an undertaking that literally takes years. An AI might end up doing it in minutes. But what an AI could cook up in years could take a h…

I agree "one AI" might produce output that keeps humans busy for decades. But that doesn't make us cockroaches. Cockroaches can't understand language, at all. You can't teach a cockroach calculus if had a trillion years. That's not our position relative to AIs. We will be learning shit-tons from them constantly. I think people who say humans will be "cockroaches" or "ants" or "bacteria" are fear-mongering, or just confused.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#293

Post by anigbrowl »

JimtheCoder wrote: post #227
tyre wrote: post #164

Earlier quoted context omitted.

This is a great way to put it. The counter to "current generation AI is terrifying" seems to fall along the lines of it not being nearly as close to AGI as the layperson believes. But I don't think that matters. I don't believe that LLMs or image/voice/video generative models need to do much beyond what they can do today in order to wreck civilization level disaster. They don't need to become Skynet, learn to operate…

Doesn't this only work if everyone is stupid and believes everything they see/read/hear on the internet? You have to have a little more faith in humanity than that...

No. You just need a minority of people. And they don't have to be stupid; they can be equally motivated by mendacity, spreading untruths because its fun or because they will discomfit political opponents. About 30% of humans are willing to suffer some sort of loss or disadvantage in order to inflict a larger one on a counterparty; that might seem irrational, but some people are just mean.

You have to have a little more faith in humanity than that...

I used to, but then I grew out of it.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#294

Post by effed3 »

Apart the expertise, is hard to say if is best to be an optimist and hope for the best, or to be a pessimist an hope being wrong.

Anyway i suspect that one true risk, among others, is to loose the true ability to think, if we delegate on a large scale to some LLM the production of the language, because the capacity to use our languages IS the capacity to think, and bad use of technology is a norm in recent (and not so recent) times.

I suspect LLM (this kind of LLM) does not really -generate- (this will be intelligence?), but only mimic on a vaste scale, but nothing more. If our brain/mind is a result of a long evolution, where this LLM are not, builded only on the final results, the language, this will be a great difference in the inner deep working, so the question is: we are feeding in our minds a massive amount of nothing more than our same intellectual productions, recycled, and nothing more? (apart all the distorsions and biases?)

A parallelism i see is in the social-networks: simply, humans cannot sustain a indifferentiated and massive amount of opinion/information/news (apart all the fakes). Even the small scale message communication is impacting the abilit of understanding long texts..

Even it there LLM are -benign-, sure their (indiscriminare) use will not cause some troubles in our beings? On a scale as big as this i an not sure at all.

im' not sure i'm expressing my doubts (and without using a LLM) clearly enough...

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#295

Post by facu17y »

Marcan wrote: post #235
facu17y wrote: post #181

There is no AI risk. There is only the risk of bad or "unfavorable" actors using AI. AI in a killer drone unleashed on civilians? The bad actor is the one who deployed this weapon. AI given agency and goal maximization ending up gaining physical form all on its own and killing people? or hacking into bank accounts to enrich its creator? The latter more likely than the former, but for cyber-offensive AI there is cyber…

Your first sentence literally mirrors this slogan: https://en.m.wikipedia.org/wiki/Guns_don%27t_kill_people,_pe...

Show me how I can kill you with my LLM, or my GAN or Diffusion Model.

I dare you!

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#296

Post by hyperthesis »

pbw wrote: post #147

Hofstadter says humans will be like cockroaches compared to AI. This is an oft-repeated line: sometimes we are ants or bacteria. But I think these comparisons might be totally wrong. I think it's very possible there's a Intelligence Completeness theorem that's analogous to Turing Completeness. A theorem that says intelligence is in some ways universal, and that our intelligence will be compatible with all other forms…

Humans have limited "working memory". We manage to cram more into it via hierarchical decomposition into "chunks", a single concept that is more complex inside.

I submit that not everything can be hierarchically decomposed in a way that's useful - i.e. any "abstraction" you try to force on it is more leaky than non-leaky; in that it doesn't simplify its interactions with other chunks. You might say it's the wrong abstraction - but there's no guarantee there is a right abstraction. Some things are just complex. (This is hypothetical, since I don't think we can conceive of any concepts we can't understand.)

An AI could have an arbitrarily large working memory.

Note: I'm talking about intuitive understanding. We could use it mechanically, just never "get it", cowering before icons, being the one in Searle's Chinese Room https://wikipedia.org/wiki/Chinese_room

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#297

Post by ke88y »

maizeq wrote: post #220
ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

I work in a AI research lab for a large tech company and my observations are completely opposed to yours. Almost every researcher I have spoken believes that real risk exists, to some degree or other. Recent surveys of people in industry have largely borne this out - your anecdote sounds more like an anomaly to me.

> real risk exists, to some degree or other

Risk of extinction? Or of bad outcomes?

I think everyone understands there are near-certain risks of bad outcomes. That's already happening all around us. Totally uncontroversial.

My post was about risk of extinction due to AI (x-risk), and risk of extinction due in particular to run-away AGI (as opposed to eg shit software accidentally launching a nuke, which isn't really an AI-specific concern). I think that view is still pretty eccentric. But please lmk if that's what you meant.

I've been at several ai labs and large corps. You at deepmind or openai by any chance? Just a guess ;)

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#298

Post by timmytokyo »

phillipcarter wrote: post #157
ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

I've come to learn that a lot of the rationalist crowd are really just fanfiction authors. That's fine - people should be able to do that - but I don't like how they're given the limelight on things that they generally have little expertise and hands-on knowledge with. I want to like them and want to have them engaged in discourse, but I find them so insufferable. Not to mention the sub-crowd of rationalists that is…

The "rationalists" are more like a strange AI cult than a scientific or philosophical program. (I always put "rationalist" in quotes when referring to this group of people, because based on the things these people believe, it's a total misnomer.) At this point they're not even really trying to hide it anymore, with their prophet, Eliezer Yudkowsky, wailing in the wilderness about the imminent end of humanity. We've seen similar eschatological belief systems in recent history and they don't usually end well.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#299

Post by pbw »

jimbokun wrote: post #268
pbw wrote: post #147

Hofstadter says humans will be like cockroaches compared to AI. This is an oft-repeated line: sometimes we are ants or bacteria. But I think these comparisons might be totally wrong. I think it's very possible there's a Intelligence Completeness theorem that's analogous to Turing Completeness. A theorem that says intelligence is in some ways universal, and that our intelligence will be compatible with all other forms…

Isn't the difference between us and cockroaches just "we can think much quicker, see patterns and make insights better and faster than cockroaches"?

I think the key difference is language. I think human language is above a key threshold. Our syntaxes are infinitely generative and we have large vocabularies (100,000+ words) which are fully extensible. No other animals have that. My claim is AIs will be able to express any complex ideas in our language. But we cannot express our ideas in "cockroach language". So the analogy is not a good one.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#300

Post by roody15 »

The issue with AI is that it may put too much power into a small groups hand. Imagine if you wanted to develop a weaponized prion or weaponized virus. In todays world this is possible but requires a state actor with systems of control, committees, over sight, testing facilities. Due to human limits it also takes time to complete.

Insert AI generation XIV .. a small group of cult fanatics with only slightly above average IQ’s band together and now get to skip all these limitations and are able to jump ahead to a killer aerosol prion delivery weapon system.

This group of people who follow their great leader (Jimbo)decide to release the weapon to save innocent souls before an evil daemon comet flies past the earth and turns all remaining humans into evil spirits.

My silly story is to just illustrate that there are many people with high IQ’s that also have emotional issues and can fall prey to cults, extremism, etc. Can humans be trusted using a AI with a human IQ of 9000 which is able to simulate reality in seconds.

Post reply on HN