Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Douglas Hofstadter changes his mind on Deep Learning and AI risk

lesswrong.com

Post reply on HN
Print view View original

161–170 of 520 posts

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#161

Post by [deleted] »

phillipcarter wrote: post #157
ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

I've come to learn that a lot of the rationalist crowd are really just fanfiction authors. That's fine - people should be able to do that - but I don't like how they're given the limelight on things that they generally have little expertise and hands-on knowledge with. I want to like them and want to have them engaged in discourse, but I find them so insufferable. Not to mention the sub-crowd of rationalists that is…

[deleted]

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#162

Post by arisAlexis »

ke88y wrote: post #155
arisAlexis wrote: post #145

Earlier quoted context omitted.

Loling something that serious with several top scientists as you said warning is at least very idiotic regardless where the truth lies. Let's start from that.

I'm laughing at the study methodology, not the underlying topic. Please don't call me idiotic. I'd wager I have spent at least 10,000 hours more than you working on (real) safety analyses for (real) AI systems, so refraining from insinuating I'm dismissive would also be nice. But refraining from "idiotic" seems like a minimum baseline.

[flagged]

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#163

Post by ctoth »

lolinder wrote: post #87
DennisP wrote: post #62

Earlier quoted context omitted.

As long as the AI is less generally intelligent than people, it makes sense to be more afraid of the people. Once the AI is more intelligent than the smartest people, it's more sensible to be most afraid of the AI.

The implication of OP's statement is that they don't believe that AGI is on the horizon, and I'm inclined to agree. This feels a lot like the hype surrounding self-driving cars a few years back, where everyone was convinced fully autonomous vehicles were ~5 years away. It turned out that, while the results we had were impressive, getting the rest of the way to fully replacing humans was much, much harder than was gen…

A few years back (let's call it 2020) and autonomous cars, which are being used for taxi trips today, would be five years in the future. In fact they would be three. Unless something major happens in the next two years, there will still be self-driving cars, even more of them, driving and picking up people in 2025. This is not the argument you think it is.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#164

Post by tyre »

ralphc wrote: post #107

When I read about the dangers of AI I'm reminded of the feeling I had after reading Jeff Hawkin's "On Intelligence". He talked about simulating the neocortex to do many of the things that deep learning and LLM's are doing now. His research may or may not be a dead end but his work, and this work, to me seems like we're building the neocortex layer without building the underlying "lizard brain" that higher animals' br…

This is a great way to put it.

The counter to "current generation AI is terrifying" seems to fall along the lines of it not being nearly as close to AGI as the layperson believes.

But I don't think that matters.

I don't believe that LLMs or image/voice/video generative models need to do much beyond what they can do today in order to wreck civilization level disaster. They don't need to become Skynet, learn to operate drone armies, hack critical infrastructure, or engineer pandemics. LLMs allow dynamic, adaptive, scalable, and targeted propaganda.

Already we have seen the effects of social media's reach combined with brute forced content generation. LLMs allow this to happen faster and at a higher fidelity. That could be enough to tip the balance and trigger a world war.

I don't think it takes a huge amount of faked primary material (generated phone calls, fuzzy video, etc.) that's massively amplified until it becomes "true enough" to drive a Chinese invasion of Taiwan, a Russian tactical nuclear strike in Ukraine, an armed insurrection in the United States.

We're close to there already.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#165

Post by wslh »

pbw wrote: post #147

Hofstadter says humans will be like cockroaches compared to AI. This is an oft-repeated line: sometimes we are ants or bacteria. But I think these comparisons might be totally wrong. I think it's very possible there's a Intelligence Completeness theorem that's analogous to Turing Completeness. A theorem that says intelligence is in some ways universal, and that our intelligence will be compatible with all other forms…

> A theorem that says intelligence is in some ways universal, and that our intelligence will be compatible with all other forms of intelligence, even if they are much "smarter".

I would not be so reductionist. Intelligence doesn't seem to be an universal thing, even IQ (a human invented metric) is measured in terms of some statistics. If you have an IQ of ~60 you have intelligence but a completely different one from an IQ >85.

> But humans are intelligent, human languages have a universal grammar and can be indefinitely extended with new words. I think this puts us in the intelligence species club, and all species in that club can all discuss any idea.

Humans have different intelligences. You can be intelligent (per the human intelligence definition) but a math ignorant. Again, this implies intelligence as we know it is not an universal thing at higher levels: not all people can have a physics Ph.D. as not all people could be a good artist where good techniques are recognizable, same for music, etc.

Yes, a cockroach is in another level of intelligence (or non-intelligence) but that does not mean there is not a super-intelligence that makes us relative cockroach.

Also, without any intention of talking about religion or "intelligent design", we can theorize that the Universe is supersmart because it creates intelligent creatures, even if it is not conscious about that. I would be very catious to define intelligence in an universal way.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#166

Post by ethanbond »

jrflowers wrote: post #158
llamaimperative wrote: post #141

Earlier quoted context omitted.

If or when AI is capable of doing all the great things people proclaim it to be able to do, then it will also be capable of doing immense harm, and we should be putting more work into mitigating that. Like it really is that simple. AI generally, LLMs specifically, and certainly this crop of LLMs in particular might end up being inert pieces of technology. But to the precise extent that they are not inert, they carry…

> The AI skeptics don't need to know which outcome the future holds. But they need to interpret a benign point about the undisputed fact that an LLM has never convinced anybody to fly a plane into a building as some sort of dangerous ignorance of risk that needs correcting.

Person observing Trinity test: "Wow this is really dangerous and could wipe out a city"

Very astute response: "An atomic weapon has never wiped out a city"

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#167

Post by ke88y »

arisAlexis wrote: post #162
ke88y wrote: post #155

Earlier quoted context omitted.

I'm laughing at the study methodology, not the underlying topic. Please don't call me idiotic. I'd wager I have spent at least 10,000 hours more than you working on (real) safety analyses for (real) AI systems, so refraining from insinuating I'm dismissive would also be nice. But refraining from "idiotic" seems like a minimum baseline.

[flagged]

> I repeat everyone that is loling at x-risk an idiot

I mean this is the nicest possible way and hope that you consider it constructive: you're willfully misconstruing what I am saying and then calling me an idiot. This isn't a nice thing to do and shuts down the conversation.

If I believed you were operating in good faith, I might take the time to explain why my practical/scientific experience makes me incredulous of what's technically possible, and why my life experience makes me suspicious of the personalities involved in pushing AI x-risk.

I might also provide a history lesson on Einstein and Oppenheimer, which is instructive to how we should think about x-risk (spoiler: both advocated for and were involved in the development of atomic weapons).

But since you're calling me an idiot and misconstruing my words, I have no interest in conversing with you. Have a good day.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#168

Post by zone411 »

echelon wrote: post #138
zone411 wrote: post #128

Earlier quoted context omitted.

If you read the survey, you'll find that many concerned researchers don't believe we're at a 90% chance of doom, but e.g. 10%. So, this type of response wouldn't be rational if they're thinking that things will go fine most of the time. If these researchers are thinking logically, they would also realize that this kind of reaction has little chance of success, especially if research continues in places like China. It…

> If you read the survey Not a nice way to engage in debate. I've spent more time listening to and refuting these arguments than most. Including debating Eliezer. > many concerned researchers don't believe we're at a 90% chance of doom, but e.g. 10%. A 10% chance of an asteroid hitting the earth would result in every country in the world diverting all of their budgets into building a means to deflect it. > So, this t…

Yudkowsky represents the extreme end of risk concern. You can't fault others who estimate the risk at 10%, with huge uncertainty about when this risk will materialize and its chances, for not behaving as you'd expect him to.

Believing that people will take extreme actions, which would ruin their careers and likely backfire, based on a 10% chance of things going terribly wrong in maybe 30 years is strange.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#169

Post by petermcneeley »

dist-epoch wrote: post #131
ke88y wrote: post #93

This is something weird happening around Rationalism/X-Risk/AGI prognostications. The "Great Minds And Great Leaders" types are rushing to warn about the risks, as are a large number of people who spend a lot of time philosophizing. But the actual scientists on the ground -- the PhDs and engineers I work with every day and who have been in this field, at the bench, doing to work on the latest generation of generative…

Nassim Taleb makes a great point: - The best person to judge the risk of playing roulette is not the carpenter who built it. - The best person to judge the risks of a global pandemic is not a virologist working with viruses daily. You can extend that: - The best person to judge the cybersecurity risks of an application is not the programmer implementing it.

Either Nassim is missing the point or you are misquoting him. I think this is more about should experts be in charge of their own domain.

Re: Douglas Hofstadter changes his mind on Deep Learning and AI risk

#170

Post by esafak »

jurgenaut23 wrote: post #9

I read Hofstadter's GEB and Tegmark's Our Mathematical Universe, and of course I developed a rather fond admiration of these brilliant minds. For some reason, both of them have developed a profound aversion and fear of what they consider an existential threat. I have a solid theoretical understanding of these systems, and I spent 15 years studying, building, and deploying them at scale and for diverse use cases. The…

LLMs are not going to do anything but the next class of models may well. They will have a physical understanding of the world, like us. Slap some sensors and a body on it and you will have something that might look scary.
Post reply on HN