Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

811–820 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#811

Post by earleybird »

dghlsakjg wrote: post #642
bondarchuk wrote: post #632

Earlier quoted context omitted.

> Every time I see Yudkowsky’s writing I wonder why people take him seriously. Because he identified the potential risk of superhuman AI 20 years before almost everyone else. Sure, science fiction also identified that risk (as people here seem eager to point out), but he identified it as an actual real-world risk and has been trying to take real-world action on it for 20 years. No matter what else you think about him…

Hal 9000, the malevolent AI in 2001: a space Odyssey, was in one of the most respected films of all time three years before Yudkowsky was born. Plenty of people have been beating that drum for years.

Including Colossus: The Forbin Project; two years after 2001 (1968). In some ways, Colossus hits much closer than 2001 with respect to unfettered AI.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#812

Post by hackinthebochs »

vidarh wrote: post #772
hackinthebochs wrote: post #766

Earlier quoted context omitted.

There are physical limits to what can make an effective and efficient computational substrate. There are physical limits to how fast/cheap these GPUs can be made. It is highly doubtful that some rogue nation is going to invent some entirely unknown but equally effective computational method or substrate. Controlling the source of the known substrate is possible and effective. Most nations aren't in a position to just…

Unless you're going to ban the sales of current capacity gaming level gpus and destroy all of the ones already in the market, the horse bolted a long time ago, and even if you managed to do that it'd still not be enough. As it is, we keep seeing researchers with relatively modest funding steadily driving down the amount of compute required for equivalent quality models month by month. Couple that with steady improvem…

>Unless you're going to ban the sales of current capacity gaming level gpus and destroy all of the ones already in the market, the horse bolted a long time ago, and even if you managed to do that it'd still not be enough.

That's silly. We can detect the acquisition of tens of thousands of high end GPUs by a single entity. And smaller nations training their own models is besides the point. The issue isn't to stop random nation from training their own GPT-4, it's to short circuit the possibility of training a potentially dangerous AGI, which is at least multiple game changing innovations down the line. The knowledge to train GPT-4 is already out there. The knowledge to train an AGI doesn't exist yet. We can ensure that only a few entities are even in a position to take a legitimate stab at it, and ensure that the knowledge is tightly controlled. We just have to be willing.

The precedence is nuclear arms controls and the monitoring of the raw materials to develop deadly pathogens. The claim that this monitoring isn't possible doesn't pass the smell test.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#813

Post by hackinthebochs »

breuleux wrote: post #700
hackinthebochs wrote: post #679

Earlier quoted context omitted.

Yes, every point in favor of the possibility of AGI comes with an asterisk. That's not all that interesting. We need to be competent at reasoning under uncertainty, something few people seem to be capable of. When the utility of a runaway AGI is infinitely negative, while the possibility of that outcome is substantially non-zero, rationality demands we act to prevent that outcome. >How would they force perfect alignm…

> Lack of independence implies low coordination costs Perhaps, but it's not obvious. Lack of independence implies more back-and-forth communication with the central coordinator, whereas independent agents could do more work before communication is required. It's a tradeoff. > the features of an artificial intelligence implies the ability to maximally utilize the abilities of the sub-components Does it? Can you elabor…

>Does it? Can you elaborate?

I don't intend to say anything controversial here. The consideration is the tradeoff between independence and tight constraints of the subcomponents. Independent entities have their own interests, as well as added computational and energetic costs involved in managing a whole entity. These are costs that can't be directed towards the overarching goal. On the other hand, tightly constrained components do not have this extra overhead and so their capacity can be fully directed towards the goal as determined by the control system. In terms of utilization of compute and energy towards the principle goal, a unified AI will be more efficient.

>If an intelligence can coordinate "mini-brains" fully reliably (a big if, by the way), presumably I can do something similar with a Python script or narrow AI.

This is plausible, and I'm totally in favor of exploiting narrow AI to maximal effect. If the only AI we ever had to worry about was narrow AI, I wouldn't have any issue aside from the mundane issues we get with the potential misuse of any new technology. But we know people (e.g. open AI) are explicitly aiming towards AGI so we need to be planning for this eventuality.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#814

Post by dghlsakjg »

TeMPOraL wrote: post #777
dghlsakjg wrote: post #642

Earlier quoted context omitted.

Hal 9000, the malevolent AI in 2001: a space Odyssey, was in one of the most respected films of all time three years before Yudkowsky was born. Plenty of people have been beating that drum for years.

Kubrick created a memorable character and a plot twist, but the movie itself wasn't meant to be a warning about AGI/ASI. "${my favorite work of fiction} mentioned/alluded to it too!" does not make that work of fiction equivalent to a serious take on the topic in real-world context.

The gp was edited.

It originally credited Mr. Yudkowsky with being the first person to imagine ai as a serious threat to humans, without mentioning that the trope existed in sci-fi, or mentioning sci-fi at all

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#815

Post by dghlsakjg »

bondarchuk wrote: post #662
dghlsakjg wrote: post #642

Earlier quoted context omitted.

Hal 9000, the malevolent AI in 2001: a space Odyssey, was in one of the most respected films of all time three years before Yudkowsky was born. Plenty of people have been beating that drum for years.

I addressed science fiction in my comment. The crucial difference is taking it seriously and actually trying to do something tangible about slowing/stopping AI progress, which is not something Stanley Kubrick did.

You edited the comment after I posted.

Your original made no mention of sci-fi, and made it sound as if no one had imagined a malicious AI before yudkowski

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#816

Post by FeepingCreature »

jrflowers wrote: post #702
FeepingCreature wrote: post #173

Earlier quoted context omitted.

It turns out if you're thinking about a problem twenty years before everyone else you tend to be the first person to make lots of arguments. So I don't see how "without restating his specific points" is supposed to be feasible. If Eliezer has already made all the strongest points, are people supposed to invent new ones?

> If Eliezer has already made all the strongest points, are people supposed to invent new ones? I like this post because the rebuttal to “there’s a kind of cult of personality around this guy (1) where everybody just sort of agrees that he’s categorically correct to the extent that world leaders are stupid or dangerous not to defer to the one guy on policy” is “Yeah he really can see the future. The proof of that is…

Yeah the thing about the future is that it's in the future? Considering all the people who say things like "AGI will take a century" and "superintelligence is impossible", like, Eliezer also did not call ChatGPT but this world surely looks more like the one he predicted than the one his detractors predicted. There's a reason so many of the people working at current AI orgs got into AI by reading Eliezer.

Also: how dare they have fun.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#818

Post by [deleted] »

FeepingCreature wrote: post #816
jrflowers wrote: post #702

Earlier quoted context omitted.

> If Eliezer has already made all the strongest points, are people supposed to invent new ones? I like this post because the rebuttal to “there’s a kind of cult of personality around this guy (1) where everybody just sort of agrees that he’s categorically correct to the extent that world leaders are stupid or dangerous not to defer to the one guy on policy” is “Yeah he really can see the future. The proof of that is…

Yeah the thing about the future is that it's in the future? Considering all the people who say things like "AGI will take a century" and "superintelligence is impossible", like, Eliezer also did not call ChatGPT but this world surely looks more like the one he predicted than the one his detractors predicted. There's a reason so many of the people working at current AI orgs got into AI by reading Eliezer. Also: how da…

[deleted]

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#819

Post by cjbprime »

hodgesrm wrote: post #806
cjbprime wrote: post #643

Earlier quoted context omitted.

I don't think I was doing either? Assume they are widely used. Does every human die?

Oh sorry! I meant that the people to whom you referred above focus on the "one true catastrophe" (whatever that is) without truly thinking what it would mean to have a few dozen nuclear weapons used. It's not just the immediate effect of the explosions. It's also the destruction of power distribution, industrial capacity, food and medicine production, and other things that millions of people depend on.

No worries, but I think we're still dancing around the same point. I think those people would say that they agree that nuclear war is very bad, and that it has many nth-order consequences which are all clearly catastrophic, but that (for example) while disrupted supply of medicine causes a great deal of human suffering and unnecessary deaths, it remains difficult to imagine how it might result in literal human extinction.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#820

Post by roryokane »

m3kw9 wrote: post #546
homefree wrote: post #70

I think Yann is probably wrong. He refuses to engage earnestly with the “doomer” arguments. The same type of motivated reasoning could also be attributed to himself and Meta’s financial goals - it’s not a persuasive framing. The attempts I’ve seen from him to discuss the issue that aren’t just name calling are things like saying he knows smart people and they aren’t president - or even that his cat is pretty smart an…

Yet nobody has a clear practical and reasonable scenario where an AI will cause human extinction. Pls don’t say AI will control the launch codes and start firing randomly.

According to a survey summarized in this post from 2021 (https://www.astralcodexten.com/p/updated-look-at-long-term-a...), “people who work in ‘AI safety and governance’” “mostly assign “low” probability (~10%) that unaligned AI will result in human extinction”. They more commonly worry about other scenarios that are still very bad for humanity.

To give one example, an AI could be instructed to reduce the crime rate, but rather than preventing people from committing crimes, the AI could decide it’s easier to prevent all victims from reporting crimes to anybody. That would remove the threat of law enforcement that currently prevents many people from turning to crime. Such a society probably wouldn’t cause human extinction, but would still be horrific and worth preventing.

If you are skeptical that AI that is trapped in software could gain any significant real-world power (not necessarily the power to kill all of humanity), https://slatestarcodex.com/2015/04/07/no-physical-substrate-... describes some possible mechanisms.

Post reply on HN