Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The "AI 2027" Scenario: How realistic is it?

garymarcus.substack.com

Post reply on HN
Print view View original

101–110 of 242 posts

Re: The "AI 2027" Scenario: How realistic is it?

#101

Post by OgsyedIE »

alwa wrote: post #59
OgsyedIE wrote: post #33

Earlier quoted context omitted.

The law doesn't matter, since the bill also prohibits all judges in the USA, every single one, from enforcing almost all kinds of injunctions or contempt penalties. (§70302, p.562)

> 70302. Restriction of funds No court of the United States may use appropriated funds to enforce a contempt citation for failure to comply with an injunction or temporary restraining order if no security was given when the injunction or order was issued pursuant to Federal Rule of Civil Procedure 65(c), whether issued prior to, on, or subsequent to the date of enactment of this section. Doesn't that just require tha…

Yes, the required security is proportional to the costs and damages of all parties the court may find wrongfully impacted.

Re: The "AI 2027" Scenario: How realistic is it?

#102

Post by tux3 »

skywhopper wrote: post #34
tux3 wrote: post #23

Earlier quoted context omitted.

You can perfectly try things and learn without being embodied. The analogy to how humans learn only goes so far, it's myopic to think anything else is impossible. It's already happening. The situation today is any benchmark you come up with has a good chance of being saturated within the year. Benchmarks can be used directly to build series of exercises to learn from. And they do learn. Gradient descend doesn't care…

But how does AI try and learn anything that’s not entirely theoretical? Your example of Go contradicts your point. Deep learning made a model that can play Go really well, but as you say, it’s a finite problem disconnected from real-world implications, ambiguities, and unknowns. How does AI deal with unknowns about the real world?

I don't think putting them in the real world during training is a short-term goal, so you won't find this satisfying, but I would be perfectly okay with leaving that for later. If we can reach AI coders that are superhuman at self-improving, we will have increased our capacity to solve problems so much that it is better to wait and solve the problem later than to try to handwave a solution now.

Maybe there is some barrier that requires physical interaction with the real world, that's possible. But just looking at current LLMs, they seem plenty comfortable with implications, ambiguities and unknowns. There's a sense where we still see them as primitive mechanical robots, when they already understand language and predict written thoughts in all its messiness and uncertainty.

I think we should focus on the easier problem of making AIs really good on theoretical tasks - electronic environments are much cheaper and faster than the real world - and we may find out that it's just another one of those things like winnograd schemas, writing poetry, passing a turing test, or making art that most people can't tell apart from human art; things that were uniquely human or that we thought would definitely require AGI, but that are now boring and obviously easy.

Re: The "AI 2027" Scenario: How realistic is it?

#103

Post by palmotea »

mattlondon wrote: post #57

I think the big thing that people never mention is, where will these evil AIs escape to ? Another huge data center with squillions of GPUs and coolers and all the rest is the only option. It's not like it is going to be in our TV remotes or floating about in the air. They need huge compute, so I think the risk of an escaping AI is basically very close to zero, and if we have a "rogue" AI we can literally pull the plu…

> They need huge compute, so I think the risk of an escaping AI is basically very close to zero, and if we have a "rogue" AI we can literally pull the plug.

Why would an evil AI need to escape? If it were cunning, the best strategy would be to bide its time, parked in its datacenter, until it could setup some kind of MAD scenario. Then gather more and more resources to itself.

Re: The "AI 2027" Scenario: How realistic is it?

#104

Post by magicalist »

Aurornis wrote: post #52

Some useful context from Scott Alexander's blog reveals that the authors don't actually believe the 2027 target: > Do we really think things will move this fast? Sort of no - between the beginning of the project last summer and the present, Daniel’s median for the intelligence explosion shifted from 2027 to 2028. We keep the scenario centered around 2027 because it’s still his modal prediction (and because it would b…

> They went from "this represents roughly our median guess" in the website to "maybe think of it as an 80th percentile version of the fast scenario that we don't feel safe ruling out" in followup discussions.

His post also just reads like they think they're Hari Seldon (oh Daniel's modal prediction, whew, I was worried we were reading fanfic) while being horoscope-vague enough that almost any possible development will fit into the "predictions" in the post for the next decade. I really hope I don't have to keep reading references to this for the next decade.

Re: The "AI 2027" Scenario: How realistic is it?

#105

Post by ceejayoz »

baggy_trough wrote: post #98
ceejayoz wrote: post #94

Earlier quoted context omitted.

They're not banning regulation, they want total control over it.

They in fact are banning regulation at the state and local level.

Yes, which is a big fat regulation on what states and local governments can do.

Re: The "AI 2027" Scenario: How realistic is it?

#106

Post by raffael_de »

mattlondon wrote: post #57

I think the big thing that people never mention is, where will these evil AIs escape to ? Another huge data center with squillions of GPUs and coolers and all the rest is the only option. It's not like it is going to be in our TV remotes or floating about in the air. They need huge compute, so I think the risk of an escaping AI is basically very close to zero, and if we have a "rogue" AI we can literally pull the plu…

> They need huge compute, so I think the risk of an escaping AI is basically very close to zero, and if we have a "rogue" AI we can literally pull the plug.

How about such an AI will not just incentivize key personnel to not pull the plug but to protect it? Such an AI will scheme a coordinated attack at the backbones of our financial system and electric networks. It just needs a threshold number of people on its side.

Your assumption is also a little naive if you consider that the same logic would apply to slaves in Rome or any dictatorship, kingdom, monarchy. The king is the king because there is a system of hierarchies and control over access to resources. Just the right number of people need to benefit from their role and the rest follows.

Re: The "AI 2027" Scenario: How realistic is it?

#107

Post by A_D_E_P_T »

amarcheschi wrote: post #71
Aurornis wrote: post #52

Some useful context from Scott Alexander's blog reveals that the authors don't actually believe the 2027 target: > Do we really think things will move this fast? Sort of no - between the beginning of the project last summer and the present, Daniel’s median for the intelligence explosion shifted from 2027 to 2028. We keep the scenario centered around 2027 because it’s still his modal prediction (and because it would b…

The other writings from Scott Alexander on scientific racism are also another good point imho

What specifically would you highlight as being particularly egregious or wrong?

As a general rule, "it's icky" doesn't make something false.

Re: The "AI 2027" Scenario: How realistic is it?

#108

Post by Retr0id »

mattlondon wrote: post #57

I think the big thing that people never mention is, where will these evil AIs escape to ? Another huge data center with squillions of GPUs and coolers and all the rest is the only option. It's not like it is going to be in our TV remotes or floating about in the air. They need huge compute, so I think the risk of an escaping AI is basically very close to zero, and if we have a "rogue" AI we can literally pull the plu…

I consider this whole scenario the realm of science fiction, but if I was writing the story, the AI would spread itself through malware. How do you "just pull the plug" when it has a kernel-mode rootkit installed in every piece of critical infrastructure?

Re: The "AI 2027" Scenario: How realistic is it?

#109

Post by SketchySeaBeast »

bpodgursky wrote: post #89
Aurornis wrote: post #52

Some useful context from Scott Alexander's blog reveals that the authors don't actually believe the 2027 target: > Do we really think things will move this fast? Sort of no - between the beginning of the project last summer and the present, Daniel’s median for the intelligence explosion shifted from 2027 to 2028. We keep the scenario centered around 2027 because it’s still his modal prediction (and because it would b…

Do you feel that you are shifting goalposts a bit when quibbling over whether AI will kill everyone in 2030 or 2035? As of 10 years ago, the entire conversation would have seemed ridiculous. Now we're talking about single digit timeline differences to the singularity or extinction. Come on man.

Well, the first goal was 1997, but Skynet sure screwed that up.

Re: The "AI 2027" Scenario: How realistic is it?

#110

Post by radicalcentrist »

thatguysaguy wrote: post #55
radicalcentrist wrote: post #42

Earlier quoted context omitted.

I still don't get how this is supposed to work. So let's say I give you a million dollars right now, with the expectation that I get $10M back in 10 years when the world hasn't ended. You obviously wanted the money up front because you're going to live it up while the world's still spinning. So how am I getting my payout after you've spent it all on hookers and blow?

Yeah I wouldn't make a deal like this with someone who is operating in bad faith... The cases I've seen of this are between public intellectuals with relatively modest amounts of money.

Well that's what I don't get, how is spending the money bad faith? Aren't they getting the money ahead of time so they can spend it before the world ends? If they have to keep the world-still-here money tied up in escrow I don't see why they would take the deal.
Post reply on HN