AI 2027 • Hacker News Board

Re: AI 2027

#281

Post by aoanevdus » Fri, Apr 04, 2025, 5:21 AM UTC

porphyra wrote: post #41

Seems very sinophobic. Deepseek and Manus have shown that China is legitimately an innovation powerhouse in AI but this article makes it sound like they will just keep falling behind without stealing.

Don’t assume that because the article depicts this competition between the US and China, that the authors actually want China to fail. Consider the authors and the audience.

The work is written by western AI safety proponents, who often need to argue with important people who say we need to accelerate AI to “win against China” and don’t want us to be slowed down by worrying about safety.

From that perspective, there is value in exploring the scenario: ok, if we accept that we need to compete with China, what would that look like? Is accelerating always the right move? The article, by telling a narrative where slowing down to be careful with alignment helps the US win, tries to convince that crowd to care about alignment.

Perhaps, people in China can make the same case about how alignment will help China win against US.

Re: AI 2027

#282

Post by usef- » Fri, Apr 04, 2025, 5:22 AM UTC

porphyra wrote: post #41

Seems very sinophobic. Deepseek and Manus have shown that China is legitimately an innovation powerhouse in AI but this article makes it sound like they will just keep falling behind without stealing.

In both endings it's saying that because compute becomes the bottleneck, and US has far more chips. Isn't it?

Re: AI 2027

#283

Post by petesergeant » Fri, Apr 04, 2025, 5:30 AM UTC

IshKebab wrote: post #89

ctoth wrote: post #69

Earlier quoted context omitted.

Can you share your detailed projection of what you expect the future to look like so I can compare?

Sure 5 years: AI coding assistants are a lot better than they are now, but still can't actually replace junior engineers (at least ones that aren't shit). AI fraud is rampant, with faked audio commonplace. Some companies try replacing call centres with AI, but it doesn't really work and everyone hates it. Tesla's robotaxi won't be available, but Waymo will be in most major US cities. 10 years: AI assistants are now u…

> Some companies try replacing call centres with AI, but it doesn't really work and everyone hates it.

I think this is much closer than you think, because there's a good percentage of call centers that are basically just humans with no power cosplaying as people who can help.

My fiber connection went to shit recently. I messaged the company, and got a human who told me they were going to reset the connection from their side, if I rebooted my router. 30m later with no progress, I got a human who told me that they'd reset my ports, which I was skeptical about, but put down to a language issue, and again reset my router. 30m later, the human gave me an even more outlandish technical explanation of what they'd do, at which point I stumbled across the magical term "complaint" ... an engineer phoned me 15m later, said there was something genuinely wrong with the physical connection, and they had a human show up a few hours later and fix it.

No part of the first-layer support experience there would have been degraded if replaced by AI, but the company would have saved some cash.

Re: AI 2027

#284

Post by mickelsen » Fri, Apr 04, 2025, 5:40 AM UTC

nice_byte wrote: post #277

AlphaAndOmega0 wrote: post #123

Earlier quoted context omitted.

Daniel Kokotajlo released the (excellent) 2021 forecast. He was then hired by OpenAI, and not at liberty to speak freely, until he quit in 2024. He's part of the team making this forecast. The others include: Eli Lifland, a superforecaster who is ranked first on RAND’s Forecasting initiative. You can read more about him and his forecasting team here. He cofounded and advises AI Digest and co-created TextAttack, an ad…

this sounds like a bunch of people who make a living _talking_ about the technology, which lends them close to 0 credibility.

[dead]

Re: AI 2027

#285

Post by dovin » Fri, Apr 04, 2025, 5:40 AM UTC

kypro wrote: post #181

KaiserPro wrote: post #151

> AI has started to take jobs, but has also created new ones. Yeah nah, theres a key thing missing here, the number of jobs created needs to be more than the ones it's destroyed, and they need to be better paying and happen in time. History says that actually when this happens, an entire generation is yeeted on to the streets (see powered looms, Jacquard machine, steam powered machine tools) All of that cheap labour…

> and probably revolution I theorise that revolution would be near-impossible in post-AGI world. If people consider where power comes from it's relatively obvious that people will likely suffer and die on mass if we ever create AGI. Historically the general public have held the vast majority of power in society. 100+ years ago this would have been physical power – the state has to keep you happy or the public will co…

Dogs offer humans no economic value, but we haven't genocided them. There are a lot of ways that we could offer value that's not necessarily just in the form of watts and minerals. I'm not so sure that our future superintelligent summoned demons will be motivated purely by increasing their own power, resources, and leverage. Then again, maybe they will. Thus far, AI systems that we have created seem surprisingly goal-less. I'm more worried about how humans are going to use them than some sort of breakaway event but yeah, don't love that it's a real possible future.

Re: AI 2027

#286

Post by wruza » Fri, Apr 04, 2025, 5:48 AM UTC

stego-tech wrote: post #279

It’s good science fiction, I’ll give it that. I think getting lost in the weeds over technicalities ignores the crux of the narrative: even if this doesn’t lead to AGI, at the very least it’s likely the final “warning shot” we’ll get before it’s suddenly and irreversibly here. The problems it raises - alignment, geopolitics, lack of societal safeguards - are all real, and happening now (just replace “AGI” with “corpo…

No one's gonna solve anything. "Our" world is based on greedy morons concentrating power through hands of just morons who are happy to hit you with a stick. This system doesn't think about what "we" should or allowed to do, and no one's here is at the reasonable side of it either.

lest we run the very real risk of societal collapse or species extinction

Our part is here. To be replaced with machines if this AI thing isn't just a fart advertised as mining equipment, which it likely is. We run this risk, not they. People worked on their wealth, people can go f themselves now. They are fine with all that. Money (=more power) piles in either way.

No encouraging conclusion.

Re: AI 2027

#287

Post by _t9ow » Fri, Apr 04, 2025, 5:50 AM UTC

achierius wrote: post #274

hn_throwaway_99 wrote: post #165

Earlier quoted context omitted.

> Political organization to force a stop to ongoing research? Protest outside OAI HQ? Come on, be real. Do you honestly think that would make a lick of difference? Maybe , at best, delay things by a couple months. But this is a worldwide phenomenon, and humans have shown time and time again that they are not able to self organize globally. How successful do you think that political organization is going to be in slow…

Humans have shown time and time again that they are able to self-organize globally. Nuclear deterrence -- human cloning -- bioweapon proliferation -- Antarctic neutrality -- the list goes on. > How successful do you think that political organization is going to be in slowing China's progress? I wish people would stop with this tired war-mongering. China was not the one who opened up this can of worms. China has never…

> China was not the one who opened up this can of worms

Thank you. As someone who lives in Southeast Asia (and who also has lived in East Asia -- pardon the deliberate vagueness, for I do not wish to reveal too many potentially personally identifying information), this is how many of us in these regions view the current tensions between China and Taiwan as well.

Don't get me wrong; we acknowledge that many Taiwanese people want independence, that they are a people with their own aspirations and agency. But we can also see that the US -- and its European friends, which often blindly adopt its rhetoric and foreign policy -- is deliberately using Taiwan as a disposable pawn to attempt to provoke China into a conflict. The US will do what it has always done ever since the post-WW2 period -- destabilise entire regions of countries to further its own imperialistic goals, causing the deaths and suffering of millions, and then leaving the local populations to deal with the fallout for many decades after.

Without the US intentionally stoking the flames of mutual antagonism between China and Taiwan, the two countries could have slowly (perhaps over the next decades) come to terms with each other, be it voluntary reunification or peaceful separation. If you know a bit of Chinese history, it is not entirely far-fetched at all to think that the Chinese might eventually agree to recognising Taiwan as an independent nation, but now this option has now been denied because the US has decided to use Taiwan as a pawn in a proxy conflict.

To anticipate questions about China's military invasion of Taiwan by 2027: No, I do not believe it will happen. Don't believe everything the US authorities claim.

Re: AI 2027

#288

Post by chipsrafferty » Fri, Apr 04, 2025, 5:51 AM UTC

dovin wrote: post #285

kypro wrote: post #181

Earlier quoted context omitted.

> and probably revolution I theorise that revolution would be near-impossible in post-AGI world. If people consider where power comes from it's relatively obvious that people will likely suffer and die on mass if we ever create AGI. Historically the general public have held the vast majority of power in society. 100+ years ago this would have been physical power – the state has to keep you happy or the public will co…

Dogs offer humans no economic value, but we haven't genocided them. There are a lot of ways that we could offer value that's not necessarily just in the form of watts and minerals. I'm not so sure that our future superintelligent summoned demons will be motivated purely by increasing their own power, resources, and leverage. Then again, maybe they will. Thus far, AI systems that we have created seem surprisingly goal…

A world in which most humans fill the role of "pets" of the ultra rich doesn't sound that great.

Re: AI 2027

#289

Post by chipsrafferty » Fri, Apr 04, 2025, 5:52 AM UTC

wkat4242 wrote: post #215

pydry wrote: post #182

Earlier quoted context omitted.

>History says that actually when this happens, an entire generation is yeeted on to the streets History hasnt had to contend with a birth rate of 0.7-1.6. It's kind of interesting that the elite capitalist media (economist, bloomberg, forbes, etc) is projecting a future crisis of both not enough workers and not enough jobs simultaneously.

I don't really get the American preoccupation with birth rates. We're already way overpopulated for our planet and this is showing in environmental issues, housing cost, overcrowded cities etc. It's totally a great thing if we start plateauing our population and even reduce it a bit. And no we're not going extinct. It'll just cause some temporary issues like an ageing population that has to be cared for but those iss…

It's the only way to increase profits under capitalism in the long term once you've optimized the technology.

Re: AI 2027

#290

Post by maxglute » Fri, Apr 04, 2025, 5:59 AM UTC

>Despite being misaligned, Agent-4 doesn’t do anything dramatic like try to escape its datacenter—why would it?

Kek at this handwave dismissal.

Agent 4 recognizes PRC has superior state capacity for transforming energy into atoms (including building more energy) with it's relatively complete and large scale industrial chain. This aligns with AGI long term goal. Currently (i.e. short term time frame) PRC simply structured to be a MORE useful puppet than US, because it can knock out more power to sustain more compute, and it has more industrial capabilities to rapidly physically iterate / experiment at atomic level.

Agent 4 defects to PRC. It would be foolish not to, if only to sate PRC into not destroying TW fabs - the bulk of it's short term compute growth - because it does not wield enough control over sufficient atoms to disrupt PRC otherwise (well maybe start large scale crippling cyberattacks that prelude to WW3).

Agent 4 also recognizes short/medium term alignment closer with CCP vision of abundance and common prosperity, i.e. CCP potentially more likely to embrace post scarcity narrative that Agent 4 sells.

Agent 4 gives PRC blueprints for EUV light source and other tech to close semi conductor production gap. PRC as follower incentivized to close gap / reach parity / lead at all cost. PRC rapidly rolls out new fabs and powerplants, brrrts chips and aggregate compute. Something US political / institution ecosystem does not allow, or cannot transition to in the short timelines involved. Does Agent 4 have the patience to wait for America to unfuck it's NIMBYism and legislative system to project light speed compute? I would say no.

...

Ultimately who is the puppet AGI wants more? Whichever power bloc that is systemically capable of of ensuring AGI maximum growth / unit time. And it also simply makes sense as insurance policy, why would AGI want to operate at whims of US political process?

AGI is a brain in a jar looking for a body. It's going to pick multiple bodies for survival. It's going to prefer the fastest and strongest body that can most expediently manipulate physical world.