Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI 2040: Plan A

ai-2040.com

Post reply on HN
Print view View original

381–390 of 567 posts

Re: AI 2040: Plan A

#381

Post by fwipsy »

taurath wrote: post #267

This is religious fervor folks, as AI 2027 was. I grew up in evangelical christianity, and to them the end of the world is just around the corner, the same way it has been since I was a small child and likely will be when we are all gone. This isn't science. This isn't hypothesis experiment record results. This is very expensive astrology, shiny rock collecting, ritualistic meaning-making and self-justification. Yall…

I remember thinking exactly the same thing around 5-7 years ago, in the GPT-2/GPT-3 era. "Oh sure they can produce semi-coherent output, but truly intelligent behavior is still far away. This isn't science fiction, they're just falling prey to Pascal's Mugging same as my religious friends did." Now I'm not so sure. I give the AI safety subculture as a whole a lot of credit for putting it on my radar back when it was otherwise still science fiction. I don't know if they're right about what comes next, but I think their case deserves to be evaluated on its merits, rather than assumed to be the result of psychological flaws.

> Yall, with your incredible wealth and resources you could do real good in this world and make society better, healthier, better educated, and the whole world more equal, just, and reduce the desperation and suffering. Reject the false and self-serving narratives that empathy doesn't matter, that altruism isn't "effective". You can change a person's whole life in a moment.

Confused at who this is directed towards. I'm fairly certain that the article was written by people who (at some point) identified as effective altruists, most of whom would enthusiastically agree with this. This community didn't start as AI researchers and later choose effective altruism; they were effective altruists who chose AI safety research as the most effective way to improve the world. Given that you apparently share their goals (a better world,) isn't it worth at least hearing them out on their methods?

Re: AI 2040: Plan A

#382

Post by davemp »

derektank wrote: post #360
27183 wrote: post #336

Earlier quoted context omitted.

I used the example of 1G constant acceleration space flight in another thread which got downvoted to oblivion, but I think it's a good one. That's a technology we know how to build. We just need superconducting electronics and miniaturized fusion reactors, or a ship which is built like Project Orion to use nuclear bombs for propulsion. Now write down a blueprint for superintelligence. So I've given you two impossible…

As early as the late 19th century, Louis Pasteur’s work had inspired a belief in the scientific community that it must, in principle be possible to selectively exterminate bacteria. The German physician Paul Ehrlich expounded on this in greatest detail in 1907 when he described his “magic bullet” (or Zauberkugel) theory for effectively targeting pathogens without harming the human host, similar to the immune system.…

I don’t think people are arguing to stop researching AGI. Moreso against sales people trying to use the concept of AGI to sell products that are very much not AGI. Or devoting so many of our resources into such a pursuit that it causes harm to real people.

This is obviously complicated by the fact that LLMs/Agents are useful by themselves, but that’s not really the topic at hand.

Re: AI 2040: Plan A

#383

Post by romanhounds »

fwipsy wrote: post #381
taurath wrote: post #267

This is religious fervor folks, as AI 2027 was. I grew up in evangelical christianity, and to them the end of the world is just around the corner, the same way it has been since I was a small child and likely will be when we are all gone. This isn't science. This isn't hypothesis experiment record results. This is very expensive astrology, shiny rock collecting, ritualistic meaning-making and self-justification. Yall…

I remember thinking exactly the same thing around 5-7 years ago, in the GPT-2/GPT-3 era. "Oh sure they can produce semi-coherent output, but truly intelligent behavior is still far away. This isn't science fiction, they're just falling prey to Pascal's Mugging same as my religious friends did." Now I'm not so sure. I give the AI safety subculture as a whole a lot of credit for putting it on my radar back when it was…

>Given that you apparently share their goals (a better world,) isn't it worth at least hearing them out on their methods?

Their methods are about convincing others that things that enrich and empower themselves at the expense of others is "improving the world". This isn't the stance of serious people who want to improve the world.

Re: AI 2040: Plan A

#384

Post by fwipsy »

romanhounds wrote: post #383
fwipsy wrote: post #381

Earlier quoted context omitted.

I remember thinking exactly the same thing around 5-7 years ago, in the GPT-2/GPT-3 era. "Oh sure they can produce semi-coherent output, but truly intelligent behavior is still far away. This isn't science fiction, they're just falling prey to Pascal's Mugging same as my religious friends did." Now I'm not so sure. I give the AI safety subculture as a whole a lot of credit for putting it on my radar back when it was…

>Given that you apparently share their goals (a better world,) isn't it worth at least hearing them out on their methods? Their methods are about convincing others that things that enrich and empower themselves at the expense of others is "improving the world". This isn't the stance of serious people who want to improve the world.

Please don't confuse AI researchers and AI safety researchers (although there is some overlap.) That's like confusing the people from the Manhattan project with the people who protested 3 Mile Island, because they're both focused on nuclear technology. One effect of EA is that every AI researcher says they're trying to make the world better. Some of them are full of shit, but not all.

I'm fairly certain the authors would be happy to see AI shut down indefinitely. They just don't believe that the coordination problem is solvable. This is their best attempt to come up with something workable in the real world, or at least get people started thinking about it.

Re: AI 2040: Plan A

#385

Post by Der_Einzige »

lelanthran wrote: post #279
rayiner wrote: post #39

Earlier quoted context omitted.

The alternative isn’t “everyone dying.” It’s us holding all the cards.

> The alternative isn’t “everyone dying.” It’s us holding all the cards. That's one outcome, certainly, but not the only one nor, I contend, the most likely one. A most likely outcome of ASI is human extinction, because there's more paths to an ELE outcome for humans from ASI than there is for non-extinction level outcome. Your outcome is only possible if: 1. ASI is never able to escape the confines it is placed in.…

Human extinction is good. Finally we built a benevolent world exploder. Oh no! The negative utilitarians get what they wanted finally!

If ASI is trying to wipe out all humans, we probably deserved it. Unironically!

Re: AI 2040: Plan A

#386

Post by mattwiese »

Ironically, the origin of "cognitive dissonance" as a concept is attributable to Leon Festinger who (with others) studied a UFO cult called The Seekers in the 1950s who believed in imminent apocalypse. As other commenters have noted, pushing the date back was inevitable. Time is a flat circle and we repeat the mistakes of our ancestors just with different coats of paint...

Re: AI 2040: Plan A

#387

Post by erwald »

VladVladikoff wrote: post #329

>We already wrote a scenario about this, called AI 2027. It depicts takeoff happening in 2027 instead of 2030 So in less than 3 years, their exponential growth curve doomsday prediction has moved back 3 years. This seems to be the opposite of exponential growth.

No, 2027 was never their median forecast; it was their modal forecast. See https://blog.aifutures.org/p/clarifying-how-our-ai-timelines...

It's more like it moved from 2028-2032 to 2030-2035 (depending on the author).

Re: AI 2040: Plan A

#388

Post by sudb »

27183 wrote: post #361
sudb wrote: post #351

Earlier quoted context omitted.

Is an implication of this that models are incapable of producing entirely novel code? Also, not to get too reductionist about this, but what do you posit is special about what is happening when humans think? Intelligence is hard to define so clearly, I reckon.

> Is an implication of this that models are incapable of producing entirely novel code? No, it does not imply that at all. Google "temperature in LLMs". > what do you posit is special about what is happening when humans think? I don't. And IIUC nobody knows, but I'm not a brain scientist. There have been some wild theories over the years (recall Penrose's). I don't really have a dog in the hunt, except that probably…

> Google "temperature in LLMs".

No need to condescend, I'm very aware of what temperature is for LLMs. But I'm going to push back - if you're claiming all LLMs simply do is a stochastic _search_, how can that produce novelty, in the conceptual sense? (I'm not, for example, talking about novel rearrangement of existing ideas and code)

> We know enough about what an LLM does, and what a brain does, to be quite certain they don't work the same.

I don't think the claim is that LLMs do what brains do - I think the correct form of the counterargument is that _whatever LLMs seem to be doing_ produces end results that were previously only possible through the application of human intelligence, so there must be some axis of however you define human intelligence that LLMs currently seem to display as an emergent behaviour.

Re: AI 2040: Plan A

#389

Post by 27183 »

stalfie wrote: post #378
27183 wrote: post #336

Earlier quoted context omitted.

I used the example of 1G constant acceleration space flight in another thread which got downvoted to oblivion, but I think it's a good one. That's a technology we know how to build. We just need superconducting electronics and miniaturized fusion reactors, or a ship which is built like Project Orion to use nuclear bombs for propulsion. Now write down a blueprint for superintelligence. So I've given you two impossible…

No one imagined LLMs in their current format, it was simply a result of discovering that scaling compute and tokens produced better and better results with the Transformer architecture. The inventors of the Transformer architecture were working on better translation, and probably did not imagine that their architecture would lead to modern LLMs. Imagining something in advance is not necessary at all for scientific ad…

> Imagining something in advance is not necessary at all for scientific advancement. This is particularily true in AI, and no one expects to imagine what superintelligence is until after it is created.

Then why does anyone expect to create it? I'll take a stab at an answer: they think an LLM is some kind of "incremental improvement" and therefore a step along the inevitable path to discovering AI. But that seems delusional to me. I can't imagine anyone sound of mind who knows how an LLM works thinks it's actually intelligent. So in what sense is it an "advancement" on the path to AI?

The concept of an incremental improvement in an objectiveless search in a high dimensional space is.. absurd.

Re: AI 2040: Plan A

#390

Post by ekelsen »

yosame wrote: post #227
ekelsen wrote: post #107

Earlier quoted context omitted.

How is this different from farming? It takes the limited resources of land and water from a community and sells the result for profit as food or fuel. The vast majority of profit is made downstream and outside the community. Golf courses being a traditional green place where people gather seems a bit far fetched to me when most of them are elite private clubs.

People are more lenient on farming because humans need agriculture to survive. Obviously not all agriculture is necessary for survival, but it's something that in some way provides real tangible benefits for everyone. On the other hand, if AI data centres all disappeared today, humanity would continue on completely fine.

The test of "if it disappeared today, we'd all be fine", would eliminate most of the economy.
Post reply on HN