Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI 2040: Plan A

ai-2040.com

Post reply on HN
Print view View original

371–380 of 567 posts

Re: AI 2040: Plan A

#371

Post by beepbooptheory »

themgt wrote: post #339
davemp wrote: post #321

Earlier quoted context omitted.

There is a vast gulf between theoretically possible and technologically feasible. If you can’t provide a realistic path to achieve something, you’re asking people to believe in science fiction. You could tell me that a rock’s molecules are comprised of protons, neutrons, and electrons. Blood is also entirely protons, neutrons, and electrons; so theoretically, one could rearrange stone into blood. But without an actua…

What's strange to me about these comments is they're timeless. They could have been written in 2026 or 2016 or 1966. Like, afaict, for many on HN going from ELIZA->Fable 5 just didn't cause any update to priors regarding this whole philosophical question. The argument against has remained unchanged. I don't see any point in arguing about it, I just find it very strange.

Unpack this a little bit. Why is it strange or interesting to you? What specific priors need to be updated for us here? What is the philosophical questions at play for you?

Re: AI 2040: Plan A

#372

Post by KaiserPro »

throw310822 wrote: post #337
KaiserPro wrote: post #251

The biggest issue with 2027 was that it didn't understand the economy. For AI2027 to be real, the money has to come from somewhere to carry on building the economy. If >10% of the workers suddenly become unemployed, and the rest taking paycuts, then money supply dries up. (unless central banks do something, but then that can be highly inflationary) Without massive amounts of investment, AI development stops dead. In…

> Without massive amounts of investment, AI development stops dead. Would the US government not pour enormous resources in AI labs if needed, knowing that China might be doing the same? What happens if an adversary develops an AI capable of finding and implementing exploits in every software run by your country's strategic infrastructure?

> Would the US government not pour enormous resources in AI labs

I mean they might, but its not clear how they would do it, especially as they are reaching the point where its going to be expensive to borrow.

Re: AI 2040: Plan A

#373

Post by [deleted] »

fulafel wrote: post #291
KaiserPro wrote: post #251

The biggest issue with 2027 was that it didn't understand the economy. For AI2027 to be real, the money has to come from somewhere to carry on building the economy. If >10% of the workers suddenly become unemployed, and the rest taking paycuts, then money supply dries up. (unless central banks do something, but then that can be highly inflationary) Without massive amounts of investment, AI development stops dead. In…

The economy doesn't need workers as consumers necessarily. It would of course be a huge shock to the economy but the economy could adjust to it eventually. Maybe it compromises the 2040 timeline. Still, billionaires are increasingly holding the assets. The money supply drying up can be countered, as it is controlled by central banks and can be arbitrarily increased.

[deleted]

Re: AI 2040: Plan A

#374

Post by Alpha3031 »

arjie wrote: post #15
tfirst wrote: post #4

If carbon taxes are already a lethal policy for an political campaign, it's absurd to think that fears of ASI will create any real movement around pausing AI. If there is any movement to pause AI development, it will come from the general public's dislike of these companies. Not from the AI safety angle.

If that is true and one cares about a moratorium on progress in the US then it seems like the number one way is to meet people where they are: so water use misinformation, degrowth, power supply constraints. That does place all the people who push for these things in a different light. They may well be attempting to do what the AI safety labs are ostensibly trying to do. As an AI safetyist, one’s closest ally (in a d…

It's possible the general public wouldn't care enough one way or the other such that elected representatives could just do it without any change to their electoral fortunes.

Re: AI 2040: Plan A

#376

Post by davemp »

throw310822 wrote: post #334
davemp wrote: post #321

Earlier quoted context omitted.

There is a vast gulf between theoretically possible and technologically feasible. If you can’t provide a realistic path to achieve something, you’re asking people to believe in science fiction. You could tell me that a rock’s molecules are comprised of protons, neutrons, and electrons. Blood is also entirely protons, neutrons, and electrons; so theoretically, one could rearrange stone into blood. But without an actua…

Jesus... This morning while I was drinking coffee and staring at the screen (it's Saturday) an agent did the equivalent of days of my work, reading code, understanding, hypothesizing, comparing, using tools, writing scripts, launching compilers and running tests, identifying problems and proposing solutions, and more. Only someone who hasn't spent a second reflecting about what it means to think and to be intelligent…

> clueless and stubborn and confidently wrong

Uhuh. I really shouldn’t be replying to this type of comment from a throwaway.

But the extremely powerful semantic search that we get from LLMs isn’t enough. I don’t think anyone is credibly arguing otherwise?

Agents already are a layer on top trying to bridge the gap. But they’re really just using LLMs as a heuristic to explore extremely NP problem spaces. The notable successes with agents so far are when we can provide them with a solid verifier and preferably additional context hints on the steps to take in the problem space. See the test oracle problem on where this gets us.

So forgive me if I think that it would be enough of a jump in computational complexity to remove those guard rails that it’s not feasible. But don’t say that I’m clueless, stubborn, or confidently wrong.

Re: AI 2040: Plan A

#377

Post by moffkalast »

VladVladikoff wrote: post #329

>We already wrote a scenario about this, called AI 2027. It depicts takeoff happening in 2027 instead of 2030 So in less than 3 years, their exponential growth curve doomsday prediction has moved back 3 years. This seems to be the opposite of exponential growth.

Fast takeoff was always only possible if you ignore all of physics and everything about how practical reality works, nothing exponential lasts for long in a finite system. If they believe that, then a certain level of self delusion is required anyway and you can ignore anything you like. Sort of like communists saying socialism will definitely work this time.

The book Superintelligence was so highly praised years ago but if you actually go and read the thing today practically all cases it presents read like raypunk retro-futurism that makes up imaginary fantastical nonsense in place of the missing knowledge it would've needed to make any sensible predictions. Practically none of its assumptions apply to LLMs as they currently exist, and some we've learned since about human inteligence are wrong too.

Re: AI 2040: Plan A

#378

Post by stalfie »

27183 wrote: post #336
davemp wrote: post #321

Earlier quoted context omitted.

There is a vast gulf between theoretically possible and technologically feasible. If you can’t provide a realistic path to achieve something, you’re asking people to believe in science fiction. You could tell me that a rock’s molecules are comprised of protons, neutrons, and electrons. Blood is also entirely protons, neutrons, and electrons; so theoretically, one could rearrange stone into blood. But without an actua…

I used the example of 1G constant acceleration space flight in another thread which got downvoted to oblivion, but I think it's a good one. That's a technology we know how to build. We just need superconducting electronics and miniaturized fusion reactors, or a ship which is built like Project Orion to use nuclear bombs for propulsion. Now write down a blueprint for superintelligence. So I've given you two impossible…

No one imagined LLMs in their current format, it was simply a result of discovering that scaling compute and tokens produced better and better results with the Transformer architecture. The inventors of the Transformer architecture were working on better translation, and probably did not imagine that their architecture would lead to modern LLMs.

Imagining something in advance is not necessary at all for scientific advancement. This is particularily true in AI, and no one expects to imagine what superintelligence is until after it is created. You set up your datasets, your architecture tweaks, and measure the results on some set of benchmarks. There never was a blueprint, no plan beyond the experiment itself. We're not even close to understanding the things we have already created, and yet we created them. So why expect anything else for the next step?

Re: AI 2040: Plan A

#379

Post by 27183 »

throw310822 wrote: post #367
27183 wrote: post #352

Earlier quoted context omitted.

> Where do you think intelligence comes from, some mysterious realm? It's physical, computational. Well, no. I don't think it comes from some mysterious realm. I think that which is not physical does not exist [edit: and if you like I'll follow that one right down the rabbit hole--continuity and infinity are useful delusions ]. But that eminently does not mean we know what intelligence is, let alone how to build one.…

You do realise then that "we know how it works, there is no "extra" there" is an argument that can be used against any artificial intelligence, now or in a thousand years, as well as (at some level) against human intelligence (no magic, it's all physics, just dumb cells exchanging signals). This should be enough to give you pause- you immediately reached for an argument that is entirely empty. > I see a lucky stochas…

> But what matters is the result- this machine, matrix multiplier, stochastic parrot, consistently displays intelligence, to the point of being able to perform very complex, open-ended tasks that integrate discovery, planning, tool usage, decision and even some aesthetic sense, understanding and using natural language, context awareness, you name it.

IDK, it doesn't seem like they actually do any of that. To me it seems like they have good enough semantic embeddings that they can kind of approximate those things, sometimes, well enough if you don't look too hard. This is enough to fool people. Of course there's gold in them hills--some recent mathematical results were found there. But to say that's "intellgence" is to say that lossy compression is intelligence. It's static. It does not learn. It does not adapt.

> Unless by "these toys" you mean "any computer program vaguely AI-related".

Not "vaguely AI related". I mean stochastic computer programs that can do things that look awful thinky. They've existed for a long time, but only recently (due to word2vec and other advances) have the results been words that mostly go together well instead of numbers. For some reason people seem to think a lot less critically when the output is words. IDGI but it's a whole thing.

Re: AI 2040: Plan A

#380

Post by derektank »

gcr wrote: post #355
derektank wrote: post #346

Earlier quoted context omitted.

>Reject the false and self-serving narratives that empathy doesn't matter, that altruism isn't "effective". You can change a person's whole life in a moment. You can change more people’s lives, more substantially, if you donate effectively. Effective altruism started out as (and the majority of effective altruist financing is committed to) an effort to rationalize what has historically been a very emotionally driven…

But in the public image, the EA community is synonymous with doubling down on AI / AGI to the exclusion of the other projects. OpenPhil changing its name to Coefficient Giving, 80000 hours and bluedot and (to a lesser extent) CFAR dropping other initiatives and switching to AGI promotion… to my knowledge GiveWell is the only other big name that continues to advance other initiatives. Then look at figureheads like SBF…

>My critique of the EA community is that it’s myopic and unregularized. If you really think AGI is make-or-break for civilization, it’s completely rational to deprioritize side bets.

I’d be curious to hear you expand on this. What binds the EA community together, from the shrimp welfare enthusiasts and wild animal initiative, to the longtermist lightcone obsessive, to the people funding vitamin A supplementation, is simply a commitment to maximizing the number of quality adjusted life years saved each year and a belief that empirical observation can be used to improve that number.

To my mind, this is a valuable insight on its own. Yes, if you come to such a heuristic with absurd prior beliefs, such as whether 100k neurons alone have QALYs in the first place or by placing equal value on people actually alive today and hypothetical people in the far flung future, you will get absurd results. Garbage in, garbage out. But that’s not an indictment of the fundamental insight, especially when you consider how poorly allocated the roughly $2 trillion in global charitable spending is.

Post reply on HN