Live data from Hacker News

The Economics of Recursive Self-Improvement [pdf]

elasticity.institute

11–20 of 103 posts

Re: The Economics of Recursive Self-Improvement [pdf]

#11
Reminder that while there are many naysayers who have been on the wrong side of AGI development progress the last decade…

There are two $1T companies who are all-in on RSI internally right now. They are supported by $20T of market cap plowing R&D into their efforts. You can think it’s dumb money at your own peril, however the market rewards intelligent allocation…

Re: The Economics of Recursive Self-Improvement [pdf]

#12
post #9

> But our models make it clear that such an [intelligence] explosion may not follow if there are diminishing returns (“ideas become harder to find”) or if feedback loops become bottlenecked. How is this not obvious to everyone? As we advance it becomes more difficult to advance. You obviously make most advancements around the things that are easiest to improve. Then all the easy things are done. So you go onto the ne…

> How is this not obvious to everyone? Because everyone's thinking around intelligence is incredibly muddled by a variety of factors, and no one is particularly motivated to actually correct anyone's mistaken notions on the matter.

I'm conflicted. On one hand I think we should more openly call people idiots and push back. On the other hand there's Descartes argument for idiots in good company.

I just wish all the people that claimed to care only about truth would actually care about truth. Feels like society is more that trope where someone says a joke to a crowd and no one hears it except one charismatic person who repeats it and gets all the laughs. In reality it feels like the repeated version of the joke doesn't even make sense, it is just vibes.

Re: The Economics of Recursive Self-Improvement [pdf]

#13
post #9

> But our models make it clear that such an [intelligence] explosion may not follow if there are diminishing returns (“ideas become harder to find”) or if feedback loops become bottlenecked. How is this not obvious to everyone? As we advance it becomes more difficult to advance. You obviously make most advancements around the things that are easiest to improve. Then all the easy things are done. So you go onto the ne…

> How is this not obvious to everyone?

That RSI can be bottlenecked? I guess this is obvious to many people. Whether RSI will be bottlenecked (at some not very interesting stage) is another question.

Re: The Economics of Recursive Self-Improvement [pdf]

#14

Reminder that while there are many naysayers who have been on the wrong side of AGI development progress the last decade… There are two $1T companies who are all-in on RSI internally right now. They are supported by $20T of market cap plowing R&D into their efforts. You can think it’s dumb money at your own peril, however the market rewards intelligent allocation…

The market can also be quick and brutal to punish mistakes, especially when leverage is high. We can be in 1998, but we can be in 1995; you stand to make a ton of money if you know which one we’re in

Re: The Economics of Recursive Self-Improvement [pdf]

#15

RSI isn't anything new though; computers have been used to make computers better for about 80 years now. Imagine having a secretary who could read 1 million records and give you back your answer in 100 microseconds, for just 10 cents an hour. That's Postgres. So I'd imagine that if R&D can be automated, everything becomes better and cheaper but we'd all lose our jobs, as secretaries did to postgres. UBI season

That would be a failure of imagination when it comes to everything secretaries actually do. Which is a problem with the idea that AI is coming for all our jobs anytime soon. It totally undersells everything humans do on the job.

Re: The Economics of Recursive Self-Improvement [pdf]

#16

Reminder that while there are many naysayers who have been on the wrong side of AGI development progress the last decade… There are two $1T companies who are all-in on RSI internally right now. They are supported by $20T of market cap plowing R&D into their efforts. You can think it’s dumb money at your own peril, however the market rewards intelligent allocation…

How do the markets know it's intelligent allocation before RSI is actually a thing? What makes those companies immune to failure?

Re: The Economics of Recursive Self-Improvement [pdf]

#17
post #9

> But our models make it clear that such an [intelligence] explosion may not follow if there are diminishing returns (“ideas become harder to find”) or if feedback loops become bottlenecked. How is this not obvious to everyone? As we advance it becomes more difficult to advance. You obviously make most advancements around the things that are easiest to improve. Then all the easy things are done. So you go onto the ne…

Because they depend on whether the rate of improvement of self-improvement outpaces the rate of increase in difficulty or not, and at some points they clearly do - e.g. a lot of skills makes the relative rate of subsequent improvement easier for a while.

It may seem obvious that it can't last, but showing the conditions where it can't still matters.

Re: The Economics of Recursive Self-Improvement [pdf]

#18
post #9

> But our models make it clear that such an [intelligence] explosion may not follow if there are diminishing returns (“ideas become harder to find”) or if feedback loops become bottlenecked. How is this not obvious to everyone? As we advance it becomes more difficult to advance. You obviously make most advancements around the things that are easiest to improve. Then all the easy things are done. So you go onto the ne…

The goal of a modeling exercise like this, which you don’t have to buy, is to generate a simple set of initial conditions that can explain things we already know. Then, we can manipulate some initial parameter value to make predictions about things we don’t see, but might.

Likewise, it is obvious that gravity exists, but a simple model that explains where it comes from (in quantum terms) would be a big breakthrough iff it came with plausibly testable implications that could be tested via experiment.

Re: The Economics of Recursive Self-Improvement [pdf]

#19
post #9

> But our models make it clear that such an [intelligence] explosion may not follow if there are diminishing returns (“ideas become harder to find”) or if feedback loops become bottlenecked. How is this not obvious to everyone? As we advance it becomes more difficult to advance. You obviously make most advancements around the things that are easiest to improve. Then all the easy things are done. So you go onto the ne…

Progress is often lumpy, modulated by new discoveries and their applications.

Re: The Economics of Recursive Self-Improvement [pdf]

#20

Reminder that while there are many naysayers who have been on the wrong side of AGI development progress the last decade… There are two $1T companies who are all-in on RSI internally right now. They are supported by $20T of market cap plowing R&D into their efforts. You can think it’s dumb money at your own peril, however the market rewards intelligent allocation…

“The market rewards intelligent allocation” is such a straightforwardly false statement that I can’t believe anyone still says this with a straight face. The last ten years of the US economy have just been scam after scam after scam, and people just keep saying this.
Post reply on HN