Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Economics of Recursive Self-Improvement [pdf]

elasticity.institute

Post reply on HN
Print view View original

21–30 of 103 posts

Re: The Economics of Recursive Self-Improvement [pdf]

#21

Post by tossandthrow »

godelski wrote: post #9

> But our models make it clear that such an [intelligence] explosion may not follow if there are diminishing returns (“ideas become harder to find”) or if feedback loops become bottlenecked. How is this not obvious to everyone? As we advance it becomes more difficult to advance. You obviously make most advancements around the things that are easiest to improve. Then all the easy things are done. So you go onto the ne…

> As we advance it becomes more difficult to advance

I don't think this follows.

We have advanced tremendously over the past 200 years, and we are likely going into a time with rapid advancement again.

With advancement, we also develop tools (eg. Llms) that assist advancing.

Re: The Economics of Recursive Self-Improvement [pdf]

#22

Post by exe34 »

godelski wrote: post #9

> But our models make it clear that such an [intelligence] explosion may not follow if there are diminishing returns (“ideas become harder to find”) or if feedback loops become bottlenecked. How is this not obvious to everyone? As we advance it becomes more difficult to advance. You obviously make most advancements around the things that are easiest to improve. Then all the easy things are done. So you go onto the ne…

> How is this not obvious to everyone?

Probably because it's not true. We had shitty neural networks for decades before the recent explosion. That particular branch may be a dead end, but there could be others lurking and waiting for their time.

Re: The Economics of Recursive Self-Improvement [pdf]

#23

Post by blitzar »

mbgerring wrote: post #20
reducesuffering wrote: post #11

Reminder that while there are many naysayers who have been on the wrong side of AGI development progress the last decade… There are two $1T companies who are all-in on RSI internally right now. They are supported by $20T of market cap plowing R&D into their efforts. You can think it’s dumb money at your own peril, however the market rewards intelligent allocation…

“The market rewards intelligent allocation” is such a straightforwardly false statement that I can’t believe anyone still says this with a straight face. The last ten years of the US economy have just been scam after scam after scam, and people just keep saying this.

NFTs were worth more than $1 trillion so we know that they were "better" than the Ai efforts of today because “The market rewards intelligent allocation”

Re: The Economics of Recursive Self-Improvement [pdf]

#24

Post by RandomLensman »

red75prime wrote: post #13
godelski wrote: post #9

> But our models make it clear that such an [intelligence] explosion may not follow if there are diminishing returns (“ideas become harder to find”) or if feedback loops become bottlenecked. How is this not obvious to everyone? As we advance it becomes more difficult to advance. You obviously make most advancements around the things that are easiest to improve. Then all the easy things are done. So you go onto the ne…

> How is this not obvious to everyone? That RSI can be bottlenecked? I guess this is obvious to many people. Whether RSI will be bottlenecked (at some not very interesting stage) is another question.

Why would it not be bottlenecked? Is intelligence so valuable as such?

Re: The Economics of Recursive Self-Improvement [pdf]

#25

Post by ACCount37 »

goatlover wrote: post #15
vatsachak wrote: post #4

RSI isn't anything new though; computers have been used to make computers better for about 80 years now. Imagine having a secretary who could read 1 million records and give you back your answer in 100 microseconds, for just 10 cents an hour. That's Postgres. So I'd imagine that if R&D can be automated, everything becomes better and cheaper but we'd all lose our jobs, as secretaries did to postgres. UBI season

That would be a failure of imagination when it comes to everything secretaries actually do. Which is a problem with the idea that AI is coming for all our jobs anytime soon. It totally undersells everything humans do on the job.

I think humans totally oversell just how special "everything they do on the job" is.

"No, you don't get it, it's me specifically who has a job that's way too special to be automated away!"

Re: The Economics of Recursive Self-Improvement [pdf]

#26

Post by not-a-llm »

blitzar wrote: post #23
mbgerring wrote: post #20

Earlier quoted context omitted.

“The market rewards intelligent allocation” is such a straightforwardly false statement that I can’t believe anyone still says this with a straight face. The last ten years of the US economy have just been scam after scam after scam, and people just keep saying this.

NFTs were worth more than $1 trillion so we know that they were "better" than the Ai efforts of today because “The market rewards intelligent allocation”

they never were

if I sell 1 token at $1 and there are a trillion of them minted, that does not make a $1T market cap

but if you want to prove a point against AI with fake data, sure

Re: The Economics of Recursive Self-Improvement [pdf]

#27

Post by Windchaser »

tossandthrow wrote: post #21
godelski wrote: post #9

> But our models make it clear that such an [intelligence] explosion may not follow if there are diminishing returns (“ideas become harder to find”) or if feedback loops become bottlenecked. How is this not obvious to everyone? As we advance it becomes more difficult to advance. You obviously make most advancements around the things that are easiest to improve. Then all the easy things are done. So you go onto the ne…

> As we advance it becomes more difficult to advance I don't think this follows. We have advanced tremendously over the past 200 years, and we are likely going into a time with rapid advancement again. With advancement, we also develop tools (eg. Llms) that assist advancing.

To put it another way, the cost per advancement increases. That’s “as we advance it becomes more difficult to advance”. However, because of the prior advances, you also have more resources to throw at future advancements.

So, then, the question is whether the “profits” on the last advance are enough to pay for the next one. We can define a new term, “affordability”, as what % “profit” you can expect from each advance relative to its cost, telling us whether it becomes relatively easier or harder to continue to advance.

Re: The Economics of Recursive Self-Improvement [pdf]

#28

Post by not-a-llm »

goatlover wrote: post #16
reducesuffering wrote: post #11

Reminder that while there are many naysayers who have been on the wrong side of AGI development progress the last decade… There are two $1T companies who are all-in on RSI internally right now. They are supported by $20T of market cap plowing R&D into their efforts. You can think it’s dumb money at your own peril, however the market rewards intelligent allocation…

How do the markets know it's intelligent allocation before RSI is actually a thing? What makes those companies immune to failure?

them having an actual product with huge demand makes them immune

Just like Tesla was immune from all the naysayers which were saying its a highly unprofitable company which will 100% go bankrupt because its economics dont make any sense, and they lost huge amounts of money shorting the stock

Re: The Economics of Recursive Self-Improvement [pdf]

#29

Post by Windchaser »

ACCount37 wrote: post #25
goatlover wrote: post #15

Earlier quoted context omitted.

That would be a failure of imagination when it comes to everything secretaries actually do. Which is a problem with the idea that AI is coming for all our jobs anytime soon. It totally undersells everything humans do on the job.

I think humans totally oversell just how special "everything they do on the job" is. "No, you don't get it, it's me specifically who has a job that's way too special to be automated away!"

We could have coffee made today by automated coffee vending machines, but many people still prefer to go to a coffee shop. Why is that?

Sometimes we don’t choose the cheaper automated product, and instead opt for the “human experience”. I don’t imagine this will change any time soon. Humans like being around humans.

Re: The Economics of Recursive Self-Improvement [pdf]

#30

Post by ACCount37 »

Windchaser wrote: post #29
ACCount37 wrote: post #25

Earlier quoted context omitted.

I think humans totally oversell just how special "everything they do on the job" is. "No, you don't get it, it's me specifically who has a job that's way too special to be automated away!"

We could have coffee made today by automated coffee vending machines, but many people still prefer to go to a coffee shop. Why is that? Sometimes we don’t choose the cheaper automated product, and instead opt for the “human experience”. I don’t imagine this will change any time soon. Humans like being around humans.

If that is true, then, why is it that the "human experience" has been on decline for decades - since the very moment it became possible to start cutting down on it?

If "humans like being around humans", why does human behavior say otherwise?

Post reply on HN