Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Economics of Recursive Self-Improvement [pdf]

elasticity.institute

Post reply on HN
Print view View original

81–90 of 103 posts

Re: The Economics of Recursive Self-Improvement [pdf]

#81

Post by FrustratedMonky »

breuleux wrote: post #78
FrustratedMonky wrote: post #77

Earlier quoted context omitted.

I was sloppy. I didn't mean to include science fiction authors, or extreme futurists. "everyone", being, a typical average of realistic predictions. In the 90's. I remember people talking about voice recognition will take 50 years. Let alone understanding. Now this is an easy base line feature. Let alone protein folding.

> I didn't mean to include science fiction authors, or extreme futurists. Science fiction authors are quite influential, and I don't think they were the only ones either. I expect that plenty of actual scientists had similar views. It wasn't extreme. > In the 90's. I remember people talking about voice recognition will take 50 years. Sure. Who? How many people said that? How many people said the opposite? If we're do…

Technically correct. But kind of backs every argument into a corner. I'm assuming also, anything like a paper with data, can also be questioned in the same way. Don't object, I've seen that done. So basically, don't talk about anything, because anything you say is contradicted by something somewhere. Everything is True, Everything is False, everything is subjective. There is no reality. Every person has there own point of reference that is completely subjective and invalid.

Re: The Economics of Recursive Self-Improvement [pdf]

#82

Post by red-iron-pine »

ACCount37 wrote: post #30
Windchaser wrote: post #29

Earlier quoted context omitted.

We could have coffee made today by automated coffee vending machines, but many people still prefer to go to a coffee shop. Why is that? Sometimes we don’t choose the cheaper automated product, and instead opt for the “human experience”. I don’t imagine this will change any time soon. Humans like being around humans.

If that is true, then, why is it that the "human experience" has been on decline for decades - since the very moment it became possible to start cutting down on it? If "humans like being around humans", why does human behavior say otherwise?

> why does human behavior say otherwise?

because economics and practical realities demand it.

most Gen Z don't drink because they can't afford it, not because they don't like bars.

a movie ticket w/ drinks and popcorn is now $55 for me and the wife. I'd love to see it in the theater w/ people but netflix is half that per month, unlimited viewing.

starbucks championed the 3rd place but is aggressively culling their own stores to chase profits

public transport isn't there in many places so if you don't have a car you can't go anywhere, whomp whomp

lots of people would go out more -- they just can't. so you do the best w/ what you have, and that's usually smartphone, streaming, and staying in

Re: The Economics of Recursive Self-Improvement [pdf]

#83

Post by breuleux »

FrustratedMonky wrote: post #81
breuleux wrote: post #78

Earlier quoted context omitted.

> I didn't mean to include science fiction authors, or extreme futurists. Science fiction authors are quite influential, and I don't think they were the only ones either. I expect that plenty of actual scientists had similar views. It wasn't extreme. > In the 90's. I remember people talking about voice recognition will take 50 years. Sure. Who? How many people said that? How many people said the opposite? If we're do…

Technically correct. But kind of backs every argument into a corner. I'm assuming also, anything like a paper with data, can also be questioned in the same way. Don't object, I've seen that done. So basically, don't talk about anything, because anything you say is contradicted by something somewhere. Everything is True, Everything is False, everything is subjective. There is no reality. Every person has there own poi…

> I'm assuming also, anything like a paper with data, can also be questioned in the same way. Don't object, I've seen that done.

Unless you've seen me do it, I don't see why I can't object. What a strange thing to say.

Look. You've supported your argument with vibes and anecdotes, so I'm offering contradicting vibes and anecdotes of my own. All that means is that it's not as obvious as you thought, and it is worth digging deeper (or not, it's not that important). A paper with data would certainly make a stronger argument. And of course it can be questioned, but it cannot be questioned in the same way. That you've "seen it done" is irrelevant.

Re: The Economics of Recursive Self-Improvement [pdf]

#84

Post by godelski »

WJW wrote: post #37
bryanrasmussen wrote: post #36

Earlier quoted context omitted.

>How is this not obvious to everyone? because people have other ideas https://en.wikipedia.org/wiki/Technological_singularity

People have other dreams , and have written about them in science fiction. But science fiction can just handwave away resource problems that exist in real life so that the plot can happen. Just as the popular "grey goo" nanomachine disaster can easily be shown to be impossible due to resource imbalances and energy shortages, AI recursive self improvements rapidly slows down due to problems with complexity, training d…

The problem is in cases of the singularity it's being treated like reality rather than fiction

Re: The Economics of Recursive Self-Improvement [pdf]

#85

Post by godelski »

skinfaxi wrote: post #67
windexh8er wrote: post #60

Earlier quoted context omitted.

Then use batteries as an example. Going on 200+ years now most of the major changes have been an easy swap: chemistry. Lead acid --> nickel cadium --> nickel metal hydride --> lithium ion. Improvements in batteries are hard and the surrounding ecosystem has gotten better. Better BMS and better motors have also made gains to increase the efficiency of the battery. Compounding gains within the ecosystem doesn't truly i…

It seems the rate of advancement has increased over time though. Like from ancient times to 1800s the rate of progress was slow, then basically a relative explosion of progress compressed in 200 years.

  > It seems the rate of advancement has increased over time though.
Effort and expenditures are not fixed. You cannot hand wave away these variables.

Here's an example let's compare two investment strategies:

  - Alice puts $1 into VOO every day. 
  - Bob puts in $10000 into VOO every day.
You don't get to say "wow, Bob is a much better investor than Alice. He's so smart, look how fast his money grows!" You get to say "Bob is investing more money than Alice."

Now pretend Bob is investing into a slower growing account than Alice. His account will still grow faster as long as his account is not losing >99.9% per day. He could have a worse strategy and still grow faster because he's throwing more money at it

Re: The Economics of Recursive Self-Improvement [pdf]

#86

Post by godelski »

bobson381 wrote: post #72
skinfaxi wrote: post #67

Earlier quoted context omitted.

It seems the rate of advancement has increased over time though. Like from ancient times to 1800s the rate of progress was slow, then basically a relative explosion of progress compressed in 200 years.

Because of access to a giant store of energy. Our advancement has also created a future need to consume more and more energy to fuel increased complexity.

Which has created all sorts of other problems even if we exclude climate change.

I'm not arguing we shouldn't live in a modern world, but rather that complexity increases

Re: The Economics of Recursive Self-Improvement [pdf]

#87

Post by godelski »

dinfinity wrote: post #70
jamesrcole wrote: post #61

Earlier quoted context omitted.

> As we advance it becomes more difficult to advance. You obviously make most advancements around the things that are easiest to improve. Then all the easy things are done. This isn't some foregone conclusion. It completely depends on the rate at which the intelligence and abilities of the AI increases. If that rate was high enough, then the harder and harder problems would become easier and easier for it.

Actually, evolution seems to show the opposite : The rate of advancement has only sped up, with billions of years between significant changes going to millions, to thousands, to tens and arguably to mere years now. Having said that, we're probably looking at an S-curve with the physical limits of reality getting in the way in the end.

You're ignoring a lot of important factors. I'll give you most obvious ones. We often do genetics testing on fruit flies because they reproduce quickly and the initial population is large.

Your claim ignores this fact. The base population and number of interactions has increased.

Re: The Economics of Recursive Self-Improvement [pdf]

#88

Post by godelski »

ben_w wrote: post #73
defmacr0 wrote: post #66

Earlier quoted context omitted.

Well at least it seems pretty implausible to me that a machine learning model trained to reproduce human text can in principle generate something that is significantly above human text production ability. If the "pea-sized semiconductor brain" is not a surprisingly shallow problem that you can just solve by interpolating existing research, I don't really see the LLM-approach to AI being the thing that makes something…

> Well at least it seems pretty implausible to me that a machine learning model trained to reproduce human text can in principle generate something that is significantly above human text production ability. These are not the only currently-in-use AI models. However, recent news has even this particular category of AI solving multiple previously unsolved Erdős problems. (Even if they also do stupid things on a frequen…

  > solving multiple previously unsolved Erdős problems.
So what?

Machines have solved tons of unsolved problems in mathematics. That's not a proof of intelligence.

If you brute force a solution, we congratulate you on your effort.

If you stumble into a solution, we congratulate you for being lucky (if we can distinguish)

If you find a unique solution that no one else imagined, we congratulate you on your intelligence.

These are categorically different things and the difference matters. It's the whole distinction. Though in the real world success usually requires all three (and more), complicating evaluation.

When we're talking about intelligence you can't distill it to "getting the answer". If you do then I'll direct you at an abacus, a calculator, a watch, or google search if you want to look at super intelligence.

  > natural selection did it
Did what? Gradient descent? If that's the argument, you need to read more

Re: The Economics of Recursive Self-Improvement [pdf]

#89

Post by godelski »

FrustratedMonky wrote: post #58
godelski wrote: post #9

> But our models make it clear that such an [intelligence] explosion may not follow if there are diminishing returns (“ideas become harder to find”) or if feedback loops become bottlenecked. How is this not obvious to everyone? As we advance it becomes more difficult to advance. You obviously make most advancements around the things that are easiest to improve. Then all the easy things are done. So you go onto the ne…

"How is this not obvious to everyone? As we advance it becomes more difficult to advance. " It's a type of recent change blindness. Because for the last few years there are seemingly impossible breakthroughs every few months. Literally, things everyone said would take 30 or 100 years, happen within months. It is really easy to think this will continue. The new normal. Really, we haven't seen it slow down at all, so w…

Fair but there's two things I think need to be added

1) predictions are usually made assuming either constant effort or continued growth. They will be extremely conservative when the environment changes and you start dumping trillions of dollars into solving those problems.

2) while there's been real progress that doesn't mean it isn't being over sold. We do need to differentiate the signal from the noise

Re: The Economics of Recursive Self-Improvement [pdf]

#90

Post by breuleux »

tossandthrow wrote: post #74
breuleux wrote: post #68

Earlier quoted context omitted.

> We have advanced tremendously over the past 200 years Would most of that have happened if we hadn't found oil, though? It's difficult to precisely identify how much of progress is owed to intelligence, and how much is simply energy availability. Energy is intrinsically transformative -- the dumbest of organisms can take over the world in a blitz if they can process available energy better than the others, whereas t…

Reducing progress to increasing entropy makes it even more clear that making progress will likely become easier, not harder. Yes, oil was important back then. Now we can augment with wind, solar, nuclear, etc. And more technologies will come.

Maybe, maybe not. The point is that you can't simply will energy into existence. Its availability, including the ease of exploitation, is a physical given. Which is true of resources in general. Arbitrarily advanced technology can't simply make exploitation arbitrarily easy, so for example there can always be situations where the very best of all possible technologies will be less productive than setting fire to petrol would have been, if there had been any. It's complicated. It's not just what we do, it's not just what we have, it's both.

Probably the best test of how the future might trend long term is to see how well we can extract value from all the stuff and waste we are currently producing, because these will inevitably become the primary resources from which we have to build everything new. Unfortunately that shift is likely to happen extremely quickly and catastrophically, because typically you only start processing bad materials once you're done with the good.

Post reply on HN