71–80 of 103 posts
When I hear recursive self improvement all I remember are the ridiculous articles a few years ago about how 3d printers were going to make themselves and take over the world.
Earlier quoted context omitted.
Then use batteries as an example. Going on 200+ years now most of the major changes have been an easy swap: chemistry. Lead acid --> nickel cadium --> nickel metal hydride --> lithium ion. Improvements in batteries are hard and the surrounding ecosystem has gotten better. Better BMS and better motors have also made gains to increase the efficiency of the battery. Compounding gains within the ecosystem doesn't truly i…
It seems the rate of advancement has increased over time though. Like from ancient times to 1800s the rate of progress was slow, then basically a relative explosion of progress compressed in 200 years.
Earlier quoted context omitted.
Our intelligence was enough to turn some balding apes into atom-bomb wielders and astronauts over a few thousand generations. Consider a hypothetical: it is possible to make a semiconductor clone of a human brain, that runs at semiconductor speeds, has semiconductor size-scale, and has a power requirement of the Landauer limit. This copy would be smaller than a pea, and you'd get to pick on a sliding scale between "s…
Well at least it seems pretty implausible to me that a machine learning model trained to reproduce human text can in principle generate something that is significantly above human text production ability. If the "pea-sized semiconductor brain" is not a surprisingly shallow problem that you can just solve by interpolating existing research, I don't really see the LLM-approach to AI being the thing that makes something…
These are not the only currently-in-use AI models. However, recent news has even this particular category of AI solving multiple previously unsolved Erdős problems.
(Even if they also do stupid things on a frequent basis).
> Well, not really. For large-scale AI models it's almost exclusively some form of gradient-based non-linear optimization. Genetic algorithms (which is just hill-climbing optimization with extra steps) and genetic programming (which is really cool and not well-understood) do not perform all that well in practice and I'm not aware of any notable applications.
I said "standard technique" rather than "best" for a reason ;)
This is proof-of-possibility: natural selection did it, we know how to mimic that, but we don't know enough to be sure we're doing it with a reward function that will actually give us minds like ours on an interesting timescale with a probability high enough to care about.
Earlier quoted context omitted.
> As we advance it becomes more difficult to advance I don't think this follows. We have advanced tremendously over the past 200 years, and we are likely going into a time with rapid advancement again. With advancement, we also develop tools (eg. Llms) that assist advancing.
> We have advanced tremendously over the past 200 years Would most of that have happened if we hadn't found oil, though? It's difficult to precisely identify how much of progress is owed to intelligence, and how much is simply energy availability. Energy is intrinsically transformative -- the dumbest of organisms can take over the world in a blitz if they can process available energy better than the others, whereas t…
Yes, oil was important back then. Now we can augment with wind, solar, nuclear, etc.
And more technologies will come.
Earlier quoted context omitted.
Our intelligence was enough to turn some balding apes into atom-bomb wielders and astronauts over a few thousand generations. Consider a hypothetical: it is possible to make a semiconductor clone of a human brain, that runs at semiconductor speeds, has semiconductor size-scale, and has a power requirement of the Landauer limit. This copy would be smaller than a pea, and you'd get to pick on a sliding scale between "s…
Well at least it seems pretty implausible to me that a machine learning model trained to reproduce human text can in principle generate something that is significantly above human text production ability. If the "pea-sized semiconductor brain" is not a surprisingly shallow problem that you can just solve by interpolating existing research, I don't really see the LLM-approach to AI being the thing that makes something…
So, it stands to reason (and observations) that such a model captures not only surface statistics of the data, but a part of functionality of the system that generated the data (the human brain for text, physics for video).
> But our models make it clear that such an [intelligence] explosion may not follow if there are diminishing returns (“ideas become harder to find”) or if feedback loops become bottlenecked. How is this not obvious to everyone? As we advance it becomes more difficult to advance. You obviously make most advancements around the things that are easiest to improve. Then all the easy things are done. So you go onto the ne…
>How is this not obvious to everyone? because people have other ideas https://en.wikipedia.org/wiki/Technological_singularity
In a weak sense, singularities are common and should be expected every so often. In a mathematical sense, a singularity is where a model 'blows up' and neglected terms become part of the dominant balance.
Industrialization hit a point of diminishing returns, but the industrial revolution was nonetheless a 'singularity,' where life afterwards was qualitatively unpredictable to people who lived before. Likewise, agriculture was such a technological singularity to hunter-gatherer ancestors.
I could even make a decent argument that writing and literacy were such a singularity, making inconceivable social organizations routine.
In that weak sense I expect AI to be a singularity, recursive self improvement or no. Life in 2050 may be completely unpredictable to someone who was taken out of time in the year 2000.
The remaining questions are speed and intensity, and both of these questions are related to RSI. If RSI works, then the 'fast takeoff' visions become more plausible where society transforms over months to a few years – at least locally where the enabling technologies have diffused. If not, it might take a couple of decades.
Earlier quoted context omitted.
"How is this not obvious to everyone? As we advance it becomes more difficult to advance. " It's a type of recent change blindness. Because for the last few years there are seemingly impossible breakthroughs every few months. Literally, things everyone said would take 30 or 100 years, happen within months. It is really easy to think this will continue. The new normal. Really, we haven't seen it slow down at all, so w…
> Literally, things everyone said would take 30 or 100 years, happen within months. "Everyone"? Plenty of people in the 50s said we'd have sentient humanoid robots before the year 2000. And flying cars. Sure, many people underestimate the rate of change. But techno-optimists have consistently overestimated it, and they still do .
"everyone", being, a typical average of realistic predictions.
In the 90's. I remember people talking about voice recognition will take 50 years. Let alone understanding. Now this is an easy base line feature.
Let alone protein folding.
Earlier quoted context omitted.
> Literally, things everyone said would take 30 or 100 years, happen within months. "Everyone"? Plenty of people in the 50s said we'd have sentient humanoid robots before the year 2000. And flying cars. Sure, many people underestimate the rate of change. But techno-optimists have consistently overestimated it, and they still do .
I was sloppy. I didn't mean to include science fiction authors, or extreme futurists. "everyone", being, a typical average of realistic predictions. In the 90's. I remember people talking about voice recognition will take 50 years. Let alone understanding. Now this is an easy base line feature. Let alone protein folding.
Science fiction authors are quite influential, and I don't think they were the only ones either. I expect that plenty of actual scientists had similar views. It wasn't extreme.
> In the 90's. I remember people talking about voice recognition will take 50 years.
Sure. Who? How many people said that? How many people said the opposite? If we're doing anecdotes, I remember that circa 2010 it was a fairly normal belief that fully self-driving cars would be widespread by 2020. I mean, I thought they would.
Point is, many people say many different things, there's never been any widespread agreement about what the future holds. They underestimate, they overestimate, but mostly reality moves sideways in directions nobody thought of and which are somehow simultaneously way more impressive and way less impressive than our expectations.
RSI isn't anything new though; computers have been used to make computers better for about 80 years now. Imagine having a secretary who could read 1 million records and give you back your answer in 100 microseconds, for just 10 cents an hour. That's Postgres. So I'd imagine that if R&D can be automated, everything becomes better and cheaper but we'd all lose our jobs, as secretaries did to postgres. UBI season
> So I'd imagine that if R&D can be automated, everything becomes better and cheaper but we'd all lose our jobs, as secretaries did to postgres. UBI season Perhaps, but there's a thing in economics, the "resource curse": https://en.wikipedia.org/wiki/Resource_curse I don't have any formal training in economics, so I can't argue the for and against arguments listed in the Wikipedia page, but it is at least possible th…
Earlier quoted context omitted.
I think humans totally oversell just how special "everything they do on the job" is. "No, you don't get it, it's me specifically who has a job that's way too special to be automated away!"
We could have coffee made today by automated coffee vending machines, but many people still prefer to go to a coffee shop. Why is that? Sometimes we don’t choose the cheaper automated product, and instead opt for the “human experience”. I don’t imagine this will change any time soon. Humans like being around humans.
Plus a coffee shop is about the experience, not the price.
But most jobs that aren't service are not about the experience.