Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The sigmoids won't save you

astralcodexten.com

Post reply on HN
Print view View original

71–80 of 297 posts

Re: The sigmoids won't save you

#71

Post by energy123 »

throwaway27448 wrote: post #41
LarsDu88 wrote: post #31

I think an interesting thing about recent AI developments is that its all happening right as we hit the diminishing returns side of another "exponential that's actually a sigmoid" which is Moore's law. The naive expectation is that AI will slow down b/c Moore's law is coming to an end, but if you really think about the models and how they are currently implemented in silicon, they are still inefficient as hell. At so…

Even at orders of magnitude greater speed, we've still hit diminishing returns for quality of output. We simply haven't found anything like superhuman reasoning ability, just superhuman (potentially) reasoning speed.

It's not that easy to assess diminishing returns with saturated benchmarks where asymptoting to 100% is mathematically baked in. I could point to the number of Erdos proofs being solved by AI going from 0 to many very recently as evidence for acceleration.

Re: The sigmoids won't save you

#72

Post by jsmcgd »

> It’s true that birth rates must eventually flatten out and become sigmoid

All positive growth eventually flattens out and becomes sigmoid, but a lot of phenomena experience negative growth and nose dive. No gentle curve, but a hard kink and perfect flat line at zero. Forever. I think it would be a stretch to categorize that pattern as sigmoid. Predicting a sigmoid pattern for negative growth implies some sort of a soft landing (depending on your definition of soft).

We can think of many populations that are no longer with us. So just a caution about over applying this reasoning in the negative case.

Re: The sigmoids won't save you

#73

Post by jsnell »

cyanydeez wrote: post #47
lukan wrote: post #23

Earlier quoted context omitted.

"It is a bit arbitrary, but I think this is what they're tracking." I don't know if they can get their numbers right this way, but this seems a way more useful metric, than theoretic capabilities.

ok, but arn't you just measuring efficiency and not the big I in AGI improvements.

No? I think you're misunderstanding what is being measured.

It is purely a test of capabilities (can it do a thing that takes a human $X hours), not efficiency (how fast will it do it).

Re: The sigmoids won't save you

#74

Post by bedobi »

ngriffiths wrote: post #67
bedobi wrote: post #56

[flagged]

I think there are many ways someone with his lack of expertise can still be valuable, including: - Making connections to other subjects that an expert would miss. The hall of fame of sigmoid predictions is just excellent, I already know I'm going to be reminded of it some time in the future. Very entertaining way to get the point across. - Writing about tricky concepts in a very accessible and elegant way, which expe…

[flagged]

Re: The sigmoids won't save you

#75

Post by adleyjulian »

kubb wrote: post #50
adleyjulian wrote: post #35

Earlier quoted context omitted.

1. It's not inevitable. 2. Those that see AI as an existential risk don't generally think it's a guarantee, but if it's say a 5% chance then that's worth addressing/mitigating. 3. That's not what this article was even about.

Sounds like the burden is on you to explain either 1. If you're not treating my claim as a black box, explain explicitly what is your model of what the article was about? Are you aware, for example of the last paragraph of the article? I think that WAS what the article was about. Do you have specific opinions on e.g. how I went wrong and where my model differs? 2. If you are treating it as a black box, what's your de…

The point of the article is that people are historically bad at predicting when exponential curves plateau, even if they're correct that there will be a plateau.

This does *not* imply the inevitability of AGI. It does not imply AGI is necessarily bad.

It does mean that "the capabilities of AI will eventually plateau" offers no meaningful predictive power or relevance to the overall AI discussion.

Re: The sigmoids won't save you

#76

Post by mitthrowaway2 »

kubb wrote: post #53
mitthrowaway2 wrote: post #48

Earlier quoted context omitted.

By that reasoning, why even warn people about anything? Why do road construction crews put up signs saying "ROAD CLOSED AHEAD" when you can just drive on and see for yourself?

Indeed, why warn people about real things that exist in the world? That is EXACTLY the same as inciting fear about something imaginary (not even projected).

In your mind, dangers from AI are imaginary and not even projected, therefore, you don't see any reason to warn about them, because you don't think the dangers are real. You don't believe the road is actually closed up ahead, so you don't think it's necessary to post the sign.

In Scott's mind, dangers from AI are not a known fact, but are somewhere between highly probable and a near-certainty. In his mind, there are well-grounded justifications for believing that AI poses substantial future dangers to the public. Therefore he also believes he should inform people about this, and strives to convince skeptics, so that we might steer clear.

It's easy to understand why someone who believes what you believe about AI would of course not warn people about AI. It's also easy to understand why someone who believes what Scott believes about AI would want to warn people about AI. Your contention is with his confidence for being worried about AI, not his reason for wanting to warn people.

Re: The sigmoids won't save you

#77

Post by btilly »

jerf wrote: post #58
btilly wrote: post #37

Lindy’s Law is an absolute gem, that I'm keeping. If we don't understand the fundamental limits to any particular kind of trend, our default assumption should be that it will continue for about as long as it has gone on already. We can, in fact, easily put a confidence interval on this. With 90% odds we're not in the first 5% of the trend, or the last 5% of the trend. Therefore it will probably go on between 1/19th l…

It's an interesting idea, and it may be something that could be mathematically justified, but I do think this is an abuse of Lindy's Law in the absence of such a justification. Per Wikipedia [1]: "The Lindy effect applies to non-perishable items, like books, those that do not have an "unavoidable expiration date"." And later in the article you can see the mathematical formulation which says the law holds for things w…

The analysis in the article explains why it applies to any phenomena that we might be able to notice.

The argument given is the same as the one that I first ran across, not by that name, in https://www.nature.com/articles/363315a0. https://en.wikipedia.org/wiki/Doomsday_argument claims that it was a rediscovery of something that was hypothesized a decade article.

I hadn't tried to give it a name, or thought to apply it outside of that context.

As for the mathematical qualms, I'm a big believer in not letting formal mathematical technicalities get in the way of adopting an effective heuristic. And the heuristic reasoning here is compelling enough that I would like to adopt it.

Re: The sigmoids won't save you

#78

Post by btilly »

skybrian wrote: post #68
btilly wrote: post #37

Lindy’s Law is an absolute gem, that I'm keeping. If we don't understand the fundamental limits to any particular kind of trend, our default assumption should be that it will continue for about as long as it has gone on already. We can, in fact, easily put a confidence interval on this. With 90% odds we're not in the first 5% of the trend, or the last 5% of the trend. Therefore it will probably go on between 1/19th l…

You can do that but you're laundering ignorance into precise-seeming mathematics. Better to just say "we're probably somewhere in the middle, not at the beginning or end" and leave it at that. Calling a peak is hard.

You speak about laundering ignorance into precise-seeming mathematics as if it was a bad thing.

But that's the entire idea of Bayesian reasoning. Which has proven to be surprisingly effective in a wide range of domains.

I'm all for quantifying my ignorance, and using it as an outside view to help guide my expectations. Read the book Superforecasting to understand how effective forecasters use an outside view to adjust their inside view, to allow them to forecast things more precisely.

Re: The sigmoids won't save you

#80

Post by hirvi74 »

Aurornis wrote: post #26
energy123 wrote: post #21

Earlier quoted context omitted.

The individual who is the best at predicting the future is predicting ASI and full labor automation by 2040: https://xcancel.com/peterwildeford/status/202963666232244661...

> The individual who is the best at predicting the future Going to need a big citation for that claim

No need. That man predicted he would be the best at predicting the future.
Post reply on HN