Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The sigmoids won't save you

astralcodexten.com

Post reply on HN
Print view View original

51–60 of 297 posts

Re: The sigmoids won't save you

#51

Post by inglor_cz »

jeffreyrogers wrote: post #28
inglor_cz wrote: post #13

Earlier quoted context omitted.

That is true, but I generally enjoy reading a lot of words from Scott, who has a talent for writing. The entire plot of the Lord of the Rings could probably be compressed into less than 10 kB of text too. Edit: this seems to be a controversial comment, but IMHO a blog of Scott Alexander's type is an art form, not just a communication channel.

I find him more interesting when he talks about non-AI topics. Lots of other interesting people are like this too. I'd rather get my knowledge on AI from people who have unique insights into it. Scott has a lot of unique perspectives of his own, but his views on AI are bog-standard for his social group.

Frankly, me too, but he is still smart enough to introduce some grains of original thought even into those bog-standard views.

Re: The sigmoids won't save you

#52

Post by layer8 »

energy123 wrote: post #21
krupan wrote: post #17

News flash: predicting the future is hard

The individual who is the best at predicting the future is predicting ASI and full labor automation by 2040: https://xcancel.com/peterwildeford/status/202963666232244661...

Predicting who will predict the future best is hard.

Re: The sigmoids won't save you

#53

Post by kubb »

mitthrowaway2 wrote: post #48
kubb wrote: post #27

If the scary AI is so inevitable, why do you feel such an overwhelming need to convince people about that? Surely you can just wait a bit, and they'll see for themselves.

By that reasoning, why even warn people about anything? Why do road construction crews put up signs saying "ROAD CLOSED AHEAD" when you can just drive on and see for yourself?

Indeed, why warn people about real things that exist in the world? That is EXACTLY the same as inciting fear about something imaginary (not even projected).

Re: The sigmoids won't save you

#54

Post by lukan »

cyanydeez wrote: post #47
lukan wrote: post #23

Earlier quoted context omitted.

"It is a bit arbitrary, but I think this is what they're tracking." I don't know if they can get their numbers right this way, but this seems a way more useful metric, than theoretic capabilities.

ok, but arn't you just measuring efficiency and not the big I in AGI improvements.

Yes, but this study was not about that and "just efficiency" is actually what most people are after.

At least I want AI to solve my problems, not score high on a academic leaderboard.

Re: The sigmoids won't save you

#55

Post by mitthrowaway2 »

inglor_cz wrote: post #10

Hmmm, this is quite an interesting take by Scott. Lindy's Law is not actually a law and many exact minds will be provoked by the very name; it also fails spectacularly in certain contexts (e.g. lifetime of a single organism, though not necessarily existence of entire species). But at the same time, I am willing to take its invocation in the context of AI somewhat seriously. There is an international arms race with Ch…

It's not a law per se, but there are rules for reasoning under uncertainty to get the most out of what limited knowledge you have, and Lindy's law arises from that. To do better than Lindy's law requires having additional information about the problem beyond just the one data point.

Re: The sigmoids won't save you

#57

Post by adw »

gm678 wrote: post #6

I don't know what the Y-axis is supposed to be on that Wharton AI capabilities graph, but I am not really convinced that Opus 4.6 has more than double the intelligence/capability/whatever of GPT 5.1 Max.

https://podcasts.apple.com/us/podcast/machine-learning-stree... is a pretty good primer on METR, what it measures, and its limitations.

Re: The sigmoids won't save you

#58

Post by jerf »

btilly wrote: post #37

Lindy’s Law is an absolute gem, that I'm keeping. If we don't understand the fundamental limits to any particular kind of trend, our default assumption should be that it will continue for about as long as it has gone on already. We can, in fact, easily put a confidence interval on this. With 90% odds we're not in the first 5% of the trend, or the last 5% of the trend. Therefore it will probably go on between 1/19th l…

It's an interesting idea, and it may be something that could be mathematically justified, but I do think this is an abuse of Lindy's Law in the absence of such a justification. Per Wikipedia [1]:

"The Lindy effect applies to non-perishable items, like books, those that do not have an "unavoidable expiration date"."

And later in the article you can see the mathematical formulation which says the law holds for things with a Pareto distribution [2]. I'd want to see some sort of good analysis that "the life span of exponential growth curves" is drawn from some Pareto distribution. I don't think it's completely out of the question. But I'm also nowhere near confident enough that it is a true statement to casually apply Lindy's Law to it.

[1]: https://en.wikipedia.org/wiki/Lindy_effect

[2]: https://en.wikipedia.org/wiki/Pareto_distribution

Re: The sigmoids won't save you

#59

Post by janalsncm »

> What if you don’t fully understand the process? AI forecasters know some things (like how data centers work and how much it costs to build them). But they’re unsure about other things (researchers keep inventing new paradigms of data generation that get over data walls, but for how long?), and other things are entirely opaque (What is intelligence really? Why do scaling laws work? Might they just stop working at some point?) Is there anything you can do here?

This is the crux of the article. To a large extent continued progress depends on a stable increase in compute, an increase in training data, and an increase in good ideas to squeeze more out of both of them.

One calculation you could do is a survival function: for each of the above, how long before it is disrupted? For example, China could crack down on AI or invade Taiwan. Or data centers become politically unpopular in the US. Or, we could run out of great ideas. Very hard to predict.

Re: The sigmoids won't save you

#60

Post by skybrian »

myhf wrote: post #30
gm678 wrote: post #6

I don't know what the Y-axis is supposed to be on that Wharton AI capabilities graph, but I am not really convinced that Opus 4.6 has more than double the intelligence/capability/whatever of GPT 5.1 Max.

According to this article: whenever someone games a benchmark to make an upward chart on some y-axis, it's YOUR responsibility to prove how and why that trend can't continue indefinitely. emoji face with eyes rolling upward

Seems to me that the default is "I don't know what's going to happen" and if you're making a confident prediction, bring evidence.

Scott makes a Lindy effect argument which is plausible, but don't let that fool you, we still don't know what's going to happen.

Post reply on HN