Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The sigmoids won't save you

astralcodexten.com

Post reply on HN
Print view View original

81–90 of 297 posts

Re: The sigmoids won't save you

#81

Post by kubb »

mitthrowaway2 wrote: post #76
kubb wrote: post #53

Earlier quoted context omitted.

Indeed, why warn people about real things that exist in the world? That is EXACTLY the same as inciting fear about something imaginary (not even projected).

In your mind, dangers from AI are imaginary and not even projected, therefore, you don't see any reason to warn about them, because you don't think the dangers are real . You don't believe the road is actually closed up ahead, so you don't think it's necessary to post the sign. In Scott's mind, dangers from AI are not a known fact, but are somewhere between highly probable and a near-certainty. In his mind , there ar…

Gosh it's quite embarassing to have to spell it out, but you inserted the part about Scott's motivations. It can't be found in the text.

Neither can any specific discussion of what the dangers are and how we can steer clear. It all comes preplanted in your head. The only thing that Scott is playing on (as far as we can see) is your ingrained fear, by using an ominous headline, and a vague reference to something "scary" in the conclusion.

Of course there was no reason to "warn" you, you already believed in the scary future. Scott is just giving you fuel, which you seem to appreciate.

Re: The sigmoids won't save you

#82

Post by mitthrowaway2 »

kubb wrote: post #81
mitthrowaway2 wrote: post #76

Earlier quoted context omitted.

In your mind, dangers from AI are imaginary and not even projected, therefore, you don't see any reason to warn about them, because you don't think the dangers are real . You don't believe the road is actually closed up ahead, so you don't think it's necessary to post the sign. In Scott's mind, dangers from AI are not a known fact, but are somewhere between highly probable and a near-certainty. In his mind , there ar…

Gosh it's quite embarassing to have to spell it out, but you inserted the part about Scott's motivations. It can't be found in the text. Neither can any specific discussion of what the dangers are and how we can steer clear. It all comes preplanted in your head. The only thing that Scott is playing on (as far as we can see) is your ingrained fear, by using an ominous headline, and a vague reference to something "scar…

Is this the first essay of Scott's that you've read?

Re: The sigmoids won't save you

#83

Post by LarsDu88 »

throwaway27448 wrote: post #41
LarsDu88 wrote: post #31

I think an interesting thing about recent AI developments is that its all happening right as we hit the diminishing returns side of another "exponential that's actually a sigmoid" which is Moore's law. The naive expectation is that AI will slow down b/c Moore's law is coming to an end, but if you really think about the models and how they are currently implemented in silicon, they are still inefficient as hell. At so…

Even at orders of magnitude greater speed, we've still hit diminishing returns for quality of output. We simply haven't found anything like superhuman reasoning ability, just superhuman (potentially) reasoning speed.

I disagree with this. Reinforcement learning with verifiable rewards training is actually the secret sauce that is leading Claude and GPT to automating software engineering tasks.

All the easily verifiable domains such as mathematics, coding, and things that can be run inside a reasonable simulation are falling very very fast.

By next year if not sooner, mathematicians will be wildly outpaced by LLMs for reasoning.

Re: The sigmoids won't save you

#84

Post by LarsDu88 »

cyanydeez wrote: post #33
LarsDu88 wrote: post #31

I think an interesting thing about recent AI developments is that its all happening right as we hit the diminishing returns side of another "exponential that's actually a sigmoid" which is Moore's law. The naive expectation is that AI will slow down b/c Moore's law is coming to an end, but if you really think about the models and how they are currently implemented in silicon, they are still inefficient as hell. At so…

they already did put a model into the silicon and it's crazy fast. https://chatjimmy.ai/ I'm pretty sure there's a 3 year design goal starting this year that'll do that to any of the qwen, deepseek, etc models. There's a lot you could do with sped up models of these quality. It might even be bad enough that the real bubble is how much we don't need giant data centers when 80-90% of use cases could just be a silicon c…

And this is an asic that is still operating digitally. Imagine a chip with baked it weights that does its math analogue with 20x reduction in number of circuit elements needed to do a multiplication op.

If there's a breakthrough in memristors, you could end up with another 20x reduction in circuit elements (get rid of memory bottlnecks, start doing multiplication ops as log transform voltage addition)

The ceiling is ultra high for how far AI can go.

Re: The sigmoids won't save you

#85

Post by stymaar »

I don't know when the sigmoid is going to kick in, but Nvidia's Quaterly datacenters revenues have been grown 15 folds over the past 3 years[1], and nobody including Scott believes this is sustainable for 3 more years otherwise Nvidia's market cap would conservatively be at least an order of magnitude higher than it is.

All exponential eventually becomes a sigmoid because exponential growth always expose limiting factors that weren't limiting at the beginning. Silicon manufacturing had lots of room for high-margin customers like Nvidia even a year ago (by the mere virtue of outbidding lower-margin customers), but now it is mostly gone, and no amount of money will make fabs build themselves overnight.

[1]: https://stockanalysis.com/stocks/nvda/metrics/revenue-by-seg...

Re: The sigmoids won't save you

#86

Post by Leynos »

cyanydeez wrote: post #47
lukan wrote: post #23

Earlier quoted context omitted.

"It is a bit arbitrary, but I think this is what they're tracking." I don't know if they can get their numbers right this way, but this seems a way more useful metric, than theoretic capabilities.

ok, but arn't you just measuring efficiency and not the big I in AGI improvements.

It also measures task coherence—ability to plan, form contingencies, recover from errors, mitigate accumulation of errors, and reconcile findings across a long context window.

Re: The sigmoids won't save you

#87

Post by Leynos »

jrumbut wrote: post #49
NitpickLawyer wrote: post #11

Earlier quoted context omitted.

IIRC that graph tracks capabilities as time_to_solve a task for humans (i.e. the model can now handle tasks that usually take a human ~8h). Which, depending on what tasks you look at, could be a reasonable finding. I could see Opus 4.6 handling tasks that take ~8h for humans, and that 5.1 couldn't previously handle (with 5.1 being "limited" at 4h tasks let's say). It is a bit arbitrary, but I think this is what they'…

Without knowing more about their methodology, it seems like a lot of the recent improvements have involved the AI itself taking time to complete the task. At first the models turned a 5 minute task into a 5 second task (by 5 seconds I mean a very short amount of time, not precisely 5 seconds). Then they turned a 15 minute task into a 5 second task. Opus 4.6 completes 8 hour tasks all the time but (at least in my expe…

It measures ability to complete (with a given success rate) a task with a known human benchmark time to complete. I.e., they set the task to human volunteers and timed how long they took the complete that task.

Re: The sigmoids won't save you

#88

Post by devmor »

aspenmartin wrote: post #29
devmor wrote: post #7

"Exponentials all tend to become sigmoids but you can't predict exactly when" is a true statement, but I'm not sure it needed an article. This doesn't say much, and the author fights their own points a couple times, suggesting that they maybe didn't think through what they wanted to write until they were in the middle of writing it and started realizing their assumptions didn't match what they expected the data to sa…

The point is the tiring arguments from AI skeptics saying “things are flattening, they have to” which while technically correct says nothing because no one knows when that will happen and we see no mechanism for this yet. Lindy’s law as a reasonable prediction under total uncertainty is interesting and insightful and a lot of people don’t know about it or why it holds. I did enjoy the reference to this!

But those skeptics are initially responding to the constant AI hype claims that we are exponentially growing to AGI. So this article is in fact just a (very poorly thought through) attempt at saying “nuh uh, the hype might be true, you can’t prove it’s not yet!

Re: The sigmoids won't save you

#89

Post by Leynos »

MadxX79 wrote: post #65
NitpickLawyer wrote: post #11

Earlier quoted context omitted.

IIRC that graph tracks capabilities as time_to_solve a task for humans (i.e. the model can now handle tasks that usually take a human ~8h). Which, depending on what tasks you look at, could be a reasonable finding. I could see Opus 4.6 handling tasks that take ~8h for humans, and that 5.1 couldn't previously handle (with 5.1 being "limited" at 4h tasks let's say). It is a bit arbitrary, but I think this is what they'…

I don't know why people are so impressed by 8h. I trained an LLM to write the whole Harry Potter series, and that took JK Rowling like 17 years. For my next point on the graph, I'll train the LLM to write the Bible, something that took humans >1500 years.

Look at the tasks in the benchmark (see §2 https://arxiv.org/html/2503.14499v3)

Re: The sigmoids won't save you

#90

Post by renticulous »

dsign wrote: post #69
energy123 wrote: post #21

Earlier quoted context omitted.

The individual who is the best at predicting the future is predicting ASI and full labor automation by 2040: https://xcancel.com/peterwildeford/status/202963666232244661...

My own bet is end of that decade: somewhere between 2045 and 2050. Ofc "full labor automation" has a certain spread of meaning. A sliver of population will always find ways to hold to a job or run one or many businesses. But there will be "enough" labor automation for it to be a social ticking bomb. That, in fact, does not depend on better models nor better AI than we have today. By 2045 there will be a couple of gen…

Complexity of our human world has gone up so much that humanity actually needs something like AI to ensure further progress. It's impossible to expect a human to learn all the fields in a shallow manner (and be a generalist politician) or one field in full depth (ie expert to push the frontier).
Post reply on HN