Live data from Hacker News

The sigmoids won't save you

astralcodexten.com

181–190 of 297 posts

Re: The sigmoids won't save you

#181
post #105

AI has scaled well according to convenient measures. It (neural networks) have the property that whatever you define, they can rapidly be trained master it. We’re able to show that various tasks of increasing complication do not require intelligence and can be framed as autoregressive RL problems. I personally don’t think AI is any closer to sentient intelligence than LeNet; it’s almost trivially clear, we know how i…

> basically how well a universal function approximator can fit to a function we define

That's what you've got wrong. We don't define functions that an LLM approximates. Autoregressive pretraining approximates an unknown function that produces text (that is what the brain does). RL doesn't approximate functions, it optimizes objective by finding an unknown function that performs better.

Re: The sigmoids won't save you

#182
post #142
post #131

Earlier quoted context omitted.

The lifespan argument shows how important it is to have more than zero information.

Yes, it is valuable to have more than zero information. But often we don't have the information that we wish. Even more often, the information that we have leads us to a story, that severely misleads us. Reminding ourselves of the zero information version of the story, can be an antidote to being mislead that way. Therefore it is valuable to know how to make the most out of zero information. And if we have informatio…

The argument is that it's not very valuable because it can be incredibly wrong and the priority is to get more information.

"And we have information to think" - then we don't have zero information right?

Re: The sigmoids won't save you

#183
I’m not saying he’s wrong about the core thesis here, but using Claude Opus 4.6 as a “mic drop” with a chart showing it being twice as good as the last model feels in my experience way off.

Re: The sigmoids won't save you

#184

I felt the better takeaway from this was that it's impossible to know for certainty how long this will or will not continue regardless of the data or models you're using, because if you (or anyone else) could predict that accurately they'd be one of the richest people on the planet. I don't know when (or if) AI will implode or succeed with any degree of provable certainty, because that's not my area of expertise. Rat…

I think his agenda here is to point out that your probability distribution for AI outcomes should be broad (what you said), but most importantly: this means you must take seriously the possibility that we are gonna get superintelligence quite soon.

Basically a lot of people say "but isn't it also pretty likely that we DON'T get superintelligence?" And, yes, it is. But superintelligence being even a remotely plausible outcome is a big fucking deal. Your investment choices in that context are not important.

People really struggle to think rationally in the face of this shape of uncertainty.

Re: The sigmoids won't save you

#185
post #89
post #65

Earlier quoted context omitted.

I don't know why people are so impressed by 8h. I trained an LLM to write the whole Harry Potter series, and that took JK Rowling like 17 years. For my next point on the graph, I'll train the LLM to write the Bible, something that took humans >1500 years.

Look at the tasks in the benchmark (see §2 https://arxiv.org/html/2503.14499v3 )

Yeah, what about them? As far as I read it the tasks are fixed. The AI companies should know the tasks by now, and have overfitted their models on the tests by now, in the same way I'm implying I overfitted my model to reproduce Harry Potter.

Re: The sigmoids won't save you

#186
post #37

Lindy’s Law is an absolute gem, that I'm keeping. If we don't understand the fundamental limits to any particular kind of trend, our default assumption should be that it will continue for about as long as it has gone on already. We can, in fact, easily put a confidence interval on this. With 90% odds we're not in the first 5% of the trend, or the last 5% of the trend. Therefore it will probably go on between 1/19th l…

I feel like Lindy's law doesn't work for things whose observation is partly controlled by the thing itself. For example, take something like a fad or trend; they don't have a hard end date like human lifespan, so it should follow Lindy's law. However, the likelihood, on average across the population, that you observe a trend is going to be higher at the end of a trend lifecycle than at the beginning. This is baked in…

Similarly, if you are a random person being alive, it likely means that the world population is near its peak and extinction is at hand, or at least the start of a permanent decline.

We have at least global warming and impending WW3, so that line of reasoning seems to work.

Re: The sigmoids won't save you

#187

I felt the better takeaway from this was that it's impossible to know for certainty how long this will or will not continue regardless of the data or models you're using, because if you (or anyone else) could predict that accurately they'd be one of the richest people on the planet. I don't know when (or if) AI will implode or succeed with any degree of provable certainty, because that's not my area of expertise. Rat…

I think his agenda here is to point out that your probability distribution for AI outcomes should be broad (what you said), but most importantly: this means you must take seriously the possibility that we are gonna get superintelligence quite soon. Basically a lot of people say "but isn't it also pretty likely that we DON'T get superintelligence?" And, yes, it is. But superintelligence being even a remotely plausible…

You’re 100% correct, which is why I opted for a broad investment approach rather than trying to pick “winners”.

My thought process RE: superintelligence/AGI is generally this:

* I personally don’t believe it’s likely to happen with silicon-based computing due to the immense power and resource costs involved just to get to where we are now; hence why I invest broadly to capitalize on what gains we actually attain using this current branch of AI research across all possible sectors and exposure rates

* If we do achieve AGI using silicon-based computing, its limited scale (requiring vast amounts of compute only deliverable via city-scale data centers) will limit its broader utility until more optimizations can be achieved or a superior compute platform delivered that improves access and dramatically lowers cost; again, investing broadly covers a general uplift rather than hoping for a specific winner

* If AGI is achieved, nobody - doomer or booster alike - will know what comes next other than complete and total destruction of existing societal structures or institutions. The stock market won’t explode with growth so much as immediately collapse from the disintegration of the consumptive base as a result of AGI quite literally annihilating a planet’s worth of jobs and associated business transactions. In this case, a broad spread protects me from harm by spreading the risk around; AGI will annihilate the market globally, but not all at once barring a significant global catastrophe instigated by it

* Which brings me to the worst outcome, where AGI follows the “if anybody builds it everyone dies” thought process: investment is irrelevant because we’re all fucked anyway.

And that’s just my investment approach. I’m too pragmatic to believe we’re at the bottom of the sigmoid curve, but too wise to begin guessing where we actually exist on it at present or how much is left in the current LLM-arm of AI research; I’m an IT dinosaur, not an AI scientist.

What I can point to is the continued demand destruction of consumer compute through higher costs and limited availability due to rampant AI speculation as proof that the harm is already here in a manner most weren’t predicting, while at the same time actual job displacement by AI is limited to the empty boasting of executives using it as a smoke screen for layoffs after RTO mandates failed to thin headcount sufficiently.

In the USA in particular, we’re facing a perfect storm of:

* consumer confidence collapse leading to a decline in spending on all goods, especially luxury ones, by all but the most monied demographics

* data center-driven cost increases (energy) and resource destruction (land, water, fossil fuel use)

* the eradication of government support for renewable energy that would’ve kept these costs in check

* the widening wealth gaps creating a new underclass not seen since before WW2

In other words, most of the discourse continues to revolve around hypotheticals of tomorrow rather than realities of today. That would be the lesson I’d hope more people take away from something like this, so we can finally begin addressing issues themselves rather than empty online circle jerking about who is right or wrong.

Re: The sigmoids won't save you

#188

This article answers the question in the second paragraph then completely ignores the answer for the rest of it. >My understanding is that this represents 3-4 “generations” of different technology (propellers, turbojets, etc). Each technology went through normal iterative improvement, then, when it reached its fundamental limits, got replaced by a better technology. The last technology, ramjets, reached its limit at…

The book "Origins of Efficiency" by Brian Potter discusses this. Stacked sigmoids are a well-understood idea in innovation. The idea that exponential growth will continue with stacked sigmoids is also not a given. An example is the nail. Nails used to be about half a percent of US GDP. That's a pretty big number! A series of innovations stacked on each other (each innovation having its own sigmoid) to reduce the cost…

I don't disagree with you, but your example of nails and their cost reductions made me wonder whether we reached a meaningful limit in say, some fundamental material terms, or whether we just reached a limit in terms of return on investment.

Return on investment can be too low because the investment required is really high, but it can also be too low because the returns are just limited. If prices had dropped 90%, surely nails became even more ubiquitous, but at that stage there's only so much more money to dig out of the cost reduction hole. It feels plausible that there may have been ideas about more digging that could be done, but the reward just wasn't there in the market, especially versus just selling what worked.

I bring it up because the distinction in one specimen may speak to a larger trend: do new sigmoid developments tend to fail to materialize more often because of serious physical limits / lack of good ideas, or because of limitations to ROI? (Or, other things?)

In the arena of AI, the ROI on more intelligence/unit-cost seems pretty high right now. So, it seems like the difficulty of applying any potential innovations would have to be staggering for none to be pursued. Or, there'd have to just not be any good ideas to try.

Overall, I think there's ideas to try. So in my opinion, that shapes out to justify a bullish sentiment on sigmoids continuing to stack until the perceived potential gains from more intelligence/unit-cost somehow fall off.

Like I said, I don't disagree, we really don't know. But I feel it's a good bet that there's more coming.

Re: The sigmoids won't save you

#189
> But if someone claims that the trend toward increasing AI capabilities will never reach some particular scary level, then the burden is on them to explain either:…

This is not the context in which I hear about sigmoids vs exponentials. I hear it in regards to “the singularity”, not that AI won’t reach some pre-specified level. You may get AGI, you aren’t getting a singularity.

Re: The sigmoids won't save you

#190

FYI: The author has predicted that "AGI" will be here in 1-2 years and has staked his public reputation on it. He is personally invested in trendlines being lindy rather than sigmoid. I don't think you can use lindy on trends as if trends are static objects, but that's another conversation.

AGI has become such a meaningless nondescript term, arguing when or how it is here has become pointless. Even OpenAI caved in and removed their AGI clause from their contract with Microsoft because they weren't fully sure that we are not there yet. The original ARC AGI was hailed as proof that AGI is not here yet, but now that ARC 1 and 2 got saturated, noone wanted to consider that perhaps we crossed the point where…

To your point, if we had truly unlimited context to the point where at least that instance of a model could “learn” and have what seems like a continuous “consciousness” I think many of us would think that we’ve attained AGI.

Right now we have an incredibly smart thing with severe short term memory loss, and it’s hard for us to reconcile that as it’s so different from us.

Post reply on HN