Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The sigmoids won't save you

astralcodexten.com

Post reply on HN
Print view View original

251–260 of 297 posts

Re: The sigmoids won't save you

#251

Post by coldtea »

bjackman wrote: post #184
stego-tech wrote: post #163

I felt the better takeaway from this was that it's impossible to know for certainty how long this will or will not continue regardless of the data or models you're using, because if you (or anyone else) could predict that accurately they'd be one of the richest people on the planet. I don't know when (or if) AI will implode or succeed with any degree of provable certainty, because that's not my area of expertise. Rat…

I think his agenda here is to point out that your probability distribution for AI outcomes should be broad (what you said), but most importantly: this means you must take seriously the possibility that we are gonna get superintelligence quite soon. Basically a lot of people say "but isn't it also pretty likely that we DON'T get superintelligence?" And, yes, it is. But superintelligence being even a remotely plausible…

>but most importantly: this means you must take seriously the possibility that we are gonna get superintelligence quite soon.

So, his point with all the demand for rigor is to end on a hand-waved jump of faith from "improved AI models" to the mythical "superintelligence"?

Re: The sigmoids won't save you

#252

Post by coldtea »

topherhunt wrote: post #232
bjackman wrote: post #184

Earlier quoted context omitted.

I think his agenda here is to point out that your probability distribution for AI outcomes should be broad (what you said), but most importantly: this means you must take seriously the possibility that we are gonna get superintelligence quite soon. Basically a lot of people say "but isn't it also pretty likely that we DON'T get superintelligence?" And, yes, it is. But superintelligence being even a remotely plausible…

I think his agenda / point is that, viewed from Lindy's Law, given the SOTA in 2026, superintelligent AI arriving soon is vastly more probable than not, right? To make the case that "sure, AI capability and intelligence have grown exponentially over the past several years, but don't worry, they're about to abruptly level off and in fact won't blatantly surpass human-level intelligence within the coming decades" seems…

>I think his agenda / point is that, viewed from Lindy's Law, given the SOTA in 2026, superintelligent AI arriving soon is vastly more probable than not, right

Why would that be? Nothing about Lindy's Law makes that promise. And even the SOTA in 2026 is over-estimated thanks to a trillion dollar industry trusted to not influence benchmarks.

Re: The sigmoids won't save you

#253

Post by coldtea »

stego-tech wrote: post #187
bjackman wrote: post #184

Earlier quoted context omitted.

I think his agenda here is to point out that your probability distribution for AI outcomes should be broad (what you said), but most importantly: this means you must take seriously the possibility that we are gonna get superintelligence quite soon. Basically a lot of people say "but isn't it also pretty likely that we DON'T get superintelligence?" And, yes, it is. But superintelligence being even a remotely plausible…

You’re 100% correct, which is why I opted for a broad investment approach rather than trying to pick “winners”. My thought process RE: superintelligence/AGI is generally this: * I personally don’t believe it’s likely to happen with silicon-based computing due to the immense power and resource costs involved just to get to where we are now; hence why I invest broadly to capitalize on what gains we actually attain usin…

>we’re facing a perfect storm of

Add

Total collapse in government quality AND public trust to politicians

Total collapse of news media to slop and paid-for

Total collapse of culture

(Not just the US either)

Re: The sigmoids won't save you

#254

Post by coldtea »

XorNot wrote: post #196
bjackman wrote: post #184

Earlier quoted context omitted.

I think his agenda here is to point out that your probability distribution for AI outcomes should be broad (what you said), but most importantly: this means you must take seriously the possibility that we are gonna get superintelligence quite soon. Basically a lot of people say "but isn't it also pretty likely that we DON'T get superintelligence?" And, yes, it is. But superintelligence being even a remotely plausible…

That's literally the singularity though - the point past which predictions are meaningless. My "plan" is hope for a benevolent intelligence that establishes a post-human government and then enjoy poat-scarcity society doing wood working or something. Billionaires should probably be more worried.

Soylent Green and Mad Max with billionaires in private communities and bunkers sounds more likely than "paid to exist and woodwork"

Re: The sigmoids won't save you

#255

Post by boxed »

suddenlybananas wrote: post #191
boxed wrote: post #155

Earlier quoted context omitted.

Just to play devils advocate: are we sure humans have demonstrated the ability to go beyond their training data? Like.. are we sure-sure about that?

Planes, trains, and automobiles are not natural features of the environment.

And all were invented piece by piece. Each human had to have updated training data to be able to handle the next tweak.

Re: The sigmoids won't save you

#256

Post by boxed »

rotis wrote: post #220
boxed wrote: post #155

Earlier quoted context omitted.

Just to play devils advocate: are we sure humans have demonstrated the ability to go beyond their training data? Like.. are we sure-sure about that?

Are you suggesting intelligent design got us here?

The exact opposite. Improvements are step by step and small.

Re: The sigmoids won't save you

#257

Post by boxed »

whatshisface wrote: post #158
boxed wrote: post #155

Earlier quoted context omitted.

Just to play devils advocate: are we sure humans have demonstrated the ability to go beyond their training data? Like.. are we sure-sure about that?

I'm not asking for anything close enough to the boundary for questions like that to be difficult. There are some ML systems like AlphaGo that have crossed the line in specific domains. It's just that making self-play and online learning work for huge LLMs is highly non obvious. The idea is simply that the basic idea behind LLMs, that you're distilling the entropy out of the entire available world of text, is antithet…

> Further developing on the theme of self-play, humans have the ability to sense what we want (intellectually) and reach for it communally over thousands of years. It's an innate quality, and if AI starts participating (contrast to giving people psychosis) we will all be able to tell.

Well.. that's not true is it. There are human cultures that haven't reached for anything for thousands of years even though they clearly saw what western culture was doing and that they were being left behind badly.

Re: The sigmoids won't save you

#258

Post by coldtea »

btilly wrote: post #37

Lindy’s Law is an absolute gem, that I'm keeping. If we don't understand the fundamental limits to any particular kind of trend, our default assumption should be that it will continue for about as long as it has gone on already. We can, in fact, easily put a confidence interval on this. With 90% odds we're not in the first 5% of the trend, or the last 5% of the trend. Therefore it will probably go on between 1/19th l…

>If we don't understand the fundamental limits to any particular kind of trend, our default assumption should be that it will continue for about as long as it has gone on already. We can, in fact, easily put a confidence interval on this. With 90% odds we're not in the first 5% of the trend, or the last 5% of the trend. Therefore it will probably go on between 1/19th longer, and 19 times longer. With a median of as long as it has gone on so far.

People would confidently cite Lindy's law all the way near the end of a trend. Nothing would stop a Roman saying that just before the Fall.

We don't always need to "understand the fundamental limits" to a trend to see where it's going. Just to observe more than a random blind guess about them.

I also wouldn't trust the "see how much we're improving" benchmarks of a trillion dollar pre-IPO industry to begin with.

Re: The sigmoids won't save you

#259

Post by t43562 »

aspenmartin wrote: post #227
t43562 wrote: post #134

Earlier quoted context omitted.

Why is the evidence on the side of the hype? Why do you assume something is size X just because nobody has proved it's smaller yet? The evidence is just whatever it is - we cannot make predictions with it.

We have very stable trends in observed performance in a LOT of different measurements and stable scaling laws that continue to hold true across at the largest scales we have built. These scaling laws have a reasonable theoretical justification and make predictions which continue to hold true. Pretraining perplexity does seem related to downstream measures of performance but those too show very stable trends in perfor…

> So: yes we can and do make predictions with it and that’s how we get funding internally and externally to build at these scales

Re: The sigmoids won't save you

#260

Post by winwang »

ordu wrote: post #119
andy99 wrote: post #105

AI has scaled well according to convenient measures. It (neural networks) have the property that whatever you define, they can rapidly be trained master it. We’re able to show that various tasks of increasing complication do not require intelligence and can be framed as autoregressive RL problems. I personally don’t think AI is any closer to sentient intelligence than LeNet; it’s almost trivially clear, we know how i…

> We’re able to show that various tasks of increasing complication do not require intelligence and can be framed as autoregressive RL problems. The important thing we can show it in hindsight only. We don't know which other tasks we are currently mistaken about requiring intelligence. Maybe none of them are? We don't know. We don't know what intelligence is. If we look at decades and even centuries of attempts to def…

Yep. No one bats an eye at eyewitnesses "hallucinating" details, or that I'd rather have Opus as a coworker vs a random middle schooler (err, labor laws notwithstanding). I think perhaps too much of the dialogue around intelligence has to do with the word (and its connotations) itself.

The poster you replied to even used the word "sentient", which is quite interesting (warning: opinionated tangent ahead). Merriam-Webster defines it as "capable of sensing or feeling: conscious of or responsive to the sensations of seeing, hearing, feeling, tasting, or smelling". Feels like qualia. Or if we don't want to go the qualia route... Of course, we wouldn't call Helen Keller non-sentient, so presumably we "really" mean "can it sense or feel" -- well, sense is just "act/feel according to the environment", which you could argue in the case of an LLM would be their context... so we should "really" remove "sense" from the definition, probably. So "do LLMs feel" is probably closer to what "sentient" is being used for here. Since we don't have the obvious symmetry of "you are like me and I feel (therefore you probably feel)", it's way better/easier/feel-good-ier to prefer "LLMs don't feel" rather than "oh shit, it feels and model training is actually just torturing it into the right shape". LLMs as fundamentally non-intelligent also avoids the problems of "what does that say about people" or "we may have made 'AGI' and it wasn't what we thought it would be" or "we're not ready to talk about this yet".

Post reply on HN