Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The sigmoids won't save you

astralcodexten.com

Post reply on HN
Print view View original

231–240 of 297 posts

Re: The sigmoids won't save you

#231

Post by torginus »

AvAn12 wrote: post #223

Forecasts are a thought exercise, not the revelation of something foretold. Best thing to do is think of the outcome you wish for and then try to take whatever actions you can to help make it so. Like with climate change for example.

It can be both. In the field of high-technology (like semiconductor) manufacturing, the techinques of making chips with feature sizes many generations ahead of current SOTA exist in various states of completion, so extrapolation can be made from that.

But what's going on in here is not that - it's reading tea leaves for maximum dramatic effect.

Re: The sigmoids won't save you

#232

Post by topherhunt »

bjackman wrote: post #184
stego-tech wrote: post #163

I felt the better takeaway from this was that it's impossible to know for certainty how long this will or will not continue regardless of the data or models you're using, because if you (or anyone else) could predict that accurately they'd be one of the richest people on the planet. I don't know when (or if) AI will implode or succeed with any degree of provable certainty, because that's not my area of expertise. Rat…

I think his agenda here is to point out that your probability distribution for AI outcomes should be broad (what you said), but most importantly: this means you must take seriously the possibility that we are gonna get superintelligence quite soon. Basically a lot of people say "but isn't it also pretty likely that we DON'T get superintelligence?" And, yes, it is. But superintelligence being even a remotely plausible…

I think his agenda / point is that, viewed from Lindy's Law, given the SOTA in 2026, superintelligent AI arriving soon is vastly more probable than not, right? To make the case that "sure, AI capability and intelligence have grown exponentially over the past several years, but don't worry, they're about to abruptly level off and in fact won't blatantly surpass human-level intelligence within the coming decades" seems to have a high burden of proof unless your model is less "sigmoid" and more "abrupt plateau".

Re: The sigmoids won't save you

#234

Post by anon7725 »

XorNot wrote: post #196
bjackman wrote: post #184

Earlier quoted context omitted.

I think his agenda here is to point out that your probability distribution for AI outcomes should be broad (what you said), but most importantly: this means you must take seriously the possibility that we are gonna get superintelligence quite soon. Basically a lot of people say "but isn't it also pretty likely that we DON'T get superintelligence?" And, yes, it is. But superintelligence being even a remotely plausible…

That's literally the singularity though - the point past which predictions are meaningless. My "plan" is hope for a benevolent intelligence that establishes a post-human government and then enjoy poat-scarcity society doing wood working or something. Billionaires should probably be more worried.

On our present trajectory, any new super intelligence will be the billionaire’s plaything. Hope is indeed needed to see how it would benefit the common person.

Re: The sigmoids won't save you

#235

Post by Balgair »

torginus wrote: post #226
bjackman wrote: post #184

Earlier quoted context omitted.

I think his agenda here is to point out that your probability distribution for AI outcomes should be broad (what you said), but most importantly: this means you must take seriously the possibility that we are gonna get superintelligence quite soon. Basically a lot of people say "but isn't it also pretty likely that we DON'T get superintelligence?" And, yes, it is. But superintelligence being even a remotely plausible…

You want to go to the store to get ice cream. Ice cream is delicious and the value of eating ice cream is a small positive, let's say x. There's a one in ten million chance you'll get hit by a car on the way and die, and your life is infinitely precious, therefore the expected value of going is x times 1 = x, and the one of not going is 1/10m times negative infinity which is negative infinity. You are a rational pers…

Aside:

The singularity framing is really tough here, right? It comes from black hole physics. Essentially, at the event horizon, the way we know how to do physics stops working, and we rightly conclude that we can't currently say anything about the other side of the event horizon. It is not saying that nothing is occurring there. Matter, time, space, energy, whatever, that still is there (maaaaybe?) and is still undergoing something. It's just that we don't know what that is.

The same is true with using these tech singularity arguments. Like, in the age of superintelligence (if that happens), there will still be thing happening, the dawn will still come every day and the dusk will still too. It's just that we say our current ideas about that new day aren't that applicable to that new age (God, this sounds like a hippie).

However, unlike with black hole physics where we aren't even sure time can exist like we know, we are likely all going to be there in that new superintelligence age. We're still going to be making coffee and remembering bad cartoons from our youth. Like, the analogy to black hole physics breaks down here and maybe does a disservice to us. It's not a stark boundary at the Schwartzchild radius, it is a continuous thing, a messy thing, a volatile thing, and very importantly for the HN userbase, a thing that we control and have the choice to participate in.

We are not passively falling into the AGI world like the gnawing grinding gravity of a black hole.

Re: The sigmoids won't save you

#236

Post by naasking »

stego-tech wrote: post #187
bjackman wrote: post #184

Earlier quoted context omitted.

I think his agenda here is to point out that your probability distribution for AI outcomes should be broad (what you said), but most importantly: this means you must take seriously the possibility that we are gonna get superintelligence quite soon. Basically a lot of people say "but isn't it also pretty likely that we DON'T get superintelligence?" And, yes, it is. But superintelligence being even a remotely plausible…

You’re 100% correct, which is why I opted for a broad investment approach rather than trying to pick “winners”. My thought process RE: superintelligence/AGI is generally this: * I personally don’t believe it’s likely to happen with silicon-based computing due to the immense power and resource costs involved just to get to where we are now; hence why I invest broadly to capitalize on what gains we actually attain usin…

I think this is a bit too pessimistic. Progress in algorithms has matched or exceeded progress in hardware, so the same number of FLOPS spent training GPT-3 years ago would produce a much better model today. Ditto for energy use, and hardware is more efficient at delivering FLOPS.

> the widening wealth gaps creating a new underclass not seen since before WW2

I go back and forth on this. I think the reality is that "underclass" is a moving target. AI and automation makes things so cheap that today's underclass lives better than kings ever did.

Re: The sigmoids won't save you

#238

Post by naasking »

jbjbjbjb wrote: post #222
andy99 wrote: post #105

AI has scaled well according to convenient measures. It (neural networks) have the property that whatever you define, they can rapidly be trained master it. We’re able to show that various tasks of increasing complication do not require intelligence and can be framed as autoregressive RL problems. I personally don’t think AI is any closer to sentient intelligence than LeNet; it’s almost trivially clear, we know how i…

Is it not the case that a lot of the recent gains are just the coding harness that directs the LLM? That coding harness isn’t all that intelligent, simple pattern matching that maps to well defined tasks a programmer might do.

I wouldn't say "just", but harnesses are a big deal responsible for a lot of improvements, yes. Models are also fine-tuned on these, like MetaClaw.

Re: The sigmoids won't save you

#239

Post by naasking »

grey-area wrote: post #217
dannyobrien wrote: post #176

Earlier quoted context omitted.

So, this is not quite right: Alexander contributed to the report, but his personal opinion is more like the mid-2030s[1]. Freddie feels like this is him backing down from the original statement, but in fact he said this at the time the report was published, and in fact pointed out a graf below the quote that Freddie claims does tie him to 2027: > Do we really think things will move this fast? Sort of no - between the…

AI boosters really are detached from reality. LLMs are nothing close to AGI and not going to lead to it, they can’t distinguish right from wrong, they can’t count, they can’t reason, they generate plausible text from a vast databank of connected text. Apparently that is enough to fool many people but it’s nothing close to AGI which would require internal models of the world, reasoning etc. We are nowhere close to AGI…

This is all speculative. We don't understand intelligence, so you literally have no idea whether what we recognize as intelligence is some suitable arrangement of "statistical token generation", especially once you add feedbacks loops.

Re: The sigmoids won't save you

#240

Post by naasking »

scott01 wrote: post #224
dreambuffer wrote: post #99

FYI: The author has predicted that "AGI" will be here in 1-2 years and has staked his public reputation on it. He is personally invested in trendlines being lindy rather than sigmoid. I don't think you can use lindy on trends as if trends are static objects, but that's another conversation.

If I'm not mistaken, he's either affiliated with or otherwise connected to the effective altruist movement, hence he can't be unbiased. I find this article tells an interesting perspective on it: https://www.noemamag.com/the-politics-of-superintelligence/

Nobody's unbiased.
Post reply on HN