Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The sigmoids won't save you

astralcodexten.com

Post reply on HN
Print view View original

261–270 of 297 posts

Re: The sigmoids won't save you

#261

Post by RaftPeople »

naasking wrote: post #239
grey-area wrote: post #217

Earlier quoted context omitted.

AI boosters really are detached from reality. LLMs are nothing close to AGI and not going to lead to it, they can’t distinguish right from wrong, they can’t count, they can’t reason, they generate plausible text from a vast databank of connected text. Apparently that is enough to fool many people but it’s nothing close to AGI which would require internal models of the world, reasoning etc. We are nowhere close to AGI…

This is all speculative. We don't understand intelligence, so you literally have no idea whether what we recognize as intelligence is some suitable arrangement of "statistical token generation", especially once you add feedbacks loops.

> "We don't understand intelligence, so you literally have no idea whether what we recognize as intelligence is some suitable arrangement of "statistical token generation""

Do you mean "token" as in the LLM sense?

Or are you thinking that thoughts in the human brain are also constructed out of some sort of underlying "token" even though the abstract thought happens and is held before any words are used to try to communicate that thought to an external party?

Re: The sigmoids won't save you

#262

Post by woeirua »

yorwba wrote: post #218
woeirua wrote: post #107

Earlier quoted context omitted.

Ok, but you can just look at the METR curve. Mythos saturated the 50% time horizon. The 80% is now at 3 hours. The rate of progress is accelerating not slowing down. There’s no indication yet that this is a sigmoid!

The METR task set contains no tasks with a duration greater than 32 hours (conservatively eyeballed from Figure 3: https://arxiv.org/abs/2503.17354 ), so any prediction that naively forecasts a longer time horizon is trivially incorrect. I guess that won't lead to a sigmoid-looking graph though, since METR will likely switch to a different evaluation methodology at that point and stop updating the old curve.

METR themselves say that any estimate >16 is highly suspect because there are too few tasks.

I expect benchmarks like ProgramBench will replace METR this year.

Re: The sigmoids won't save you

#263

Post by muvlon »

Balgair wrote: post #235
torginus wrote: post #226

Earlier quoted context omitted.

You want to go to the store to get ice cream. Ice cream is delicious and the value of eating ice cream is a small positive, let's say x. There's a one in ten million chance you'll get hit by a car on the way and die, and your life is infinitely precious, therefore the expected value of going is x times 1 = x, and the one of not going is 1/10m times negative infinity which is negative infinity. You are a rational pers…

Aside: The singularity framing is really tough here, right? It comes from black hole physics. Essentially, at the event horizon, the way we know how to do physics stops working, and we rightly conclude that we can't currently say anything about the other side of the event horizon. It is not saying that nothing is occurring there. Matter, time, space, energy, whatever, that still is there (maaaaybe?) and is still unde…

I don't know, are we actually going to be making coffee in the case of the AGI singularity?

If you listen to the hardcore doomers, the misaligned superintelligence will curl a finger on its monkey paw and turn the planet into paperclips or something. If you listen to the most depraved boosters, AGI will remove the need for 99.999% of human workers and so we all get turned into biofuel to churn out more tokens.

Yes those are really extremely scenarios but that's how I think of the singularity. It's so alien that we cannot rule out anything.

Re: The sigmoids won't save you

#264

Post by cellis »

stego-tech wrote: post #163

I felt the better takeaway from this was that it's impossible to know for certainty how long this will or will not continue regardless of the data or models you're using, because if you (or anyone else) could predict that accurately they'd be one of the richest people on the planet. I don't know when (or if) AI will implode or succeed with any degree of provable certainty, because that's not my area of expertise. Rat…

Not to argue your broader point, but how, exactly would predicting this, in a macro sense, make you the “richest person on the planet”? A very uninformed person could tell you that “Cerebras will be the next NVIDIA”, put their life savings into it and in 10 years have 100 million. Thats not going to make them anywhere close to the richest person on the planet. Unless you can do it on a micro sense, merely having the ability to “predict” the way the wind is blowing is meaningless.

Re: The sigmoids won't save you

#265

Post by 1attice »

boxed wrote: post #249
1attice wrote: post #157

Earlier quoted context omitted.

Idk, I mean, Shakespeare never read Shakespeare, so, I mean, unless aliens?

AI images are all unique and new.

Not nearly as new as Shakespeare was when he single handedly reinvented English vocabulary, for example.

Re: The sigmoids won't save you

#266

Post by dwa3592 »

I am not disagreeing with the conclusion of the article but the way they reached the conclusion is problematic (and wrong) and here is my nuanced explanation of that : There is no attention paid to the origin (0,0) in any of the plots in the article. To see a meaningful trend you have to be be able to zoom out and zoom in, which means your origin is not a fixed point, ever. This also means that plots with fixed point origins don't tell us anything important at a societal level.

For example: the flight airspeed plot, it starts at ~1900. Now of course it should start there bc we did not have planes before that. But let's change the plot heading from airspeed to human speed a.k.a at what speed could humans move. now you can change the origin meaningfully as we had chariots, horses, bicycles, ships before that.

If you instead create a plot for last 5000 years you would see that the speed at which humans are able to move is rising exponentially, from walking on foot in a radius of 1000-2000feet in a thick jungle 5000 years ago to reentering earth at 25000 miles/hour in 1969 (yeah, read that again). Even for AI, if you zoom out the plot to last 70 years it will look exponential, if you zoom in to last 2 months it will look absolutely flat. The point is that the whole sigmoid/exponential argument is a function of the origin (0,0).

Re: The sigmoids won't save you

#268

Post by robberat7 »

btilly wrote: post #37

Lindy’s Law is an absolute gem, that I'm keeping. If we don't understand the fundamental limits to any particular kind of trend, our default assumption should be that it will continue for about as long as it has gone on already. We can, in fact, easily put a confidence interval on this. With 90% odds we're not in the first 5% of the trend, or the last 5% of the trend. Therefore it will probably go on between 1/19th l…

I am sure Lindy's law is not a general law. Because i have a completely opposite example- How many years can we go by without having a passenger plane crash? as the number of years with no crash increases the chance of something going wrong increases with it. If a crash happened last week, you'd have some sort of sense that okay, it's not going to happen the week after. I can't comment on the causal mechanism, but maybe the collective of people working in the flight industry become more aware after a crash and do their job well.

Re: The sigmoids won't save you

#269

Post by metalcrow »

torginus wrote: post #226
bjackman wrote: post #184

Earlier quoted context omitted.

I think his agenda here is to point out that your probability distribution for AI outcomes should be broad (what you said), but most importantly: this means you must take seriously the possibility that we are gonna get superintelligence quite soon. Basically a lot of people say "but isn't it also pretty likely that we DON'T get superintelligence?" And, yes, it is. But superintelligence being even a remotely plausible…

You want to go to the store to get ice cream. Ice cream is delicious and the value of eating ice cream is a small positive, let's say x. There's a one in ten million chance you'll get hit by a car on the way and die, and your life is infinitely precious, therefore the expected value of going is x times 1 = x, and the one of not going is 1/10m times negative infinity which is negative infinity. You are a rational pers…

And yet sometimes the consequences ARE too severe to ignore. Nuclear war is a serious concept and it's carefully investigated and attempted to be controlled by a lot of powerful people. Why is this situation different? Because it's unlikely? So is nuclear war.
Post reply on HN