AI 2027 • Hacker News Board

Re: AI 2027

#541

Post by EMIRELADERO » Fri, Apr 04, 2025, 10:05 PM UTC

grey-area wrote: post #531

boznz wrote: post #521

Earlier quoted context omitted.

> we haven't seen the signs of a runaway singularity as some thought was likely. The signs are not there but while we may not be on an exponential curve (which would be difficult to see), we are definitely on a steep upward one which may get steeper or may fizzle out if LLM's can only reach human level 'intelligence' but not surpass it. Original article was a fun read though and 360,000 words shorter than my very sim…

LLMs don’t have any sort of intelligence at present, they have a large corpus of data and can produce modified copies of it.

While certainly not human-level intelligence, I don't see how you could say they don't have any sort of it. There's clearly generalization there. What would you say is the threshold?

Re: AI 2027

#542

Post by reducesuffering » Fri, Apr 04, 2025, 10:08 PM UTC

hn_throwaway_99 wrote: post #129

reducesuffering wrote: post #87

Earlier quoted context omitted.

Will people finally wake up that the AGI X-Risk people have been right and we’re rapidly approaching a really fucking big deal? This forum has been so behind for too long. Sama has been saying this a decade now: “Development of Superhuman machine intelligence is probably the greatest threat to the continued existence of humanity” 2015 https://blog.samaltman.com/machine-intelligence-part-1 Hinton, Ilya, Dario Amodei,…

> Will people finally wake up that the AGI X-Risk people have been right and we’re rapidly approaching a really fucking big deal? OK, say I totally believe this. What, pray tell, are we supposed to do about it? Don't you at least see the irony of quoting Sama's dire warnings about the development of AI, without at least mentioning that he is at the absolute forefront of the push to build this technology that can dest…

> If on one hand your goal is to build "super intelligence", i.e. way smarter than any human or group of humans, how do you expect to control that super intelligence when you're just acting at the middling level of human intelligence?

That's exactly what the true AGI X-Riskers think! Sama acknowledges the intense risk but thinks the path forward is inevitable anyway so hoping that building intelligence will give them the intelligence to solve alignment. The other camp, a la Yudkowsky, believe it's futile to just hope it gets solved without AGI capabilities first becoming more intelligent, powerful, and disregarding any of our wishes. And then we've ceded any control of our future to an uncaring system that treats us as a means to achieve its original goals like how an ant is in the way of a Google datacenter. I don't see how anyone who thinks "maybe stock number go up as your only goal is not the best way to make people happy", can miss this.

Re: AI 2027

#543

Post by pertymcpert » Fri, Apr 04, 2025, 10:11 PM UTC

percentcer wrote: post #501

jug wrote: post #496

Earlier quoted context omitted.

> there are some new capabilities that are big, but they are still fundamentally next-token predictors Anthropic recently released research where they saw how when Claude attempted to compose poetry, it didn't simply predict token by token and "react" to when it thought it might need a rhyme and then looked at its context to think of something appropriate, but actually saw several tokens ahead and adjusted for where…

Isn't this just a form of next token prediction? i.e. you'll keep your options open for a potential rhyme if you select words that have many associated rhyming pairs, and you'll further keep your options open if you focus on broad topics over niche

It doesn't really make explain it because then you'd expect lots of nonsensical lines trying to make a sentence that fits with the theme and rhymes at the same time.

Re: AI 2027

#544

Post by hollerith » Fri, Apr 04, 2025, 10:14 PM UTC

reducesuffering wrote: post #542

hn_throwaway_99 wrote: post #129

Earlier quoted context omitted.

> Will people finally wake up that the AGI X-Risk people have been right and we’re rapidly approaching a really fucking big deal? OK, say I totally believe this. What, pray tell, are we supposed to do about it? Don't you at least see the irony of quoting Sama's dire warnings about the development of AI, without at least mentioning that he is at the absolute forefront of the push to build this technology that can dest…

> If on one hand your goal is to build "super intelligence", i.e. way smarter than any human or group of humans, how do you expect to control that super intelligence when you're just acting at the middling level of human intelligence? That's exactly what the true AGI X-Riskers think! Sama acknowledges the intense risk but thinks the path forward is inevitable anyway so hoping that building intelligence will give them…

Slightly more detail: until about 2001 Yudkowsky was what we would now call an AI accelerationist, then it dawned on him that creating an AI that is much "better at reality" than people are would probably kill all the people unless the AI has been carefully designed to stay aligned with human values (i.e., to want what we want) and that ensuring that it stays aligned is a very thorny technical problem, but was still hopeful that humankind would solve the thorny problem. He worked full time on the alignment problem himself. In 2015 he came to believe that the alignment problem is so hard that it is very very unlikely to be solved by the time it is needed (namely, when the first AI is deployed that is much "better at reality" than people are). He went public with his pessimism in Apr 2022, and his nonprofit (the Machine Intelligence Research Institute) fired most of its technical alignment researchers and changed its focus to lobbying governments to ban the dangerous kind of AI research.

Re: AI 2027

#545

Post by boshalfoshal » Fri, Apr 04, 2025, 11:16 PM UTC

nikisil80 wrote: post #451

visarga wrote: post #431

The story is entertaining, but it has a big fallacy - progress is not a function of compute or model size alone. This kind of mistake is almost magical thinking. What matters most is the training set. During the GPT-3 era there was plenty of organic text to scale into, and compute seemed to be the bottleneck. But we quickly exhausted it, and now we try other ideas - synthetic reasoning chains, or just plain synthetic…

Best reply in this entire thread, and I align with your thinking entirely. I also absolutely hate this idea amongst tech-oriented communities that because an AI can do some algebra and program an 8-bit video game quickly and without any mistakes, it's already overtaking humanity. Extrapolating from that idea to some future version of these models, they may be capable of solving grad school level physics problems and…

I don't necessarily think you're wrong, and in general I do agree with you to an extent that this seems like self-centeted Computer Scientist/SWE hubris to think that automating programming is ~AGI.

HOWEVER there is a case to be made that software is an insanely powerful lever for many industries, especially AI. And if current AI gets good enough at software problems that it can improve its own infrastructure or even ideate new model architectures, then we would (in this hypothetical case), potentially reach an "intelligence explosion," which would (may) _actually_ yield a true, generalized intelligence.

So as a cynic, while I think the intermediary goal of many of these so-called-agi companies is just your usual SaaS automation slop because thats the easiest industry to disrupt and extract money from (and the people at these companies only really know how software works, as opposed to having knowledge of other things like chemistry, biology, etc), I also think that in theory, being a very fast and low cost programming agent is a bit more powerful than you think.

Re: AI 2027

#546

Post by lanza » Fri, Apr 04, 2025, 11:28 PM UTC

Without reading an entire novel's worth of text, do they explain why they picked these dates? They have a separate timeline post where the 90th percentile of superhuman coder is later than 2050. Did they just go for shock value and pick the scariest timeline?

Re: AI 2027

#547

Post by dangus » Fri, Apr 04, 2025, 11:36 PM UTC

EMIRELADERO wrote: post #541

grey-area wrote: post #531

Earlier quoted context omitted.

LLMs don’t have any sort of intelligence at present, they have a large corpus of data and can produce modified copies of it.

While certainly not human-level intelligence, I don't see how you could say they don't have any sort of it. There's clearly generalization there. What would you say is the threshold?

Seems like you’d have to prove the inverse.

The threshold would be “produce anything that isn’t identical or a minor transfiguration of input training data.”

In my experience my AI assistant in my code editor can’t do a damn thing that isn’t widely documented and sometimes botches tasks that are thoroughly documented (such as hallucinating parameters names that don’t exist). I can witness this when I reach the edge of common use cases where extending beyond the documentation requires following an implication.

For example, AI can’t seem to understand how to help me in any way with Terraform dynamic credentials because the documentation is very sparse, and it is not part of almost any blog posts or examples online. My definition the variable is populated dynamically and real aren’t shown anywhere. I get a lot of irrelevant nonsense suggestions on how to fix it.

AI is a great “amazing search engine” and it can string together combinations of logic that already exist in documentation and examples while changing some names here and there, but what looks like true understanding really is just token prediction.

IMO the massive amount of training data is making the man behind the curtain look way better than he is.

Re: AI 2027

#548

Post by EMIRELADERO » Fri, Apr 04, 2025, 11:45 PM UTC

dangus wrote: post #547

EMIRELADERO wrote: post #541

Earlier quoted context omitted.

While certainly not human-level intelligence, I don't see how you could say they don't have any sort of it. There's clearly generalization there. What would you say is the threshold?

Seems like you’d have to prove the inverse. The threshold would be “produce anything that isn’t identical or a minor transfiguration of input training data.” In my experience my AI assistant in my code editor can’t do a damn thing that isn’t widely documented and sometimes botches tasks that are thoroughly documented (such as hallucinating parameters names that don’t exist). I can witness this when I reach the edge o…

That's creativity, not intelligence. LLMs can be intelligent while having very little (or even none at all) creativity. I don't believe one necessarily requires the other.

Re: AI 2027

#549

Post by dangus » Fri, Apr 04, 2025, 11:45 PM UTC

MoonGhost wrote: post #517

jenny91 wrote: post #512

Late 2025, "its PhD-level knowledge of every field". I just don't think you're going to get there. There is still a fundamental limitation that you can only be as good as the sources you train on. "PhD-level" is not included in this dataset: in other words, you don't become PhD-level by reading stuff. Maybe in a few fields, maybe a masters level. But unless we come up with some way to have LLMs actually do original r…

> Late 2025, "its PhD-level knowledge of every field". I just don't think you're going to get there. You think too much of PhDs. They are different. Some of them are just repackaging of existing knowledge. Some are just copy-paste like famous Putin's. Not sure he even rad, to be honest.

I’m pretty sure a PhD wouldn’t confidently hallucinate configuration parameters that don’t exist like my AI coding tool does.

A PhD also wouldn’t be biased toward agreeing with me all the time.

Re: AI 2027

#550

Post by dangus » Fri, Apr 04, 2025, 11:50 PM UTC

ImHereToVote wrote: post #447

dughnut wrote: post #391

I don’t know about you, but my takeaway is that the author is doing damage control but inadvertently tipped a hand that OpenAI is probably running an elaborate con job on the DoD. “Yes, we have a super secret model, for your eyes only, general. This one is definitely not indistinguishable from everyone else’s model and it doesn’t produce bullshit because we pinky promise. So we need $1T.” I love LLMs, but OpenAI’s ma…

How do you know this?

I don’t think that was their claim that they knew this.

I think it’s hilarious that apparently few have learned from Theranos or WeWork.

OpenAI is in a precarious position. Anything less than AGI will make them look like a bust. They are backed into a situation where they are heavily incentivized to lie and Theranos their way out of this and hope they can actually deliver something that resembles their pie in the sky predictions.

We are at the point where GPT-5 is starting to look like the iPhone 5.