Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The changing goalposts of AGI and timelines

mlumiste.com

Post reply on HN
Print view View original

361–370 of 411 posts

Re: The changing goalposts of AGI and timelines

#361

Post by Otterly99 »

pu_pe wrote: post #344
sulam wrote: post #331

Earlier quoted context omitted.

Because there are some really fundamental things they cannot do with next token prediction. For instance, their memory is akin to someone who reads the phone book and memorizes the entire thing, but can't tell you what a phone number is for. Moreover, they can mimic semantic knowledge, because they have been trained on that knowledge, but take them out of their training distribution and they get into a "creative stor…

You didn't actually give an example of what the issue with next token prediction is. You just mentioned current constraints (ie generalization and learning are difficult, needs mountains of data to train, can't play chess very well) that are not fundamental problems. You can trivially train a transformer to play chess above the level any human can play at, and they would still be doing "next token prediction". I woul…

But chess models aren't trained the same way LLMs are trained. If I am not mistaken, they are trained directly from chess moves using pure reinforcement learning, and it's definitely not trivial as for instance AlphaZero took 64 TPUs to train.

Re: The changing goalposts of AGI and timelines

#362

Post by tim333 »

ozgung wrote: post #28

"Artificial general intelligence (AGI) is a type of artificial intelligence that matches or surpasses human capabilities across virtually all cognitive tasks." [Wikipedia] One can argue that they have already achieved this. At least for short termed tasks. Humans are still better at organization, collaboration and carrying out very long tasks like managing a project or a company.

I think the term "artificial general intelligence" is deliberately ambiguous as it doesn't specify any levels. I mean my cat was generally intelligent.

LLMs can't be swapped in for human workers in general because there are still a lot of things they don't do like learning as they go. So that's missing from the Wikipedia thing.

Re: The changing goalposts of AGI and timelines

#363

Post by ACCount37 »

Otterly99 wrote: post #361
pu_pe wrote: post #344

Earlier quoted context omitted.

You didn't actually give an example of what the issue with next token prediction is. You just mentioned current constraints (ie generalization and learning are difficult, needs mountains of data to train, can't play chess very well) that are not fundamental problems. You can trivially train a transformer to play chess above the level any human can play at, and they would still be doing "next token prediction". I woul…

But chess models aren't trained the same way LLMs are trained. If I am not mistaken, they are trained directly from chess moves using pure reinforcement learning, and it's definitely not trivial as for instance AlphaZero took 64 TPUs to train.

You can train them in a very similar way.

Modern LLMs often start at "imitation learning" pre-training on web-scale data and continue with RLVR for specific verifiable tasks like coding. You can pre-train a chess engine transformer on human or engine chess parties, "imitation learning" mode, and then add RL against other engines or as self-play - to anneal the deficiencies and improve performance.

This was used for a few different game engines in practice. Probably not worth it for chess unless you explicitly want humanlike moves, but games with wider state and things like incomplete information benefit from the early "imitation learning" regime getting them into the envelope fast.

Re: The changing goalposts of AGI and timelines

#364

Post by ACCount37 »

RugnirViking wrote: post #359
ACCount37 wrote: post #217

Earlier quoted context omitted.

What makes you think that text is inherently a worse reflection of the world than light is? All world models are lossy as fuck, by the way. I could give you a list of chess moves and force you to recover the complete board state from it, and you wouldn't fare that much better than an off the shelf LLM would. An LLM trained for it would kick ass though.

> I could give you a list of chess moves and force you to recover the complete board state from it, and you wouldn't fare that much better than an off the shelf LLM would idk, I would expect anyone with an understanding of the rules of chess, and an understanding of whatever notation the moves are in, would be able to do it reasonably well? does that really sound so hard to you? people used to play correspondance che…

I would expect that off the shelf GPT-5.4 would be able to do it when prompted carefully, yes. Through reasoning - by playing every move step by step and updating the board one move at a time to arrive at a final board state.

On the other hand, recovering the full board state in a single forward pass? That takes some special training.

Same goes for meatbag chess. A correspondence chess aficionado might be able to take a glance at a list of moves and see the entire game unfold in his mind's eye. A casual player who only knows how to play chess at 600 ELO on a board that's in front of him would have to retrace every move carefully, and might make errors while at it.

Re: The changing goalposts of AGI and timelines

#365

Post by VorpalWay »

Wowfunhappy wrote: post #336
VorpalWay wrote: post #274

Earlier quoted context omitted.

Machine learning: that definitely includes SVM and regression models. Oh and decision trees. Probably a few other things I'm not thinking of right now. Many people will unfortunately be thinking of just neural networks though. (By the way, if something like a regression model or decision tree can solve your problem, you should prefer those. Much cheaper to train and to run inference with those than with neural networ…

Wait, a decision tree is machine learning?

Yes: you fit a decision tree to your dataset in an automated fashion, that fits the definition of machine learning. Just as you would use backpropagation to fit a neural network to your data.

This is what I learnt at university some decades ago, and it matches what wikipedia says today: https://en.wikipedia.org/wiki/Decision_tree_learning

Re: The changing goalposts of AGI and timelines

#366

Post by casey2 »

ghoblin wrote: post #350
casey2 wrote: post #345

Earlier quoted context omitted.

When did search replace all research jobs? How can an AI replace a masseuse? How far away away are we getting from an LLM that scratches your back for less than it costs you to do it? People have unrealistic expectations because they literally think they are summoning god instead of accelerating a few concurrent tasks. If you want to break causality you need to pay the entropy demon it's due.

Everyone cites some niche "human-only" jobs to argue AI won't replace labor. But most of the economy runs on things like document processing, logistics, retail, and factories. High-volume, repeatable, rule-driven tasks, and in those areas, we're already on the brink of full automation. Autonomous retail stores, delivery fleets, and smart factories are either here or imminent. It's not about AI scratching backs, it's…

You're describing valueless automation. We can build an assembly line and mass produce cars, that only has value if society is restructured. The food delivery industry only moves a trillion (globally) because it's incredibly wasteful, not due to value. Most of the value in going to a restaurant was in the experience and culture, the food is just a blend of fats, carbs and protein, but you pay more for the "luxury" of eating in your house or at work.

You'll have cities made to serve cars and food made to serve delivery and worker drones. In the pursuit of optimization you'll end up back at the same place, when there was only one cafeteria in walking distance.

Anyway we aren't "on the brink of full automation" that's ridiculous, people always think this, because they have no idea how brittle automated systems are. To get a generally intelligent robot that operates in the real world you have to go WAY beyond replacing knowledge workers. The brain only uses 1W more when it's working at full tilt, 5% more. For any physical job the body is using. The full body at rest uses 100W, walking that's 300W, manual labor 600W a full sprint could peak at 2000W. That's an absurd range made only possible due to trillions of cells packed with ATP and billions of microscopic capillaries full of glucose that get sucked into your muscles the second you use them. Automation only works in closed systems, give it 2000 years maybe someone makes AGSI, then the robotics problem becomes approachable, but if it were smart it'd just declare it impossible without biotech.

Re: The changing goalposts of AGI and timelines

#367

Post by sulam »

pu_pe wrote: post #344
sulam wrote: post #331

Earlier quoted context omitted.

Because there are some really fundamental things they cannot do with next token prediction. For instance, their memory is akin to someone who reads the phone book and memorizes the entire thing, but can't tell you what a phone number is for. Moreover, they can mimic semantic knowledge, because they have been trained on that knowledge, but take them out of their training distribution and they get into a "creative stor…

You didn't actually give an example of what the issue with next token prediction is. You just mentioned current constraints (ie generalization and learning are difficult, needs mountains of data to train, can't play chess very well) that are not fundamental problems. You can trivially train a transformer to play chess above the level any human can play at, and they would still be doing "next token prediction". I woul…

I use the chess example because it’s especially instructive. It would NOT be trivial to train an LLM to play chess, next token prediction breaks down when you have so many positions to remember and you can’t adequately assign value to intermediate positions. Chess bots work by being trained on how to assign value to a position, something fundamentally different than what an LLM is doing.

A simpler example — without tool use, the standard BPE tokenization method made it impossible for state of the art LLMs to tell you how many ‘r’s are in strawberry. This is because they are thinking in tokens, not letters and not words. Can you think of anything in our intelligence where the way we encode experience makes it impossible for us to reason about it? The closest thing I can come to is how some cultures/languages have different ways of describing color and as a result cannot distinguish between colors that we think are quite distinct. And yet I can explain that, think about it, etc. We can reason abstractly and we don’t have to resort to a literal deus ex machina to do so.

Not being able to explain our brain to you doesn’t mean I can’t notice things that LLMs can’t do, and that we can, and draw some conclusions.

Re: The changing goalposts of AGI and timelines

#368

Post by palmotea »

atomicnumber3 wrote: post #183
politelemon wrote: post #179

Earlier quoted context omitted.

The enskibidification of AI

Honestly, not enough of a joke. I was thinking something similar - this isn't AI, and none of "those people" care if it is or isn't. They don't care philosophically, or even pragmatically. They're selling a product. That product is the IDEA of replacement of the majority of human labor with what's basically slave labor but with substantially disregardable ethical quandaries. It's honestly a genius product. I'm not su…

> I'm vaguely surprised so many people who don't stand to benefit in any way shape or form, or who will even potentially starve if it works out, are so keen on it. But there are always bootlickers.

I've been getting more and more disappointed by software engineers (in aggregate) as the years go by. They don't even have to be bootlickers to do what you describe, I think a lot of it is pride in their "intelligence," which they express by believing and regurgitating the propaganda they've consumed. They prove their smarts by (among other things) having opinions that align with a zeitgeist of some group of powerful elites. They're too-easily manipulated.

And it's not just AI, it's also things like libertarianism. You've got workers identifying as capitalist tycoons, because they read a book and have some shares in a 401k.

Re: The changing goalposts of AGI and timelines

#369

Post by bonesss »

datsci_est_2015 wrote: post #354
devonkelley wrote: post #264

[flagged]

“Judgment” is close to my mental model, but I prefer “liability”. All meatspace-based employment comes down to liability. A McDonald’s shift worker is liable for any mistakes made during their shift that are considered their responsibility. A SWE is liable for any mistakes made during their employment that are considered their responsibility. Accountant. Paralegal. CEO. AI will need to be able to experience consequen…

Grok “apologizing” for generating potentially illegal images and related cases working through various legal systems is an early test of that lack of liability shield.

There are a number of industries where I don’t think “kinda…” is an acceptable answer to “was this code read before deploying?”. Humans aren’t great at repeating boring tasks ad nauseam.

Re: The changing goalposts of AGI and timelines

#370

Post by SpicyLemonZest »

onlyrealcuzzo wrote: post #232
sulam wrote: post #122

The reality is that current models are simply nowhere near AGI. Next token prediction has been pushed very far, and proven to have applicability far beyond the original domain it was designed for (reasoning models are an application I would not have predicted) but it is fundamentally not AGI. It has no real world model, no ability to learn in any but superficial ways, and without extensive scaffolding this is all ver…

How many months has it been since we were told there would be zero software engineers left in the world in 12 months?

What Dario Amodei said 12 months ago is that AI would be "writing essentially all of the code", and the job of software engineers would become guiding and reviewing the code generation process. That's come true at a number of companies.

The important context I think people may miss is, this does not require AI to be 10x or 5x or even 1x as good as a human programmer. Claude is worse than me in meaningful ways at the kind of code I need to write, but it’s still doing almost all my coding because after 4.6 it’s smart enough to understand when I explain what program it should have written.

Post reply on HN