Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Model intelligence is no longer the constraint for automation

latentintent.substack.com

Post reply on HN
Print view View original

11–20 of 156 posts

Re: Model intelligence is no longer the constraint for automation

#11

Post by jondwillis »

threecheese wrote: post #3

Author IMO correctly recognizes that access to context needs to scale (“latent intent” which I love), but I’m not sure I’m convinced that current models will be effective even if given access to all priors needed for a complex task. The ability to discriminate valuable from extraneous context will need to scale with size of available context, it will be pulling needles from haystacks that aren’t straightforward simil…

We’re already steering, during pre-training (e.g. reasoning RLHF), as well as test-time (structured outputs, tool calls, agents…)

Re: Model intelligence is no longer the constraint for automation

#12

Post by HappMacDonald »

dismalaf wrote: post #8
miller24 wrote: post #5

It 100% is still intelligence. GPT-5 with Thinking still can't win at tic-tac-toe.

Tic-tac-toe is solved and a draw can be forced 100% of the time...

.. and you can "program" a neural network — so simple it can be implemented by boxes full of marbles and simple rules about how to interact with the boxes — to learn by playing tictactoe until it always plays perfect games. This is frequently chosen as a lesson in how neural network training even works.

But I have a different challenge for you: train a human to play tictactoe, but never allow them to see the game visually, even in examples. You have to train them to play only by spoken words.

Point being that tictactoe is a visual game and when you're only teaching a model to learn from the vast sea of stream-of-tokens (similar to stream-of-phonemes) language, visual games like this aren't going to be well covered in the training set, nor is it going to be easy to generalize to playing them.

Re: Model intelligence is no longer the constraint for automation

#13

Post by mathiaspoint »

gjm11 wrote: post #10
Kuinox wrote: post #6

It's specific model that run for maths. GPT-5 and Gemini 2.5 still cannot compute an arbitrary length sum of whole number without a calculator. I have a proceduraly generated benchmark of basic operations, LLMs gets better at it with time, but they cant still solve basic maths or logic problems. BTW I'm open to selling it, my email is on my hn profile.

> GPT-5 and Gemini 2.5 still cannot compute an arbitrary length sum of whole number without a calculator. Neither can many humans, including some very smart ones. Even those who can will usually choose to use a calculator (or spreadsheet or whatever) rather than doing the arithmetic themselves.

Right but most (competent) humans will reliably use a calculator. It's difficult to get these to reliably make lots of tool calls like that.

Re: Model intelligence is no longer the constraint for automation

#14

Post by HappMacDonald »

Kuinox wrote: post #6

It's specific model that run for maths. GPT-5 and Gemini 2.5 still cannot compute an arbitrary length sum of whole number without a calculator. I have a proceduraly generated benchmark of basic operations, LLMs gets better at it with time, but they cant still solve basic maths or logic problems. BTW I'm open to selling it, my email is on my hn profile.

Have you ever seen what these arbitrary length whole numbers look like once they are tokenized? They don't break down to one-digit-per-token, and the same long number has no guarantee of breaking down into tokens the same way every time it is encountered.

But the algorithms they teach humans in school to do long-hand arithmetic (which are liable to be the only algorithms demonstrated in the training data) require a single unique numeral for every digit.

This is the same source as the problem of counting "R"'s in "Strawberry".

Re: Model intelligence is no longer the constraint for automation

#15

Post by Kuinox »

bt1a wrote: post #9
Kuinox wrote: post #6

It's specific model that run for maths. GPT-5 and Gemini 2.5 still cannot compute an arbitrary length sum of whole number without a calculator. I have a proceduraly generated benchmark of basic operations, LLMs gets better at it with time, but they cant still solve basic maths or logic problems. BTW I'm open to selling it, my email is on my hn profile.

i'd wager your benchmark problems require cumbersome arithmetic or are poorly worded / inadequately described. or, you're mislabeling them as basic math and logic (a domain within which LLMs have proven their strengths!) i only call this out because you're selling it and don't hypothesize* on why they fail your simple problems. i suppose an easily aced bench wouldn't be very marketable

This is a simple sum of 2 whole number, the number are simply big.

Most of the time they make a correct summation table but fail to copy correctly the sum result into a final result. That is not a tokenisation problem (you can change the output format to make sure of it). I have a separated benchmark that test specifically this, when the input is too large, the LLMs fails to accuratly copy the correct token. I suppose the positional embedding, are not perfectly learned and it sometimes cause a mistake.

The prompt is quite short, it use structured output, and I can generate a nice graph of % of good response accross difficulity of the question (which is just the total digit count of the input numbers.

LLMs have 100% success rate on theses sum until they reach a frontier, past that their accuracy collapse at various speed depending of the model.

Re: Model intelligence is no longer the constraint for automation

#16

Post by Kuinox »

HappMacDonald wrote: post #14
Kuinox wrote: post #6

It's specific model that run for maths. GPT-5 and Gemini 2.5 still cannot compute an arbitrary length sum of whole number without a calculator. I have a proceduraly generated benchmark of basic operations, LLMs gets better at it with time, but they cant still solve basic maths or logic problems. BTW I'm open to selling it, my email is on my hn profile.

Have you ever seen what these arbitrary length whole numbers look like once they are tokenized? They don't break down to one-digit-per-token, and the same long number has no guarantee of breaking down into tokens the same way every time it is encountered . But the algorithms they teach humans in school to do long-hand arithmetic (which are liable to be the only algorithms demonstrated in the training data) require a…

That's was the initial thinking of anyone which I explained this, it was also my speculation, but when you look in it's reasoning where it do the mistake, it correctly extract the digits out of the input token. As I say in another comments, most of the mistakes her happen when it recopy the answer it calculated from the summation table. You can avoid tokenization issue when it extract the answer by making it output an array of digits of the answer, it will still fail at simply recopying the correct digit.

Re: Model intelligence is no longer the constraint for automation

#17

Post by miller24 »

HappMacDonald wrote: post #12
dismalaf wrote: post #8

Earlier quoted context omitted.

Tic-tac-toe is solved and a draw can be forced 100% of the time...

.. and you can "program" a neural network — so simple it can be implemented by boxes full of marbles and simple rules about how to interact with the boxes — to learn by playing tictactoe until it always plays perfect games. This is frequently chosen as a lesson in how neural network training even works. But I have a different challenge for you: train a human to play tictactoe, but never allow them to see the game vis…

Well whatever your story is, I know with near certainty that no amount of scaffolding is going to get you from an LLM that can't figure out tic-tac-toe (but will confidently make bad moves) to something that can replace a human in an economically important job.

Re: Model intelligence is no longer the constraint for automation

#18

Post by miller24 »

dismalaf wrote: post #8
miller24 wrote: post #5

It 100% is still intelligence. GPT-5 with Thinking still can't win at tic-tac-toe.

Tic-tac-toe is solved and a draw can be forced 100% of the time...

That's exactly why it's so crazy that GPT-5 with Thinking still loses...

Re: Model intelligence is no longer the constraint for automation

#19

Post by Kuinox »

mathiaspoint wrote: post #13
gjm11 wrote: post #10

Earlier quoted context omitted.

> GPT-5 and Gemini 2.5 still cannot compute an arbitrary length sum of whole number without a calculator. Neither can many humans, including some very smart ones. Even those who can will usually choose to use a calculator (or spreadsheet or whatever) rather than doing the arithmetic themselves.

Right but most (competent) humans will reliably use a calculator. It's difficult to get these to reliably make lots of tool calls like that.

I do think that competent humans can solve any arbitrary sum of 2 whole number with a pen, paper and time. LLMs can't do that.

Re: Model intelligence is no longer the constraint for automation

#20

Post by dismalaf »

miller24 wrote: post #18
dismalaf wrote: post #8

Earlier quoted context omitted.

Tic-tac-toe is solved and a draw can be forced 100% of the time...

That's exactly why it's so crazy that GPT-5 with Thinking still loses...

Ah, your first comment said "can't win". Which is different than "always loses".
Post reply on HN