Model intelligence is no longer the constraint for automation
latentintent.substack.com
Model intelligence is no longer the constraint for automation
1–10 of 156 posts
Re: Model intelligence is no longer the constraint for automation
#2Same same human problems. Regardless of their inherent intelligence...humans perform well only when given decent context and clear specifications/data. If you place a brilliant executive into a scenario without meaningful context.... an unfamiliar board meeting where they have no idea of the company’s history, prior strategic discussions, current issues, personel dynamics...expectations..etc etc, they will struggle just as a model does surly. They may still manage something reasonably insightful, leveraging general priors, common sense, and inferential reasoning... their performance will never match their potential had they been fully informed of all context and clearly data/objectives. I think context is the primary primitive property of intelligent systems in general?
Re: Model intelligence is no longer the constraint for automation
#3Author IMO correctly recognizes that access to context needs to scale (“latent intent” which I love), but I’m not sure I’m convinced that current models will be effective even if given access to all priors needed for a complex task. The ability to discriminate valuable from extraneous context will need to scale with size of available context, it will be pulling needles from haystacks that aren’t straightforward similarity. I think we will need to steer these things.
Re: Model intelligence is no longer the constraint for automation
#4Providing more context is difficult for a number of reasons. If you do it RAG style you need to know which context is relevant. LLMs are notorious for knowing that a factor is relevant if directly asked about that factor, but not bringing it up if it's implicit. In business things like people's feelings on things, historical business dealings, relevance to trending news can all be factors. If you fine tune... well... there have been articles recently about fine tuning on specific domains causing overall misalignment. The more you fine tune, the riskier.
Re: Model intelligence is no longer the constraint for automation
#5It 100% is still intelligence. GPT-5 with Thinking still can't win at tic-tac-toe.
Re: Model intelligence is no longer the constraint for automation
#6It's specific model that run for maths.
GPT-5 and Gemini 2.5 still cannot compute an arbitrary length sum of whole number without a calculator.
I have a proceduraly generated benchmark of basic operations, LLMs gets better at it with time, but they cant still solve basic maths or logic problems.
BTW I'm open to selling it, my email is on my hn profile.
Re: Model intelligence is no longer the constraint for automation
#7This article is insightful, but I blinked when I saw the headline “Reducing the human bottleneck” used without any apparent irony.
At some point we should probably take a step back and ask “Why do we want to solve this problem?” Is a world where AI systems are highly intelligent tools, but humans are needed to manage the high level complexity of the real world… supposed to be a disappointing outcome?
Re: Model intelligence is no longer the constraint for automation
#8It 100% is still intelligence. GPT-5 with Thinking still can't win at tic-tac-toe.
Tic-tac-toe is solved and a draw can be forced 100% of the time...
Re: Model intelligence is no longer the constraint for automation
#9It's specific model that run for maths. GPT-5 and Gemini 2.5 still cannot compute an arbitrary length sum of whole number without a calculator. I have a proceduraly generated benchmark of basic operations, LLMs gets better at it with time, but they cant still solve basic maths or logic problems. BTW I'm open to selling it, my email is on my hn profile.
i'd wager your benchmark problems require cumbersome arithmetic or are poorly worded / inadequately described. or, you're mislabeling them as basic math and logic (a domain within which LLMs have proven their strengths!)
i only call this out because you're selling it and don't hypothesize* on why they fail your simple problems. i suppose an easily aced bench wouldn't be very marketable
Re: Model intelligence is no longer the constraint for automation
#10It's specific model that run for maths. GPT-5 and Gemini 2.5 still cannot compute an arbitrary length sum of whole number without a calculator. I have a proceduraly generated benchmark of basic operations, LLMs gets better at it with time, but they cant still solve basic maths or logic problems. BTW I'm open to selling it, my email is on my hn profile.
> GPT-5 and Gemini 2.5 still cannot compute an arbitrary length sum of whole number without a calculator.
Neither can many humans, including some very smart ones. Even those who can will usually choose to use a calculator (or spreadsheet or whatever) rather than doing the arithmetic themselves.