Model intelligence is no longer the constraint for automation • Hacker News Board

Re: Model intelligence is no longer the constraint for automation

#41

Post by simoncion » Sat, Aug 16, 2025, 3:48 AM UTC

storus wrote: post #24

miller24 wrote: post #5

It 100% is still intelligence. GPT-5 with Thinking still can't win at tic-tac-toe.

What if it's the desired outcome? Become more human-like (i.e. dumb) to make us feel better about ourselves? NI beats AI again!

> What if it's the desired outcome?

To be able to reason about the rules of a game so trivial that it has been solved for ages, so that it can figure out enough strategy to never not bring the game to a draw (if played against one who is playing to not lose), or a win (if played against someone who is leaving the bot an opening to win), as mentioned in [0] and probably a squillion other places?

Duh?

[0] https://news.ycombinator.com/item?id=44919138>

Re: Model intelligence is no longer the constraint for automation

#42

Post by simoncion » Sat, Aug 16, 2025, 3:57 AM UTC

simoncion wrote: post #38

neom wrote: post #2

Same same human problems. Regardless of their inherent intelligence...humans perform well only when given decent context and clear specifications/data. If you place a brilliant executive into a scenario without meaningful context.... an unfamiliar board meeting where they have no idea of the company’s history, prior strategic discussions, current issues, personel dynamics...expectations..etc etc, they will struggle j…

> I think context is the primary primitive property of intelligent systems in general? What do you mean by 'context' in this context? As written, I believe that I could knock down your claim by pointing out that there exist humans who would do catastrophically poorly at a task that other humans would excel at, even if both humans have been fully informed of all of the same context.

To clarify what I'm thinking here by analogy...

Imagine that someone said:

> I think wood is the primary primitive property of sawmills in general.

An obvious observation would be that it is dreadfully difficult to produce the expected product of a sawmill without tools to cut or sand or otherwise shape the wood into the desired shapes.

One might also notice that while a sawmill with no wood to work on will not produce any output, a sawmill with wood but without woodworking tools is vanishingly unlikely to produce any output... and any it does manage to produce is not going to be good enough for any real industrial purpose.

Re: Model intelligence is no longer the constraint for automation

#43

Post by themanmaran » Sat, Aug 16, 2025, 4:10 AM UTC

mrlongroots wrote: post #23

I very much disagree. To attempt a proof by contradiction: Let us assume that the author's premise is correct, and LLMs are plenty powerful given the right context. Can an LLM recognize the context deficit and frame the right questions to ask? They can not: LLMs have no ability to understand when to stop and ask for directions. They routinely produce contradictions, fail simple tasks like counting the letters in a wo…

This depends on whether you mean LLMs in the sense of single shot, or LLMs + software built around it. I think a lot of people conflate the two.

In our application e use a multi-step check_knowledge_base workflow before and after each LLM request. Pretty much, make a separate LLM request to check the query against the existing context to see if more info is needed, and a second check after generation to see if output text exceeded it's knowledge base.

And the results are really good. Now coding agents in your example are definitely stepwise more complex, but the same guardrails can apply.

Re: Model intelligence is no longer the constraint for automation

#44

Post by mdaniel » Sat, Aug 16, 2025, 4:20 AM UTC

stephc_int13 wrote: post #40

This is because we tend to use a human-centric reference to evaluate the difficulty of a task : playing chess at grand master level is a lot harder than folding laundry, except that it is the opposite, and this weird bias is well known as Moravec’s Paradox. Intelligence is the bottleneck, but not the kind of intelligence you need to solve puzzles.

For others who also hadn't heard of that: https://en.wikipedia.org/wiki/Moravec%27s_paradox

Re: Model intelligence is no longer the constraint for automation

#45

Post by bubblyworld » Sat, Aug 16, 2025, 4:25 AM UTC

mrlongroots wrote: post #23

I very much disagree. To attempt a proof by contradiction: Let us assume that the author's premise is correct, and LLMs are plenty powerful given the right context. Can an LLM recognize the context deficit and frame the right questions to ask? They can not: LLMs have no ability to understand when to stop and ask for directions. They routinely produce contradictions, fail simple tasks like counting the letters in a wo…

This is not a proof by contradiction - you have stated an assumption followed by a bunch of non-sequitors about what LLMs can and can't do, also known as begging the question. Under the conditions of your assumption (namely that LLMs are plenty powerful with the right context) why would you believe anything in your last paragraph? That's how a proof by contradiction works.

(not saying you are wrong, necessarily, but I don't think this argument holds water)

Re: Model intelligence is no longer the constraint for automation

#46

Post by therein » Sat, Aug 16, 2025, 4:41 AM UTC

bubblyworld wrote: post #45

mrlongroots wrote: post #23

I very much disagree. To attempt a proof by contradiction: Let us assume that the author's premise is correct, and LLMs are plenty powerful given the right context. Can an LLM recognize the context deficit and frame the right questions to ask? They can not: LLMs have no ability to understand when to stop and ask for directions. They routinely produce contradictions, fail simple tasks like counting the letters in a wo…

This is not a proof by contradiction - you have stated an assumption followed by a bunch of non-sequitors about what LLMs can and can't do, also known as begging the question. Under the conditions of your assumption (namely that LLMs are plenty powerful with the right context) why would you believe anything in your last paragraph? That's how a proof by contradiction works. (not saying you are wrong, necessarily, but…

I agree it isn't really proof by contradiction. It is more like proof by demonstration of concrete failures in real life demonstrations, which is stronger.

It is like the author is saying 12 is a prime number and I am like but I divided it by 2 just the other day.

Re: Model intelligence is no longer the constraint for automation

#47

Post by bubblyworld » Sat, Aug 16, 2025, 4:46 AM UTC

therein wrote: post #46

bubblyworld wrote: post #45

Earlier quoted context omitted.

This is not a proof by contradiction - you have stated an assumption followed by a bunch of non-sequitors about what LLMs can and can't do, also known as begging the question. Under the conditions of your assumption (namely that LLMs are plenty powerful with the right context) why would you believe anything in your last paragraph? That's how a proof by contradiction works. (not saying you are wrong, necessarily, but…

I agree it isn't really proof by contradiction. It is more like proof by demonstration of concrete failures in real life demonstrations, which is stronger. It is like the author is saying 12 is a prime number and I am like but I divided it by 2 just the other day.

True, but in this case these are hardly globally applicable facts about LLM-based systems (not nearly to the same degree as "12 divides 2" anyway). Different systems have different properties on all those fronts.

I don't think no argument is the right substitute for a bad one!

Re: Model intelligence is no longer the constraint for automation

#48

Post by patrickhogan1 » Sat, Aug 16, 2025, 4:49 AM UTC

themanmaran wrote: post #43

mrlongroots wrote: post #23

I very much disagree. To attempt a proof by contradiction: Let us assume that the author's premise is correct, and LLMs are plenty powerful given the right context. Can an LLM recognize the context deficit and frame the right questions to ask? They can not: LLMs have no ability to understand when to stop and ask for directions. They routinely produce contradictions, fail simple tasks like counting the letters in a wo…

This depends on whether you mean LLMs in the sense of single shot, or LLMs + software built around it. I think a lot of people conflate the two. In our application e use a multi-step check_knowledge_base workflow before and after each LLM request. Pretty much, make a separate LLM request to check the query against the existing context to see if more info is needed, and a second check after generation to see if output…

Do you have a concrete example of what you mean?

For example, the article above was insightful. But the authors pointing to 1,000s of disparate workflows that could be solved with the right context, without actually providing 1 concrete example of how he accomplishes this makes the post weaker.

Re: Model intelligence is no longer the constraint for automation

#49

Post by Davidzheng » Sat, Aug 16, 2025, 4:52 AM UTC

thorum wrote: post #7

This article is insightful, but I blinked when I saw the headline “Reducing the human bottleneck” used without any apparent irony. At some point we should probably take a step back and ask “Why do we want to solve this problem?” Is a world where AI systems are highly intelligent tools , but humans are needed to manage the high level complexity of the real world… supposed to be a disappointing outcome?

it actually doesn't matter what we want. Because eliminating it will in long run increase yield, economic forces will automate humans away by capitalistic forces.

Re: Model intelligence is no longer the constraint for automation

#50

Post by Fade_Dance » Sat, Aug 16, 2025, 4:53 AM UTC

Kuinox wrote: post #16

HappMacDonald wrote: post #14

Earlier quoted context omitted.

Have you ever seen what these arbitrary length whole numbers look like once they are tokenized? They don't break down to one-digit-per-token, and the same long number has no guarantee of breaking down into tokens the same way every time it is encountered . But the algorithms they teach humans in school to do long-hand arithmetic (which are liable to be the only algorithms demonstrated in the training data) require a…

That's was the initial thinking of anyone which I explained this, it was also my speculation, but when you look in it's reasoning where it do the mistake, it correctly extract the digits out of the input token. As I say in another comments, most of the mistakes her happen when it recopy the answer it calculated from the summation table. You can avoid tokenization issue when it extract the answer by making it output a…

I recently saw someone that posted a leaked system prompt for GPT5 (and regardless of the truth of the matter since I can't confirm the authenticity of the claim, the point I'm making stands alone to some degree).

A portion of the system prompt was specifically instructing the LLM that math problems are, essentially, "special", and that there is zero tolerance for approximation or imprecision with these queries.

To some degree I get the issue here. Most queries are full of imprecision and generalization, and the same type of question may even get a different output if asked in a different context, but when it comes to math problems, we have absolutely zero tolerance for that. To us this is obvious, but when looking from the outside, it is a bit odd that we are so loose and sloppy with, well basically everything we do, but then we put certain characters in a math format, and we are hyper obsessed with ultra precision.

The actual system prompt section for this was funny though. It essentially said "you suck at math, you have a long history of sucking at math in all contexts, never attempt to do it yourself, always use the calculation tools you are provided."