Model intelligence is no longer the constraint for automation • Hacker News Board

Re: Model intelligence is no longer the constraint for automation

#101

Post by woooooo » Sat, Aug 16, 2025, 3:03 PM UTC

OrderlyTiamat wrote: post #59

therein wrote: post #46

Earlier quoted context omitted.

I agree it isn't really proof by contradiction. It is more like proof by demonstration of concrete failures in real life demonstrations, which is stronger. It is like the author is saying 12 is a prime number and I am like but I divided it by 2 just the other day.

Nit pick, but proof by contradiction is necessarily stronger as it is deductive reasoning, and this kind of "proof" by anecdotal evidence doesn't rise above abductive reasoning. Still useful, very much not a proof.

We don't have a formal model of how/why any given LLM works, and incidentally we're also short on proofs for real-world software and organizations.

Empirical facts are the strongest thing we have in this domain.

Re: Model intelligence is no longer the constraint for automation

#102

Post by GauntletWizard » Sat, Aug 16, 2025, 3:17 PM UTC

vladms wrote: post #77

GauntletWizard wrote: post #36

Earlier quoted context omitted.

We can automate plenty in physiological needs, and in fact have already. There's plenty of food and housing for everyone to have them, but a bunch of people will immediately destroy them if provided with such. I don't think "Dispose of a full house every 3 months" will ever be practical, but we might be able to "solve" physiological needs. Safety needs might be possible to solve. Totalitarian states with ubiquitous p…

> There will always be a limited quantity of being "The Best" at anything Still, to pick a simple example, we do have different sports at which different people are "The Best". One solution would be to multiple the categories, which I feel is already happening to some extent with all the computer games or niche artistical trends. And I would claim that very few people are "The Best", it's mostly about not being "the…

You would think, but you've never seen drama like single-speedrunner games. They know they're unfulfilled and kings of a molehill, and as soon as there's the slightest competition - a single other "run" from someone who bothers with a little practice - there's a blowup. Super-niche-ing is not the solution you think it is.

Re: Model intelligence is no longer the constraint for automation

#103

Post by neom » Sat, Aug 16, 2025, 3:18 PM UTC

simoncion wrote: post #42

simoncion wrote: post #38

Earlier quoted context omitted.

> I think context is the primary primitive property of intelligent systems in general? What do you mean by 'context' in this context? As written, I believe that I could knock down your claim by pointing out that there exist humans who would do catastrophically poorly at a task that other humans would excel at, even if both humans have been fully informed of all of the same context.

To clarify what I'm thinking here by analogy... Imagine that someone said: > I think wood is the primary primitive property of sawmills in general. An obvious observation would be that it is dreadfully difficult to produce the expected product of a sawmill without tools to cut or sand or otherwise shape the wood into the desired shapes. One might also notice that while a sawmill with no wood to work on will not produ…

My perspective ("context as primary primitive") was about context as the foundational prerequisite of intelligent performance. I'm discussing a scenario with the minimum conditions for any intelligent action, whether small scale or large scale. At risk of talking past each other due to nuance methinks and I'm a bit lazy to think it through properly but... I think there is something in saw vs sawmill? Like a scale thing? Either way I wasn't trying to be profound or anything, I was just saying I think context abilities is likely the first prerequisite for any minimally intelligent thing (maybe I shouldn't have used the word system in my original comment).

Re: Model intelligence is no longer the constraint for automation

#104

Post by fmbb » Sat, Aug 16, 2025, 3:33 PM UTC

agentcoops wrote: post #97

fmbb wrote: post #74

> It’s because the bottleneck isn’t in intelligence, but in human tasks: specifying intent and context engineering. So the bottleneck is intelligence. Junior engineers are intelligent enough to understand when they don't understand. They interrogate the intent and context of the tasks they are given. This is intelligence. Solving math questions is not intelligence, computers have been better than humans at that for l…

I'm not sure your argument applies only to AI. Intelligence is certainly not knowing through, say, divine inspiration what another person wants you to do. This bottleneck of "describing the problem" is the same bottleneck faced when working with junior (or senior) engineers, especially in a team. One need only consider the classic of our field, Mythical Man-Month, which is really dedicated to this precise and, in som…

All living things have means and ends and pursue goals to completion. That does not make us call them intelligent.

Whenever the LLM fails to act intelligently, we blame the person who gave it the task. So we don't expect them to be able to figure anything out, we are just treating them as easily reconfigurable Skinner boxes.

I'm not an expert or even very interested in the field so I cannot judge what you propose, only intuit from the word "intelligence" and how these machines are described to work and how I observe them working. Reading a bit of https://en.wikipedia.org/wiki/Intelligence leads me to believe these machines have even less to do with any classical definition of intelligence, but I did notice that

> Scholars studying artificial intelligence have proposed definitions of intelligence that include the intelligence demonstrated by machines

which seems rather relevant. Yeah when the AI researchers describe intelligence the machines are intelligent.

Re: Model intelligence is no longer the constraint for automation

#105

Post by variadix » Sat, Aug 16, 2025, 3:34 PM UTC

nradov wrote: post #29

fnordpiglet wrote: post #27

Earlier quoted context omitted.

Assuming you buy the idea of a post scarcity society and assuming we can separate our long ingrained notion that spending your existence in toil to survive is a moral imperative and not working is deserving of punishment if not death, I personally look forward to a time we can get off the hamster wheel. Most buttons that get pushed by people are buttons not worth spending your existence pushing. This includes an awfu…

We will never have a post scarcity society. Automation can make certain foodstuffs and manufactured goods somewhat cheaper but the things that people really want will always be in short supply, for example real estate in geographically favorable areas.

You’re not imagining what post scarcity can really look like. If you have abundant energy, automation, etc. you could manipulate geography and climate, you could build artificial land mass, and so on. It really depends on what people mean by post scarcity.

Re: Model intelligence is no longer the constraint for automation

#106

Post by Jensson » Sat, Aug 16, 2025, 4:22 PM UTC

diggan wrote: post #71

bobbylarrybobby wrote: post #32

Earlier quoted context omitted.

Claude routinely stops and asks me clarifying questions before continuing, especially when the given extended thinking or doing research.

Indeed, the ability to do so seems to depend more on how well your system prompt is laying out that workflow, than how "intelligent" the model is.

Prompting it to ask clarifying questions will make it ask questions it has seen before, not ask questions it needs you to clarify. So that doesn't solve the problem, it just causes other problems.

If it actually did solve the problem then they would train the models to act that way by default, so anything that you need to make smart prompts for has to be dumb.

Re: Model intelligence is no longer the constraint for automation

#107

Post by kbrkbr » Sat, Aug 16, 2025, 4:24 PM UTC

hdgvhicv wrote: post #96

djrj477dhsnv wrote: post #34

Earlier quoted context omitted.

Do you really want to live in this "post scarcity" world? With no effort required to meet your needs and desires, what motivation will you have to do anything? Kaczynski's warnings seem more apt with every year that passes.

Plenty of weathy people do things, not because they have to, but because they want to Plenty of retired people carry on doing things too

Here is another view: some of them maybe do things to perform richness. And others are probably so bored that they just try new extreme things, but nothing fills that inner void. I can't get no satisfaction.

Or maybe not. I'll never know.

Re: Model intelligence is no longer the constraint for automation

#108

Post by ankit219 » Sat, Aug 16, 2025, 4:25 PM UTC

saint_yossarian wrote: post #66

ankit219 wrote: post #56

The bottleneck for automation is verification. With human work, verification was fast(er) because you know where to look with certain assumptions that your upstream tasker would not have made trivial mistakes. For automation, AI needs to verify it's own work, review, and self correct to be able to automate any given work. Where this works, it will also change the abstraction layer compared to what it is today. The pr…

One thing that comes to mind: You still have to verify that the tests are exhaustive, and that the code isn't just gaming specific test scenarios. I guess fuzzing and property-based testing could mitigate this to some extent.

Yes, we are getting there. I think compiler is a bigger problem than unit tests given most verticals don't even have that. With unit tests, there would be some reward hacking but would be controlled at the model level + tests. (this is one of the reason i dont believe in transformer based llm as a judge for a verifier)

Re: Model intelligence is no longer the constraint for automation

#109

Post by Jensson » Sat, Aug 16, 2025, 4:25 PM UTC

KoolKat23 wrote: post #68

mrlongroots wrote: post #25

Earlier quoted context omitted.

> they will struggle just as a model does surly A human will struggle, but they will recognize the things they need to know, and seek out people who may have the relevant information. If asked "how are things going" they will reliably be able to say "badly, I don't have anything I need".

That's just additional context.

That the person go and get themselves. If a model could to that we wouldn't need you to drive them. Basically every human is self going that way, you don't need to go and pick them up since they got stuck in a loop of unknowns at a grocery store etc.

Re: Model intelligence is no longer the constraint for automation

#110

Post by Jensson » Sat, Aug 16, 2025, 4:29 PM UTC

HappMacDonald wrote: post #14

Kuinox wrote: post #6

It's specific model that run for maths. GPT-5 and Gemini 2.5 still cannot compute an arbitrary length sum of whole number without a calculator. I have a proceduraly generated benchmark of basic operations, LLMs gets better at it with time, but they cant still solve basic maths or logic problems. BTW I'm open to selling it, my email is on my hn profile.

Have you ever seen what these arbitrary length whole numbers look like once they are tokenized? They don't break down to one-digit-per-token, and the same long number has no guarantee of breaking down into tokens the same way every time it is encountered . But the algorithms they teach humans in school to do long-hand arithmetic (which are liable to be the only algorithms demonstrated in the training data) require a…

> But the algorithms they teach humans in school to do long-hand arithmetic (which are liable to be the only algorithms demonstrated in the training data) require a single unique numeral for every digit.

But humans don't see single digits, we learn to parse noisy visual data into single digits and then use those single digits to do the math.

It is much easier for these models to understand what the number is based on the tokens and parse that than it is for a visual model to do it based on an image, so getting those tokens streamed straight into its system makes its problem to solve much much simpler than what humans do. We weren't born able to read numbers, we learn that.