Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Model intelligence is no longer the constraint for automation

latentintent.substack.com

Post reply on HN
Print view View original

131–140 of 156 posts

Re: Model intelligence is no longer the constraint for automation

#131

Post by mrlongroots »

antithesizer wrote: post #113
mrlongroots wrote: post #23

I very much disagree. To attempt a proof by contradiction: Let us assume that the author's premise is correct, and LLMs are plenty powerful given the right context. Can an LLM recognize the context deficit and frame the right questions to ask? They can not: LLMs have no ability to understand when to stop and ask for directions. They routinely produce contradictions, fail simple tasks like counting the letters in a wo…

The word you're looking for is "rebuttal" since this is neither proof nor refutation of anything, but merely an argument against the thesis.

A rebuttal is just an alias for "counterargument", it does not define the structure of the counterargument.

However flawed, what I said did have a structure (please refer to my other response in this thread for why).

Re: Model intelligence is no longer the constraint for automation

#132

Post by mrlongroots »

Wowfunhappy wrote: post #83
mrlongroots wrote: post #23

I very much disagree. To attempt a proof by contradiction: Let us assume that the author's premise is correct, and LLMs are plenty powerful given the right context. Can an LLM recognize the context deficit and frame the right questions to ask? They can not: LLMs have no ability to understand when to stop and ask for directions. They routinely produce contradictions, fail simple tasks like counting the letters in a wo…

> LLMs have no ability to understand when to stop and ask for directions. I haven't read TFA so I may be missing the point. However, I have had success getting Claude to stop and ask for directions by specifically prompting it to do so. "If you're stuck or the task seems impossible, please stop and explain the problem to me so I can help you."

Ok I think the confusion arises because of the probabilistic nature of LLM responses that blurs the line between "intelligent vs not".

Let's take driving a car as an example, and a random decision generator as a lower bound on the intelligence of the driver.

- A professionally trained human, who is not fatigued or unhealthy or substance-impaired, rarely makes a mistake, and when they do, there are reasonable mitigating factors.

- ML models, OTOH, are very brittle and probabilistic. A model trained on blue tinted windshields may suffer a dramatic drop in performance if ran on yellow-tinted windshields.

Models are unpredictably probabilistic. They do not learn a complete world model, but the very specific conditions and circumstances of their training dataset.

They continue to get better, and you are able to induce a behavior similar to true intelligence more and more often. In your case, you are able to get them to stop and ask, but if they had the ability to do this reliably, they would not make mistakes as agents at all. Right now they resemble intelligence under a very specific light, and as the regimes under which they resemble one get bigger, they will get to AGIs. But we're not there yet.

Re: Model intelligence is no longer the constraint for automation

#133

Post by exe34 »

djrj477dhsnv wrote: post #34
fnordpiglet wrote: post #27

Earlier quoted context omitted.

Assuming you buy the idea of a post scarcity society and assuming we can separate our long ingrained notion that spending your existence in toil to survive is a moral imperative and not working is deserving of punishment if not death, I personally look forward to a time we can get off the hamster wheel. Most buttons that get pushed by people are buttons not worth spending your existence pushing. This includes an awfu…

Do you really want to live in this "post scarcity" world? With no effort required to meet your needs and desires, what motivation will you have to do anything? Kaczynski's warnings seem more apt with every year that passes.

I'm practically living in a post scarcity situation - my work is stuff I'd do for fun anyway, other than a bit of paperwork now and again. nothing is compulsory if you want to do it anyway. even then I only need to work part time to survive.

the rest of the time I spend studying and doing sports. I've tried doing nothing - but boredom is actually worse than work.

what I really want is for other people to also be in a similar situation. I also want to be able to afford to just not work for 6 months and travel the world - but I've got a mortgage to pay. so I think further reductions in scarcity in my life would not reduce my drive to do, learn, experience one bit.

I suspect that most people would be the same if they weren't accustomed to not having the energy to look after themselves and growing their mind.

Re: Model intelligence is no longer the constraint for automation

#134

Post by exe34 »

kbrkbr wrote: post #107
hdgvhicv wrote: post #96

Earlier quoted context omitted.

Plenty of weathy people do things, not because they have to, but because they want to Plenty of retired people carry on doing things too

Here is another view: some of them maybe do things to perform richness. And others are probably so bored that they just try new extreme things, but nothing fills that inner void. I can't get no satisfaction. Or maybe not. I'll never know.

so the only way to get that inner satisfaction is to do work that you hate? is there another option I'm missing here?

Re: Model intelligence is no longer the constraint for automation

#135

Post by JumpCrisscross »

diggan wrote: post #71
bobbylarrybobby wrote: post #32

Earlier quoted context omitted.

Claude routinely stops and asks me clarifying questions before continuing, especially when the given extended thinking or doing research.

Indeed, the ability to do so seems to depend more on how well your system prompt is laying out that workflow, than how "intelligent" the model is.

Haven’t we all met a smart person who never learned to think critically or in structured ways?

Re: Model intelligence is no longer the constraint for automation

#136

Post by CamperBob2 »

swivelmaster wrote: post #67
Davidzheng wrote: post #49

Earlier quoted context omitted.

it actually doesn't matter what we want. Because eliminating it will in long run increase yield, economic forces will automate humans away by capitalistic forces.

We should stop considering it a given that capitalistic forces will do this and start considering how we build systems that optimize for the maximum amount of human good rather than the maximum amount of abstract economic good (which nowadays usually means an increase in wealth disparity).

Because no one has come up with a system that's better than capitalism at accomplishing that.

Re: Model intelligence is no longer the constraint for automation

#137

Post by elevatortrim »

exe34 wrote: post #118
fmbb wrote: post #74

> It’s because the bottleneck isn’t in intelligence, but in human tasks: specifying intent and context engineering. So the bottleneck is intelligence. Junior engineers are intelligent enough to understand when they don't understand. They interrogate the intent and context of the tasks they are given. This is intelligence. Solving math questions is not intelligence, computers have been better than humans at that for l…

> Junior engineers are intelligent enough to understand when they don't understand. They interrogate the intent and context of the tasks they are given Eh, I wouldn't apply that as if it's a general thing. yes, the really good ones do. many will equally plough through into the mud with albeit admirable determination.

Yes but for the purposes of these conversations, we do not need to say "good juniors" "good engineers" every time. After all, for every task, it is possible to find someone who is really bad at it, and we should not need to keep repeating we are not talking about _that_ person.

Re: Model intelligence is no longer the constraint for automation

#138

Post by aledalgrande »

I don't agree with the author. Where is the part about nondeterminism and hallucinations? Drawing a pretty chart doesn't make the argument true. All these benchmarks and competitions are on problems that have a _right answer_. I write most my code entirely through Claude at work and have Claude Max for personal, and I can see every day that even with the right context, it's not certain that the model is going to converge to a decent answer on complex real life issues. At least one thing I do agree on: model growth is not an exponential, like everyone thought when we were on the first leg of it, but a logarithmic.

Re: Model intelligence is no longer the constraint for automation

#139

Post by Kuinox »

gjm11 wrote: post #99
Kuinox wrote: post #93

Earlier quoted context omitted.

https://i.imgur.com/l2elIAv.png Difficulty is the amount of digits, small models struggle with 10 digits numbers, gemini and gpt-5 are very good recent models, gemini start failing before 40 digits, GPT-5 (the one by api, the online chat version is worse and I didn't tested it) can do more than 120 digits (at this point it's pointless to test for more).

My tests of GPT-5 were using the online chat version. Of course, I only ran it once; I can't at all rule out the possibility that sometimes it gets it wrong. But, again, the same is true of humans.

The online version is way worse, it also have a router that could route it to a random model.

Re: Model intelligence is no longer the constraint for automation

#140

Post by themanmaran »

patrickhogan1 wrote: post #48
themanmaran wrote: post #43

Earlier quoted context omitted.

This depends on whether you mean LLMs in the sense of single shot, or LLMs + software built around it. I think a lot of people conflate the two. In our application e use a multi-step check_knowledge_base workflow before and after each LLM request. Pretty much, make a separate LLM request to check the query against the existing context to see if more info is needed, and a second check after generation to see if output…

Do you have a concrete example of what you mean? For example, the article above was insightful. But the authors pointing to 1,000s of disparate workflows that could be solved with the right context, without actually providing 1 concrete example of how he accomplishes this makes the post weaker.

Sure, concrete example. We do conversational AI for banks, and spend a lot of time on the compliance side. Biggest thing is we don't want the LLM to ever give back an answer that could violate something like ECOA.

So every message that gets generated by the first LLM is then passed to a second series of LLM requests + a distilled version of the legislation. ex: "Does this message imply likelihood of credit approval (True/False)". Then we can score the original LLM response based on that rubric.

All of the compliance checks are very standardized, and have very little reasoning requirements, since they can mostly be distilled into a series of ~20 booleans.

Post reply on HN