Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Model intelligence is no longer the constraint for automation

latentintent.substack.com

Post reply on HN
Print view View original

111–120 of 156 posts

Re: Model intelligence is no longer the constraint for automation

#111

Post by bwfan123 »

Kuinox wrote: post #15
bt1a wrote: post #9

Earlier quoted context omitted.

i'd wager your benchmark problems require cumbersome arithmetic or are poorly worded / inadequately described. or, you're mislabeling them as basic math and logic (a domain within which LLMs have proven their strengths!) i only call this out because you're selling it and don't hypothesize* on why they fail your simple problems. i suppose an easily aced bench wouldn't be very marketable

This is a simple sum of 2 whole number, the number are simply big. Most of the time they make a correct summation table but fail to copy correctly the sum result into a final result. That is not a tokenisation problem (you can change the output format to make sure of it). I have a separated benchmark that test specifically this, when the input is too large, the LLMs fails to accuratly copy the correct token. I suppos…

This is close to what the apple paper [1] also found on constraint satisfaction problems. As an example, on towers of hanoi, past a frontier, accuracy collapses.

Even when the algorithm steps are laid out precisely, they cannot be followed. Perhaps, LLMs should be trained on turing machine specs and be given a tape lol.

Constraint satisfaction and combinatorics are where the search space is exponential, and the techniques are not formalized (not enough data in training set), and remain hard for machines as seen in the Problem 6 of IMO which could not be solved by LLMs. I suspect, there is this aspect of human intelligence which is not yet captured in LLMs.

[1] - https://machinelearning.apple.com/research/illusion-of-think...

Re: Model intelligence is no longer the constraint for automation

#112

Post by Jensson »

KoolKat23 wrote: post #95
Kuinox wrote: post #87

Earlier quoted context omitted.

Simple maths problems are simple logic problem. Here it doesn't even have to come up with a reasoning, it probably already memorised how to solve sums. Yet it fails at that, it shows it cannot solve logic problems if there are too much steps. > All indications are that it will continue to become smarter. I'm not disputing that, every new model score better at my benchmark, but right now, none truly "solve" one of the…

If it can frame the question for the tool, it therefore has the logic (whether that was static recall or deductive). LLM's struggle with simple maths by nature of their architecture not due to a lack of logic. Yes it struggles with logic questions too but they're not directly related here.

> LLM's struggle with simple maths by nature of their architecture not due to a lack of logic.

No, if it was good at logic it would have overcame that tiny architectural hurdle, its such a trivial process to convert tokens to numbers that it is ridiculous for you to suggest that is the reason it fails at math.

The reason it fails at math is because it fails at logic, and math is the most direct set of logic we have. It doesn't fail at converting between formats, it can convert strawberry to correct Base64 encoding, meaning it does know exactly what letters are there, it just lacks to logic to actually understand what "count letters" means.

Re: Model intelligence is no longer the constraint for automation

#113

Post by antithesizer »

mrlongroots wrote: post #23

I very much disagree. To attempt a proof by contradiction: Let us assume that the author's premise is correct, and LLMs are plenty powerful given the right context. Can an LLM recognize the context deficit and frame the right questions to ask? They can not: LLMs have no ability to understand when to stop and ask for directions. They routinely produce contradictions, fail simple tasks like counting the letters in a wo…

The word you're looking for is "rebuttal" since this is neither proof nor refutation of anything, but merely an argument against the thesis.

Re: Model intelligence is no longer the constraint for automation

#114

Post by bwfan123 »

visarga wrote: post #52

Verification is the bottleneck, not ideation. LLMs can generate anything on tap, but solving any non-trivial problem requires iteration between thinking, doing and observing outcomes. The real world is too complex to be simulated by AI or humans. The scientific method works the same way, we are not exempt from having to validate our ideas. But as humans we have better feedback and access to context and we can assume…

> Verification is the bottleneck

In my use of cursor as a coding assistant, this is the primary problem. The code is 90% on the mark, but still buggy, and needs verification, and the feedback it gets from me is not with full fidelity as something is lost in translation.

But, a bigger issue is that AI has only some solution templates for problems that it is trained on, and being able to generate new templates is beyond its capability as that requires training on datasets of higher levels of abstration.

Re: Model intelligence is no longer the constraint for automation

#115

Post by dash2 »

I'm not sure about the assumption that science is context-free. Maths maybe, but a lot of practical science has tons of unformalized contextual knowledge that is "handed down" by practitioners. It's one reason why replication can be so hard.

OTOH, I also think a lot of science is like 1% inspiration, 99% very mundane tasks like data cleaning. So no reason the AI can't help with that. And scientists write terrible code, so the bar is low :-)

Re: Model intelligence is no longer the constraint for automation

#116

Post by bwfan123 »

HappMacDonald wrote: post #12
dismalaf wrote: post #8

Earlier quoted context omitted.

Tic-tac-toe is solved and a draw can be forced 100% of the time...

.. and you can "program" a neural network — so simple it can be implemented by boxes full of marbles and simple rules about how to interact with the boxes — to learn by playing tictactoe until it always plays perfect games. This is frequently chosen as a lesson in how neural network training even works. But I have a different challenge for you: train a human to play tictactoe, but never allow them to see the game vis…

llm maximalists' apologies:

- but tokens are not letters - but humans fail too - just wait, we are on an S curve to AGI - but your prompt was incorrect - but I tried and here it works

Meanwhile, their claims:

- LLMs are performing at PhD levels. - AGI is around the corner - humanity will be wiped out - situational awareness report

Re: Model intelligence is no longer the constraint for automation

#117

Post by exe34 »

visarga wrote: post #53
themanmaran wrote: post #43

Earlier quoted context omitted.

This depends on whether you mean LLMs in the sense of single shot, or LLMs + software built around it. I think a lot of people conflate the two. In our application e use a multi-step check_knowledge_base workflow before and after each LLM request. Pretty much, make a separate LLM request to check the query against the existing context to see if more info is needed, and a second check after generation to see if output…

> Pretty much, make a separate LLM request to check the query against the existing context to see if more info is needed, and a second check after generation to see if output text exceeded it's knowledge base. They are unreliable at that. They can't reliably judge LLM outputs without access to the environment where those actions are executed and sufficient time to actually get to the outcomes that provide feedback si…

are the different LLMs correlated in what they get wrong? I suspect they are, given how much incest there's been in their training, but if they each have some edge in one particular area, you could use a committee. would cost that much more tokens, obviously.

Re: Model intelligence is no longer the constraint for automation

#118

Post by exe34 »

fmbb wrote: post #74

> It’s because the bottleneck isn’t in intelligence, but in human tasks: specifying intent and context engineering. So the bottleneck is intelligence. Junior engineers are intelligent enough to understand when they don't understand. They interrogate the intent and context of the tasks they are given. This is intelligence. Solving math questions is not intelligence, computers have been better than humans at that for l…

> Junior engineers are intelligent enough to understand when they don't understand. They interrogate the intent and context of the tasks they are given

Eh, I wouldn't apply that as if it's a general thing. yes, the really good ones do. many will equally plough through into the mud with albeit admirable determination.

Re: Model intelligence is no longer the constraint for automation

#119

Post by KoolKat23 »

Jensson wrote: post #112
KoolKat23 wrote: post #95

Earlier quoted context omitted.

If it can frame the question for the tool, it therefore has the logic (whether that was static recall or deductive). LLM's struggle with simple maths by nature of their architecture not due to a lack of logic. Yes it struggles with logic questions too but they're not directly related here.

> LLM's struggle with simple maths by nature of their architecture not due to a lack of logic. No, if it was good at logic it would have overcame that tiny architectural hurdle, its such a trivial process to convert tokens to numbers that it is ridiculous for you to suggest that is the reason it fails at math. The reason it fails at math is because it fails at logic, and math is the most direct set of logic we have.…

It can't see that data so how can it convert it? It can only see the token input.

An analogy (probably poor) is like asking a human to see UV light. We can do so but only with tools or by removing our lense.

The fact that SOTA models (not yet publicly available) can achieve gold at IOM implies otherwise.

Re: Model intelligence is no longer the constraint for automation

#120

Post by exe34 »

parineum wrote: post #35
lll-o-lll wrote: post #30

Earlier quoted context omitted.

With a stable population, post scarcity is surely possible technically . Just invest resource into improving everything that already exists. I also agree that we will never have a post scarcity society; but this is more about humanity than technology.

There will always be scarcity for goods whose value is derived from their scarcity. Maybe food won't be scarce (we wre actually very close to that) and shelter may not be scarce but, even if you invent the replicator, there will still be things that are bespoke.

there are levels of post scarcity. if food, shelter, medicine and leisure are available to all for almost no toil, then we're in post-scarcity. You'll (probably) never have your own planet. You might never be able to convince a certain artist to produce something for you personally.
Post reply on HN