Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are still surprisingly bad at some simple tasks

shkspr.mobi

Post reply on HN
Print view View original

61–70 of 107 posts

Re: LLMs are still surprisingly bad at some simple tasks

#61

Post by dgfitz »

jw1224 wrote: post #2

> “To stave off some obvious comments: > yoUr'E PRoMPTiNg IT WRoNg! > Am I though?” Yes. You’re complaining that Gemini “shits the bed”, despite using 2.5 Flash (not Pro), without search or reasoning. It’s a fact that some models are smarter than others. This is a task that requires reasoning so the article is hard to take seriously when the author uses a model optimised for speed (not intelligence), and doesn’t even…

> … all of which appears to be accurate.

Isn’t that the whole goddamn rub? You don’t _know_ if they’re accurate.

Re: LLMs are still surprisingly bad at some simple tasks

#62

Post by anthonylevine »

jstrieb wrote: post #48
mexicocitinluez wrote: post #25

Earlier quoted context omitted.

> When LLMs say something true, it’s a coincidence of the training data that the statement of fact is also a likely sequence of words; Do you know what a "coincidence" actually is? The definition you're using is wrong. It's not a coincidence that I train a model on healthcare regulations and it answers a question about healthcare regulations correctly. None of that is coincidental. If I trained it on healthcare regul…

LLMs are trained on text, only some of which includes facts. It's a coincidence when the output includes new facts not explicitly present in the training data.

> It's a coincidence when the output includes facts,

That's not what a coincidence is.

A coincidence is: "a remarkable concurrence of events or circumstances without apparent causal connection."

Are you saying that training it on a subset of specific data and it responding with that data "does not have a causal connection"> Do you know how statistical pattern matching works?

Re: LLMs are still surprisingly bad at some simple tasks

#63

Post by K0balt »

The training data is not automatically in the context scope, and on list tasks LLMs have nearly no way to ensure completeness due to their fundamental characteristics.

To do a task like this with LLMs, you need to use a document for your source lists or bring them directly into context, then a smart model with good prompting might zero-shot it.

But if you want any confidence in the answer, you need to use tools: “here is two lists, write a python script to find the exact matches, and return a new list with only the exact matches. Write a test dataset and verify that there are no errors, omissions, or duplicates.”

LLMs plus tools / code are amazing. LLMs on their own are a professor with an intermittent heroin problem.

Re: LLMs are still surprisingly bad at some simple tasks

#64

Post by anthonylevine »

delusional wrote: post #52
mexicocitinluez wrote: post #44

Earlier quoted context omitted.

And that's not a coincidence. That's not what the word "coincidence" means. It's a complete misunderstanding of how these tools works.

I don't think you're the right person to make any claim of "complete misunderstanding" when you claim that training an LLM on regulations would produce a system capable of answering questions about that regulation.

> you claim that training an LLM on regulations would produce a system capable of answering questions about that regulation.

Huh? But it does do that? What do you think training an LLM entails?

Are you of the belief that an LLM trained on non-medical data would have the same statical chance of answering a medical question correctly?

we're at the "Redefining what words means in order to not have to admit I was wrong" stage of this argument

Re: LLMs are still surprisingly bad at some simple tasks

#65

Post by edent »

simianwords wrote: post #49

Author seemed to have used a weak model since the strong models get the answer. They should have put more thought into it and at least provide a comparison. As a ChatGPT user I would have reached for the thinking model for such questions. I understand if the “auto” model doesn’t pick the right model here - but confident claims from the author should be backed up by at least this much.

How do you think most people use tools?

Go sit on public transport and look at how people use their devices. They don't fiddle with settings or dive deep into configuration menus.

I literally just opened the tools and used what they gave me. They're sold on the promise that "this thing is really clever and will answer any question!!" so why should I have to spend time futzing with it?

Re: LLMs are still surprisingly bad at some simple tasks

#66

Post by Lapel2742 »

Blahah wrote: post #47
roxolotl wrote: post #27

Earlier quoted context omitted.

That’s the point the author is making. The LLMs don’t have the raw correct information required to accomplish the task so all they can do is provide a plausible sounding answer. And even if it did the way they are architected still can only results in a plausible sounding answer.

Not really. This works great in Claude Sonnet 4.1: 'Please could you research a list of valid TLDs and a list of valid HTML5 elements, then cross reference them to produce a list of HTML5 elements which are also valid TLDs. Use search to find URLs to the lists, then use the analysis tool to write a script that downloads the lists, normalises and intersects them.' Ask a stupid question, get a stupid answer.

> This works great in Claude Sonnet 4.1: 'Please could you research a list of valid TLDs and a list of valid HTML5 elements, then cross reference them to produce a list of HTML5 elements which are also valid TLDs. Use search to find URLs to the lists, then use the analysis tool to write a script that downloads the lists, normalises and intersects them.'

Ok, I only have to:

1. Generally solve the problem for the AI

2. Make a step by step plan for the AI to execute

3. Debug the script I get back and check by hand if it uses reliable sources.

4. Run that script.

For what do I need the AI?

Re: LLMs are still surprisingly bad at some simple tasks

#69

Post by vinc »

The other day I found that they were struggling with "find me two synonyms of 'downloading' and 'extracting' that are the same length" because I was writing a script and wanted to see if could align the next path parameter.

First there's the tokenization issue, the same old "how many R in STRAWBERRY" where they are often confidently wrong, but I also asked not to mix tense (-ing and -ed for example) and that was very hard for them.

Re: LLMs are still surprisingly bad at some simple tasks

#70

Post by lsecondario »

jstrieb wrote: post #14

The point from the end of the post that AI produces output that sounds correct is exactly what I try to emphasize to friends and family when explaining appropriate uses of LLMs. AI is great at tasks where sounding correct is the essence of the task (for example "change the style of this text"). Not so great when details matter and sounding correct isn't enough, which is what the author here seems to have rediscovered…

I like this analogy a lot for non-technical...erm...audiences. I do hope that anyone using this analogy will pair it with loud disclaimers about not anthropomorphizing LLMs; they do not "lie" in any real sense, and I think framing things in those terms can give the impression that you should interpret their output in terms of "trust". The emergent usefulness of LLMs is (currently at least) fundamentally opaque to human understanding and we shouldn't lead people to believe otherwise.
Post reply on HN