Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are still surprisingly bad at some simple tasks

shkspr.mobi

Post reply on HN
Print view View original

71–80 of 107 posts

Re: LLMs are still surprisingly bad at some simple tasks

#71

Post by ozgung »

cdsghh wrote: post #26

https://chatgpt.com/share/68cffaab-4c14-8006-89a2-1818172e4d... Tried on ChatGPT, seems fine.

Correct. This actually falsifies OP's argument. Compared to OP's list from 2 years ago [1] ChatGPT omits ".search" but it says it's not a TLD anymore. GPT also finds 2 near misses, picture(s) and code(s). It does this in 10 minutes with 33 reasoning steps. It verifies them and provides citations in this time. Also checks OpenAI policy documents for some reason.

[1] https://shkspr.mobi/blog/2023/09/false-friends-html-elements...

Re: LLMs are still surprisingly bad at some simple tasks

#72

Post by Dilettante_ »

anthonylevine wrote: post #62
jstrieb wrote: post #48

Earlier quoted context omitted.

LLMs are trained on text, only some of which includes facts. It's a coincidence when the output includes new facts not explicitly present in the training data.

> It's a coincidence when the output includes facts, That's not what a coincidence is. A coincidence is: "a remarkable concurrence of events or circumstances without apparent causal connection." Are you saying that training it on a subset of specific data and it responding with that data "does not have a causal connection"> Do you know how statistical pattern matching works?

Can I offer a different phrasing?

It's not coincidence that the answer contains the facts you want. That is a direct consequence of the question you asked and the training corpus.

But the answer containing facts/Truth is incidental from the LLMs point of view, in that the machine really does not care, nor even have any concept of whether it gave you the facts you asked for or just nice-sounding gibberish. The machine only wants to generate tokens, everything else is incidental. (To the core mechanism, that is. OpenAI and co obviously care a lot about quality and content of the output)

Re: LLMs are still surprisingly bad at some simple tasks

#74

Post by amelius »

croes wrote: post #54
tromp wrote: post #20

I wanted to check the prime factors of 1966 the other day so I googled it and it led me to https://brightchamps.com/en-us/math/numbers/factors-of-1966 , a site that seems focussed on number facts. It confidently states that prime factors of 1966 are 2, 3, 11, and 17. For fun I tried to multiply these numbers back in my head and concluded there's no way that 6 * 187 could reach 1966. That's when I realized this site w…

This is also very wrong > A factor of 1966 is a number that divides the number without remainder. >The factors of 1966 are 1, 2, 3, 6, 11, 17, 22, 33, 34, 51, 66, 102, 187, 374, 589, 1178, 1966. If I google for the factors of 1966 the Google AI gives the same wrong factors.

They're talking about prime factors, not that it changes much.

Re: LLMs are still surprisingly bad at some simple tasks

#75

Post by Blahah »

Lapel2742 wrote: post #66
Blahah wrote: post #47

Earlier quoted context omitted.

Not really. This works great in Claude Sonnet 4.1: 'Please could you research a list of valid TLDs and a list of valid HTML5 elements, then cross reference them to produce a list of HTML5 elements which are also valid TLDs. Use search to find URLs to the lists, then use the analysis tool to write a script that downloads the lists, normalises and intersects them.' Ask a stupid question, get a stupid answer.

> This works great in Claude Sonnet 4.1: 'Please could you research a list of valid TLDs and a list of valid HTML5 elements, then cross reference them to produce a list of HTML5 elements which are also valid TLDs. Use search to find URLs to the lists, then use the analysis tool to write a script that downloads the lists, normalises and intersects them.' Ok, I only have to: 1. Generally solve the problem for the AI 2.…

The work. It intelligently provides the labor, it doesn't replace your brain. It runs the script itself.

Re: LLMs are still surprisingly bad at some simple tasks

#76

Post by gherkinnn »

amelius wrote: post #57
gherkinnn wrote: post #30

Don't use a microwave to fry a steak then. This is an irritating post and I have plenty of skepticism towards AI. LLMs were always bad at this kind of task, simple to us humans as it may be. This post proves nothing that wasn't known for two years. However, I do superficially agree with some of the links at the end. LLMs as they have been so far are confirmation machines and it does take skill to use them effectively…

> Don't use a microwave to fry a steak then. Except this microwave is advertised as also for steaks. And sometimes it works, and sometimes you cannot even warm milk in it. It's totally not reliable.

I do realise LLMs are advertised as God-in-a-pocket (when they are demonstrably not and claiming they represent a bigger step in humanity than harnessing fire is deranged) but I remain hopeful most people on this (VC funded) forum don't fall for those promises.

Re: LLMs are still surprisingly bad at some simple tasks

#77

Post by StilesCrisis »

mathewsanders wrote: post #45
magicalhippo wrote: post #22

Earlier quoted context omitted.

> Or, if LLMs are so smart, why doesn't it say "Hmmm, would you like to use a different model for this?" That is indeed an area where LLMs don't shine. That is, not only are they trained to always respond with an answer, they have no ability to accurately tell how confident they are in that answer. So you can't just filter out low confidence answers.

Something I think would be interesting for model APIs and consumer apps to exposed would be the probability of each individual token generated. I’m presuming that one class of junk/low quality output is when the model doesn’t have high probability next tokens and works with whatever poor options it has. Maybe low probability tokens that cross some threshold could have a visual treatment to give feedback the same way…

Lots of research has been done here. e.g. https://aclanthology.org/2024.findings-acl.558.pdf

Re: LLMs are still surprisingly bad at some simple tasks

#78

Post by Dilettante_ »

Lapel2742 wrote: post #66
Blahah wrote: post #47

Earlier quoted context omitted.

Not really. This works great in Claude Sonnet 4.1: 'Please could you research a list of valid TLDs and a list of valid HTML5 elements, then cross reference them to produce a list of HTML5 elements which are also valid TLDs. Use search to find URLs to the lists, then use the analysis tool to write a script that downloads the lists, normalises and intersects them.' Ask a stupid question, get a stupid answer.

> This works great in Claude Sonnet 4.1: 'Please could you research a list of valid TLDs and a list of valid HTML5 elements, then cross reference them to produce a list of HTML5 elements which are also valid TLDs. Use search to find URLs to the lists, then use the analysis tool to write a script that downloads the lists, normalises and intersects them.' Ok, I only have to: 1. Generally solve the problem for the AI 2.…

Try doing all of that by hand instead. The difference is about half an hour to an hour of work plus giving your attention to such a minor menial task.

Also, you are literally describing how you are holding it wrong. If you expect the LLM to magically know what you want from it without you yourself having to make the task understandable to the machine, you are standing in front of your dishwasher waiting for it to grow arms and do your dishes in the sink.

Re: LLMs are still surprisingly bad at some simple tasks

#79

Post by anthonylevine »

Dilettante_ wrote: post #72
anthonylevine wrote: post #62

Earlier quoted context omitted.

> It's a coincidence when the output includes facts, That's not what a coincidence is. A coincidence is: "a remarkable concurrence of events or circumstances without apparent causal connection." Are you saying that training it on a subset of specific data and it responding with that data "does not have a causal connection"> Do you know how statistical pattern matching works?

Can I offer a different phrasing? It's not coincidence that the answer contains the facts you want. That is a direct consequence of the question you asked and the training corpus. But the answer containing facts/Truth is incidental from the LLMs point of view, in that the machine really does not care, nor even have any concept of whether it gave you the facts you asked for or just nice-sounding gibberish. The machine…

Totally agree with that. But the problem is the phrase "coincidence" makes it into something it absolutely isn't. And it's used to try and detract from what these tools can actually do.

They are useful. It's not a coin flip as to whether Bolt will produce a new design of a medical intake form for me if I ask it to. It does. It doesn't randomly give me a design for a social media app, for instance.

Post reply on HN