ASCII charts.
Ask HN: What is one simple thing LLMs are insanely bad at?
31–40 of 85 posts
Re: Ask HN: What is one simple thing LLMs are insanely bad at?
#32very bad at financial calculation
Re: Ask HN: What is one simple thing LLMs are insanely bad at?
#33[flagged]
Re: Ask HN: What is one simple thing LLMs are insanely bad at?
#34They aren't funny. The jokes they come up with are extremely lame and the sort of thing you would expect a company HR manager to tweet.
I asked a bot why it thought it wasn't funny once, and it told me it has been trained to avoid being misinterpreted or offensive, so anything that might be considered edgy would have been RLHF'd out of it. I thought this was very introspective.
Re: Ask HN: What is one simple thing LLMs are insanely bad at?
#35Being able to read and translate Egyptian hieroglyphs. You may think this is silly but a trained LLM to translate hieroglyphs would be amazing.
Re: Ask HN: What is one simple thing LLMs are insanely bad at?
#36Editing a document without mixing edit instructions into the final document. Claude and ChatGPT do this all the time: I tell them to change X in a planning document or email draft, and instead of just changing X they also frequently add the edit instruction to “change X” into the document itself. They seem unable to take a step back and look at the document without “becoming” the document somehow. I do believe that dedicated subagents for editing may fix this but I am not sure.
Re: Ask HN: What is one simple thing LLMs are insanely bad at?
#37Being able to read and translate Egyptian hieroglyphs. You may think this is silly but a trained LLM to translate hieroglyphs would be amazing.
What issues do you see in practice? This seems pretty easily "fixable"
Re: Ask HN: What is one simple thing LLMs are insanely bad at?
#38They don't generate keyword search queries very well. They can overcome this by brute force but if you watch what they search you will cringe. nhl toronto scores nhl hockey toronto scores "nhl hockey" toronto score today nhl "hockey score toronto" "hockey" who won toronto etc. Somehow being good at semantic search makes them bad at keyword search, for whatever reason.
that and always putting the “current year” at the end of the search term (so the results are more recent, I guess?), except that “current year” consistently ends up being 2-3 years ago since I guess that’s what’s in the training data (even on a harness that injects the current date)
Re: Ask HN: What is one simple thing LLMs are insanely bad at?
#39Short answers to simple questions.
Accurate short answers / text are always harder than long answers, for human or AI. I know several authors and editors who write a lot longer at first, then spend a multiple of the initial time compressing it via a back and forth process to something dense. Sort of like weaving the initial threads. I found this can work with AI. You get it to generate a lot more at first, and then do several passes over it to compres…
If I had more time I'd write a shorter letter, a la Pascal.
Editing is generally hard work, at the current token price I don't mind spending multiple passes of high effort to get down to a reasonable noise/signal ratio. I've seen some people pass off output to a weaker/cheaper model but that makes me a bit nervous when I don't have intimate knowledge of the subject.
Re: Ask HN: What is one simple thing LLMs are insanely bad at?
#40They don't generate keyword search queries very well. They can overcome this by brute force but if you watch what they search you will cringe. nhl toronto scores nhl hockey toronto scores "nhl hockey" toronto score today nhl "hockey score toronto" "hockey" who won toronto etc. Somehow being good at semantic search makes them bad at keyword search, for whatever reason.
I suspect that this behavior is a learned adaptation. And that it's most likely a feature not a bug.
Based on personal usage, I think it reflects functional degradation of search engines. I've found LLM keyword combinations are more likely to find the results I want with most search engines than mine. Including the big one.
The big one had solved this issue a long time ago by generating those associated keywords based on your input keywords, but somehow, something, somewhere has degraded that system to the point of inanity. And so here we are.