Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What is one simple thing LLMs are insanely bad at?

news.ycombinator.com

Post reply on HN
Print view View original

71–80 of 80 posts

Re: Ask HN: What is one simple thing LLMs are insanely bad at?

#71

Post by busyant »

Identifying bird species from photos that I upload.

To be clear, it depends on what your definition of "insanely bad" is.

I'd say ChatGPT/Gemini make egregious mistakes on ~10% of my photo uploads.

I recently uploaded a photo of a short-billed dowitcher and ChatGPT told me that it was a Wilson's snipe, explaining all sorts of details about the legs and tail feathers (neither of which were visible in my pic!).

I then followed up explaining that a Wilson's snipe hadn't been seen at my location since last November (and that Wilson's snipe was out of season at my location) and Chat revised its estimate downward to 85% Wilson's snipe.

Again, I followed up and I revealed the precise location of the bird and ChatGPT said something like "oh yeah, 99% short-billed dowitcher"!

I've had similar experiences w/ Gemini (haven't tested Claude).

Again, 90% success rate is pretty good, but the other 10% of the time, the 2 LLMs that I use fail on species ID and often hallucinate features on bird photos.

edits for typos, plus another example from the same "birding outing" the other day.

I uploaded a very clear photo of a sparrow.

* ChatGPT says "song sparrow"

* I explain, "no way. this sparrow has yellow over its eye and the breast is wrong for song sparrow."

* ChatGPT: Oh yeah, savannah sparrow

* I explain, beak is too big for savannah sparrow.

* ChatGPT: Oh yeah, saltmarsh sparrow.

* I expalin, "no orange on the bird's face."

* ChatGPT: oh yeah, seaside sparrow (finally correct!)

Re: Ask HN: What is one simple thing LLMs are insanely bad at?

#72

Post by humanrebar »

honr wrote: post #18
humanrebar wrote: post #8

Short answers to simple questions.

Accurate short answers / text are always harder than long answers, for human or AI. I know several authors and editors who write a lot longer at first, then spend a multiple of the initial time compressing it via a back and forth process to something dense. Sort of like weaving the initial threads. I found this can work with AI. You get it to generate a lot more at first, and then do several passes over it to compres…

Sometimes it's not hard. "What color is the tongue of a giraffe?" can be answered in less than eight words trivially. Most models will give you paragraphs, bullet points, and followup questions.

Re: Ask HN: What is one simple thing LLMs are insanely bad at?

#73

Post by kichik »

realitysballs wrote: post #58
jampa wrote: post #43

Serious answer: no model ever gets close to writing an architectural floor plan that makes sense. They understand all the rules and best practices, they can (sometimes) spot a bad idea in a floor plan, they can describe a good floor plan. But ask them to make one, even if you give it every detail (even a "node graph" of rooms), they will still output nonsense. Same for text and image models. Floor plans should be the…

10000% , imho opinion core issue is that cd-level architectural plan-sets en masse are overly shielded by design firms and clients. Diffusion/AI vision has a data problem in this regard. Also, LLMs fundamentally lacks a spatial intuition or comprehension of orthographic /sectional drawings.

Don't you have to submit plans to the city for approval? You might be able to get the data from that angle. Some cities let you pull the plans for a fee.

Re: Ask HN: What is one simple thing LLMs are insanely bad at?

#74

Post by antilimit »

nunez wrote: post #50
humanrebar wrote: post #8

Short answers to simple questions.

I've had some success with adding something like "answer in no more than n words" to the prompt

this is my go-to as well using n of sentences, always trailing every session of work/response after whatever it wants to write. essentially a tldr and then I can review details if/when necessary.

Re: Ask HN: What is one simple thing LLMs are insanely bad at?

#75

Post by Yizahi »

elliotto wrote: post #34

They aren't funny. The jokes they come up with are extremely lame and the sort of thing you would expect a company HR manager to tweet. I asked a bot why it thought it wasn't funny once, and it told me it has been trained to avoid being misinterpreted or offensive, so anything that might be considered edgy would have been RLHF'd out of it. I thought this was very introspective.

Oh, that we have already figured out. It's because...

https://www.scribd.com/doc/290970915/The-Jokester-by-Isaac-A...

Re: Ask HN: What is one simple thing LLMs are insanely bad at?

#76

Post by zingababba »

nilsherzig wrote: post #63
jgb1984 wrote: post #61

Following instructions. I've got a modest sized CLAUDE.md containing some simple rules to follow. Things to always do, things to never do. Not a day goes by where Claude Opus violates one or several of the instructions. He keeps making Django multi line template comment bugs. He keeps using -r with ripgrep thinking that means recursive, when actually that's a replacement instruction, he hits that problem several time…

That's a Claude thing btw. Try using a harness which does not inject half a novel of instructions in combination with a different model. I would recommend Pi + GPT 5.6 Luna for a very capable and cheap test. After using Claude (paid by work) for a couple of months, I was amazed how well instruction following works in other setups.

Yes. The orgs that have gone all in on Claude specifically (Claude Enterprise lets say) have an extremely distinct smell to them. It is basically one of things have gone quite off the rails and no one seems to know how to clean it up.

Re: Ask HN: What is one simple thing LLMs are insanely bad at?

#77

Post by lanstin »

Swankivo wrote: post #66

I'm working on generating slides with our own model, and one thing general LLMs can't do is leave empty space. Whitespace is the core of design that actually feels good, but they keep trying to add "distinctive design elements," and you end up with that AI-flavored excess everywhere. They only know how to add. What LLMs seem unusually bad at is taking things away.

LOL: from the RFCs: “In protocol design, perfection has been reached not when there is nothing left to add, but when there is nothing left to take away.” (from https://www.rfc-editor.org/info/rfc1925/ )

Re: Ask HN: What is one simple thing LLMs are insanely bad at?

#78

Post by lanstin »

zarify wrote: post #60

Determining important from human speech. I’ve had a few summaries from meetings I’ve been involved in and there’s always been a real mismatch from what was actually focused on and how it comes across in the summary.

And the interesting details are often wrong or missed.

Re: Ask HN: What is one simple thing LLMs are insanely bad at?

#79

Post by TZubiri »

mojuba wrote: post #46

One unexpected discovery that I have made while building an AI-based system: the LLM's are bad at designing prompts. We tend to think that the AI has some sort of self-knowledge and should be good at designing prompts for itself but it's really not. Been struggling with a task that heavily depended on prompts, ended up rewriting all my prompts from scratch in my own words, and it finally worked. Then every time I ask…

> LLM's are bad at designing prompts.

You are going down a maddening rabbit hole. Prompts are the things humans write, you are building gas town but unironically

Re: Ask HN: What is one simple thing LLMs are insanely bad at?

#80

Post by TZubiri »

sandcat_ wrote: post #16

Video game tips. Constant mistakes and hallucinations, in my experience. Seen this across a lot of different games. Even in really well documented games, such as OSRS (which has multiple fantastic wikis). Anno 1800 was a recent one I had trouble with, using Claude Opus. Completely made up game mechanics. Rainbow Six Siege, too.

Probably not benchmarked on games. Doing so might sacrifice quality on other more important benchmarks, and when it's used for legal and medical purposes, it's the right choice.
Post reply on HN