Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5: "How many times does the letter b appear in blueberry?"

bsky.app

Post reply on HN
Print view View original

231–240 of 339 posts

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#231

Post by weregiraffe »

tibbar wrote: post #161
windowshopping wrote: post #144

Earlier quoted context omitted.

What a terrible analogy. Illusions don't fool our intelligence, they fool our senses, and we use our intelligence to override our senses and see it for what it for it actually is - which is exactly why we find them interesting and have a word for them. Because they create a conflict between our intelligence and our senses. The machine's senses aren't being fooled. The machine doesn't have senses. Nor does it have int…

Really? I thought the analogy was pretty good. Here senses refer to how the machines perceive text, IE as tokens that don't correspond 1:1 to letters. If you prefer a tighter comparison, suppose you ask an English speaker how many vowels are in the English transliteration of a passage of Chinese characters. You could probably figure it out, but it's not obvious, and not easy to do correctly without a few rounds of ca…

I can't even

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#232

Post by what »

storus wrote: post #179

This is a tokenizer issue. Just ask it to first spell out the word letter by letter and then ask the same question. Not sure why people use it as a proof of LLMs being stupid. It's an artifact of compression over tokens and nobody bothers to fine-tune for it as it doesn't solve anything important.

If it can spell the word letter by letter when you ask it, it must know what letters are in the word and be able to count them.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#233

Post by seanhunter »

rainsford wrote: post #178
mikehearn wrote: post #132

This is a well known blindspot for LLMs. It's the machine version of showing a human an optical illusion and then judging their intelligence when they fail to perceive the reality of the image (the gray box example at the top of https://en.wikipedia.org/wiki/Optical_illusion is a good example). The failure is a result of their/our fundamental architecture.

Sure, but I think the point is why do LLM's have a blindspot for performing a task that a basic python script could get right 100% of the time using a tiny fraction of the computing power? I think this is more than just a gotcha. LLMs can produce undeniably impressive results, but the fact that they still struggle with weirdly basic things certainly seems to indicate something isn't quite right under the hood. I have…

If you think this is more than just a gotcha that’s because you don’t understand how LLMs are structured. The model doesn’t operate on words it operates on tokens. So the structure of the text in the word that the question relies on has been destroyed by the tokenizer before the model gets a chance to operate on it.

It’s as simple as that- this is a task that exploits the design of llms because they rely on tokenizing words and when llms “perform well” on this task it is because the task is part of their training set. It doesn’t make them smarter if they succeed or less smart if they fail.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#234

Post by seanhunter »

Chinjut wrote: post #172
mikehearn wrote: post #132

This is a well known blindspot for LLMs. It's the machine version of showing a human an optical illusion and then judging their intelligence when they fail to perceive the reality of the image (the gray box example at the top of https://en.wikipedia.org/wiki/Optical_illusion is a good example). The failure is a result of their/our fundamental architecture.

Presumably you are referencing tokenization, which explains the initial miscount in the link, but not the later part where it miscounts the number of "b"s in "b l u e b e r r y".

Do you think “b l u e b e r r y” is not tokenized somehow? Everything the model operates on is a token. Tokenization explains all the miscounts. It baffles me that people think getting a model to count letters is interesting but there we are.

Fun fact, if you ask someone with French, Italian or Spanish as a first language to count the letter “e” in an english sentence with a lot of “e’s” at the end of small words like “the” they will often miscount also because the way we learn language is very strongly influenced by how we learned our first language and those languages often elide e’s on the end of words.[1] It doesn’t mean those people are any less smart than people who succeed at this task — it’s simply an artefact of how we learned our first language meaning their brain sometimes literally does not process those letters even when they are looking out for them specifically.

[1] I have personally seen a French maths PhD fail at this task and be unbelievably frustrated by having got something so simple incorrect.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#235

Post by vrighter »

danpalmer wrote: post #227
mdp2021 wrote: post #122

I have done this test extensively days ago, on a dozen models: no one could count - all of them got results wrong, all of them suggested they can't check and will just guess. Until they will be able of procedural thinking they will be radically, structurally unreliable. Structurally delirious. And it is also a good thing that we can check in this easy way - if the producers patched the local fault only, then the abse…

Gemini 2.5 Flash got it right for me first time. It’s just a few anecdotes, not data, but that’s two examples of first time correctness so certainly doesn’t seem like luck. If you have more general testing data on this I’m keen to see the results and methodology though.

throwing a pair of dice and getting exactly 2 can also happen on the first try. Doesn't mean the dice are a 1+1 calculating machine

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#236

Post by energy123 »

zahlman wrote: post #159
mikehearn wrote: post #132

This is a well known blindspot for LLMs. It's the machine version of showing a human an optical illusion and then judging their intelligence when they fail to perceive the reality of the image (the gray box example at the top of https://en.wikipedia.org/wiki/Optical_illusion is a good example). The failure is a result of their/our fundamental architecture.

In an optical illusion, we perceive something that isn't there due to exploiting a correction mechanism that's meant to allow us to make better practical sense of visual information in the average case. Asking LLMs to count letters in a word fails because the needed information isn't part of their sensory data in the first place (to the extent that a program's I/O can be described as "sense"). They reason about text…

The real criticism should be the AI doesn't say "I don't know.", or even better, "I can't answer this directly because my tokenizer... But here's a python snippet that calculates this ...", so exhibiting both self-awareness of limitations combined with what an intelligent person would do absent that information.

We do seem to be an architectural/methodological breakthrough away from this kind of self-awareness.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#237

Post by KeplerBoy »

xenotux wrote: post #155
mikehearn wrote: post #132

This is a well known blindspot for LLMs. It's the machine version of showing a human an optical illusion and then judging their intelligence when they fail to perceive the reality of the image (the gray box example at the top of https://en.wikipedia.org/wiki/Optical_illusion is a good example). The failure is a result of their/our fundamental architecture.

OpenAI codenamed one of their models "Project Strawberry" and IIRC, Sam Altman himself was taking a victory lap that it can count the number of "r"s in "strawberry". Which I think goes to show that it's hard to distinguish between LLMs getting genuinely better at a class of problems versus just being fine-tuned for a particular benchmark that's making rounds.

It gets strawberry right though, so I guess we are only one project blueberry from getting one step closer to AGI.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#239

Post by flowerthoughts »

mikehearn wrote: post #132

This is a well known blindspot for LLMs. It's the machine version of showing a human an optical illusion and then judging their intelligence when they fail to perceive the reality of the image (the gray box example at the top of https://en.wikipedia.org/wiki/Optical_illusion is a good example). The failure is a result of their/our fundamental architecture.

No need to anthropomorphize. This is a tool designed for language understanding, that is failing at basic language understanding. Counting wrong might be bad, but this seems like a much deeper issue.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#240

Post by spidersouris »

energy123 wrote: post #236
zahlman wrote: post #159

Earlier quoted context omitted.

In an optical illusion, we perceive something that isn't there due to exploiting a correction mechanism that's meant to allow us to make better practical sense of visual information in the average case. Asking LLMs to count letters in a word fails because the needed information isn't part of their sensory data in the first place (to the extent that a program's I/O can be described as "sense"). They reason about text…

The real criticism should be the AI doesn't say "I don't know.", or even better, "I can't answer this directly because my tokenizer... But here's a python snippet that calculates this ...", so exhibiting both self-awareness of limitations combined with what an intelligent person would do absent that information. We do seem to be an architectural/methodological breakthrough away from this kind of self-awareness.

For the AI to say this or to produce the correct answer would be easily achievable with post-training. That's what was done for the strawberry problem. But it's just telling the model what to reply/what tools to use in that exact situation. There's nothing about "self-awareness".
Post reply on HN