Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5: "How many times does the letter b appear in blueberry?"

bsky.app

Post reply on HN
Print view View original

191–200 of 339 posts

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#191

Post by amalcon »

rainsford wrote: post #186
zahlman wrote: post #159

Earlier quoted context omitted.

In an optical illusion, we perceive something that isn't there due to exploiting a correction mechanism that's meant to allow us to make better practical sense of visual information in the average case. Asking LLMs to count letters in a word fails because the needed information isn't part of their sensory data in the first place (to the extent that a program's I/O can be described as "sense"). They reason about text…

I can't take credit for coming up with this, but LLMs have basically inverted the common Sci-Fi trope of the super intelligent robot that struggles to communicate with humans. It turns out we've created something that sounds credible and smart and mostly human well before we made something with actual artificial intelligence. I don't know exactly what to make of that inversion, but it's definitely interesting. Maybe…

Searle seems to have been right: https://en.m.wikipedia.org/wiki/Chinese_room

(Not that I am the first to notice this either)

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#192

Post by hansvm »

zahlman wrote: post #183
hansvm wrote: post #103

Earlier quoted context omitted.

Common misconception. That just means the algorithm for counting letters can't be as simple as adding 1 for every token. The number of distinct tokens is tiny compared to the parameter space, and it's not infeasible to store a mapping from token type to character count in those weights. If you're fine appealing to less concrete ideas, transformers are arbitrary function approximators, tokenization doesn't change that…

> The number of distinct tokens is tiny compared to the parameter space, and it's not infeasible to store a mapping from token type to character count in those weights. You seem to suppose that they actually perform addition internally, rather than simply having a model of the concept that humans sometimes do addition and use it to compute results. Why? > For any finite-length function (like counting letters in a bou…

> You seem to suppose that they actually perform addition internally, rather than simply having a model of the concept that humans sometimes do addition and use it to compute results. Why?

Nothing of the sort. They're _capable_ of doing so. For something as simple as addition you can even hand-craft weights which exactly solve it.

> The problem is that the question space grows exponentially in the length of input. If you want a non-coincidentally-correct answer to "how many t's in 'correct horse battery staple'?" then you need to actually add up the per-token counts.

Yes? The architecture is capable of both mapping tokens to character counts and of addition with a fraction of their current parameter counts. It's not all that hard.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#193

Post by hansvm »

viraptor wrote: post #157
hansvm wrote: post #103

Earlier quoted context omitted.

Common misconception. That just means the algorithm for counting letters can't be as simple as adding 1 for every token. The number of distinct tokens is tiny compared to the parameter space, and it's not infeasible to store a mapping from token type to character count in those weights. If you're fine appealing to less concrete ideas, transformers are arbitrary function approximators, tokenization doesn't change that…

> They just haven't bothered. Or they don't see the benefit. I'm sure they could train the representation of every token and make spelling perfect. But if you have real users spending money on useful tasks already - how much money would you spend on training answers to meme questions that nobody will pay for. They did it once for the fun headline already and apparently it's not worth repeating.

That's just a potential explanation for why they haven't bothered. I don't think we're disagreeing.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#194

Post by kgeist »

mdp2021 wrote: post #169
kgeist wrote: post #158

Earlier quoted context omitted.

Did you enable reasoning? Qwen3 32b with reasoning enabled gave me the correct answer on the first attempt.

> Did you enable reasoning Yep. > gave me the correct answer Try real-world tests that cannot be covered by training data or chancey guesses.

Counting letters is a known blindspot in LLMs because of how tokenization works in most LLMs - they don't see individual letters. I'm not sure it's a valid test to make any far-reaching conclusions about their intelligence. It's like saying a blind person is an absolute dumbass just because they can't tell green from red.

The fact that reasoning models can count letters, even though they can't see individual letters, is actually pretty cool.

>Try real-world tests that cannot be covered by training data

If we don't allow a model to base its reasoning on the training data it's seen, what should it base it on? Clairvoyance? :)

> chancey guesses

The default sampling in most LLMs uses randomness to feel less robotic and repetitive, so it’s no surprise it makes “chancey guesses.” That’s literally what the system is programmed to do by default.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#195

Post by mdp2021 »

skeledrew wrote: post #176
lottin wrote: post #28

Earlier quoted context omitted.

The fact that it can't count.

That isn't a flaw though. Counting is orthogonal to the functioning of LLMs, which are merely completing patterns based on their training data and available context. If you want an LLM to count reliably, give it a tool.

Anything articulate (hence possibly convincing) which could be «merely [guessing]» should either be locked out of consequential questions, or fixed.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#196

Post by themafia »

mikehearn wrote: post #132

This is a well known blindspot for LLMs. It's the machine version of showing a human an optical illusion and then judging their intelligence when they fail to perceive the reality of the image (the gray box example at the top of https://en.wikipedia.org/wiki/Optical_illusion is a good example). The failure is a result of their/our fundamental architecture.

The difference being that you can ask a human to prove it and they'll actually discover the illusion in the process. They've asked the model to prove it and it has just doubled down on nonsense or invented a new spelling of the word. These are not even remotely comparable.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#197

Post by chromaton »

rainsford wrote: post #186
zahlman wrote: post #159

Earlier quoted context omitted.

In an optical illusion, we perceive something that isn't there due to exploiting a correction mechanism that's meant to allow us to make better practical sense of visual information in the average case. Asking LLMs to count letters in a word fails because the needed information isn't part of their sensory data in the first place (to the extent that a program's I/O can be described as "sense"). They reason about text…

I can't take credit for coming up with this, but LLMs have basically inverted the common Sci-Fi trope of the super intelligent robot that struggles to communicate with humans. It turns out we've created something that sounds credible and smart and mostly human well before we made something with actual artificial intelligence. I don't know exactly what to make of that inversion, but it's definitely interesting. Maybe…

Moravec strikes again.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#199

Post by pxc »

zahlman wrote: post #173
pxc wrote: post #127

Earlier quoted context omitted.

Here's an example of what gpt-oss-20b (at the default mxfp4 precision) does with this question: > How many "s"es are in the word "Mississippi"? The "thinking portion" is: > Count letters: M i s s i s s i p p i -> s appears 4 times? Actually Mississippi has s's: positions 3,4,6,7 = 4. The answer is: > The word “Mississippi” contains four letter “s” s. They can indeed do some simple pattern matching on the query, separ…

I'll be impressed when you can reliably give them a random four-word phrase for this test. Because I don't think anyone is going to try to teach them all those facts; even if they're trained to know letter counts for every English word (as the other comment cites as a possibility), they'd then have to actually count and add, rather than presenting a known answer plus a rationalization that looks like counting and add…

You can already do it with arbitrary strings that aren't in the dictionary. But I wonder if the pattern matching will break once strings are much longer than any word in the dictionary, even if there's plenty of room left in context and all that.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#200

Post by throwaway1004 »

rainsford wrote: post #186
zahlman wrote: post #159

Earlier quoted context omitted.

In an optical illusion, we perceive something that isn't there due to exploiting a correction mechanism that's meant to allow us to make better practical sense of visual information in the average case. Asking LLMs to count letters in a word fails because the needed information isn't part of their sensory data in the first place (to the extent that a program's I/O can be described as "sense"). They reason about text…

I can't take credit for coming up with this, but LLMs have basically inverted the common Sci-Fi trope of the super intelligent robot that struggles to communicate with humans. It turns out we've created something that sounds credible and smart and mostly human well before we made something with actual artificial intelligence. I don't know exactly what to make of that inversion, but it's definitely interesting. Maybe…

Very interesting, cognitive atrophy is a serious concern that is simply being handwaved away. Assuming the apparent trend of diminishing returns continues, and LLMs retain the same abilities and limitations we see today, there's a considerable chance that they will eventually achieve the same poor reputation as smartphones and "iPad kids". "Chewing gum for the mind".

Children increasingly speak in a dialect I can only describe as "YouTube voice", it's horrifying to imagine a generation of humans adopting any of the stereotypical properties of LLM reasoning and argumentation. The most insidious part is how the big player models react when one comes within range of a topic it considers unworthy or unsafe for discussion. The thought of humans being in any way conditioned to become such brick walls is frightening.

Post reply on HN