Live data from Hacker News

GPT-5: "How many times does the letter b appear in blueberry?"

bsky.app

131–140 of 339 posts

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#131
post #109

Is scrolling down the page on this website extremely laggy for anyone else? It's bizarre

Actual scrolling seems normal speed, more or less, but it sorta looks rough (almost like dropped FPS or something). Using Fennec F-Droid (Firefox mobile). One quick thumb flick still gets me between the top and bottom, though.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#132
This is a well known blindspot for LLMs. It's the machine version of showing a human an optical illusion and then judging their intelligence when they fail to perceive the reality of the image (the gray box example at the top of https://en.wikipedia.org/wiki/Optical_illusion is a good example). The failure is a result of their/our fundamental architecture.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#133
post #22

Isn't that just an artifact caused by the tokenization of the training and input data? See https://platform.openai.com/tokenizer https://github.com/openai/tiktoken

It can spell the word (writing each letter in uppercase followed by a whitespace, which should turn each letter with its whitespace into a separate token). It also has reasoning tokens to use as scratch space, and previous models have demonstrated knowledge of the fact that spelling words is a useful step to counting letters. Tokenization makes the problem difficult, but not solving it is still a reasoning/intelligen…

> It also has reasoning tokens to use as scratch space

For GPT 5, it would seem this depends on which model your prompt was routed to.

And GPT 5 Thinking gets it right.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#134

Why don't people here on HN understand that LLMs never see ASCII or other raw characters as input? Expecting spelling, rhyming, arithmetic or other character oriented responses will always yield disappointing results.

Because the damn things are marketed under the word "intelligence". That word used to mean something.

It's an umwelt problem. Bats think we're idiots because we don't hear ultrasonic sound, and thus can't echolocate. And we call the LLMs idiots because they consume tokenized inputs, and don't have access to the raw character stream.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#135

Why don't people here on HN understand that LLMs never see ASCII or other raw characters as input? Expecting spelling, rhyming, arithmetic or other character oriented responses will always yield disappointing results.

And which other objectual ideas cannot they instance? Their task is to check, for all important mental activities - world simulation, "telling yourself reliable stories: that is what intelligence is" (Prof. Patrick Winston).

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#136

Earlier quoted context omitted.

Except a human would say "I don't know" instead up making up some nonsense.

It's ironic that saying "I don't know" could be the ultimate sign of superior intelligence just like Socrates told us millennia ago.

Two and a half millennia ago (he died 2424 years ago)

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#137
post #131
post #109

Is scrolling down the page on this website extremely laggy for anyone else? It's bizarre

Actual scrolling seems normal speed, more or less, but it sorta looks rough (almost like dropped FPS or something). Using Fennec F-Droid (Firefox mobile). One quick thumb flick still gets me between the top and bottom, though.

on Firefox on my older Windows laptop it's like 5fps. Maybe mostly a Firefox thing?

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#138

Why don't people here on HN understand that LLMs never see ASCII or other raw characters as input? Expecting spelling, rhyming, arithmetic or other character oriented responses will always yield disappointing results.

Because the damn things are marketed under the word "intelligence". That word used to mean something.

What did it used to mean? I was under the impression that it has always be a little vague.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#139
post #98

Just asked ChatGPT5 "Are you told to 'think' when someone asks you how many of a certain letter are in a word?" >Yes — when you ask something like “How many r’s are in blueberry?” I’m basically told to slow down, not just blurt out the first number that pops into my “mind.” Seems somewhat suspicious that it would confirm this in reality given how much they typically try to prevent system prompt disclosure, but there…

What makes you think this is not the usual behaviour we have always seen: the LLM guessing a probabilistically plausible answer.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#140

This is a well known blindspot for LLMs. It's the machine version of showing a human an optical illusion and then judging their intelligence when they fail to perceive the reality of the image (the gray box example at the top of https://en.wikipedia.org/wiki/Optical_illusion is a good example). The failure is a result of their/our fundamental architecture.

Except we realize they’re illusions and don't argue back. Instead we explore why and how these illusions work
Post reply on HN