Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5: "How many times does the letter b appear in blueberry?"

bsky.app

Post reply on HN
Print view View original

321–330 of 339 posts

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#321

Post by ehsankia »

anon_e-moose wrote: post #279
ehsankia wrote: post #184

Earlier quoted context omitted.

Analogies are just that, they are meant to put things in perspective. Obviously the LLM doesn't have "senses" in the human way, and it doesn't "see" words, but the point is that the LLM perceives (or whatever other word you want to use here that is less anthropomorphic) the word as a single indivisible thing (a token). In more machine learning terms, it isn't trained to autocomplete answers based on individual letter…

> Obviously the LLM doesn't have "senses" in the human way, and it doesn't "see" words > A different analogy could be, imagine a being that had a sense that you "see" magnetic lines, and they showed you an object and asked you If my grandmother had wheels she would have been a bicycle. At some point to hold the analogy, your mind must perform so many contortions that it defeats the purpose of the analogy itself.

> If my grandmother had wheels she would have been a bicycle.

That's irrelevant here, that was someone trying to convert one dish into another dish.

> your mind must perform so many contortions that it defeats the purpose

I disagree, what contortions? The only argument you've provided is that "LLMs don't have senses". Well yes, that's the whole point of an analogy. I still hold that the way LLMs interpret tokens is analogous to a "sense".

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#323

Post by skeledrew »

mdp2021 wrote: post #316
skeledrew wrote: post #302

Earlier quoted context omitted.

> Following patterns is what an hearsay machine does. That's also how the brain works, at least partially. Primary differences are it takes and processes (trains itself on) raw sensory data instead of character tokens, and it continually does so for every conscious moment from at least birth until death.

> how the brain works, at least partially With the difference, which have us go back to the original point, that the human mind has a crucial property of going beyond "pattern-based" intuition and check mental items lucidly and consciously. > and it continually does so It also persistently evaluates consciously and "store" and "learn" (which must be noted because it is the second main thing that LLMs don't do, after…

> check mental items lucidly and consciously

Capabilities that evolved over millennia. We don't even have a decent, universally-agreed upon definition for consciousness yet.

> "store" and "learn"

Actually there are tools for that. Again, the core LLM functionality is best left on its own, and augmented on the fly with various tools which can be easily specialized and upgraded independently of the model. Consider too that the brain itself has multiple sections dedicated to different kinds of processing, instead of anything just happening anywhere.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#324

Post by mdp2021 »

skeledrew wrote: post #323
mdp2021 wrote: post #316

Earlier quoted context omitted.

> how the brain works, at least partially With the difference, which have us go back to the original point, that the human mind has a crucial property of going beyond "pattern-based" intuition and check mental items lucidly and consciously. > and it continually does so It also persistently evaluates consciously and "store" and "learn" (which must be noted because it is the second main thing that LLMs don't do, after…

> check mental items lucidly and consciously Capabilities that evolved over millennia. We don't even have a decent, universally-agreed upon definition for consciousness yet. > "store" and "learn" Actually there are tools for that. Again, the core LLM functionality is best left on its own, and augmented on the fly with various tools which can be easily specialized and upgraded independently of the model. Consider too…

> Capabilities that evolved over millennia

Means nothing. Now they are urgent.

> consciousness

I meant "conscious" as the wake opposed to deliriousness, as the ability "to be sure about what you have in front of you with the degree of accomplished clarity and substantially unproblematic boundaries of definition",

not as that quality that intrigues (and obsesses) less pragmatic intellectuals in directions e.g. at the "Closer to Truth" channel.

When I ask somebody, it has to be sure to a high degree. When implementing a mind, the property of "lucid conscious check" is fundamental.

> tools for that

The "consciously check, then store and learn" is structural in the proper human mental process - a high level functioning, not just a module; i.e. "it's what we do".

Which means, the basic LLM architecture is missing important features that we need if we want to implement a developed interlocutor. And we need and want that.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#325

Post by alternatex »

jcgrillo wrote: post #318
alternatex wrote: post #312

Earlier quoted context omitted.

They believe in waiting for their stock to vest, just as anyone else in their uniquely fortunate position would.

What good is OpenAI stock? Is anyone buying it?

I mean vested stock doesn't necessarily need to go up in value. In some places the stock comp is higher than the actual pay, so even if you sell on the day of vesting you're doubling your income. So the investment viability of stock comp doesn't matter too much, it only matters how much of it you receive.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#326

Post by davidguetta »

visarga wrote: post #301
davidguetta wrote: post #292

Earlier quoted context omitted.

I can do it if I write the word once and look at it, which is exactly what a transformer based llm is supposed to do.

It sees tokens not letters like us. And has to recite tokens in reverse order, and their letters in reverse order, over a set of 200K tokens. Token codes are arbitrary numbers associated with word fragments, they convey no letters.

This is true but not as impacting as you think. Ask GPT to rewrite ANY text with spaces between lettres and he will do it. He DOES know how to spell and read letters

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#327

Post by OrangeMusic »

Minor49er wrote: post #308

I'm not sure if someone did this already, but I reproduced this in GPT-5 and asked it to explain how it got this wrong. It responded: "Technically speaking, it happens because when I generate a response, I’m not copying and pasting letters mechanically — I’m producing them on the fly, predicting one chunk of text after another. When I tried to highlight the b’s in “blueberry,” my output process went something like th…

Reminder that LLMs are just as bad at explaining how they work, as they are at counting the number of Bs in blueberry.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#328

Post by dvrj101 »

kgeist wrote: post #158
mdp2021 wrote: post #122

I have done this test extensively days ago, on a dozen models: no one could count - all of them got results wrong, all of them suggested they can't check and will just guess. Until they will be able of procedural thinking they will be radically, structurally unreliable. Structurally delirious. And it is also a good thing that we can check in this easy way - if the producers patched the local fault only, then the abse…

Did you enable reasoning? Qwen3 32b with reasoning enabled gave me the correct answer on the first attempt.

2b granite model can do this in first attempt

ollama run hf.co/ibm-granite/granite-3.3-2b-instruct-GGUF:F16 >>> how many b’s are there in blueberry? The word "blueberry" contains two 'b's.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#329

Post by dvrj101 »

andai wrote: post #18

My phone still has gpt-4o which gets it right: https://files.catbox.moe/0yg6cu.jpg But my browser has gpt-5 which says 3: https://files.catbox.moe/63qkce.jpg Claude spells it out letter by letter: https://files.catbox.moe/f1irfx.jpg So I thought GPT-5 Thinking might get it right, and it does: https://files.catbox.moe/xlchnr.jpg It refuses to show the thinking process for this question though, so its unclear if it eve…

tested with 2b and 4b models https://files.catbox.moe/6q0v4t.png https://files.catbox.moe/5t32fl.png

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#330

Post by jcgrillo »

alternatex wrote: post #325
jcgrillo wrote: post #318

Earlier quoted context omitted.

What good is OpenAI stock? Is anyone buying it?

I mean vested stock doesn't necessarily need to go up in value. In some places the stock comp is higher than the actual pay, so even if you sell on the day of vesting you're doubling your income. So the investment viability of stock comp doesn't matter too much, it only matters how much of it you receive.

well since openai is private probably the employees are getting stock options not RSUs--which, since it's private, might be hard to sell even if they do exercise them..
Post reply on HN