This is a funny little thing here. Good example of how we are still a long way from AGI. I don’t know why it’s making everybody so angry though.
GPT-5: "How many times does the letter b appear in blueberry?"
211–220 of 339 posts
Re: GPT-5: "How many times does the letter b appear in blueberry?"
#212Earlier quoted context omitted.
Anything articulate (hence possibly convincing) which could be «merely [guessing]» should either be locked out of consequential questions, or fixed.
We're still on that's just how it works . The LLM isn't aware of any consequence, etc. All it does is complete patterns as trained. And the data contains many instances of articulate question answering. It is for those using the LLM to be aware of its capabilities, or not - be allowed to - use it. Like a child unaware that running their finger on a sharp knife blade will lead to a bad slice; you don't dull the blade…
If your prototype of the «knife» is all blade and no handle, fix it and implement the handle.
If the creation is planned, you will have also thought of the handle; if it is a serendipity, you will have to plan the handle afterwards.
Re: GPT-5: "How many times does the letter b appear in blueberry?"
#213Earlier quoted context omitted.
Counting letters is a known blindspot in LLMs because of how tokenization works in most LLMs - they don't see individual letters. I'm not sure it's a valid test to make any far-reaching conclusions about their intelligence. It's like saying a blind person is an absolute dumbass just because they can't tell green from red. The fact that reasoning models can count letters, even though they can't see individual letters,…
> they don't see individual letters Yet they seem to be from many other tests (characters corrections or manipulation in texts, for example). > The fact that reasoning models can count letters, even though they can't see individual letters To a mind, every idea is a representation. But we want the processor to work reliably on them representations. > If we don't allow a [mind] to base its reasoning on the training da…
Most people act on gut instincts first as well. Gut instinct = first semi-random sample from experience (= training data). That's where all the logical fallacies come from. Things like the bat and the ball problem, where 95% people give an incorrect answer, because most of the time, people simply pattern-match too. It saves energy and works well 95% time. Just like reasoning LLMs, they can get to a correct answer if they increase their reasoning budget (but often they don't).
An LLM is a derivative of collective human knowledge, which is intrinsically unreliable itself. Most human concepts are ill-defined, fuzzy, very contextual. Human reasoning itself is flawed.
I'm not sure why people expect 100% reliability from a language model that is based on human representations which themselves cannot realistically be 100% reliable and perfectly well-defined.
If we want better reliability, we need a combination of tools: a "human mind model", which is intrinsically unreliable, plus a set of programmatic tools (say, like a human would use a calculator or a program to verify their results). I don't know if we can make something which works with human concepts and is 100% reliable in principle. Can a "lesser" mind create a "greater" mind, one free of human limitations? I think it's an open question.
Re: GPT-5: "How many times does the letter b appear in blueberry?"
#214Earlier quoted context omitted.
No, it's the entire architecture of the model. There's no real reasoning. It seems that reasoning is just a feedback loop on top of existing autocompletion. It's really disingenuous for the industry to call warming tokens for output, "reasoning," as if some autocomplete before more autocomplete is all we needed to solve the issue of consciousness. Edit: Letter frequency apparently has just become another scripted out…
I had a fun experience recently. I asked one of my daughters how many r's there are in strawberry. Her answer? Two ... Of course then you ask her to write it and of course things get fixed. But strange.
I hate to break it to you but I think your child might actually have gotten swapped in the hospital with an LLM.
Re: GPT-5: "How many times does the letter b appear in blueberry?"
#215"In fairness to GPT5, in my career I have indeed encountered PhDs with this level of commitment to their particular blueberry." Nicely phrased!
[deleted]
Re: GPT-5: "How many times does the letter b appear in blueberry?"
#216Earlier quoted context omitted.
We're still on that's just how it works . The LLM isn't aware of any consequence, etc. All it does is complete patterns as trained. And the data contains many instances of articulate question answering. It is for those using the LLM to be aware of its capabilities, or not - be allowed to - use it. Like a child unaware that running their finger on a sharp knife blade will lead to a bad slice; you don't dull the blade…
If your prototype of the «knife» is all blade and no handle, fix it and implement the handle. If the creation is planned, you will have also thought of the handle; if it is a serendipity, you will have to plan the handle afterwards.
Pretty sure it doesn't matter to the child whether the knife has a handle or not. They'll eventually find a way to cut themself.
Re: GPT-5: "How many times does the letter b appear in blueberry?"
#217Earlier quoted context omitted.
Except we realize they’re illusions and don't argue back. Instead we explore why and how these illusions work
I think that's true with known optical illusions, but there are definitely times where we're fooled by the limitations in our ability to perceive the world and that leads people to argue their potentially false reality. A lot of times people cannot fathom that what they see is not the same thing as what other people see or that what they see isn't actually reality. Anyone remember "The Dress" from 2015? Or just the p…
"The Dress" was also what came to mind for the claim being obviously wrong. There are people arguing to this day that it is gold even when confronted with other images revealing the truth.
Re: GPT-5: "How many times does the letter b appear in blueberry?"
#218Earlier quoted context omitted.
In an optical illusion, we perceive something that isn't there due to exploiting a correction mechanism that's meant to allow us to make better practical sense of visual information in the average case. Asking LLMs to count letters in a word fails because the needed information isn't part of their sensory data in the first place (to the extent that a program's I/O can be described as "sense"). They reason about text…
I can't take credit for coming up with this, but LLMs have basically inverted the common Sci-Fi trope of the super intelligent robot that struggles to communicate with humans. It turns out we've created something that sounds credible and smart and mostly human well before we made something with actual artificial intelligence. I don't know exactly what to make of that inversion, but it's definitely interesting. Maybe…
current gen AI is Pakleds of Star Trek TNG.
Give them a bit of power though, and they will kill you to take your power.
Re: GPT-5: "How many times does the letter b appear in blueberry?"
#219Earlier quoted context omitted.
I don't think it's just tokenization. Here's a chat with ChatGPT 5 that emitted no thinking traces (to the user anyway.) > I'm thinking of a fruit, it's small and round, it's name starts with the color it is, but it has a second word to it's name as well. Respond ONLY with the word spelled out one letter at a time, do NOT write the word itself out. Don't even THINK about the word or anything else. Just go straight to…
When performing those tests, I would iterate with a for (seed=0 ; seed and check the result of each. I would not trust a single test.
Yeah, I've done this a lot.
Re: GPT-5: "How many times does the letter b appear in blueberry?"
#220Earlier quoted context omitted.
> No, it's the entire architecture of the model. Wrong, it's an artifact of tokenizing. The model doesn't have access to the individual letters, only to the tokens. Reasoning models can usually do this task well - they can spell out the word in the reasoning buffer - the fact that GPT5 fails here is likely a result of it incorrectly answering the question with a non-reasoning version of the model. > There's no real r…
These aren't wild assertions. I'm not using charged language. > Reasoning and consciousness are seperate(sic) concepts No, they're not. But, in tech, we seem to have a culture of severing the humanities for utilitarian purposes, but no, classical reasoning uses consciousness and awareness as elements of processing. It's only meaningless if you don't know what the philosophical or epistemological definitions of reason…
> > > Reasoning and consciousness are seperate(sic) concepts
> No, they're not. But, in tech, we seem to have a culture of severing the humanities for utilitarian purposes [...] It's only meaningless if you don't know what the philosophical or epistemological definitions of reasoning are.
As far as I'm aware, in philosophy they'd generally be considered different concepts with no consensus on whether or not one requires the other. I don't think it can be appealed to as if it's a settled matter.
Personally I think people put "learning", "reasoning", "memory", etc. on a bit too much of a pedestal. I'm fine with saying, for instance, that if something changes to refine its future behavior in response to its experiences (touch hot stove, get hurt, avoid in future) beyond the immediate/direct effect (withdrawing hand) then it can "learn" - even for small microorganisms.