71–80 of 339 posts
Q: How many times does the letter b appear in blueberry? The letter “b” appears 3 times in the word blueberry: • 1st b – at the beginning: Blueberry • 2nd b – in the middle: blueberry • 3rd b – again in the middle: blueberry (wait… no — actually…) Let’s break it down carefully: blueberry → b, l, u, e, b, e, r, r, y Letters: 1. b 2. l 3. u 4. e 5. b 6. e 7. r 8. r 9. y So the letter b appears twice. Final answer: 2 ti…
I think the concrete issue this points to is the thing that dynamically decides when to use reasoning failed to choose it in this instance. Sam Altman said it was broken on release day.
Even if it’s pointing to a weaker GPT-5 like gpt-5-nano, it should still be able to answer this question correctly.
Earlier quoted context omitted.
Even if it’s pointing to a weaker GPT-5 like gpt-5-nano, it should still be able to answer this question correctly.
If you know how GPT architectures work, why would you think this?
Nicely phrased!
These are always amazing when juxtaposed with apparently impressive LLM reasoning, knowledge, and creativity. You can trivially get them to make the most basic mistakes about words and numbers, and double down on those mistakes, repeatedly explaining that they're totally correct. Have any systems tried prompting LLMs with a warning like "You don't intuitively or automatically know many facts about words, spelling, or…
You can’t just prompt your way out of a systemic flaw
For things where factuality is even just a little important, you need to treat these things like asking a toddler that got their hands on a thesaurus and an encyclopaedia (that's a few years out of date): go through everything it produces and fact check any statement it makes that you're not confident about already.
Unfortunately, people seem to be mistaking LLMs for search engines more and more (no doubt thanks to attempts from LLM companies to make people think exactly that) so this will only get worse in the future. For now we can still catch these models out with simple examples, but as AI fuckups grow sparser, more people will think these things tell the actual truth.
I tried and was unable to replicate. Me: How many R's in strawberry ChatGPT said: 3 Me: How many B's in blueberry? ChatGPT said: 2 Me: How many C's in coconut? ChatGPT said: 2 Me: How many D's in Diamond? ChatGPT said: 2 Me: How many A's in Banana? ChatGPT said: 3 https://chatgpt.com/share/6897cc40-6650-8006-aae3-ea2b8278d5...
They patched it, asking it to count letters now switches it to thinking mode. It'll still make basic mistakes for other queries, though.
And people think we're 2 years away from humanity's extinction by AI. Lol.
Earlier quoted context omitted.
No, it's the entire architecture of the model. There's no real reasoning. It seems that reasoning is just a feedback loop on top of existing autocompletion. It's really disingenuous for the industry to call warming tokens for output, "reasoning," as if some autocomplete before more autocomplete is all we needed to solve the issue of consciousness. Edit: Letter frequency apparently has just become another scripted out…
> No, it's the entire architecture of the model. Wrong, it's an artifact of tokenizing. The model doesn't have access to the individual letters, only to the tokens. Reasoning models can usually do this task well - they can spell out the word in the reasoning buffer - the fact that GPT5 fails here is likely a result of it incorrectly answering the question with a non-reasoning version of the model. > There's no real r…
> Reasoning and consciousness are seperate(sic) concepts
No, they're not. But, in tech, we seem to have a culture of severing the humanities for utilitarian purposes, but no, classical reasoning uses consciousness and awareness as elements of processing.
It's only meaningless if you don't know what the philosophical or epistemological definitions of reasoning are. Which is to say, you don't know what reasoning is. So you'd think it was a meaningless statement.
Do computers think, or do they compute?
Is that a meaningless question to you? I'm sure given your position it's irrelevant and meaningless, surely.
And this sort of thinking is why we have people claiming software can think and reason.
Earlier quoted context omitted.
This is a great idea. Like, if someone asked me to count the number of B's in your paragraph, I'd yeet it through `grep -o 'B' file.txt | wc -l` or similar, why would I sit there counting it by hand? As a human, if you give me a number on screen like 100000000, I can't be totally sure if that's 100 Million or 1 Billion without getting close and counting carefully. Should ought have my glasses. Mouse pointer helps som…
If you have to build an MCP for every system you aren’t building intelligence in the first place.