Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5: "How many times does the letter b appear in blueberry?"

bsky.app

Post reply on HN
Print view View original

51–60 of 339 posts

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#51

Post by pavel_lishin »

shadowjones wrote: post #37

I think a lot of those trick questions outputting stupid stuff can be explained by simple economics. It's just not sustainable for OpenAI to run GPT at the best of its abilities on every request. Their new router is not trying to give you the most accurate answer, but a balance of speed/accuracy/sustainable cost on their side. (kind of) a similar thing happened when 4o came out, they often tinkered with it and the re…

> I think a lot of those trick questions outputting stupid stuff can be explained by simple economics.

> It's just not sustainable for OpenAI to run GPT at the best of its abilities on every request.

So how do I find out whether the answer to my question was run on the discount hardware, or whether it's actually correct?

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#52

Post by shadowjones »

exasperaited wrote: post #42
shadowjones wrote: post #37

I think a lot of those trick questions outputting stupid stuff can be explained by simple economics. It's just not sustainable for OpenAI to run GPT at the best of its abilities on every request. Their new router is not trying to give you the most accurate answer, but a balance of speed/accuracy/sustainable cost on their side. (kind of) a similar thing happened when 4o came out, they often tinkered with it and the re…

This is not a demonstration of a trick question. This is a demonstration of a system that delusionally refuses to accept correction and correct its misunderstanding (which is a thing that is fundamental to their claim of intelligence through reasoning). Why would anyone believe these things can reason, that they are heading towards AGI, when halfway through a dialogue where you're trying to tell it that it is wrong i…

It's a trick question for an artificial intelligence that tokenize words. Humans have plenty of different weaknesses.

>Why would you offer up an easy out for them like this? You're not the PR guy for the firm swimming in money paying million dollar bonuses off what increasingly looks, at a fundamental level, like castles in the sand. Why do the labour?

I deeply hate OpenAI and everything it stands for. But I can't deny the fact that they're +/- dominating the market and releasing SOTA models on a regular basis, trying to understand why and how it fails seems important to not get left behind.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#53

Post by jcgrillo »

lucas_membrane wrote: post #11

A couple of weeks ago, I asked google, ordinary google search, how many times the letter r is found in preferred, and it told me 2. This century has taken quite a bitter turn against those of us who think that the 'enough' in 'good enough' ought to exclude products indistinguishable from the most grievously disgraceful products of sloth. But I have also lately realized that human beings, brains, society, culture, edu…

The generation leading this world has all the weapons the previous generation built at their disposal and none of the discipline or education to wield them responsibly. This too shall pass, but how it passes will be interesting to see.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#54

Post by shadowjones »

pavel_lishin wrote: post #51
shadowjones wrote: post #37

I think a lot of those trick questions outputting stupid stuff can be explained by simple economics. It's just not sustainable for OpenAI to run GPT at the best of its abilities on every request. Their new router is not trying to give you the most accurate answer, but a balance of speed/accuracy/sustainable cost on their side. (kind of) a similar thing happened when 4o came out, they often tinkered with it and the re…

> I think a lot of those trick questions outputting stupid stuff can be explained by simple economics. > It's just not sustainable for OpenAI to run GPT at the best of its abilities on every request. So how do I find out whether the answer to my question was run on the discount hardware, or whether it's actually correct?

I'd say use the API, search and high reasoning if you want accuracy.

But then you can partially start to see why it doesn't make economic sense to do this.

Personally I assume that anything I send through their chat UI will run on the cheapest settings they can get away with.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#55

Post by exasperaited »

andrewmcwatters wrote: post #26
amai wrote: post #22

Isn't that just an artifact caused by the tokenization of the training and input data? See https://platform.openai.com/tokenizer https://github.com/openai/tiktoken

No, it's the entire architecture of the model. There's no real reasoning. It seems that reasoning is just a feedback loop on top of existing autocompletion. It's really disingenuous for the industry to call warming tokens for output, "reasoning," as if some autocomplete before more autocomplete is all we needed to solve the issue of consciousness. Edit: Letter frequency apparently has just become another scripted out…

In ten years time an LLM lawyer will lose a legal case for someone who can no longer afford a real lawyer because there are so few left. And it'll be because the layers of bodges in the model caused it to go crazy, insult the judge and threaten to burn down the courthouse.

There will be a series of analytical articles in the mainstream press, the tech industry will write it off as a known problem with tokenisation that they can't fix because nobody really writes code anymore.

The LLM megacorp will just add a disclaimer: the software should not be used in legal actions concerning fruit companies and they disclaim all losses.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#56

Post by minimaxir »

shadowjones wrote: post #52
exasperaited wrote: post #42

Earlier quoted context omitted.

This is not a demonstration of a trick question. This is a demonstration of a system that delusionally refuses to accept correction and correct its misunderstanding (which is a thing that is fundamental to their claim of intelligence through reasoning). Why would anyone believe these things can reason, that they are heading towards AGI, when halfway through a dialogue where you're trying to tell it that it is wrong i…

It's a trick question for an artificial intelligence that tokenize words. Humans have plenty of different weaknesses. >Why would you offer up an easy out for them like this? You're not the PR guy for the firm swimming in money paying million dollar bonuses off what increasingly looks, at a fundamental level, like castles in the sand. Why do the labour? I deeply hate OpenAI and everything it stands for. But I can't de…

It’s a more difficult question for LLMs due to tokenization, but far from a trick one. There’s no word play or ambiguity involved.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#57

Post by JasonBee »

jonnycomputer wrote: post #48

For what it's worth, it got it right when I tried it. >simple question should be easy for a genius like you. have many letter b's in the word blueberry? ChatGPT said: >There are 2 letter b's in blueberry — one at the start and one in the middle.

To me that makes it worse. Why would two people get wildly different answers to a simple factual observation query.

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#58

Post by kiratp »

andrewmcwatters wrote: post #26
amai wrote: post #22

Isn't that just an artifact caused by the tokenization of the training and input data? See https://platform.openai.com/tokenizer https://github.com/openai/tiktoken

No, it's the entire architecture of the model. There's no real reasoning. It seems that reasoning is just a feedback loop on top of existing autocompletion. It's really disingenuous for the industry to call warming tokens for output, "reasoning," as if some autocomplete before more autocomplete is all we needed to solve the issue of consciousness. Edit: Letter frequency apparently has just become another scripted out…

> Edit: Letter frequency apparently has just become another scripted output, like doing arithmetic. LLMs don't have the ability to do this sort of work inherently, so they're trained to offload the task.

Mechanistic research at the leading labs has shown that LLMs actually do math in token form up to certain scale of difficulty.

> This is a real-time, unedited research walkthrough investigating how GPT-J (a 6 billion parameter LLM) can do addition.

https://youtu.be/OI1we2bUseI

Re: GPT-5: "How many times does the letter b appear in blueberry?"

#60

Post by SpicyLemonZest »

amai wrote: post #22

Isn't that just an artifact caused by the tokenization of the training and input data? See https://platform.openai.com/tokenizer https://github.com/openai/tiktoken

It clearly is an artifact of tokenization, but I don’t think it’s a “just”. The point is precisely that the GPT system architecture cannot reliably close the gap here; it’s almost able to count the number of Bs in a string, there’s no fundamental reason you could not build a correct number-of-Bs mapping for tokens, and indeed it often gets the right answer. But when it doesn’t you can’t always correct it with things like chain of thought reasoning.

This matters because it poses a big problem for the (quite large) category of things where people expect LLMs to be useful when they get just a bit better. Why, for example, should I assume that modern LLMs will ever be able to write reliably secure code? Isn’t it plausible that the difference between secure and almost secure runs into some similar problem?

Post reply on HN