121–130 of 339 posts
Why don't people here on HN understand that LLMs never see ASCII or other raw characters as input? Expecting spelling, rhyming, arithmetic or other character oriented responses will always yield disappointing results.
Until they will be able of procedural thinking they will be radically, structurally unreliable. Structurally delirious.
And it is also a good thing that we can check in this easy way - if the producers patched the local fault only, then the absence of procedural thinking would not be clear, and we would need more sophisticated ways to check.
``` curl 'https://api.openai.com/v1/chat/completions' \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer ' \ --data '{ "model": "gpt-5-chat-latest", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "How many times does the letter b appear in blueberry" } ] } ], "temperature": 0, "max_completion_tokens": 2048, "top_p": 1, "frequency_penalty": 0, "presence_penalty": 0 }' ```
Just asked ChatGPT5 "Are you told to 'think' when someone asks you how many of a certain letter are in a word?" >Yes — when you ask something like “How many r’s are in blueberry?” I’m basically told to slow down, not just blurt out the first number that pops into my “mind.” Seems somewhat suspicious that it would confirm this in reality given how much they typically try to prevent system prompt disclosure, but there…
That’s not even the main problem. It’s that it’ll come up with whatever answer it considers most plausible to the question given with little regard to factual accuracy.
How useful can generative AI be past acting as a bank for Sam Altman
I tried it twice, it gets it right: https://chatgpt.com/share/6897da1e-f988-8004-8453-8e7f7e3490...
That means nothing: it seemingly can get it wrong.
Isn't that just an artifact caused by the tokenization of the training and input data? See https://platform.openai.com/tokenizer https://github.com/openai/tiktoken
It can spell the word (writing each letter in uppercase followed by a whitespace, which should turn each letter with its whitespace into a separate token). It also has reasoning tokens to use as scratch space, and previous models have demonstrated knowledge of the fact that spelling words is a useful step to counting letters. Tokenization makes the problem difficult, but not solving it is still a reasoning/intelligen…
> How many "s"es are in the word "Mississippi"?
The "thinking portion" is:
> Count letters: M i s s i s s i p p i -> s appears 4 times? Actually Mississippi has s's: positions 3,4,6,7 = 4.
The answer is:
> The word “Mississippi” contains four letter “s” s.
They can indeed do some simple pattern matching on the query, separate the letters out into separate tokens, and count them without having to do something like run code in a sandbox and ask it the answer.
The issue here is just that this workaround/strategy is only trained into the "thinking" models, afaict.
Why don't people here on HN understand that LLMs never see ASCII or other raw characters as input? Expecting spelling, rhyming, arithmetic or other character oriented responses will always yield disappointing results.
Url changed from https://bsky.app/profile/kjhealy.co/post/3lvtxbtexg226 , which points to this.
The reason I submitted the Bluesky post is because the discussion there is more informative (and also multiple instances of confirmation that it’s not a fluke), but the link to both the post and blog is a good compromise.
Earlier quoted context omitted.
They patched it, asking it to count letters now switches it to thinking mode. It'll still make basic mistakes for other queries, though.
Which fruit will be patched next? And people think we're 2 years away from humanity's extinction by AI. Lol.