Earlier quoted context omitted.
Counting letters is such a dull test. LLMs generally have a hard time with this question because letters are tokenized before they receive them, and they have to go through an involved reasoning process to figure it out. It's like asking a color blind person what color the street light is, and declaring him unintelligent because he sometimes gets the answer wrong.
I mean, if you don’t want to include tests that LLMs are, by definition, bad at, why don’t we do the same thing for humans?
Because this is a gotcha test and they are trained on a different tokenized transform of the data that makes it hard. As a system they can output it as a string and have an interpreter, as part of the system, count the letters. So no real meaningful deficiency except they maybe don't waste system prompt space to tell it to do that, as a choice.