Earlier quoted context omitted.
I just tested it with Copilot with two random 45 digit numbers and it gets it correct by translating it into Python and running it in the background. When I asked to not use any external tools, it got the first five, the last two, and a hand full more digits in the middle correct, out of 90. It also fails to calculate the 45 intermediate products - one number times one digit from the other - including multiplying by…
When kids learn multiplication, they learn it on paper, not just in their heads. LLMs don’t have access to paper. “Do long arithmetic entirely in your mind” is not a test most humans can pass. Maybe a few savants. This makes me suspect it is not a reliable test of reasoning. Humans also get a training run every night. As we sleep, our brains are integrating our experiences from the day into our existing minds, so we…
Also I am not asking to learn it in one day, you can dump everything that a child would hear and read during primary school into the context. You can even do it interactively, maybe the model has questions.