without getting into theory of mind it's a bit difficult to elaborate, and I don't have the time or the will for that. But the short version is that thinking is interconnected with BEING as well as will, and the Agent has neither, in a philosophically formal sense. The agent is deterministically bound. So it is a fancy Rube Goldberg machine that outputs letters in a way that creates the impression of thought, but it is not thought, in the same way that some birds can mimic human speech without even the slightest hint as to the words' or sentences' meaning, underlying grammar, connotations, subtext, context, intended use, likely effect, etc. Is speech speech if the speaker has no concept whatsoever of said speech's content, and can not use it to actualize itself? I'd say no. It's mimicry, but not speech. So that means speech is something more than just its outward aspect - the words. It is the relation of something invisible, some inner experience known only to the speaker, VIA the words.
Whereas a gorilla who learns sign language to communicate and use that communication to achieve aims which have direct correlation with its sense of self - that's thought in the Cogito, Ergo Sum sense of the word.
Thought as commonly concieved by the layman is a sort of isolated phenomenon that is mechanical in nature and can be judged by its outward effects; whereas in the philosophical tradition defining thought is known to be one of the hard questions for its mysterious qualia of being interconnected with will and being as described above.
Guess I gave you the long answer. (though, really, it could be much longer than this.) The Turing Test touches on this distinction between the appearance of thought and actual thought.
The question goes all the way down to metaphysics; some (such as myself) would say that one must be able to define awareness (what some call consciousness - though I think that term is too loaded) before you can define thought. In fact that is at the heart of the western philosophical tradition; and the jury consensus remains elusive after all these thousands of years.