If I recall correctly, there were some papers which suggested that LLMs favor LLM-generated passages over human written ones. I can consistently reproduce this by asking Claude which code snippet it prefers: the one it generated in a different chat, or one that I refactored for my own needs and find more useful. It always picks its own :) I've also experienced that both Claude and Codex routinely include generated we…
I discovered that LLM-generated tokens in the scratchpad were relatively stable, but injected thoughts were frequently ignored and often deleted from the scratchpad within a few turns – even when the injected thought was the literal answer to the puzzle it was stuck at!
A reader[2] then pointed me toward research similar to what you might recall: LLMs interpret text by maintaining activations for input tokens, so text that is not generated by the same LLM will seem "unlikely" to the LLM in a sense, and when given the alternative between likely and unlikely text, it's probably trained to judge the unlikely text as a weird "slip of the mind" and discredit it in favour of the more likely text. I speculate this is part of how they can be useful in the first place, despite their non-determinism.
[1]: https://entropicthoughts.com/getting-an-llm-to-play-text-adv...
[2]: https://entropicthoughts.com/getting-an-llm-to-play-text-adv...