Earlier quoted context omitted.
See, my thought would have been the opposite: in a situation like this—where nobody tries the thing because “everybody knows” it’s counterproductive—I’d expect AI literature surveys to confidently assert received wisdom. It sounds from the quote like even the researchers thought it was a mistake at first… and that on the basis of the literature PLUS their collective professional wisdom. Now, obviously, they did in fa…
LLMs don't rely completely on received wisdom. The training process works in a higher dimensional space so some data points that might seem unrelated to humans end up being clustered close together if there is a hidden or unrecognized relationship.
To hear it from the researchers, this feels like the sort of finding that, even in retrospect, is non-obvious from existing literature.
I remember hearing scientific progress described in terms of punctuated equilibrium: some Big New Idea, then a bunch of work generalizing that new idea to the rest of the problem space. I could see AI tools speeding up the second type of work: taking a new framework and chewing through everything that came before, in that new light.
But I have a hard time thinking about how the AI techniques could produce novel, surprising outcomes like this one—ones where it’s not just a permutation of existing knowledge, but where it turns out reality actually cuts against the accumulated written knowledge that came before. The “there is magic to be explained here!” aspect of science.