Earlier quoted context omitted.
isn't this very similar or a case of trading space for time or vice versa (e.g in algorithm analysis)
That is certainly an example of it, but I feel like what I'm talking about is more general. In algorithms you tend to be able to say concretely exactly how much space vs time you're trading off. But in a game like Connect 4 or chess it's a lot harder to say exactly how much you gain from, say, adding one more heuristic to your board analysis, or memorizing one more opening subtree. I don't know how to think about tha…
Do the work of indexing, or summarization or compression vs just do it retrieval on the raw records.
It could be embeddings, or presorts or indices.
But I think the parent poster is correct. There’s an inherent write vs read tradeoff as well as memory vs compute. Maybe with some generalization vs specialization.
Idk if im explaining properly. But you can always front load more effort on a narrower problem space and then utilize it on that space downstream.
I think with your llm example there is no code/data split. The llm is caching a solution to a narrow problem by writing the code.