I agree it is possible to build an LLM to play poker, with appropriate tool calling, in principle.
I think it's useful to distinguish what LLMs can do in a) theory, b) non-LLM approaches we know work and c) how to do it with LLMs.
In a) theory, LLMs with the "thinking" rollouts are equivalent to (finite-tape) Turing machine, so they can do anything a computer can, so a solution exists (given large-enough neural net/rollout). To do the sampling, I agree the LLM can use an external tool call. This a good start!
For b) to achieve strong performance in poker, we know you can do continual resolving (e.g. search + gadget)
For c) "Quantization" as you suggested is an interesting approach, but it goes against the spirit of "let's have a big neural net that can do any general task". You gave an example how to quantize for a state that has 2 actions. But what about 3? 4? Or N?
So in practice, to achieve such generality, you need to output in the token space.
On top of that, for poker, you'd need LLM to somehow implement continual resolving/ReBeL (for equilibrium guarantees). To do all of this, you need either i) LLM call the CPU implementation of the resolver or ii) the LLM to execute instructions like a CPU.
I do believe i) is practically doable today, to e.g. finetune an LLM to incorporate value function in its weights and call a resolver tool, but not something ChatGPT and others can do (to come to my original parent post).
Also, in such finetuning process, you will likely trade-off the LLM generality for specialization.
> you can do a k-NN or some other simple approximation. [..] You can say that about any other game then, no?
Yes, you can approximate value function with any model (k-NN, neural net, etc).
> In poker if you call 25% more or 35% more if the bet size is 20% smaller is unlikely to result in a huge blunder. Chess is more volatile and thus you need more "precision" telling patterns apart.
I see. The same applies for Chess however -- you can play mixed strategies there too, with similar property - you can linearly interpolate expected value between losing (-1) and winning (1).
Overall, I think being able to incorporate a value function within an LLM is super interesting research, there are some works there, e.g. Cicero [6], and certainly more should be done, e.g. have a neural net to be both a language model and be able to do AlphaZero-style search.
[6] https://www.science.org/doi/10.1126/science.ade9097