31–33 of 33 posts
Earlier quoted context omitted.
> it frustrates attempts to “detect” if LLMs are used and that is very good. Why is that good?
If you are asking, you’re the kind of person it’s designed to frustrate. Good. Stay frustrated.
If you think I’m doing something I shouldn’t, tell me what you think I’m doing that I shouldn’t, and why I shouldn’t?
Why would you just wish me ill?
Earlier quoted context omitted.
ELI6 why SPAG is better than just the default pretraining method (token context statistics?) of an LLM.
I don't think of SPAG as a replacement for pretraining. For SPAG to work effectively, I would think that it would have to start with an LLM that is pretrained with self-supervised / imitation learning on regular next-token prediction. Think of SPAG as more of a competitor to RLHF than to pretraining. RL is what gave AlphaGo the edge to finally go beyond merely imitating human games, and finally achieve something new.…
(thank you)