Earlier quoted context omitted.
You are making a very popular mistake: confusing the training procedure for LLMs (autocomplete) with "how they work"/their internal ontology (mostly unknown). When we teach children how to do arithmetic, we have them predict missing items in equations. We don't accuse them of "only doing autocomplete". The same applies for LLMs.
I was implementing my own transformer-based models and fine tuning GPT-2 in 2019, and I've kept up with every development since then. I understand the internal structure of these things better than nearly all of the "AI Engineers" who are currently working on wrapping them up as black boxes embedded in applications. I'm not making a "popular mistake", I'm literally describing how inference is done.
To make it even more concrete.... I have an LLM where I removed the last layer and fine tuned on a classification problem and the last layer now only has two outputs rather than vocab size outputs. The goal is binary classification. The output is not a completion of an idea or anything of the sort. It's still an LLM. It still works the same way all the way up until the last layer. The weights are the same all the way up to the last layer. It works because an LLM has to create a rich understanding of how a bunch of concepts work together.