LLMs are auto-regressive predictors -- so they take the text given to them (i.e., the prompt) and generate a probability estimate for what the next token should be. Suppose you gave it a quote -- "Once upon a midnight dreary, while I pondered, " and ask it to keep writing, it will generate a probability distribution across various tokens it has been trained on. I'll use words here, rather than tokens, to make the poi…
The idea of an LLM makes sense: it's ultimately a graph with statistical weights for which node we use to generate text next (my understanding here is still correct, yes?).
My issue is, where does the "learning" part come in? It feels like it's all hard-coded but I know it's not. What allows for flexibility in token generation besides that randomization from temperature that you mentioned?