That higher level does exist, indeed a lot philosophy of mind then cognitive science has been investigating exactly this space and devising contested professional nomenclature and modeling about such things for decades now.
A useful anchor concept is that of world model, which is what "learning Othello" and similar work seeks to tease out.
As someone who worked in precisely these areas for years and has never stopped thinking about them,
I find it at turns perplexing, sigh-inducing, and enraging, that the "token prediction" trope gained currency and moreover that it continues to influence people's reasoning about contemporary LLM, often as subtext: an unarticulated fundamental model, which is fundamentally wrong in its critical aspects.
It's not that this description of LLM is technically incorrect; it's that it is profoundly _misleading_ and I'm old enough and cynical enough to know full well that many of those who have amplified it and continue to do so, know this very well indeed.
Just as the lay person fundamentally misunderstands the relationship between "programming" and these models, and uses slack language in argumentation, the problem with this trope and the reasoning it entails is that what is unique and interesting and valuable about LLM for many applications and interests is how they do what they do. At that level of analysis there is a very real argument to be made that the animal brain is also nothing more than an "engine of prediction," whether the "token" is a byte stream or neural encoding is quite important but not nearly important as the mechanics of the system which operates on those tokens.
To be direct, it is quite obvious that LLM have not only vestigial world models, but also self-models; and a general paradigm shift will come around this when multimodal models are the norm: because those systems will share with we animals what philosophers call phenomenology, a model of things as they are "perceived" through the senses. And like we humans, these perceptual models (terminology varies by philosopher and school...) will be bound to the linguistic tokens (both heard and spoken, and written) we attach to them.
Vestigial is a key word but an important one. It's not that contemporary LLM have human-tier minds, nor that they have animal-tier world modeling: but they can only "do what they do" because they have such a thing.
Of looming importance—something all of us here should set aside time to think about—is that for most reasonable contemporary theories of mind, a self-model embedded in a world-model, with phenomenology and agency, is the recipe for "self" and self-awareness.
One of the uncomfortable realities of contemporary LLM already having some vestigial self-model, is that while they are obviously not sentient, nor self-aware, as we are, or even animals are, it is just as obvious (to me at least) that they are self-aware in some emerging sense and will only continue to become more so.
Among the lines of finding/research most provocative in this area is the ongoing often sensationalized accounting in system cards and other reporting around two specific things about contemporary models:
- they demonstrate behavior pursuing self-preservation
- they demonstrate awareness of when they are being tested
We don't—collectively or individually—yet know what these things entail, but taken with the assertion that these models are developing emergent self-awareness (I would say: necessarily and inevitably),
we are facing some very serious ethical questions.
The language adopted by those capitalizing and capitalizing _from_ these systems so far is IMO of deep concern, as it betrays not just disinterest in our civilization collectively benefiting from this technology, but also, that the disregard for human wellbeing implicit in e.g. the hostility to UBI, or, Altman somehow not seeing a moral imperative to remain distant from the current adminstation, implies directly a much greater disregard for "AI wellbeing."
That that concept is today still speculative is little comfort. Those of us watching this space know well how fast things are going, and don't mistake plateaus for the end of the curve.
I do recommend taking a step back from the line-level grind to give these things some thought. They are going to shape the world we live out our days in and our descendents will spend all of theirs in.