Earlier quoted context omitted.
It’s going to sound delusional, but my own understanding of my understanding is that there’s probably no such thing, only a feeling of familiarity and some heuristics. All I do is invoke “pictures” of semantic blobs in my mind, and my unconscious parts pull up more “pictures” in connection to that. Which I verbalize in turn, apply logic checks, and thus get this feeling that I’m meaningfully thinking about something,…
You don’t think there is such a thing as “understanding” a subject? Knowing the fundamental buildings blocks of that subject? Rearranging those building blocks within other domains to make new discoveries? A few days ago you commented, in reference to needing an expert for imagery: “Because someone who knows something about pixels needs to make a judgement.” Would this person not need “understanding” of how pixels em…
I probably misunderstood (lol) you, because I was thinking of the phenomenal feeling of "getting it" - which is the only thing that I was able to label as "understanding" that exists in humans, but (I think) not in machine models. But when we feel insight, that's not some "real" understanding, it's a heuristics-powered illusion. That's why I wrote that.
It's quite ironic: I genuinely thought I understand what you're referring to, but that was a misunderstanding.
> Knowing the fundamental buildings blocks of that subject? [...]
Ah, in that sense - as having a mental model of something at its structural level - yes, it exists. But then computer models can understand things too - LLMs are not just giant Markov chains with pure token statistics, they build internal representations way beyond that. Look for the Othello-GPT story, it's pretty small, but quite fascinating how a model had built a world model out of just moves.
> Would this person not need “understanding” of how pixels emerge into tasteful art?
First, let me be clear that the key point there was the judgement bit. A machine lacks personhood, thus it cannot be held responsible, thus cannot make a judgement. That classic IBM memo.
For a judgement alone, just knowledge is sufficient. Seeing an irregularity is pattern matching, "those pixels look sloppy" phase doesn't yet need a "why", just a trained eye. But - yes, understanding is necessary follow-up, how to make those pixels stop raising eyebrows. Gotta not just see that e.g. "this hand has weird fingers", but also why they're weird, and then how to correct that.
> This is quite terrifying
Yes, but - IMHO - not because of how shallow some understanding might be. Competence shifts are perfectly natural, skills that are in demand remain, skills that aren't atrophy, I don't see anything too scary about that.
What's uncanny is that a lot of people indeed pass on judgement and even agency to a tool that has none. "AI" takes jobs, "AI" destroys environment, "AI" makes people zombies, "AI" steals art - that's what's really scary, that a lot of slogans put a veil in front of the actual (very much human) agents and their actions.