101–110 of 240 posts
Earlier quoted context omitted.
It is not historical: https://kieranhealy.org/blog/archives/2025/08/07/blueberry-h... Perhaps they have a hot fix that special cases HN complaints?
They clearly RLHF out the embarrassing cases and make cheating on benchmarks into a sport.
Earlier quoted context omitted.
> Symbols, by definition, only represent a thing. This is missing the lesson of the Yoneda Lemma: symbols are uniquely identified by their relationships with other symbols. If those relationships are represented in text, then in principle they can be inferred and navigated by an LLM. Some relationships are not represented well in text: tacit knowledge like how hard to twist a bottle cap to get it to come off, etc. We…
I don’t think it’s a communication problem as much as there is no possible relation between a word and a (literal) physical experiences. They’re, quite literally, on different planes of existence.
Now, maybe there are other possible experiences that would result in me behaving identically, such that from my behavior (including what words I say) it is impossible to distinguish between different potential experiences I could have had.
But, “caused me to say” is a relation, is it not?
Unless you want to say that it wasn’t the experience that caused me to do something, but some physical thing that went along with the experience, either causing or co-occurring with the experience, and also causing me to say the word I said. But, that would still be a relation, I think.
Earlier quoted context omitted.
I don’t think it’s a communication problem as much as there is no possible relation between a word and a (literal) physical experiences. They’re, quite literally, on different planes of existence.
When I have a physical experience, sometimes it results in me saying a word. Now, maybe there are other possible experiences that would result in me behaving identically, such that from my behavior (including what words I say) it is impossible to distinguish between different potential experiences I could have had. But, “caused me to say” is a relation, is it not? Unless you want to say that it wasn’t the experience…
It's like trying to describe a color to a blind person: poetic subjective nonsense.
Language models aren't world models for the same reason languages aren't world models. Symbols, by definition, only represent a thing. They are not the same as the thing. The map is not the territory, the description is not the described, you can't get wet in the word "water". They only have meaning to sentient beings, and that meaning is heavily subjective and contextual. But there appear to be some who think that w…
> Symbols, by definition, only represent a thing. They are not the same as the thing First of all, the point isn't about the map becoming the territory, but about whether LLMs can form a map that's similar to the map in our brains. But to your philosophical point, assuming there are only a finite number of things and places in the universe - or at least the part of which we care about - why wouldn't they be represent…
That statement is problematic. It implies a metaphysical set of laws that make physical stuff relate a certain way.
The Humean way of looking at physics is that we notice relationships and model those with various symbols. They symbols form incomplete models because we can't get to the bottom of why the relationships exist.
> that doesn't mean that we can predict the future, as that would entail both measuring things precisely and simulating them faster than their operation in nature, and both of these things are... difficult.
The indeterminism of Quantum Mechanics limits how how precise measure can be and how predictable the future is.
https://deepmind.google/discover/blog/genie-3-a-new-frontier...
We also have multimodal AIs that can do both language and video. Genie 3 made multimodal with language might be pretty impressive.
Focusing only on what pure language models can do is a bit of a straw man at this point.
Earlier quoted context omitted.
> Symbols, by definition, only represent a thing. They are not the same as the thing First of all, the point isn't about the map becoming the territory, but about whether LLMs can form a map that's similar to the map in our brains. But to your philosophical point, assuming there are only a finite number of things and places in the universe - or at least the part of which we care about - why wouldn't they be represent…
> Of course, if physics does exist - i.e. the universe is governed by a finite set of laws Wouldn't physics still "exist" even if there were an infinite set of laws?
Earlier quoted context omitted.
> Symbols, by definition, only represent a thing. This is missing the lesson of the Yoneda Lemma: symbols are uniquely identified by their relationships with other symbols. If those relationships are represented in text, then in principle they can be inferred and navigated by an LLM. Some relationships are not represented well in text: tacit knowledge like how hard to twist a bottle cap to get it to come off, etc. We…
I don’t think it’s a communication problem as much as there is no possible relation between a word and a (literal) physical experiences. They’re, quite literally, on different planes of existence.
Earlier quoted context omitted.
> Symbols, by definition, only represent a thing. They are not the same as the thing First of all, the point isn't about the map becoming the territory, but about whether LLMs can form a map that's similar to the map in our brains. But to your philosophical point, assuming there are only a finite number of things and places in the universe - or at least the part of which we care about - why wouldn't they be represent…
> course, if physics does exist - i.e. the universe is governed by a finite set of laws That statement is problematic. It implies a metaphysical set of laws that make physical stuff relate a certain way. The Humean way of looking at physics is that we notice relationships and model those with various symbols. They symbols form incomplete models because we can't get to the bottom of why the relationships exist. > that…
What I meant was that since physics is the scientific search for the laws of nature, then if there's an infinite number of them, then the pursuit becomes somewhat meaningless, as an infinite number of laws aren't really laws at all.
> They symbols form incomplete models because we can't get to the bottom of why the relationships exist.
Why would a model be incomplete if we don't know why the laws are what they are? A model pretty much is a set of laws; it doesn't require an explanation (we may want such an explanation, but it doesn't improve the model).
That whole bit about color blending and transparency and LLMs "not knowing colors" is hard to believe. I am literally using LLMs every day to write image-processing and computer vision code using OpenCV. It seamlessly reasons across a range of concepts like color spaces, resolution, compression artifacts, filtering, segmentation and human perception. I mean, removing the alpha from a PNG image was a preprocessing ste…
> it wrote by itself as part of a larger task I had given it, so it certainly understands transparency Or it’s a common step or a known pattern or combination of steps that is prevalent in its training data for certain input. I’m guessing you don’t know what’s exactly in the training sets. I don’t know either. They don’t tell ;) > but it adapts and combines them well to suit my particular requirements. So they seem t…
If LLMs are stochastic parrots, but also we’re just stochastic parrots, then what does it matter? That would mean that LLMs are in fact useful for many things (which is what I care about far more than any abstract discussion of free will).