Why the most valuable things you know are things you cannot say
deadneurons.substack.com
Why the most valuable things you know are things you cannot say
1–10 of 70 posts
Re: Why the most valuable things you know are things you cannot say
#2I would call the difference intuitive knowledge versus rational knowledge.
I've never seen the word calibration used this way:
different modes of learning. The first is instruction: the transfer of explicit models, rules, and relationships from one person to another through language. The second is calibration: the development of internal models through repeated exposure to feedback in a specific environment.
Judgement is learnable through calibration. It is not transmissible through instruction.
Re: Why the most valuable things you know are things you cannot say
#3Saying certain thing hurts them like a scientist's knife that opens up an animal for a study but also kills it. Thus certain knowledge about life cannot be expressed in an analytical way: it loses the very subject it tries to catch. But if you use the knife merely to point to the animal... that is, if you use words not as an explanation but as means to guide the listener's attention, then there is a chance to convey that understanding. A parable works this way; or a work of real art.
Re: Why the most valuable things you know are things you cannot say
#4I would call the difference intuitive knowledge versus rational knowledge. I've never seen the word calibration used this way: different modes of learning. The first is instruction: the transfer of explicit models, rules, and relationships from one person to another through language. The second is calibration: the development of internal models through repeated exposure to feedback in a specific environment. Judgemen…
Intuition is just our brains' amazing pattern recognition ability at work.
Re: Why the most valuable things you know are things you cannot say
#5I would call the difference intuitive knowledge versus rational knowledge. I've never seen the word calibration used this way: different modes of learning. The first is instruction: the transfer of explicit models, rules, and relationships from one person to another through language. The second is calibration: the development of internal models through repeated exposure to feedback in a specific environment. Judgemen…
You raise an interesting question. How do we keep the meanings of words from diverging so dramatically and so rapidly?
A little bit is natural and expected, but this kind of change in meaning feels like a consequence of a culture that in the last decade has accelerated the practice of re-framing specific words and concepts as something that's "actually a positive" or "actually quite negative if you think about it".
Part of this is a result of our (in the US) culture wars and hijacking of popular terms, but it's also a symptom of social media culture that's always seeking a hot take and creators who are looking to distinguish themselves with (what seems to me) clever re-framing.
The result is a culture that is increasingly fragmented and in which a word can have dramatically different meaning and insinuations depending on it's use in certain social groups or intellectual cliques.
It increasingly feels like I need to download a massive amount of linguistic context before I step into the world of a niche online community because their tight-knit dialogues and shared experiences have now re-framed a word or concept that was largely understood to mean something else.
Re: Why the most valuable things you know are things you cannot say
#6[dead]
Re: Why the most valuable things you know are things you cannot say
#7This was a powerful read and a useful lens to apply to the legibility of the technology era we're in.
Re: Why the most valuable things you know are things you cannot say
#8I would call the difference intuitive knowledge versus rational knowledge. I've never seen the word calibration used this way: different modes of learning. The first is instruction: the transfer of explicit models, rules, and relationships from one person to another through language. The second is calibration: the development of internal models through repeated exposure to feedback in a specific environment. Judgemen…
You raise an interesting question. How do we keep the meanings of words from diverging so dramatically and so rapidly? A little bit is natural and expected, but this kind of change in meaning feels like a consequence of a culture that in the last decade has accelerated the practice of re-framing specific words and concepts as something that's "actually a positive" or "actually quite negative if you think about it". P…
> How do we keep the meanings of words from diverging so dramatically and so rapidly?
We don’t engage. It’s the only shot we have.
There was a useful article at 404 Media recently about our failure to prevent those on the extreme edges of culture from normalizing their language and behavior: We Have Learned Nothing About Amplifying Morons[0]. See the article, but essentially by engaging we cede ground. Sorta like how both-sides journalism gives space to anti-science nuts and lets them spread falsehoods.
0. https://www.404media.co/we-have-learned-nothing-about-amplif...
Re: Why the most valuable things you know are things you cannot say
#9[dead]
Re: Why the most valuable things you know are things you cannot say
#10Thinking about this in the context of machine learning.. We can discover the dimensions and relationships between them through training over a set of examples.
What we are generally getting though is a network with extremely high dimensionality trained on many domains at once, at least as far as the commonly used ones like LLMs and VLMs.
We do have mixture of experts which I guess helps to compress things.
Going back to the idea that this stuff just can't be represented by language, I wonder if someday there could be a type of more concise representation than transmitting for example a LoRA with millions of bytes.
Maybe if we keep looking at distillation of different models over and over we might come up with some highly compressed standardized hierarchical representation that optimizes subdomain or expert selection and combination to such a degree that the information for a type of domain expertise can be transmitted maybe not orally between humans but at least in very compact and standard way between models.
I guess if you just take something like a 1B 1 bit model and build a LoRA for a very narrow domain and then compress that. That's something like the idea. Or maybe a quantized NOLA.
But I wonder if someday there will be a representation that is more easily interpretable like language but is able to capture high dimensional complex functions in a standard and concise way.