11–20 of 27 posts
Open a bank account. Open your heart. Open a can. Open to new experiences. Words are a tricky thing to handle.
And that is just in English Other languages have similar but fundamentally different oddities which do not translate cleanly
Earlier quoted context omitted.
And that is just in English Other languages have similar but fundamentally different oddities which do not translate cleanly
Not sure how they're fundamentally different. What do you mean?
Try explaining why tough and rough rhyme but bough doesn't
You know? Language has a ton of idiosyncrasies.
Open a bank account. Open your heart. Open a can. Open to new experiences. Words are a tricky thing to handle.
As I see it, "Open your heart", "Open a can" and "Open to new experiences" have very similar meanings for "Open", being essentially "make a container available for external I/O", similar to the definition of an "open system" in thermodynamics. "Open a bank account" is a bit different, as it creates an entity that didn't exist before, but even then the focus is on having something that allows for external I/O - in this case deposits and withdrawals.
>> For each LLM, we extract static, token-level embeddings from its input embedding layer (the ‘E‘matrix). This choice aligns our analysis with the context-free nature of stimuli typical in human categorization experiments, ensuring a comparable representational basis. They're analyzing input embedding models, not LLMs. I'm not sure how the authors justify making claims about the inner workings of LLMs when they have…
Note that the token embeddings are also trained, therefore their values do give some hints on how a model is organizing information. They used token embeddings directly and not intermediate representations because the latter depend on the specific sentence that the model is processing. Data on human judgment was however collected without any context surrounding each word, thus using the token embeddings seem to be th…
Though it sounds odd there is no problem with it and it would indeed return the model's representation of that single word as seen by the model without any additional context.
>> For each LLM, we extract static, token-level embeddings from its input embedding layer (the ‘E‘matrix). This choice aligns our analysis with the context-free nature of stimuli typical in human categorization experiments, ensuring a comparable representational basis. They're analyzing input embedding models, not LLMs. I'm not sure how the authors justify making claims about the inner workings of LLMs when they have…
It’s mind blowing LeCun is listed as one of the authors. I would expect model size to correlate with alignment score because usually model sizes correlate with hidden dimension. But also opposite can be true - bigger models might shift more basic token classification logic into layers and hence embedding alignment can go down. Regardless feels like pretty useless research…
>> For each LLM, we extract static, token-level embeddings from its input embedding layer (the ‘E‘matrix). This choice aligns our analysis with the context-free nature of stimuli typical in human categorization experiments, ensuring a comparable representational basis. They're analyzing input embedding models, not LLMs. I'm not sure how the authors justify making claims about the inner workings of LLMs when they have…
This paper is interesting, but ultimately it's just restating that LLMs are statistical tools and not cognitive systems. The information-theoretic framing doesn’t really change that.
I have never understood broad statements that models are just (or mostly) statistical tools.
Certainly statistics apply, minimizing mismatches results in mean (or similar measure) target predictions.
But the architecture of a model is the difference between compressed statistics vs. forcing a model to translate information in a highly organized way reflecting the actual shape of the problem to get any accuracy at all.
In both cases, statistics are relevant, but in the latter it's not a particularly insightful way to talk about what a model has learned.
Statistical accuracy, prediction, etc. are basic problems to solve. The training criteria being optimized. But they don't limit the nature of solutions. They both leave problem difficulty, and solution sophistication unbounded.
Earlier quoted context omitted.
Not sure how they're fundamentally different. What do you mean?
Think about the work of localizing a joke that relies on wordplay or similar sounding words to be funny. Or simply how words rhyme Try explaining why tough and rough rhyme but bough doesn't You know? Language has a ton of idiosyncrasies.
>> For each LLM, we extract static, token-level embeddings from its input embedding layer (the ‘E‘matrix). This choice aligns our analysis with the context-free nature of stimuli typical in human categorization experiments, ensuring a comparable representational basis. They're analyzing input embedding models, not LLMs. I'm not sure how the authors justify making claims about the inner workings of LLMs when they have…
Note that the token embeddings are also trained, therefore their values do give some hints on how a model is organizing information. They used token embeddings directly and not intermediate representations because the latter depend on the specific sentence that the model is processing. Data on human judgment was however collected without any context surrounding each word, thus using the token embeddings seem to be th…
And like the other commenter said, you can absolutely feed single tokens through the model. Your point doesn’t make any sense though regardless. How about priming the model with “You’re a helpful assistant” just like everyone else does.