Is there any way to read these responsive sites on mobile when they are cut off by the column margins?
Viewing with a landscape viewport sometimes works.
Decoding the Thought Vector
11–20 of 24 posts
Re: Decoding the Thought Vector
#12[deleted]
Re: Decoding the Thought Vector
#13> Rather curiously, it turns "airplanes" into "knives". I do not understand why this happens. I thought it was pretty obvious. The atoms are complected (defined - by Rich Hickey of Clojure - as, basically, a semantic that contains multiple interdependent concepts (for example how variables complect state, values, and names)). In fact that's the conceit of the whole idea, the thought vector is being extracted from the…
This explanation reminds me of a concept in ASL (American Sign Language) called Classifiers:
https://seattlecentral.edu/faculty/baron/Summer%20Courses/AS...
They are "class" modifiers that modify different nouns in different ways.
Re: Decoding the Thought Vector
#14This is incredible and infinitely useful. For many years, I've had this hunch that Human symbols and abstractions had an algebraic quality. And, I have always wanted to substitute tags, category, labels, and attributes with global, permanent indices. Mainly, I wanted to do this because the 'view' of a thing is mutable: e.g. a word for a concept changes over time even when the conceptual meaning remains constant. I can't wait to get home and play around with this.
Re: Decoding the Thought Vector
#15When I was working on a recommender for television shows, I ran SVD on a large User/Item matrix to create a low rank approximation, essentially reducing thousands of user features (TV show preferences) to user vectors representing twenty or thirty abstract "features". Then I looked at the actual item preferences of users who expressed each feature at the greatest and least magnitude. The features, in some cases, mapp…
A question: is this much better / different than a principal component analysis (or a factor analysis)?
Re: Decoding the Thought Vector
#16When I was working on a recommender for television shows, I ran SVD on a large User/Item matrix to create a low rank approximation, essentially reducing thousands of user features (TV show preferences) to user vectors representing twenty or thirty abstract "features". Then I looked at the actual item preferences of users who expressed each feature at the greatest and least magnitude. The features, in some cases, mapp…
A question: is this much better / different than a principal component analysis (or a factor analysis)?
One way to do PCA is using SVD to find a transformation matrix of eigenvectors to project your data with, so they're similar.
Re: Decoding the Thought Vector
#17The "atom" terminology is a bit confusing to me. Isn't an "atom" just another thought vector? If "sunglasses" + "smiles" = "smiling-while-wearing-sunglasses", then "sunglasses" and "smiling-while-wearing-sunglasses" are both just vectors. Is there a reason for the distinction?
Also, if all thoughts can be described as vectors and linear combinations of vectors in "thought-space", I wonder what the axis represent and how many dimensions there are. Are all thoughts just a combination of 100 "unit thoughts"?
Really interesting post!
Re: Decoding the Thought Vector
#18Nice interactive examples but I'm afraid the basic setup here doesn't make sense to me. The "atom" is defined as the average encoding of inputs with the feature ("faces with a smile"), but I'd think the proper definition should subtract off inputs without the feature (i.e. "smile" = "faces with a smile" minus "faces without a smile"). The way it's defined you end up adding an extra "average face" along with the feature of interest, which is clearly seen in "The Geometry of Thought Vectors" example -- the non-smiling woman isn't so much forced to smile as to have her face merged with that of a generic smiling woman.
Re: Decoding the Thought Vector
#19I find it counter-intuitive that thought vectors in should have "Linear Structure" in multilevel autoencoders.
Since the whole appeal of neural networks is that they can model non-linear functions. Why would the autoencoder end up with an encoding that is essentially linear?
Re: Decoding the Thought Vector
#20Earlier quoted context omitted.
Did you ever compare that focus with a graph over the singular values?
It's a good question, FWIW I would expect a reasonably sharp "L" shaped curve in the focus. The assumption there I guess being that this metric of 'focus' is something well characterized by low-frequency type basis matrices given by the first few rows/columns of the SVD's U and V.
Exactly what I saw. Your expectation is correct.