21–30 of 43 posts
As a physicist, I always found it funny that in ML people renamed 'the angle between vectors' into something as fancy-feeling as 'cosine similarity'.
Earlier quoted context omitted.
Because op knows TS and doesn't know Rust?
What I'm asking really is what is the benefit of TS if we could use decent and improving coding AIs to help us code in Rust, even if we are new to it, or never used it, and compile to WASM. Much faster execution per my experience. I mean not even comparable.
The future is bleak.
Great post, but what struck me (again, like every time I look at cos similiarity) is how unreasonably well it works. It's just one of those things that's so "weird" about our world: why would cosine similarity work in n-dimensional semantic spaces? It's so stupid simple, and it intuitively makes sense, and it works really well. Crazy cool. I'm reminded of that old Eugene Wigner quote: "The most incomprehensible thing…
It's much more interesting that almost any set of ML-adjacent vectors can be somewhat reasonably compared via cosine distance _even without_ explicitly constructing an optimal embedding. It's not at all intuitive to me that an autoencoder's interior layer should behave well with respect to cosine similarity and not have any knots or anything warping that (associated) metric's usefulness.
As a physicist, I always found it funny that in ML people renamed 'the angle between vectors' into something as fancy-feeling as 'cosine similarity'.
(a @ b.T)/(np.linalg.norm(a)*np.linalg.norm(b))As a physicist, I always found it funny that in ML people renamed 'the angle between vectors' into something as fancy-feeling as 'cosine similarity'.
If you really wanted to be pretentious you could invent a Dimension Insensitive Euclidean Metric and call it DIEM to make it sound like you are putting some Latin into your papers.
I wonder though- isn't classic TF-IDF scoring also a type of basic vector comparison? What is the actual difference between "vector database" and something like elastic search?
Great post, but what struck me (again, like every time I look at cos similiarity) is how unreasonably well it works. It's just one of those things that's so "weird" about our world: why would cosine similarity work in n-dimensional semantic spaces? It's so stupid simple, and it intuitively makes sense, and it works really well. Crazy cool. I'm reminded of that old Eugene Wigner quote: "The most incomprehensible thing…
That cosine distance works at all as a concept isn't terribly shocking, especially given our habit of norming everything. Cosine similarity in a unit vector space is monotonic with euclidean distance, and we're using this stuff in "select the K most relevant vectors" sorts of queries, so cosine similarity behaves identically to euclidean distance. Tack on the fact that every finite set of vectors, with your favorite…
Tbh, I would argue that's also pretty surprising, as Euclidean distance is notoriously unintuitive[1] (and noisy) in higher dimensions. (I guess norming does help, so that's likely a good point.)
Meanwhile in Python this is just (a @ b.T)/(np.linalg.norm(a)*np.linalg.norm(b))
As a physicist, I always found it funny that in ML people renamed 'the angle between vectors' into something as fancy-feeling as 'cosine similarity'.