Latent Dictionary: 3D map of Oxford3000+search words via DistilBert embeddings
latentdictionary.com
Latent Dictionary: 3D map of Oxford3000+search words via DistilBert embeddings
1–10 of 43 posts
Re: Latent Dictionary: 3D map of Oxford3000+search words via DistilBert embeddings
#2I added those in succession:
> man woman king queen ruler force powerful care
and couldn't reliably determine position of any of them
Re: Latent Dictionary: 3D map of Oxford3000+search words via DistilBert embeddings
#3Is this with some sort of dimensionality reduction of the embedding space?
Re: Latent Dictionary: 3D map of Oxford3000+search words via DistilBert embeddings
#4Seems mostly nonsensical, not sure if that's a bug or some deeper point I'm missing.
Re: Latent Dictionary: 3D map of Oxford3000+search words via DistilBert embeddings
#5By running the same multiple times, I get different visualization. I don't really understand what's going on, but I like the idea of visualizing embeddings.
Re: Latent Dictionary: 3D map of Oxford3000+search words via DistilBert embeddings
#6Is this with some sort of dimensionality reduction of the embedding space?
In the bottom left "?" button it says it performs PCA down to 3 dimensions. That's going to lose a ton of information, rendering the space mostly useless.
Re: Latent Dictionary: 3D map of Oxford3000+search words via DistilBert embeddings
#7Re: Latent Dictionary: 3D map of Oxford3000+search words via DistilBert embeddings
#8I wish there were more context and maybe the ability to do math on the vectors
Eg what is the real distance between the two vectors? That should be easy to compute
Similarly: what do I get from summing two vectors and what are some nearby vectors?
Maybe just generally: what are some nearby vectors?
Without any additional context it's just a point cloud with a couple of randomly labeled elements
Re: Latent Dictionary: 3D map of Oxford3000+search words via DistilBert embeddings
#9By running the same multiple times, I get different visualization. I don't really understand what's going on, but I like the idea of visualizing embeddings.
I think the PCA dimensionality reduction is non deterministic, but I say this with really low confidence
Re: Latent Dictionary: 3D map of Oxford3000+search words via DistilBert embeddings
#10By running the same multiple times, I get different visualization. I don't really understand what's going on, but I like the idea of visualizing embeddings.
I think the PCA dimensionality reduction is non deterministic, but I say this with really low confidence
PCA is purely deterministic (and might not give great results). My guess is this is done by t-sne or UMAP, both of which depend on a seed.