Latent Dictionary: 3D map of Oxford3000+search words via DistilBert embeddings • Hacker News Board

Re: Latent Dictionary: 3D map of Oxford3000+search words via DistilBert embeddings

#41

Post by kaoD » Sun, Dec 31, 2023, 9:31 AM UTC

tikimcfee wrote: post #34

kaoD wrote: post #31

Earlier quoted context omitted.

What you're describing exists. It's called "embeddings" and it's one of the first steps ChatGPT does to do its magic and it's indeed very useful. What renders this useless is reducing the dimensionality from thousands to just 3.

Thank you.. sincerely, thank you. You just unlocked something in my brain about how this works in a certain way and I.. am incredibly excited to explore this more. If I’m intuitively understanding that, I really really really feel like I can… get this. Do something with this. Thank you. Now I need to go understand embedding a more. And use them. If you have any resources or ideas or keywords to look for (I think i ex…

People are talking about embeddings in these comments, you'll get plenty of info and keywords here.

Re: Latent Dictionary: 3D map of Oxford3000+search words via DistilBert embeddings

#42

Post by pyinstallwoes » Thu, Jan 04, 2024, 9:29 AM UTC

warkdarrior wrote: post #40

pyinstallwoes wrote: post #30

Earlier quoted context omitted.

Yeah but it doesn’t use a universal method does it? And it requires labeling. The method I’m describing requires no labeling. Labeling would be a local only translation (alias). Labels emerge based on meaning. But the labels are more of an interface - not the actual nodes themselves which arise off the not identity principle * event proximity * comparisons.

Labeling (which is typically manual and thus not scalable) is a proxy for comparisons. Two things are the same if they have the same label. The question is how else to encode the comparison information.

Right one is manual the other is automatic and my hypothesis is you can have automatic universal labeling the way I describe

Re: Latent Dictionary: 3D map of Oxford3000+search words via DistilBert embeddings

#43

Post by patcon » Mon, Jan 08, 2024, 8:31 PM UTC

tikimcfee wrote: post #35

patcon wrote: post #22

Earlier quoted context omitted.

Yep, been thinking on that paper as well: Traveling words. There's code! https://arxiv.org/abs/2309.07315 https://github.com/santiag0m/traveling-words

This is so damn cool! You just showed me something that may change the course of my ideation, current project, and who I may talk to. Thank you so much for sharing. Words in space. It’s finally freaking happening!!!

:)