Harnessing the Universal Geometry of Embeddings • Hacker News Board

Re: Harnessing the Universal Geometry of Embeddings

#41

Post by billconan » Thu, May 22, 2025, 8:03 AM UTC

jxmorris12 wrote: post #3

Hi HN, I'm Jack, the last author of this paper. It feels good to release this, the fruit of a two-year quest to "align" two vector spaces without any paired data. It's fun to look back a bit and note that at least two people told me this wasn't possible: 1. An MIT professor who works on similar geometry alignment problems didn't want to work on this with me because he was certain we would need at least a little bit o…

Thank you for sharing! I have a question about embedding versioning/migration. I'm not sure if this research solves it?

Say I want to build an app with embedding/vector search. Currently, my embeddings are generated by model A, that is not open source. Later, I find a better embedding model B, and my new data will be using this model B. Since A and B are two different vector spaces, how can I migrate A to B, or how can I make vector search work without migrating A to B?

Can your research solve this problem? Also，if all embedding models are the same, is there a point of upgrading the model at all? some must be better trained than others?

Re: Harnessing the Universal Geometry of Embeddings

#42

Post by henry-d » Thu, May 22, 2025, 5:37 PM UTC

Very cool! Do you think if an alien civilization created an embedding model for their alien corpus of text it would satisfy this?

Re: Harnessing the Universal Geometry of Embeddings

#43

Post by _1tem » Fri, May 23, 2025, 8:37 AM UTC

Huh. So Plato was right. This has many implications for philosophy. Interestingly, the 12th century Platonic-influenced Arab philosopher Ibn Arabi described methods of converting text to numbers (embeddings) and then performing operations on those numbers to yield new meanings (inference). A 12th century LLM? His books are full of these kinds of operations (called Abjad math) and a core part of his textual hermeneutics.

Re: Harnessing the Universal Geometry of Embeddings

#44

Post by lpasselin » Tue, Jun 03, 2025, 3:55 PM UTC

Hey, I read the paper in detail and presented to colleagues during our reading group.

I still do not understand exactly where D1L comes from in LGan(D1L, T(A1(u)). Is D1L simply A1(u)?

I also find that mixing notation in figure 2 and 3 makes it tricky.

Would have loved to have more insights from the results in the tables.

And more results from inversion, on more than Enron dataset. Since that is one end goals, even if reusing another method.

Thank you for the paper, very interesting!