Harnessing the Universal Geometry of Embeddings
1–10 of 44 posts
Re: Harnessing the Universal Geometry of Embeddings
#2[deleted]
Re: Harnessing the Universal Geometry of Embeddings
#3Hi HN, I'm Jack, the last author of this paper. It feels good to release this, the fruit of a two-year quest to "align" two vector spaces without any paired data. It's fun to look back a bit and note that at least two people told me this wasn't possible:
1. An MIT professor who works on similar geometry alignment problems didn't want to work on this with me because he was certain we would need at least a little bit of paired data
2. A vector database startup founder who told me about his plan to randomly rotate embeddings to guarantee user security (and ignored me when I said it might not be a good idea)
The practical takeaway is something that many people already understood, which is that embeddings are not encrypted, even if you don't have access to the model that produced them.
As one example, in the Cursor security policy (https://www.cursor.com/security#codebase-indexing) they state:
> Embedding reversal: academic work has shown that reversing embeddings is possible in some cases. Current attacks rely on having access to the model [...]
This is no longer the case. Since all embedding models are learning ~the same thing, we can decode any embedding vectors, given we have at least a few thousand of them.
Re: Harnessing the Universal Geometry of Embeddings
#4The fact that embeddings from different models can be translated into a shared latent space (and back) supports the notion that semantic anchors or guides are not just model-specific hacks, but potentially universal tools. Fantastic read, thank you.
Given the demonstrated risk of information leakage from embeddings, have you explored any methods for hardening, obfuscating, or 'watermarking' embedding spaces to resist universal translation and inversion?
Re: Harnessing the Universal Geometry of Embeddings
#5The fact that embeddings from different models can be translated into a shared latent space (and back) supports the notion that semantic anchors or guides are not just model-specific hacks, but potentially universal tools. Fantastic read, thank you. Given the demonstrated risk of information leakage from embeddings, have you explored any methods for hardening, obfuscating, or 'watermarking' embedding spaces to resist…
> Given the demonstrated risk of information leakage from embeddings, have you explored any methods for hardening, obfuscating, or 'watermarking' embedding spaces to resist universal translation and inversion?
No, we haven't tried anything like that. There's definitely a need for it. People are using embeddings all over the place, not to mention all of the other representations people pass around (kv caches, model weights, etc.).
One consideration is that's likely going to be a tradeoff between embedding usefulness and invertability. So if we watermark our embedding space somehow, or apply some other 'defense' to make inversion difficult, we will probably sacrifice some quality. It's not clear yet how much that would be.
Re: Harnessing the Universal Geometry of Embeddings
#6The fact that embeddings from different models can be translated into a shared latent space (and back) supports the notion that semantic anchors or guides are not just model-specific hacks, but potentially universal tools. Fantastic read, thank you. Given the demonstrated risk of information leakage from embeddings, have you explored any methods for hardening, obfuscating, or 'watermarking' embedding spaces to resist…
> Given the demonstrated risk of information leakage from embeddings, have you explored any methods for hardening, obfuscating, or 'watermarking' embedding spaces to resist universal translation and inversion? No, we haven't tried anything like that. There's definitely a need for it. People are using embeddings all over the place, not to mention all of the other representations people pass around (kv caches, model we…
Are you continuing research? Is there somewhere we can follow along?
Re: Harnessing the Universal Geometry of Embeddings
#7This seems like a catastrophe in the wings for legal-services-RAG companies.
Re: Harnessing the Universal Geometry of Embeddings
#8Hi HN, I'm Jack, the last author of this paper. It feels good to release this, the fruit of a two-year quest to "align" two vector spaces without any paired data. It's fun to look back a bit and note that at least two people told me this wasn't possible: 1. An MIT professor who works on similar geometry alignment problems didn't want to work on this with me because he was certain we would need at least a little bit o…
[deleted]
Re: Harnessing the Universal Geometry of Embeddings
#9Earlier quoted context omitted.
> Given the demonstrated risk of information leakage from embeddings, have you explored any methods for hardening, obfuscating, or 'watermarking' embedding spaces to resist universal translation and inversion? No, we haven't tried anything like that. There's definitely a need for it. People are using embeddings all over the place, not to mention all of the other representations people pass around (kv caches, model we…
Are you continuing research? Is there somewhere we can follow along?
Yes! For now just our Twitters:
- Rishi, the first author: x.com/rishi_d_jha
- Me: x.com/jxmnop
And there's obviously always ArXiv. Maybe we should make a blog or something, but the updates really don't come that often.
Re: Harnessing the Universal Geometry of Embeddings
#10Hi HN, I'm Jack, the last author of this paper. It feels good to release this, the fruit of a two-year quest to "align" two vector spaces without any paired data. It's fun to look back a bit and note that at least two people told me this wasn't possible: 1. An MIT professor who works on similar geometry alignment problems didn't want to work on this with me because he was certain we would need at least a little bit o…
I must admit reading the abstract made me think to myself that I should read the paper in skeptical mode.
Does this extend to being able to analytically determine which concepts are encodable in one embedding but not another? An embedding from a deft tiny stories LLM presumably cannot encode concepts about RNA replication.
Assuming that is true. If you can detect when you are trying to put a square peg into a round hole, does this mean you have the ability to remove square holes from a system?