Live data from Hacker News

Harnessing the Universal Geometry of Embeddings

arxiv.org

31–40 of 44 posts

Re: Harnessing the Universal Geometry of Embeddings

#31
I don't see how the "different data" aspect is evidenced. If the "modality" of the data is the same, we're choosing a highly specific subset of all possible data -- and, in practice, radically more narrow than just that. Any sufficiently capable LLM is going to have to be trained on a corpus not-so-dissimilar to all electronic texts which exist in the standard corpa used for LLM training.

The idea that a data set is "different" merely because its some subset of this maximal corpa is a difference without a distinction. What isnt being proposed is, say, that training just on all the works of scifi fiction lead to a zero-info translatable embedding space projectable into all the works of horror, and the like (or say that english-scifi can be bridged to japanese-scifi by way of a english-japanese-horror-corpus).

The very objective of creating LLMs with useful capabilities entials an extremely similar dataset starting point. We do not have so many petabytes of training data here that there is any meaningful sense in which OpenAI uses "only this discrete subspace" and perplextiy, "yet another". All useful LLMs sample roughly randomly across the maximal corpus that we have to hand.

Thus this hype around there being a platonic form of how word tokens ought be arranged seems wholly unevidenced. Reality has a "natural arrangement" -- this does not show that our highly lossy encoding of it in english has anything like a unique or natural correspondence. It has a circumstantial correspondance in "all recorded electronic texts" which are the basis for training all generally useful LLMs.

Re: Harnessing the Universal Geometry of Embeddings

#32

Hi HN, I'm Jack, the last author of this paper. It feels good to release this, the fruit of a two-year quest to "align" two vector spaces without any paired data. It's fun to look back a bit and note that at least two people told me this wasn't possible: 1. An MIT professor who works on similar geometry alignment problems didn't want to work on this with me because he was certain we would need at least a little bit o…

I hate to be "reviewer 2", but: I used to work on what your paper calls "unsupervised transport", that is machine translation between two languages without alignment data. You note that this field has existed since ~2016 and you provide a number of references, but you only dedicate ~4 lines of text to this branch of research. There's no comparison about why your technique is different to this prior work or why the pr…

> I strongly dislike your name of vec2vec.

Imagine having more than a passing understanding of philosophy, and then reading much of any major computer science papers. By this "No right to claim" logic, I'd have you all on trial.

Re: Harnessing the Universal Geometry of Embeddings

#34

Earlier quoted context omitted.

I hate to be "reviewer 2", but: I used to work on what your paper calls "unsupervised transport", that is machine translation between two languages without alignment data. You note that this field has existed since ~2016 and you provide a number of references, but you only dedicate ~4 lines of text to this branch of research. There's no comparison about why your technique is different to this prior work or why the pr…

Hey, I appreciate the perspective. We definitely should cite both those papers, and will do so in the next version of our draft. There are a lot of papers in this area, and they're all a few years old now, so you might understand how we missed two of them. We tested all of the methods in the Python Optimal Transport package ( https://pythonot.github.io/ ) and reported the max in most of our tables. So some of this is…

> We tested all of the methods in the Python Optimal Transport package (https://pythonot.github.io/) and reported the max in most of our tables.

Sorry if I'm being obtuse, but I don't see any mention of the POT package in your paper or of what specific algorithms you used from it to compare against. My best guess is that you used the linear map similar to the example at https://pythonot.github.io/auto_examples/domain-adaptation/p...>. The methods I mentioned are also linear, but contain a number of additional tricks that result in much better performance than a standard L2 loss, and so I would expect those methods to outperform your OT baseline.

> As for the name – the paper you recommend is called 'vecmap' which seems equally general, doesn't it? Google shows me there are others who have developed their own 'vec2vec'. There is a lot of repetition in AI these days, so collisions happen.

But both of those papers are about generic vector alignment, so the generality of the name makes sense. Your contribution here seems specifically about the LLM use case, and so a name that implies the LLM use case would be preferable.

I do agree though that in general naming is hard and I don't have a better name to suggest. I also agree that there's lots of related papers, and you can't cite/discuss them all reasonably.

And I don't mean to be overly critical... the application to LLMs is definitely cool. I wouldn't have read the paper and written up my critiques if I didn't overall like it :)

Re: Harnessing the Universal Geometry of Embeddings

#35

Earlier quoted context omitted.

I hate to be "reviewer 2", but: I used to work on what your paper calls "unsupervised transport", that is machine translation between two languages without alignment data. You note that this field has existed since ~2016 and you provide a number of references, but you only dedicate ~4 lines of text to this branch of research. There's no comparison about why your technique is different to this prior work or why the pr…

Naming things is hard. Noting the two alternative approaches that you referenced are called "vecmap" and "alignment" which "aren't the first/only algorithm for ... and you have no right to claim such a general title" could easily apply there as well.

Except those papers are 8ish years old; they actually were among the first 2-3 algs for this task; and they studied the fully general vector space alignment problem. But I agree that naming things is hard and don't have a better name.

Re: Harnessing the Universal Geometry of Embeddings

#36

Hi HN, I'm Jack, the last author of this paper. It feels good to release this, the fruit of a two-year quest to "align" two vector spaces without any paired data. It's fun to look back a bit and note that at least two people told me this wasn't possible: 1. An MIT professor who works on similar geometry alignment problems didn't want to work on this with me because he was certain we would need at least a little bit o…

I hate to be "reviewer 2", but: I used to work on what your paper calls "unsupervised transport", that is machine translation between two languages without alignment data. You note that this field has existed since ~2016 and you provide a number of references, but you only dedicate ~4 lines of text to this branch of research. There's no comparison about why your technique is different to this prior work or why the pr…

The problem solved in this paper is strictly harder than alignment. Alignment works with multiple, unmatched representations of the same inputs (e.g, different embeddings of the same words). The goal is to match them up.

The goal here is harder: given an embedding of an unknown text in one space, generate a vector in another space that's close to the embedding of the same text -- but, unlike in the word alignment problem, the texts are not known in advance.

Neither unsupervised transport, nor optimal alignment can solve this problem. Their input sets must be embeddings of the same texts. The input sets here are embeddings of different texts.

FWIW, this is all explained in the paper, including even the abstract. The comparisons with optimal assignment explicitly note that this is an idealized pseudo-baseline, and in reality OA cannot used for embedding translation (as opposed to matching, alignment, correspondence, etc.)

Re: Harnessing the Universal Geometry of Embeddings

#37
post #29
post #10

Earlier quoted context omitted.

I must admit reading the abstract made me think to myself that I should read the paper in skeptical mode. Does this extend to being able to analytically determine which concepts are encodable in one embedding but not another? An embedding from a deft tiny stories LLM presumably cannot encode concepts about RNA replication. Assuming that is true. If you can detect when you are trying to put a square peg into a round h…

I read a lot of AI papers on arxiv, and it's been a while since I read one where the first line of the abstract had me scoffing and done. > We introduce the FIRST method for translating text embeddings from one vector space to another without any paired data (emphasis mine) Nope. I'm not gonna go a literature search for you right now and find the references, but this is certainly not the first attempt to do unsupervi…

There has been plenty of work on alignment of embeddings, a lot of it cited in the paper. This paper solves the problem of translation, where (unlike in word alignment) there isn't already a set of candidate vectors in the target embedding space. It's generation, not matching.

Re: Harnessing the Universal Geometry of Embeddings

#38
This arguably means that we could translate any unknown alien message if it is sufficiently long and not encrypted: 1) Create embeddings from the Alienese message. 2) Convert them to English.

Could we also use this to read the Voynich manuscript, by converting Voynichese into embeddings and embeddings into English text? Perhaps, though I worry the manuscript is too short for that.

Re: Harnessing the Universal Geometry of Embeddings

#39

Hi HN, I'm Jack, the last author of this paper. It feels good to release this, the fruit of a two-year quest to "align" two vector spaces without any paired data. It's fun to look back a bit and note that at least two people told me this wasn't possible: 1. An MIT professor who works on similar geometry alignment problems didn't want to work on this with me because he was certain we would need at least a little bit o…

> we can decode any embedding vectors, given we have at least a few thousand of them.

Do you think that could be sufficient to translate the Voynich manuscript?

Re: Harnessing the Universal Geometry of Embeddings

#40

Hi HN, I'm Jack, the last author of this paper. It feels good to release this, the fruit of a two-year quest to "align" two vector spaces without any paired data. It's fun to look back a bit and note that at least two people told me this wasn't possible: 1. An MIT professor who works on similar geometry alignment problems didn't want to work on this with me because he was certain we would need at least a little bit o…

Does this result imply that if we had a LLM trained on a very large volume of only English data, and one trained only on a very large volume of data in another language, your technique could be used to translate between the two languages? Pretty cool. If we somehow came across a huge volume of text in an alien language, your technique could potentially translate their language into ours (although maybe the same could…

> (although maybe the same could be achieved just by training a single LLM on both languages?).

Intuitively I assume this would work even better.

Post reply on HN