Jina AI launches open-source 8k text embedding
1–10 of 217 posts
Re: Jina AI launches open-source 8k text embedding
#2[deleted]
Re: Jina AI launches open-source 8k text embedding
#3This is great news!
It feels like open-source is closing the gap with "Open"AI which is really exciting, and the acceleration towards parity is faster than more advancements made on the closed source models. Maybe it's wishful thinking though?
Re: Jina AI launches open-source 8k text embedding
#4Impressive work.
I wonder what would be the best way to use 8k embeddings. It’s a lot of information to keep in a vector, so things like “precision” of the embedding space and its ability to distinguish very similar large documents will be key.
Maybe it can be useful for coarse similarity matching, for example to detect plagiarism?
Re: Jina AI launches open-source 8k text embedding
#5Anyone got links to examples of text embedding?
Re: Jina AI launches open-source 8k text embedding
#6Is there something like oobabooga to easily run this in a click-and-run way? Where I can load up a model, a text, and ask it questions?
Re: Jina AI launches open-source 8k text embedding
#7Impressive work. I wonder what would be the best way to use 8k embeddings. It’s a lot of information to keep in a vector, so things like “precision” of the embedding space and its ability to distinguish very similar large documents will be key. Maybe it can be useful for coarse similarity matching, for example to detect plagiarism?
8K is the context length. Their vector dimension size is actual much smaller, which is great for a number of use cases, though maybe not the ones you are thinking about.
Re: Jina AI launches open-source 8k text embedding
#8Does anyone know what they are using for this comparison and ranking? And where does instruct-xl stand in the mix?
Re: Jina AI launches open-source 8k text embedding
#9Some relevant stats from the link:
8192 token input sequence length
768 embedding dimensions
0.27GB model (with 0.07GB model also available)
Tokeniser: BertTokenizer [1], 30528 token vocab [2]
Is an 8K sequence length directly comparable to text-embedding-ada-002 if the vocabulary is much smaller? I seem to remember its tokeniser has a larger vocabulary.
[1] https://huggingface.co/jinaai/jina-embeddings-v2-base-en/blo...
[2] https://huggingface.co/jinaai/jina-embeddings-v2-base-en/blo...
Re: Jina AI launches open-source 8k text embedding
#10Does anyone know what they are using for this comparison and ranking? And where does instruct-xl stand in the mix?
Oh duh, it’s right in the post and instructor-xl is number 9. And so many new participants now!