Just noticed that they (jina.ai) have offices both in Berlin and China. I am wondering how they will they operate with the presence of chip export restrictions and other side effects of USA / China tensions.
[dead]
Jina AI launches open-source 8k text embedding
161–170 of 217 posts
Re: Jina AI launches open-source 8k text embedding
#162I'm always happy to see OSS contributions but I don't quite understand why this model is so remarkable. As the leaderboard suggests it's ranking lower than OpenAI embeddings, while 14 other contributions are even better than that. Many of which feature a comparable or lower dimensionality than 768. The 8k context window is new, but isn't the 512 token limitation a soft limit anyway? I'm pretty sure I can stuff bigger…
I have been trying to understand the hype as well. Happy to see all the work happening in this space still.
I was pretty curious about the context limit. I am not an expert in this area but I always thought the biggest problem was the length of your original text. So typically you might only encode a sentence or a selection of sentences. You could always stuff more in but they you are potentially losing the specificity, I would think that is a function of the dimensionality. This model is 768, are they saying I can stuff 8k tokens worth of text and can utilize it just as well as I have with other models on a per 1-3 sentence level?
Re: Jina AI launches open-source 8k text embedding
#163I just shipped a new llm-embed-jina plugin for my LLM tool which provides access to these new Jina models: https://github.com/simonw/llm-embed-jina Here's how to try it out. First, install LLM. Use pip or pipx or brew: brew install llm Next install the new plugin: llm install llm-embed-jina You can confirm the new models are now available to LLM by running: llm embed-models You should see a list that includes "jina-e…
FYI it seems that llm install llm-embed-jina is missing yaml dependency
File "/opt/homebrew/Cellar/llm/0.11_1/libexec/lib/python3.12/site-packages/llm/default_plugins/openai_models.py", line 17, in
import yamlRe: Jina AI launches open-source 8k text embedding
#164Just quantized the models for onnx usage in e.g. transformers.js and got 4x reduced file size:
- 𝟐𝟖.𝟓 𝐌𝐁 jina-embeddings-v2-small-en (https://huggingface.co/do-me/jina-embeddings-v2-small-en)
- 𝟏𝟎𝟗 𝐌𝐁 jina-embeddings-v2-base-en (https://huggingface.co/do-me/jina-embeddings-v2-base-en)
However, I noted, that the base model is performing quite poorly on small text chunks (a few words) while the small version seems to be unaffected. Might this be some kind of side effect due to the way they deal with large contexts?
If you want to test, you can head over to SemanticFinder (https://do-me.github.io/SemanticFinder/), go to advanced settings, choose the Jina AI base model (at the very bottom) and run with "Find". You'll see that all other models perform just fine and find "food"-related chunks but the base version doesn't.
Re: Jina AI launches open-source 8k text embedding
#165this is super cool! I wish there was an easy to understand and follow guide on how to make your own embedding, for llama2 for example. All I can find are various guides that already assume you know everything there is to training an embedding. I just want to make an embedding between a conversation of me and my friend and simulate talking to them. Is this a hard thing to train to begin with? If anyone knows or could…
I will butcher this so if any experts see this please don't flame me. I think you might be conflating ideas? You could definitely fine-tune existing embedding models or train your own from scratch but the goals of embeddings models are different than a LLM conversation. Embedding models are used for things like, classifying, search, image captioning...maybe at a high level anything where you have high dimensionality that you need to condense?
What you are asking for sounds like fine tuning an existing LLM...where the data will be tokenized but the outcomes are different? There is a lot of writeups on how people have done it. You should especially follow some of the work on Huggingface. To replicate talking to your friend though, you will need a very large dataset to train off of I would think and its unclear to me if you can just fine-tune it or you would need to train a model from scratch. So a dataset with 10s of thousands of examples and then you need to train it on a GPU.
https://www.anyscale.com/blog/fine-tuning-llama-2-a-comprehe...
Re: Jina AI launches open-source 8k text embedding
#166Some relevant stats from the link: 8192 token input sequence length 768 embedding dimensions 0.27GB model (with 0.07GB model also available) Tokeniser: BertTokenizer [1], 30528 token vocab [2] Is an 8K sequence length directly comparable to text-embedding-ada-002 if the vocabulary is much smaller? I seem to remember its tokeniser has a larger vocabulary. [1] https://huggingface.co/jinaai/jina-embeddings-v2-base-en/bl…
Thanks what size gpu would you need to fine tune or do an inference?
Re: Jina AI launches open-source 8k text embedding
#167When I go to this leaderboard:
https://huggingface.co/spaces/mteb/leaderboard
I click on the "Classification" tab, then I see "jina-embeddings-v2-base-en" at number 12, with an average score of 73.45. But the highest scoring model there is llmrails/ember-v1 with 75.99 average score but it only supports 512 tokens, so if you need 8K tokens to be embedded, I guess they are the best. Do people need 8K of tokens for embedding? Maybe not but they might need more than 512 often enough. It could save a summary extraction step.
Re: Jina AI launches open-source 8k text embedding
#168I'm always happy to see OSS contributions but I don't quite understand why this model is so remarkable. As the leaderboard suggests it's ranking lower than OpenAI embeddings, while 14 other contributions are even better than that. Many of which feature a comparable or lower dimensionality than 768. The 8k context window is new, but isn't the 512 token limitation a soft limit anyway? I'm pretty sure I can stuff bigger…
I have been trying to understand the hype as well. Happy to see all the work happening in this space still. I was pretty curious about the context limit. I am not an expert in this area but I always thought the biggest problem was the length of your original text. So typically you might only encode a sentence or a selection of sentences. You could always stuff more in but they you are potentially losing the specifici…
Thinking about it some more as I read through more comments. I guess in the stated case of research papers it can make sense if your task is looking for the common themes and not specific details. If you are embedding a sentence or a paragraph you miss out on the connection between those sentences across the whole paper...or at least its harder to manage that. By encoding a large number of pages from the paper (or the entire paper) you can hopefully do a better job of capturing the theme of that paper.
This also opens up another question though, how would that compare to using a LLM to summarize that paper and then just embed on top of that summary.
Re: Jina AI launches open-source 8k text embedding
#169This is great news! It feels like open-source is closing the gap with "Open"AI which is really exciting, and the acceleration towards parity is faster than more advancements made on the closed source models. Maybe it's wishful thinking though?
Wishful thinking? Embeddings to me were never the interesting or bleeding edge thing at OpenAI. Maybe the various ada models at one point reigned supreme but there have been open-source models at the top of the leaderboard for a while and from a cost/performance perspective, often even the Bert models did a really fine job.
Re: Jina AI launches open-source 8k text embedding
#170This is great news! It feels like open-source is closing the gap with "Open"AI which is really exciting, and the acceleration towards parity is faster than more advancements made on the closed source models. Maybe it's wishful thinking though?
They compare it to OpenAI's ada model though, which is light-years away from ChatGPT.
Does that not conflate two different things though? Embedding model != LLM Model ?