Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jina AI launches open-source 8k text embedding

jina.ai

Post reply on HN
Print view View original

21–30 of 217 posts

Re: Jina AI launches open-source 8k text embedding

#21

Post by simonw »

I just shipped a new llm-embed-jina plugin for my LLM tool which provides access to these new Jina models: https://github.com/simonw/llm-embed-jina

Here's how to try it out.

First, install LLM. Use pip or pipx or brew:

    brew install llm
Next install the new plugin:

    llm install llm-embed-jina
You can confirm the new models are now available to LLM by running:

    llm embed-models
You should see a list that includes "jina-embeddings-v2-small-en" and "jina-embeddings-v2-base-en"

To embed a string using the small model, run this:

    llm embed -m jina-embeddings-v2-small-en -c 'Hello world'
That will output a JSON array of 512 floating point numbers (see my explainer here for what those are: https://simonwillison.net/2023/Oct/23/embeddings/#what-are-e...)

Embeddings are only really interesting if you store them and use them for comparisons.

Here's how to use the "llm embed-multi" command to create embeddings for the 30 most recent issues in my LLM GitHub repository:

    curl 'https://api.github.com/repos/simonw/llm/issues?state=all&filter=all' \
    | jq '[.[] | {id: .id, title: .title}]' \
    | llm embed-multi -m jina-embeddings-v2-small-en jina-llm-issues - \
    --store
This creates a collection called "jina-llm-issues" in a default SQLite database on your machine (the path to that can be found using "llm collections path").

To search for issues in that collection with titles most similar to the term "bug":

    llm similar jina-llm-issues -c 'bug'
Or for issues most similar to another existing issue by ID:

    llm similar jina-llm-issues 1922688957
Full documentation on what you can do with LLM and embeddings here: https://llm.datasette.io/en/stable/embeddings/index.html

Alternative recipe - this creates embeddings for every single README.md in the current directory and its subdirectories. Run this somewhere with a node_modules folder and you should get a whole lot of interesting stuff:

    llm embed-multi jina-readmes \
      -m jina-embeddings-v2-small-en \
      --files . '**/README.md' --store
Then search them like this:

    llm similar jina-readmes -c 'backup tools'

Re: Jina AI launches open-source 8k text embedding

#22

Post by X6S1x6Okd1st »

simonw wrote: post #21

I just shipped a new llm-embed-jina plugin for my LLM tool which provides access to these new Jina models: https://github.com/simonw/llm-embed-jina Here's how to try it out. First, install LLM. Use pip or pipx or brew: brew install llm Next install the new plugin: llm install llm-embed-jina You can confirm the new models are now available to LLM by running: llm embed-models You should see a list that includes "jina-e…

Thank you so much for all the work you've put into llm!

Re: Jina AI launches open-source 8k text embedding

#25

Post by simonw »

Nitrolo wrote: post #6

Is there something like oobabooga to easily run this in a click-and-run way? Where I can load up a model, a text, and ask it questions?

See my comment here: https://news.ycombinator.com/item?id=38020655 for a CLI tool that lets you do this.

Note that embedding models are a different kind of thing from a Large Language Model, so it's not the kind of model you can ask questions.

It's a model which can take text and turn it into an array of floating point numbers, which you can then use to implement things like semantic search and related documents.

More on that here: https://simonwillison.net/2023/Oct/23/embeddings/

Re: Jina AI launches open-source 8k text embedding

#26

Post by andy99 »

What is the use case for an 8k token embedding? My (somewhat limited) experience with long context models is they aren't great for RAG. I get the impression they are optimized for something else, like writing 8k+ tokens rather than synthesizing responses.

Isn't the normal way of using embedding to find relevant text snippets for a RAG prompt? Where is it better to have coarser retrieval?

Re: Jina AI launches open-source 8k text embedding

#28

Post by LoganDark »

RossBencina wrote: post #9

Some relevant stats from the link: 8192 token input sequence length 768 embedding dimensions 0.27GB model (with 0.07GB model also available) Tokeniser: BertTokenizer [1], 30528 token vocab [2] Is an 8K sequence length directly comparable to text-embedding-ada-002 if the vocabulary is much smaller? I seem to remember its tokeniser has a larger vocabulary. [1] https://huggingface.co/jinaai/jina-embeddings-v2-base-en/bl…

> Is an 8K sequence length directly comparable to text-embedding-ada-002 if the vocabulary is much smaller? I seem to remember its tokeniser has a larger vocabulary.

Words that aren't in the vocabulary can still be represented by multiple tokens. Some models can input and output valid UTF-8 at the byte level (rather than needing a unique token for each codepoint). For example RWKV-World.

Re: Jina AI launches open-source 8k text embedding

#29

Post by Zuiii »

Color me surprised! it looks like its actually open source (Apache 2.0) and not the usual false advertising by some two-faced company or institution. Links here:

* https://huggingface.co/jinaai/jina-embeddings-v2-base-en * https://huggingface.co/jinaai/jina-embeddings-v2-small-en

Re: Jina AI launches open-source 8k text embedding

#30

Post by space_fountain »

LoganDark wrote: post #28
RossBencina wrote: post #9

Some relevant stats from the link: 8192 token input sequence length 768 embedding dimensions 0.27GB model (with 0.07GB model also available) Tokeniser: BertTokenizer [1], 30528 token vocab [2] Is an 8K sequence length directly comparable to text-embedding-ada-002 if the vocabulary is much smaller? I seem to remember its tokeniser has a larger vocabulary. [1] https://huggingface.co/jinaai/jina-embeddings-v2-base-en/bl…

> Is an 8K sequence length directly comparable to text-embedding-ada-002 if the vocabulary is much smaller? I seem to remember its tokeniser has a larger vocabulary. Words that aren't in the vocabulary can still be represented by multiple tokens. Some models can input and output valid UTF-8 at the byte level (rather than needing a unique token for each codepoint). For example RWKV-World.

A large vocabulary means less tokens are needed to represent the same information
Post reply on HN