Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jina AI launches open-source 8k text embedding

jina.ai

Post reply on HN
Print view View original

191–200 of 217 posts

Re: Jina AI launches open-source 8k text embedding

#191

Post by egorfine »

pietz wrote: post #155

I'm always happy to see OSS contributions but I don't quite understand why this model is so remarkable. As the leaderboard suggests it's ranking lower than OpenAI embeddings, while 14 other contributions are even better than that. Many of which feature a comparable or lower dimensionality than 768. The 8k context window is new, but isn't the 512 token limitation a soft limit anyway? I'm pretty sure I can stuff bigger…

I fail to imagine a 8k-token-length piece of text that has just one single semantic coordinate and is appropriate for embedding and vector search.

In my experience, any text is better embedded using a sliding window of a few dozen words - this is the approximate size of semantic units in a written document in english; although this will wildly differ for different texts and topics.

Re: Jina AI launches open-source 8k text embedding

#192

Post by egorfine »

One thing that is missing in comparison: OpenAI's model is multilingual.

And not only it supports and embeds a variety of languages, it also computes the same coordinates for the same semantics in different languages. I.e. if you embed "russia is a terrorist state" and "россия - страна-террорист", both of these embeddings will have almost the same coordinates.

Re: Jina AI launches open-source 8k text embedding

#193

Post by simonw »

egorfine wrote: post #191
pietz wrote: post #155

I'm always happy to see OSS contributions but I don't quite understand why this model is so remarkable. As the leaderboard suggests it's ranking lower than OpenAI embeddings, while 14 other contributions are even better than that. Many of which feature a comparable or lower dimensionality than 768. The 8k context window is new, but isn't the 512 token limitation a soft limit anyway? I'm pretty sure I can stuff bigger…

I fail to imagine a 8k-token-length piece of text that has just one single semantic coordinate and is appropriate for embedding and vector search. In my experience, any text is better embedded using a sliding window of a few dozen words - this is the approximate size of semantic units in a written document in english; although this will wildly differ for different texts and topics.

What are you using those embeddings for?

I can see a sliding window working for semantic search and RAG, but not so much for clustering or finding related documents.

Re: Jina AI launches open-source 8k text embedding

#194

Post by egorfine »

simonw wrote: post #193
egorfine wrote: post #191

Earlier quoted context omitted.

I fail to imagine a 8k-token-length piece of text that has just one single semantic coordinate and is appropriate for embedding and vector search. In my experience, any text is better embedded using a sliding window of a few dozen words - this is the approximate size of semantic units in a written document in english; although this will wildly differ for different texts and topics.

What are you using those embeddings for? I can see a sliding window working for semantic search and RAG, but not so much for clustering or finding related documents.

Ah yes, clustering is indeed something that would benefit from large context, I agree.

However even so I would think about the documents themselves and figure out if it is even needed. Lets say we are talking about clustering court proceedings. I'd rather extract the abstract from these document, embed and cluster those instead of the whole text.

Re: Jina AI launches open-source 8k text embedding

#195

Post by kristopolous »

Grimburger wrote: post #70
simonw wrote: post #68

Earlier quoted context omitted.

> You could just sum it up for us all rather than do a divert to your blog? Why? Have links gone out of fashion? I even linked directly to the relevant section rather than linking to the top of the page. The paper that coined the term used the hyphen, though I think I prefer it without: https://arxiv.org/abs/2005.11401

> Have links gone out of fashion? Yes. You wrote far more words than needed to answer the comment, I did it for you instead.

You know you're responding to a programmer famous enough to have a Wikipedia page, right?

https://en.m.wikipedia.org/wiki/Simon_Willison

Re: Jina AI launches open-source 8k text embedding

#196

Post by mike_ivanov »

simonw wrote: post #105
mike_ivanov wrote: post #103

Earlier quoted context omitted.

JFYI, this is what happens on my M1 Macbook: $ brew install llm $ llm ModuleNotFoundError: No module named 'typing_extensions' Not sure where to report it.

Whoa, that is a weird one. Do you know what version of Python you have from Homebrew? It looks like that package is correctly listed in the formula: https://github.com/Homebrew/homebrew-core/blob/a0048881ba9a2...

    % python3 --version
    Python 3.11.6
    
    % which python3
    /opt/homebrew/bin/python3

    % brew info python-typing-extensions
    ==> python-typing-extensions: stable 4.8.0 (bottled)

Re: Jina AI launches open-source 8k text embedding

#197

Post by mike_ivanov »

IanCal wrote: post #144
mike_ivanov wrote: post #103

Earlier quoted context omitted.

JFYI, this is what happens on my M1 Macbook: $ brew install llm $ llm ModuleNotFoundError: No module named 'typing_extensions' Not sure where to report it.

Probably not this, but check with `which llm` what that's running. I had weird issues not matching the documentation but just had some other random python cli tool called llm I'd put in my home bin for and forgotten about it.

    % which llm
    /opt/homebrew/bin/llm

Re: Jina AI launches open-source 8k text embedding

#199

Post by minimaxir »

karxxm wrote: post #173
teaearlgraycold wrote: post #104

Earlier quoted context omitted.

You can do PCA or some other dimensionality reduction technique. That’ll reduce computation and improve signal/noise ratio when comparing vectors.

Unfortunately this is not feasible with a large amount of words due to the quadratic scaling. But thanks for the response!

Not sure what you mean by large amount of words. You can fit a PCA on millions of vectors relatively performantly, then inference from it is just a matmul.

Re: Jina AI launches open-source 8k text embedding

#200

Post by m3kw9 »

egorfine wrote: post #192

One thing that is missing in comparison: OpenAI's model is multilingual. And not only it supports and embeds a variety of languages, it also computes the same coordinates for the same semantics in different languages. I.e. if you embed "russia is a terrorist state" and "россия - страна-террорист", both of these embeddings will have almost the same coordinates.

I don’t really know what that means but it seems useful
Post reply on HN