Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jina AI launches open-source 8k text embedding

jina.ai

Post reply on HN
Print view View original

171–180 of 217 posts

Re: Jina AI launches open-source 8k text embedding

#171

Post by larodi »

pietz wrote: post #155

I'm always happy to see OSS contributions but I don't quite understand why this model is so remarkable. As the leaderboard suggests it's ranking lower than OpenAI embeddings, while 14 other contributions are even better than that. Many of which feature a comparable or lower dimensionality than 768. The 8k context window is new, but isn't the 512 token limitation a soft limit anyway? I'm pretty sure I can stuff bigger…

Potentially useful for paragraph embedding, where... well, paragraphs can grow a lot. Not sure how this model fares in comparison to other embedding engines (yet), but I can definitely tell you mpnet models fare much better for paragraph embeddings than the leader in HF's leaderboard (being thenlper/gte-large at time of writing).

I can guess the Davinci and similar embeddings work better for code than MPNET and it really matters what you are encoding, not only the context length. What features are actually being extracted by the emb.engine.

Re: Jina AI launches open-source 8k text embedding

#172

Post by cztomsik »

luke-stanley wrote: post #167

When I go to this leaderboard: https://huggingface.co/spaces/mteb/leaderboard I click on the "Classification" tab, then I see "jina-embeddings-v2-base-en" at number 12, with an average score of 73.45. But the highest scoring model there is llmrails/ember-v1 with 75.99 average score but it only supports 512 tokens, so if you need 8K tokens to be embedded, I guess they are the best. Do people need 8K of tokens for embe…

Small context window means you cannot embed the whole document, you are embedding just a part.

So, if there is some information at the bottom which is dependent on something which is at the top, your embedding could be entirely wrong.

Re: Jina AI launches open-source 8k text embedding

#173

Post by karxxm »

teaearlgraycold wrote: post #104
karxxm wrote: post #101

Earlier quoted context omitted.

You wrote „out of the box“, did you find a way to improve this?

You can do PCA or some other dimensionality reduction technique. That’ll reduce computation and improve signal/noise ratio when comparing vectors.

Unfortunately this is not feasible with a large amount of words due to the quadratic scaling. But thanks for the response!

Re: Jina AI launches open-source 8k text embedding

#174

Post by kordlessagain »

pjc50 wrote: post #135
SOLAR_FIELDS wrote: post #71

Earlier quoted context omitted.

The very definition of what constitutes open source is being called into question in these kinds of discussions about AI. Without the training details and the weights being made fully open it’s hard to really call something truly open, even if it happens to meet some arbitrary definition of “open source”. A good definition of “truly open” is whether the exact same results can be reproduced by someone with no extra in…

The old Stallman definition used the phrase "preferred form for modification" rather than the more specific "source code". What do you need to effectively modify an AI model?

Usually the datasets, not the source code.

Re: Jina AI launches open-source 8k text embedding

#175

Post by darkwater »

EGreg wrote: post #147
darkwater wrote: post #145

Earlier quoted context omitted.

Good example. And in fact you are calling the "engine" opensource, not the whole Quake game. The 'assets" in most "opensource" AI models are not available.

Imagine if the Telegram client was open source but not the backend. Imagine if Facebook open-sourced their front-end libraries like React but not the back-end. Imagine if Twitter or Google didn’t publish its Algorithm for how they rank things to display to different people. You don’t need to imagine. That’s exactly what’s happening! Would you call them open source because their front end is open source? Could you hos…

I completely agree with you (and the example you mention are singled out in the "antifeatures" list in F-Droid, to name an example)

Re: Jina AI launches open-source 8k text embedding

#176

Post by simonw »

thomasahle wrote: post #119
simonw wrote: post #87

Earlier quoted context omitted.

My previous implementation used TF-IDF - I basically took all the words in the post and turned them into a giant "word OR word OR word OR word" search query and piped that through SQLite full-text search. https://til.simonwillison.net/sqlite/related-content I jumped straight from that to OpenAI embeddings. The results were good enough that I didn't spend time investigating other approaches.

> Into a giant "word OR word OR word OR word" Does that mean you'd return other docs if they share just one word? The idea of tfidf is that it gives you a vector (maybe combined with pca or a random dimensionality reduction) that you can use just like an Ada embedding. But you still need vector search.

My goal for related articles was to first filter to every document that shared at least one word with the target - which is probably EVERY document in the set - but then rank them based on which ones share the MOST words, scoring words that are rare in the corpus more highly. BM25 does that for free.

Then I take the top ten by score and call those the "related articles".

Re: Jina AI launches open-source 8k text embedding

#177

Post by jerpint »

TeMPOraL wrote: post #99
okaram wrote: post #86

Earlier quoted context omitted.

Notice you are creating your own arbitrary definition of 'truly open', which IMHO corresponds more with 'reproducible'. We already have a definition of open source. I don't see any reason to change it.

Problem is, the literal/default definition of "open source" is meaningless/worthless in this context. It's the weights, training data and methodology that matter for those models - NOT the inference shell. It's basically like giving people a binary program and calling it open source because the compiler and runtime used are open source.

The weights are the inference and result of training. I can give you all the training details and you might not be able to reproduce what I did (google does this all the time). As a dev, I’d much rather an open model over an open recipe without weights. We can all agree having both is the best case scenario but having openly licensed weights is for me the bare minimum of open source

Re: Jina AI launches open-source 8k text embedding

#178

Post by simonw »

TeMPOraL wrote: post #127
minimaxir wrote: post #65

Earlier quoted context omitted.

That's a property of Word2Vec specifically due to how it's trained (a shallow network where most of the "logic" would be contained within the embeddings themselves). Using it for embeddings generated from LLMs or Embedding layers will not give as fun results; in practice the only thing you can do is average or cluster them.

> That's a property of Word2Vec specifically due to how it's trained (a shallow network where most of the "logic" would be contained within the embeddings themselves). Is it though? I thought the LLM-based embeddings are even more fun for this, as you have many more interesting directions to move in. I.e. not just: emb("king") - emb("man") + emb("woman") = emb("queen") But also e.g.: emb( ) + a v(sad) + b v(short) -…

That talk used a novel embeddings model trained by the speaker which does exhibit this kind of property - but that was a new (extremely cool) thing, not something that other embeddings models can do.

Re: Jina AI launches open-source 8k text embedding

#179

Post by kordlessagain »

TeMPOraL wrote: post #108
rolisz wrote: post #55

Earlier quoted context omitted.

I think the point is: if you compress 25 pages of text into 1024 floats, you will lose a ton of information, regardless of what the use case is, so you're probably still better of with chunking.

> if you compress 25 pages of text into 1024 floats, you will lose a ton of information Sure, but then if you do it one page at a time, or one paragraph at a time, you lose ton of meaning - after all, individual paragraphs aren't independent of each other. And meaning is kind of the whole point of the exercise. Or put another way, squashing a ton of text loses you some high-frequency information, while chunking cuts…

I think that the assumption that you lose a ton of meaning (of low frequency) in doing separate chunks is probably less likely to be true over doing the whole document at once (losing high frequency meaning). As you say, doing both is probably a good strategy, and I think that's why we see a lot of "summarize this text" approaches.

I use a multi-pronged approach to this based on a special type of summarization. I chunk on sentences using punctuation until they are just over 512 characters, then I embed them. After embedding, I ask a foundation model to summarize (or ask a question about the chunk) and then generate keyterms for it. Those keyterms are stored along with the vector in the database. During search, I use the user's input to do a vector search for matching chunks, then pull their keyterms in. Using those keyterms, I do set operations to find related chunks. I then run a vector search against these to the top matches from the vector search to assemble new prompt text.

This strategy is based on the idea of a "back of the book index". It is entirely plausible to look for "outliers" in the keyterms and consider throwing those chunks with those keyterms in there to see if it nets us understanding of some "hidden" meaning in the document.

There is also a means to continue doing the "keyterm" extraction trick as the system is used. Keyterms from answer as well as user prompts may be added to the existing index over time, thus helping improve the ability to return low frequency information that may be initially hidden.

Re: Jina AI launches open-source 8k text embedding

#180

Post by simonw »

Gasp0de wrote: post #150
burcs wrote: post #3

This is great news! It feels like open-source is closing the gap with "Open"AI which is really exciting, and the acceleration towards parity is faster than more advancements made on the closed source models. Maybe it's wishful thinking though?

They compare it to OpenAI's ada model though, which is light-years away from ChatGPT.

Don't confuse the current Ada embedding model the old Ada GPT3 model.

It turns out OpenAI have used the name "Ada" for several very different things, purely because they went through a phase of giving everything Ada/Babbage/Curie/DaVinci names because they liked the A/B/C/D thing to indicate which of their models were largest.

Post reply on HN