Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jina AI launches open-source 8k text embedding

jina.ai

Post reply on HN
Print view View original

111–120 of 217 posts

Re: Jina AI launches open-source 8k text embedding

#111

Post by antman »

refulgentis wrote: post #47
teaearlgraycold wrote: post #43

Earlier quoted context omitted.

You could get a facsimile to a summary for a full article or short story. Reducing an 8k token article to a summary using a completions model would cost far more. So if you need to search through collections of contracts, scientific papers, movie scripts, etc. for recommendations/clustering then bigger input sizes can do that in one shot. Think of it like skipping the square root step in Euclidean distance. Perfectly…

I think I'm missing something: like, yeah, it's vector search for bigger text chunks. But arguably vector search with bigger text chunks is _definitively_ worse -- this isn't doing summarization, just turning about 25 pages of text to 1024 floats, which you then can use cosine similarity to measure the semantic similarity to other text I'd much rather know what paragraph to look in than what 25 pages to look in

you could do both

Re: Jina AI launches open-source 8k text embedding

#112

Post by TeMPOraL »

teaearlgraycold wrote: post #93
rolisz wrote: post #55

Earlier quoted context omitted.

I think the point is: if you compress 25 pages of text into 1024 floats, you will lose a ton of information, regardless of what the use case is, so you're probably still better of with chunking.

Ever read the back of a book?

You mean the marketing blurb? Those tend to carry low information value, sometimes even negative - as in, if you didn't know anything else about the book, reading the blurb will make you even more wrong about it than you were. This is a common feature of marketing copy.

Re: Jina AI launches open-source 8k text embedding

#113

Post by pietro72ohboy »

dcastm wrote: post #110

I wonder how much better is this, compared to taking the average ( or some other aggregation) of embeddings with a smaller context length. Has anyone done a similar comparison?

The issue with averaging is that over large inputs, it drowns out small signal. For example, there is a chance that it completely loses a reference to something made only in a single sentence somewhere in a large document.

Re: Jina AI launches open-source 8k text embedding

#114

Post by TeMPOraL »

minimaxir wrote: post #34
jncraton wrote: post #14

This is great to see. It looks like the size of the embedding vector is half the size of text-embedding-ada-002 (768 vs 1536) while providing competitive performance. This will save space in databases and make lookups somewhat faster. For those unaware, if 512 tokens of context is sufficient for your use case, there are already many options that outperform text-embedding-ada-002 on common benchmarks: https://huggingf…

The 768D-sized embeddings compared to OpenAI's 1536D embeddings are actually a feature outside of index size. In my experience, OpenAI's embeddings are overspecified and do very poorly with cosine similarity out of the box as they match syntax more than semantic meaning (which is important as that's the metric for RAG). Ideally you'd want cosine similarity in the range of [-1, 1] on a variety of data but in my experi…

Unless I'm missing something, it should be possible to map out in advance which dimensions represent syntactic aspects, and then downweigh or remove them for similarity comparisons. And that map should be a function of the model alone, i.e. fully reusable. Are there any efforts to map out the latent space of ada models like that?

Re: Jina AI launches open-source 8k text embedding

#115

Post by abriosi »

SOLAR_FIELDS wrote: post #71
jerpint wrote: post #67

Earlier quoted context omitted.

It definitely is open source even if they don’t disclose all details behind the training

The very definition of what constitutes open source is being called into question in these kinds of discussions about AI. Without the training details and the weights being made fully open it’s hard to really call something truly open, even if it happens to meet some arbitrary definition of “open source”. A good definition of “truly open” is whether the exact same results can be reproduced by someone with no extra in…

Imagine someone giving you a executable binary without the source code and calling it "open source"

Re: Jina AI launches open-source 8k text embedding

#116

Post by [deleted] »

Grimburger wrote: post #60
simonw wrote: post #49

Earlier quoted context omitted.

I have an explanation of RAG in the context of embeddings here: https://simonwillison.net/2023/Oct/23/embeddings/#answering-...

You could just sum it up for us all rather than do a divert to your blog? It's Retrieval Augmented Generation btw. To quote: > The key idea is this: a user asks a question. You search your private documents for content that appears relevant to the question, then paste excerpts of that content into the LLM (respecting its size limit, usually between 3,000 and 6,000 words) along with the original question. > The LLM ca…

[deleted]

Re: Jina AI launches open-source 8k text embedding

#117

Post by [deleted] »

Grimburger wrote: post #70
simonw wrote: post #68

Earlier quoted context omitted.

> You could just sum it up for us all rather than do a divert to your blog? Why? Have links gone out of fashion? I even linked directly to the relevant section rather than linking to the top of the page. The paper that coined the term used the hyphen, though I think I prefer it without: https://arxiv.org/abs/2005.11401

> Have links gone out of fashion? Yes. You wrote far more words than needed to answer the comment, I did it for you instead.

[deleted]

Re: Jina AI launches open-source 8k text embedding

#118

Post by xnorswap »

richardw wrote: post #106
SOLAR_FIELDS wrote: post #71

Earlier quoted context omitted.

The very definition of what constitutes open source is being called into question in these kinds of discussions about AI. Without the training details and the weights being made fully open it’s hard to really call something truly open, even if it happens to meet some arbitrary definition of “open source”. A good definition of “truly open” is whether the exact same results can be reproduced by someone with no extra in…

So if someone includes images in their project they need to tell you every brush stroke that led to the final image? All sorts of intangibles end up in open source projects. This isn’t a science experiment that needs replication. They’re not trying to prove how they came up with the image/code/model.

Those "Brush Strokes" are effectively the source code. To be considered open source, yes source code needs to be provided along side the binaries (the "image").

Re: Jina AI launches open-source 8k text embedding

#119

Post by thomasahle »

simonw wrote: post #87
thomasahle wrote: post #85

Earlier quoted context omitted.

Did you compare with simple baselines like bag-of-words and word vectors?

My previous implementation used TF-IDF - I basically took all the words in the post and turned them into a giant "word OR word OR word OR word" search query and piped that through SQLite full-text search. https://til.simonwillison.net/sqlite/related-content I jumped straight from that to OpenAI embeddings. The results were good enough that I didn't spend time investigating other approaches.

> Into a giant "word OR word OR word OR word"

Does that mean you'd return other docs if they share just one word?

The idea of tfidf is that it gives you a vector (maybe combined with pca or a random dimensionality reduction) that you can use just like an Ada embedding. But you still need vector search.

Post reply on HN