Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jina AI launches open-source 8k text embedding

jina.ai

Post reply on HN
Print view View original

61–70 of 217 posts

Re: Jina AI launches open-source 8k text embedding

#61

Post by dragonwriter »

andy99 wrote: post #26

What is the use case for an 8k token embedding? My (somewhat limited) experience with long context models is they aren't great for RAG. I get the impression they are optimized for something else, like writing 8k+ tokens rather than synthesizing responses. Isn't the normal way of using embedding to find relevant text snippets for a RAG prompt? Where is it better to have coarser retrieval?

> What is the use case for an 8k token embedding?

Calculating embeddings on larger documents than smaller-window embedding models.

> My (somewhat limited) experience with long context models is they aren't great for RAG.

The only reason they wouldn't be great for RAG is that they aren't great at using information in their context window, which is possible (ISTR that some models have a strong recency bias within the window, for instance) but I don't think is a general problem of long context models.

> Isn't the normal way of using embedding to find relevant text snippets for a RAG prompt?

I would say the usual use is for search and semantic similarity comparisons generally. RAG is itself an application of search, but its not the only one.

Re: Jina AI launches open-source 8k text embedding

#62

Post by udev4096 »

burcs wrote: post #3

This is great news! It feels like open-source is closing the gap with "Open"AI which is really exciting, and the acceleration towards parity is faster than more advancements made on the closed source models. Maybe it's wishful thinking though?

Is it tho? It's not really open source if they don't give us the information regarding training datasets

Re: Jina AI launches open-source 8k text embedding

#63

Post by simonw »

simonw wrote: post #21

I just shipped a new llm-embed-jina plugin for my LLM tool which provides access to these new Jina models: https://github.com/simonw/llm-embed-jina Here's how to try it out. First, install LLM. Use pip or pipx or brew: brew install llm Next install the new plugin: llm install llm-embed-jina You can confirm the new models are now available to LLM by running: llm embed-models You should see a list that includes "jina-e…

Wrote this up on my blog: https://simonwillison.net/2023/Oct/26/llm-embed-jina/

Re: Jina AI launches open-source 8k text embedding

#64

Post by helloplanets »

e1g wrote: post #53
simonw wrote: post #46

Earlier quoted context omitted.

No, you're confusing two things here. "text-ada-001" is LLM in the GPT3 family, described as "Capable of very simple tasks, usually the fastest model in the GPT-3 series, and lowest cost" "text-embedding-ada-002" is entirely different - that page describes it as "Our second generation embedding model, text-embedding-ada-002 is a designed to replace the previous 16 first-generation embedding models at a fraction of th…

OpenAI doesn't say directly what text-embedding-ada-002 is, but in the release blog post they show that performance is comparable to davinci/curie, which places it firmly in the universe of GPT3. I understand it's not a straight line comparison, but to me it's still a useful mental heuristic about what to expect. [1] https://openai.com/blog/new-and-improved-embedding-model (see "Model improvements")

Reading through that article, the specific Davinci/Curie models they seem to be referring to are called the following: 'text-search-davinci-001', 'text-search-curie-001', 'text-similarity-davinci-001' and 'text-similarity-curie-001'.

Are you sure these have anything to do with 'text-davinci-003' or 'text-curie-001'?

Will have to agree with everyone here that OpenAI is good at being extremely confusing. It seems like the logic might be something along the lines of the 'text-search' portion being the actual type of the model, while the 'curie-001' / '-' format is just a personalized way of expressing the version of that type of model. And the whole 'GPT' category used to be a sort family of models, but now they've just switched it to the actual name of the newer gargantuan LLMs. Then, because the 'GPT' models are now that different thing altogether these days, the newest 'text-embedding' model is just named 'ada-' because it's on that iteration of the 'text-embedding' type of model, adhering to the older principle of naming their models? Not sure, ha. Definitely feels like doing some detective work.

Re: Jina AI launches open-source 8k text embedding

#65

Post by minimaxir »

osigurdson wrote: post #39
BoorishBears wrote: post #15

Earlier quoted context omitted.

Easiest example is taking three words: Universe, University, College. - University and Universe are similar alphabetically. - University and College are similar in meaning. Take embeddings for those three words and `University` will be near `College`, while `Universe` will be further away, because embeddings capture meaning: University College Universe _ With old school search you'd need to handle the special case of…

Another interesting point is that math can be performed on embedding vectors: emb("king") - emb("man") + emb("woman") = emb("queen").

That's a property of Word2Vec specifically due to how it's trained (a shallow network where most of the "logic" would be contained within the embeddings themselves). Using it for embeddings generated from LLMs or Embedding layers will not give as fun results; in practice the only thing you can do is average or cluster them.

Re: Jina AI launches open-source 8k text embedding

#66

Post by simonw »

e1g wrote: post #53
simonw wrote: post #46

Earlier quoted context omitted.

No, you're confusing two things here. "text-ada-001" is LLM in the GPT3 family, described as "Capable of very simple tasks, usually the fastest model in the GPT-3 series, and lowest cost" "text-embedding-ada-002" is entirely different - that page describes it as "Our second generation embedding model, text-embedding-ada-002 is a designed to replace the previous 16 first-generation embedding models at a fraction of th…

OpenAI doesn't say directly what text-embedding-ada-002 is, but in the release blog post they show that performance is comparable to davinci/curie, which places it firmly in the universe of GPT3. I understand it's not a straight line comparison, but to me it's still a useful mental heuristic about what to expect. [1] https://openai.com/blog/new-and-improved-embedding-model (see "Model improvements")

You mean this table here?

    text-embedding-ada-002     53.3
    text-search-davinci-*-001 52.8
    text-search-curie-*-001     50.9
    text-search-babbage-*-001 50.4
    text-search-ada-*-001     49.0
That's not comparing it to the davinci/curie/babbage GPT3 models, it's comparing to the "search-text-*" family.

Those were introduced in https://openai.com/blog/introducing-text-and-code-embeddings as the first public release of embeddings models from OpenAI.

> We’re releasing three families of embedding models, each tuned to perform well on different functionalities: text similarity, text search, and code search. The models take either text or code as input and return an embedding vector.

It's not at all clear to me if there's any relationship between those and the GPT3 davinci/curie/babbage/ada models.

My guess is that OpenAI's naming convention back then was "davinci is the best one, then curie, then babbage, then ada".

Re: Jina AI launches open-source 8k text embedding

#67

Post by jerpint »

udev4096 wrote: post #62
burcs wrote: post #3

This is great news! It feels like open-source is closing the gap with "Open"AI which is really exciting, and the acceleration towards parity is faster than more advancements made on the closed source models. Maybe it's wishful thinking though?

Is it tho? It's not really open source if they don't give us the information regarding training datasets

It definitely is open source even if they don’t disclose all details behind the training

Re: Jina AI launches open-source 8k text embedding

#68

Post by simonw »

Grimburger wrote: post #60
simonw wrote: post #49

Earlier quoted context omitted.

I have an explanation of RAG in the context of embeddings here: https://simonwillison.net/2023/Oct/23/embeddings/#answering-...

You could just sum it up for us all rather than do a divert to your blog? It's Retrieval Augmented Generation btw. To quote: > The key idea is this: a user asks a question. You search your private documents for content that appears relevant to the question, then paste excerpts of that content into the LLM (respecting its size limit, usually between 3,000 and 6,000 words) along with the original question. > The LLM ca…

> You could just sum it up for us all rather than do a divert to your blog?

Why? Have links gone out of fashion?

I even linked directly to the relevant section rather than linking to the top of the page.

The paper that coined the term used the hyphen, though I think I prefer it without: https://arxiv.org/abs/2005.11401

Re: Jina AI launches open-source 8k text embedding

#69

Post by pknerd »

clarkmcc wrote: post #59
pknerd wrote: post #54

Pardon my ignorance in advance but could it be used to "chat" with PDFs and websites? I am looking for OpenAI alternatives as I am in learning phase

Check out my little side project for chatting with PDFs. You should be able to load most models including this one. https://github.com/clarkmcc/chitchat

This looks cool so can it be used to feed Website/Products data in CSV/JSON format and "chat" with it?

Re: Jina AI launches open-source 8k text embedding

#70

Post by Grimburger »

simonw wrote: post #68
Grimburger wrote: post #60

Earlier quoted context omitted.

You could just sum it up for us all rather than do a divert to your blog? It's Retrieval Augmented Generation btw. To quote: > The key idea is this: a user asks a question. You search your private documents for content that appears relevant to the question, then paste excerpts of that content into the LLM (respecting its size limit, usually between 3,000 and 6,000 words) along with the original question. > The LLM ca…

> You could just sum it up for us all rather than do a divert to your blog? Why? Have links gone out of fashion? I even linked directly to the relevant section rather than linking to the top of the page. The paper that coined the term used the hyphen, though I think I prefer it without: https://arxiv.org/abs/2005.11401

> Have links gone out of fashion?

Yes.

You wrote far more words than needed to answer the comment, I did it for you instead.

Post reply on HN