Jina AI launches open-source 8k text embedding

Post by simonw » Thu, Oct 26, 2023, 4:51 AM UTC

e1g wrote: post #78

simonw wrote: post #66

Earlier quoted context omitted.

You mean this table here? text-embedding-ada-002 53.3 text-search-davinci-*-001 52.8 text-search-curie-*-001 50.9 text-search-babbage-*-001 50.4 text-search-ada-*-001 49.0 That's not comparing it to the davinci/curie/babbage GPT3 models, it's comparing to the "search-text-*" family. Those were introduced in https://openai.com/blog/introducing-text-and-code-embeddings as the first public release of embeddings models f…

How interesting. I assumed that a consistent codename such as Ada/Davinci refers to the lineage/DNA of the OpenAI model from which a distinct product was created. But I can see how these codenames could be "just" a revision label of A/B/C/D (Ada/Babbage/Curie/Davinci), similar to "Pro/Max/Ultra". If true, a product named "M2 Ultra" could have nothing to do with another product called "Watch Ultra".

Post by pknerd » Thu, Oct 26, 2023, 5:00 AM UTC

lofties wrote: post #79

pknerd wrote: post #54

Pardon my ignorance in advance but could it be used to "chat" with PDFs and websites? I am looking for OpenAI alternatives as I am in learning phase

No. “Chatting with PDFs” is (mostly) taking a users chat message, retrieve relevant content via e.g embedding search, then feed that into an LLM with a prompt that’s something along the lines of “given this information, can you answer this question”. This tool helps with embedding part. I’ve built a bunch of ”chat with your PDFs” bots, do reach out if you have any questions me at brian.jp.

Post by thomasahle » Thu, Oct 26, 2023, 5:12 AM UTC

simonw wrote: post #57

rolisz wrote: post #55

Earlier quoted context omitted.

I think the point is: if you compress 25 pages of text into 1024 floats, you will lose a ton of information, regardless of what the use case is, so you're probably still better of with chunking.

I've been getting great results for related documents by embedding entire blog posts, e.g. here: https://til.simonwillison.net/gis/pmtiles#related I'm not sure how I would do that after chunking.

Post by simonw » Thu, Oct 26, 2023, 5:16 AM UTC

thomasahle wrote: post #85

simonw wrote: post #57

Earlier quoted context omitted.

I've been getting great results for related documents by embedding entire blog posts, e.g. here: https://til.simonwillison.net/gis/pmtiles#related I'm not sure how I would do that after chunking.

Did you compare with simple baselines like bag-of-words and word vectors?

Post by [deleted] » Thu, Oct 26, 2023, 5:16 AM UTC

Zuiii wrote: post #29

Color me surprised! it looks like its actually open source (Apache 2.0) and not the usual false advertising by some two-faced company or institution. Links here: * https://huggingface.co/jinaai/jina-embeddings-v2-base-en * https://huggingface.co/jinaai/jina-embeddings-v2-small-en