Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You probably shouldn't use OpenAI's embeddings

iamnotarobot.substack.com

Post reply on HN
Print view View original

21–30 of 31 posts

Re: You probably shouldn't use OpenAI's embeddings

#21

Post by eternalban »

mustacheemperor wrote: post #20
eternalban wrote: post #19

Earlier quoted context omitted.

It's in the repo: You first create embeddings. What is this? It's an n-dimensional vector space with your tweets 'embedded' in that space. Each word is an n-dimensional vector in this space. The vectorization is supposed to maintain 'semantic distance'. Basically, if two words are very close in meaning or related (by say frequently appearing next to each other in corpus) they should be 'close' in some of those n-dime…

Thank you for this walkthrough, and for citing the code alongside!

hth

Re: You probably shouldn't use OpenAI's embeddings

#22

Post by jeadie »

mustacheemperor wrote: post #3

Could anyone point me towards a relatively beginner-friendly guide to do something like >download all my tweets (about 20k) and build a semantic searcher on top ? How can utilize 3rd party embeddings with OpenAI's LLM API? Am I correct to understand from this article that this is possible?

You could try https://github.com/marqo-ai/marqo

Re: You probably shouldn't use OpenAI's embeddings

#25

Post by anshumankmr »

adeelk93 wrote: post #23
breckenedge wrote: post #10

It’s fine to use their embeddings for a proof of concept, but since you don’t own it, you probably shouldn’t rely on it because it could go away at any time.

Couldn’t you make that argument against all SaaS?

Well some SaaS purists may believe that to be true about Microsoft Office 365. Hence we have, Microsoft Office 2021.

(Although there is a lot more advantages to just having Office 2021 like the flat fee)

Re: You probably shouldn't use OpenAI's embeddings

#26

Post by shirkey »

fzliu wrote: post #13

I've done some quick-and-dirty testing with OpenAI's embedding API + Zilliz Cloud. The 1st gen embeddings leave something to be desired ( https://medium.com/@nils_reimers/openai-gpt-3-text-embedding... ), but the 2nd gen embeddings are actually fairly performant relative to many open source models with MLM loss. I'll have to dig out the notebook that I created for this, but I'll try to post it here once I find it.

Please do and thanks in advance for any insights you can provide -- it would be great to understand any benchmarking improvement with ada-002 from your previous findings, and whether you tested the specific OpenAI text-search-*-{query,doc} models as a comparison for large document search.

Re: You probably shouldn't use OpenAI's embeddings

#27

Post by breckenedge »

adeelk93 wrote: post #23
breckenedge wrote: post #10

It’s fine to use their embeddings for a proof of concept, but since you don’t own it, you probably shouldn’t rely on it because it could go away at any time.

Couldn’t you make that argument against all SaaS?

Yes and no. Sometimes you get contracts that require “source code escrow” so that companies can run your source if you ever go out of business.

Re: You probably shouldn't use OpenAI's embeddings

#30

Post by nico »

KRAKRISMOTT wrote: post #29
nico wrote: post #2

Is someone doing embeddings embeddings mapping? For example, mapping embeddings of Llama to GPT-3? That way you can see how similar the models “understand the world”.

Isn't this the whole point behind the CLIP architecture?

Could you explain a bit more what the CLIP architecture is? Any good links to a short video demoing it? Thank you!
Post reply on HN