Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pinecone raises $100M Series B

pinecone.io

Post reply on HN
Print view View original

81–90 of 100 posts

Re: Pinecone raises $100M Series B

#81

Post by darkteflon »

danielmarkbruce wrote: post #79
kordlessagain wrote: post #78

Earlier quoted context omitted.

You aren't mistaken. Keeping state, or storing memories, is where it's at with prompts. The trick is knowing what to remember and what to forget. I consider vector engines to be "hot" models, given they are storing the vector representations of text already run through the "frozen" model. Having written something a while back that indexes documents and enters into discussion with them, I'm pretty sure ChatGPT is usin…

Sure - but a vector db is helping you keep your prompts to under size X. It isn't adding state and there are various mechanisms to keep your prompt to under size X - like summarization, providing a table of contents etc. It seems to me that vector db and semantic search are one trick in a pile of tricks to keep prompt sizes down until we can get the input sizes up (although gpt4 already takes 32,000 tokens). Using se…

Context constraint is a cheap way to keep the model on-topic. So rather than relying on an ever-growing context window to stuff/mapreduce more undifferentiated “context” (the entire chat history), interposing a vector search engine that only returns relevant context tends to get you better overall model performance, in addition to being scalable in a way that increasing context window size is not.

Re: Pinecone raises $100M Series B

#83

Post by MacsHeadroom »

satvikpendem wrote: post #77
gamegoblin wrote: post #47

Earlier quoted context omitted.

Vector databases make it so semantically similar sentences get mapped to be closer together in the vector space. So the sentence "I started working as a programmer" will be very close to "I began my job as a software developer". This makes it very powerful for natural language search. So when the user asks a bot "Find the text message John sent me 3 years ago about wanting to found a company, I think it was like a ha…

So you have to feed in the items to ChatGPT manually (or via some script) it looks like? In the future I guess ChatGPT with plugins could query the database on its own? Does it work for text data or can it work for other types of data as well?

It works for all types of data. You can say "Give me objects similar to water" and have it return words like "liquid" and "juice," pictures of water, the water drop emoji, and babbling brook and rain storm sound files.

It all depends on how you produce the vectors before storage. The vector database just stores them.

Re: Pinecone raises $100M Series B

#84

Post by satvikpendem »

MacsHeadroom wrote: post #83
satvikpendem wrote: post #77

Earlier quoted context omitted.

So you have to feed in the items to ChatGPT manually (or via some script) it looks like? In the future I guess ChatGPT with plugins could query the database on its own? Does it work for text data or can it work for other types of data as well?

It works for all types of data. You can say "Give me objects similar to water" and have it return words like "liquid" and "juice," pictures of water, the water drop emoji, and babbling brook and rain storm sound files. It all depends on how you produce the vectors before storage. The vector database just stores them.

How does the production of vectors work? How do you know that you should associate the water emoji to pictures of water close together in the hyperspace?

Re: Pinecone raises $100M Series B

#85

Post by danielmarkbruce »

darkteflon wrote: post #81
danielmarkbruce wrote: post #79

Earlier quoted context omitted.

Sure - but a vector db is helping you keep your prompts to under size X. It isn't adding state and there are various mechanisms to keep your prompt to under size X - like summarization, providing a table of contents etc. It seems to me that vector db and semantic search are one trick in a pile of tricks to keep prompt sizes down until we can get the input sizes up (although gpt4 already takes 32,000 tokens). Using se…

Context constraint is a cheap way to keep the model on-topic. So rather than relying on an ever-growing context window to stuff/mapreduce more undifferentiated “context” (the entire chat history), interposing a vector search engine that only returns relevant context tends to get you better overall model performance, in addition to being scalable in a way that increasing context window size is not.

Agreed.

But summarization is better to keep the model on topic for most cases. And there are other tricks.

Vectors and semantic search are one (likely questionable way given LLMs can likely reason over a table of contents or similar better) to search a large corpus or very large document. It's really only appropriate for a specific set of use cases. It's not some "general memory layer" for AI.

Re: Pinecone raises $100M Series B

#87

Post by neximo64 »

arecurrence wrote: post #23

I was using pinecone before installing pgvector in Postgres. Pinecone works and all but having the vectors in Postgres resulted in an explosion of use for us. Full relational queries with where clauses and order by etc AND vector embeddings is wicked.

What does it do is it just storing vectors and cosine similarity? Why is there a whole new category of db for that

Re: Pinecone raises $100M Series B

#88

Post by mmq »

swyx wrote: post #31

there's at least $168m being poured into vector db's this year. recent vector database fundraises: - Chroma - $18M seed https://www.trychroma.com/blog/seed - Weaviate - $50m A https://www.theinformation.com/articles/index-ventures-leads... - Pinecone - $100M B

Not recent, but the company that runs on top of milvus: https://www.businesswire.com/news/home/20220824005057/en/Vec...

Re: Pinecone raises $100M Series B

#89

Post by appenz »

danielmarkbruce wrote: post #85
darkteflon wrote: post #81

Earlier quoted context omitted.

Context constraint is a cheap way to keep the model on-topic. So rather than relying on an ever-growing context window to stuff/mapreduce more undifferentiated “context” (the entire chat history), interposing a vector search engine that only returns relevant context tends to get you better overall model performance, in addition to being scalable in a way that increasing context window size is not.

Agreed. But summarization is better to keep the model on topic for most cases. And there are other tricks. Vectors and semantic search are one (likely questionable way given LLMs can likely reason over a table of contents or similar better) to search a large corpus or very large document. It's really only appropriate for a specific set of use cases. It's not some "general memory layer" for AI.

Summarization is much more expensive than vector db's. Assume you have 1m tokens of context. You could run all through GPT-4 and summarize the information, but it would cost $60 (based on current prices) and take 10's of minutes of GPU time to do the inference.

Disclaimer: I work for a16z and on the infra team, so consider me biassed.

Re: Pinecone raises $100M Series B

#90

Post by danielmarkbruce »

appenz wrote: post #89
danielmarkbruce wrote: post #85

Earlier quoted context omitted.

Agreed. But summarization is better to keep the model on topic for most cases. And there are other tricks. Vectors and semantic search are one (likely questionable way given LLMs can likely reason over a table of contents or similar better) to search a large corpus or very large document. It's really only appropriate for a specific set of use cases. It's not some "general memory layer" for AI.

Summarization is much more expensive than vector db's. Assume you have 1m tokens of context. You could run all through GPT-4 and summarize the information, but it would cost $60 (based on current prices) and take 10's of minutes of GPU time to do the inference. Disclaimer: I work for a16z and on the infra team, so consider me biassed.

If you look through the comments here, folks are mostly referring to keeping for example a chat history. No one is doing 1m words of chat. A common pattern is to summarize a chat history and pass that in the prompt.

As for a corpus of documents (which is what you are presumably talking about), there are a couple problems with what you are saying:

First, you are implying that the content is always new - that's not true for many cases folks are talking about solving (like technical support or customer support), so it's a one time fee to summarize the corpus. You might run it periodically for updates.

Second, there is an assumption that a basic semantic search is the best way to search documents to find the most relevant content. That's questionable before the existence of LLMs, but with LLMs you are basically assuming your cosine similarity search on your vectors is better than an LLM can do with a simple table of contents and question "where should I search?" I haven't seen someone do a detailed study, but the implicit assumption that semantic search is the best idea for text could easily be a bad one.

Third, it assumes the quantum of data to search through is astronomically large and/or getting bigger compared to almost certain decreases in inference cost and increases in input tokens. This will be true for some subset of things, but unlikely to be many and in the cases it is true they'll do something more sophisticated than embeddings and embedding search. They'll probably fine tune the underlying model on an ongoing basis.

Regardless - the post you guys wrote seems... like a stretch for a definition of what this really is And, at least on the surface vector databases appear to be commodity infra. Pinecone might be growing fast now, but how do they ever make much money above their costs? But, you guys seem smart, so maybe there is something there?

Post reply on HN