Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PrivateGPT

github.com

Post reply on HN
Print view View original

1–10 of 147 posts

Re: PrivateGPT

#2

Post by simonw »

I'm always interested in seeing the prompt that drives these kinds of tools.

In this case it appears to be using RetrievalQA from LangChain, which I think is this prompt here: https://github.com/hwchase17/langchain/blob/v0.0.176/langcha...

    Use the following pieces of context to answer the question at the end. If you don't
    know the answer, just say that you don't know, don't try to make up an answer.

    {context}

    Question: {question}
    Helpful Answer:

Re: PrivateGPT

#4

Post by jstarfish »

simonw wrote: post #2

I'm always interested in seeing the prompt that drives these kinds of tools. In this case it appears to be using RetrievalQA from LangChain, which I think is this prompt here: https://github.com/hwchase17/langchain/blob/v0.0.176/langcha... Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer. {context} Questio…

Do such fail-early conditions save processing time?

Re: PrivateGPT

#5

Post by behnamoh »

When you split a document into chunks, doesn't some crucial information get cut in half? In that case, you'd probably lose that information in the context if that information was immediately followed by an irrelevant information that reduces the cosine similarity. Is there a "smarter" way to feed documents as context to LLMs?

Re: PrivateGPT

#6

Post by mabbo »

jstarfish wrote: post #4
simonw wrote: post #2

I'm always interested in seeing the prompt that drives these kinds of tools. In this case it appears to be using RetrievalQA from LangChain, which I think is this prompt here: https://github.com/hwchase17/langchain/blob/v0.0.176/langcha... Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer. {context} Questio…

Do such fail-early conditions save processing time?

If you mean the "If you don't know" part, oh no, they have a much bigger problem they're solving.

The LLM will absolutely lie if it doesn't know and you haven't made it perfectly clear that you'd rather it did not do that.

LLMs seem to be trying to give answers that make you happy. A good lie will make you happy. Unless it understands that you will not be happy with a lie.

Is this anthropomorphizing? Yep. But that's the best way I've found to reason about them.

Re: PrivateGPT

#7

Post by dragonwriter »

mabbo wrote: post #6
jstarfish wrote: post #4

Earlier quoted context omitted.

Do such fail-early conditions save processing time?

If you mean the "If you don't know" part, oh no, they have a much bigger problem they're solving. The LLM will absolutely lie if it doesn't know and you haven't made it perfectly clear that you'd rather it did not do that. LLMs seem to be trying to give answers that make you happy. A good lie will make you happy. Unless it understands that you will not be happy with a lie. Is this anthropomorphizing? Yep. But that's…

A less anthropomorphic approach might be to say that LLMs can predict the correct “shape” of an answer even when they don’t have data that gives them a clear right answer for the correct content, and since their basic design is to provide the best response they can, they’ll provide an answer of the correct shape with fairly random content if all they have good information to predict is the shape and not the content.

Re: PrivateGPT

#9

Post by haolez »

behnamoh wrote: post #5

When you split a document into chunks, doesn't some crucial information get cut in half? In that case, you'd probably lose that information in the context if that information was immediately followed by an irrelevant information that reduces the cosine similarity. Is there a "smarter" way to feed documents as context to LLMs?

Don't know if there is a smarter way, but these libraries usually offer an overlap parameter that allows you to repeat the last N characters of a chunk in the first N of the next chunk.
Post reply on HN