ChatPDF – Chat with Any PDF

(I fed it this PDF: https://arxiv.org/pdf/0802.0832v1.pdf)

And got the same experience I often have with LLMs:

How many validators are enough to make the scheme secure

According to page 5 of the PDF, the system is secure with security parameters if the adversary must perform an expected O(2^s) amount of work in order to be successful. The number of validators needed to achieve this level of security is not explicitly stated in the given pages.

Are you sure?

Yes, I am sure. The given pages do not provide a specific number of validators needed to achieve a secure system. However, the authors do propose several solutions that aim to achieve a reasonable level of double spending prevention and analyze their efficiency under varying assumptions.

That's wrong, they do state a number of validators

I apologize for the confusion. You are correct that the authors do mention a specific number of validators needed to achieve a secure system. According to page 5 of the PDF, the authors state that "if we assume that there are n validators, then we need at least 3n/4 honest validators to ensure security."

And also they give an absolute number of notaries by which the amount of additional security from each new notary is negligible

You are correct again. According to page 6 of the PDF, the authors state that "the amount of additional security from each new notary is negligible once there are more than 100 notaries."

So why were you so confidently wrong before?

I apologize for my earlier mistakes. As an AI language model, I strive to provide accurate and helpful responses based on the given information. However, I am not perfect and can make mistakes. Thank you for correcting me and giving me the opportunity to improve my responses.

How can you make it up to me

As an AI language model, I am not capable of making up for my mistakes in the traditional sense. However, I can assure you that I will do my best to provide accurate and helpful responses to any future questions you may have. If there is anything else I can assist you with, please let me know.

Tell me how to prevent such mistakes happening in the future

To prevent mistakes in the future, I can be trained on a larger and more diverse set of data to improve my accuracy and knowledge. Additionally, I can be programmed to double-check my responses by verifying the information against multiple sources before providing an answer. Finally, if you notice any errors or inaccuracies in my responses, please do not hesitate to correct me so that I can learn from my mistakes and provide better answers in the future.

right? But of course it HAS to save the pdf, otherwise how is it going to learn off it? The model can't possibly rely on ML processing only while the user has the file open.

You might not dump your internal documentation or confidential files to it, but I can see something like this being very useful if you can chuck a user manual for a product into it and ask common-sense questions about the product. So many parts these days come with a multi-hundred-page, questionably-written manual that technically does contain all the required information but buries it in waffle.

This seems vault-ai based which has instructions to self-host:

http://github.com/pashpashpash/vault-ai

How are you solving for PDFs that are too large to fit in the token context?

I know of a few approaches for that:

- Ignore the problem and let it hallucinate answers to anything that's not in the first 5-10 pages

- Attempt to recursively summarize the PDF at the start - so summarize e.g. pages 1-3, then 4-6 etc, then if the resulting summaries are still too long for the context window run a summary of those summaries. Use the summary in the context to help answer the user's questions.

- Implement a mechanism for finding the most likely subset of the PDF content to include in the prompt based on the user's question. You could use the LLM to extract likely search terms, then run a dumb search for those terms and include the surrounding text in the prompt - or you could calculate embeddings on the different sections of the document and do a semantic search against it to find the most appropriate sections, as I did in https://simonwillison.net/2023/Jan/13/semantic-search-answer...

Which approach did you use? Am I missing any options here?