81–90 of 141 posts
4th submission in 3 weeks
If you're worried about astroturfing, email the mods and they'll take a look.
Earlier quoted context omitted.
Care to elaborate? I mean, looking at the source code it clearly does use LangChain for PDF ingestion https://github.com/mayooear/gpt4-pdf-chatbot-langchain/blob/...
He might mean the site chatpdf.com does not use langchain but uses the OpenAI API.
I have tried using Mathpix to convert the formal theory papers into latex and then fed it to GPT-4, but it was not able to take the whole text in a single prompt. When I broke it down in multiple prompts, it started responding hallucinated sections of the paper. I had given preemptive instructions stating that I was going to share the paper section by section and then ask it questions.
Once I finished uploading the whole paper after multiple prompts, it did not give satisfactory answers.
Earlier quoted context omitted.
Consider adding a bit of overlap to the text chunks. Say, 300 tokens: text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=300) Otherwise, you'll likely end up with too many edge cases in which only part of a relevant context is retrieved :-)
This is actually pretty insightful - I have done something similar with splitting my obsidian data into chunks using paragraphs and headers as demarcation, but this solves a more interesting problem of nuance! I like it.
Other possibilities to fuel the ChatGPT hype train... ChatPNG - apply OCR to an image, extract text, feed it to GPT. ChatMP3 - apply speech-to-text to a recording, feed it to GPT. ChatGPS - hmm. not sure yet. something location-based obviously... If any VC's are interested, I'm selling 10% stake in these projects for only $20k right now. /s
Other possibilities to fuel the ChatGPT hype train... ChatPNG - apply OCR to an image, extract text, feed it to GPT. ChatMP3 - apply speech-to-text to a recording, feed it to GPT. ChatGPS - hmm. not sure yet. something location-based obviously... If any VC's are interested, I'm selling 10% stake in these projects for only $20k right now. /s
How are you solving for PDFs that are too large to fit in the token context? I know of a few approaches for that: - Ignore the problem and let it hallucinate answers to anything that's not in the first 5-10 pages - Attempt to recursively summarize the PDF at the start - so summarize e.g. pages 1-3, then 4-6 etc, then if the resulting summaries are still too long for the context window run a summary of those summaries…
Docalysis also shows you the PDF side-by-side, has page numbers, and overall responses are of better quality according to users that have emailed comparisons to ChatPDF.
Me: Bukowski said: "If you're going to try, go all the way". How would you do this with an AB test?
PDF: While Charles Bukowski's quote is not directly related to A/B testing, you can apply the spirit of his words to your approach to A/B testing. To "go all the way" with an A/B test, you should aim to test as many variables as possible in order to get the most accurate and comprehensive results. This means testing multiple variants, adjusting multiple parameters, and collecting data from a large sample size over a significant period of time. By doing so, you can ensure that your results are reliable and that you have a clear understanding of which variant performs better. (Page 21)
Me: But if you go all the way (in the way you described), wouldn't you be walking in a garden of forking paths?
PDF: You raise a valid concern. When conducting an A/B test with many variables and parameters, there is a risk of encountering the "garden of forking paths" problem. This refers to the issue of having many possible paths to take when analyzing the data, which can lead to false positives or overfitting. To avoid this problem, it's important to have a clear hypothesis and plan in place before conducting the test, and to limit the number of variables being tested at once. Additionally, you should use appropriate statistical methods to analyze the data and correct for multiple comparisons. By taking these precautions, you can minimize the risk of encountering the garden of forking paths problem and ensure that your results are reliable. (Page 21)
Me: So what's a better way to go all the way?
PDF: A better way to "go all the way" with an A/B test is to focus on testing a few key variables that are most likely to have a significant impact on your metrics. This approach is sometimes called "full factorial testing" or "multivariate testing". By testing a smaller number of variables, you can reduce the risk of encountering the garden of forking paths problem and increase the statistical power of your test. Additionally, you should use appropriate statistical methods to analyze the data and correct for multiple comparisons. By taking these precautions, you can ensure that your results are reliable and that you have a clear understanding of which variables have the biggest impact on your metrics. (Page 21)