I just started on a project with about 5000 pages worth of government supplied documentation in PDF form. I wish I could just throw your tool at it.
Show HN: DocAsker – Use LLMs to ask documentation questions
21–27 of 27 posts
Re: Show HN: DocAsker – Use LLMs to ask documentation questions
#22I just started on a project with about 5000 pages worth of government supplied documentation in PDF form. I wish I could just throw your tool at it.
Parsing pdfs (and powerpoints) and breaking them into "askable" chunks is definitely something we've been looking into and are keen to roll out. If you'd like to talk more about your use case definitely feel free to chuck us an email on the "reach out" email on the page!
Re: Show HN: DocAsker – Use LLMs to ask documentation questions
#23Would be awesome to have something like this that can ingest a whole GitHub repo including issues and PRs to ask when things got decifed or why they are a certain way.
Re: Show HN: DocAsker – Use LLMs to ask documentation questions
#24Would be awesome to have something like this that can ingest a whole GitHub repo including issues and PRs to ask when things got decifed or why they are a certain way.
I think this is supposed to do that https://www.gptduck.com/
Re: Show HN: DocAsker – Use LLMs to ask documentation questions
#25I just started on a project with about 5000 pages worth of government supplied documentation in PDF form. I wish I could just throw your tool at it.
Parsing pdfs (and powerpoints) and breaking them into "askable" chunks is definitely something we've been looking into and are keen to roll out. If you'd like to talk more about your use case definitely feel free to chuck us an email on the "reach out" email on the page!
don't want to jump on your brigade, but at AnyQuestions.ai we specialise in quality PDF and transcript processing for AI-answer purposes (supporting AI answers with citations, just like your tool for documentation). This comes from 3 years working on tech to parse lecture slides correctly, identifying semantic areas (e.g. what is a title, how are bullet points connected...) turns out this is useful for semantic search and other purposes of embedding. You can verify this somewhat by viewing how transcripts get bunched if you upload a youtube video, or if you search for PDF results (bullet points will be resolved to what they refer to etc as appropriate)
Would love to chat with you if you're up for it - you can test demo run our tool and contact us through the interface
Re: Show HN: DocAsker – Use LLMs to ask documentation questions
#26Hi HN! I'm the other person working on this with Ankly. This came up as a pet project as we were eager to put some LLM work into production. Currently piggy-backing off APIs, though the results with certain self-hosted models could be worse (but could be better), as he mentioned, we've ran some experiments with Flan-T5. We have a bunch of current plans as to where to keep building on this, beyond improving the model/…
I've noticed when prompting ChatGPT for code that it occasionally invents libraries that don't exist at all, or adds it own input if something about a prompt is strongly connected to its latent knowledge. For example, I asked it to write a program that would select one of my favourite dinner ideas at random, providing a list of options in my prompt. It added 4 more recipes I'd never heard of and some playful commenta…
https://platform.openai.com/docs/guides/fine-tuning
You create a series of prompts and their responses and then that tuned model is used with that implicit knowledge already stored in it.
For example a notebook for "lets train GPT on the information about the olympics - https://github.com/openai/openai-cookbook/blob/main/examples... and https://github.com/openai/openai-cookbook/blob/main/examples... and https://github.com/openai/openai-cookbook/blob/main/examples... )
The gotcha for this is that while regular Davinci is $0.02/1k tokens, training is $0.03/1k tokens and use is $0.12/1k tokens.
The other thing to consider is that Chat GPT has a session and history for that session. You can use GPT stateless which doesn't have the "it gets confused about what you were talking about before."
curl https://api.openai.com/v1/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-d '{
"model": "text-davinci-003",
"prompt": "Write a recipe based on these ingredients and instructions:\n\nFrito Pie\n\nIngredients:\nFritos\nChili\nShredded cheddar cheese\nSweet white or red onions, diced small\nSour cream\n\nInstructions:",
"temperature": 0.3,
"max_tokens": 120,
"top_p": 1,
"frequency_penalty": 0,
"presence_penalty": 0
}'
Re: Show HN: DocAsker – Use LLMs to ask documentation questions
#27I just started on a project with about 5000 pages worth of government supplied documentation in PDF form. I wish I could just throw your tool at it.
Parsing pdfs (and powerpoints) and breaking them into "askable" chunks is definitely something we've been looking into and are keen to roll out. If you'd like to talk more about your use case definitely feel free to chuck us an email on the "reach out" email on the page!
Since you are working with raw text, it shouldn't need too much effort. There are a bunch of open source tools to extract text from PDFs.
The hard part would be parsing tables and other layout-dependent semantics. You usually start with text coordinates (like HTML elements with absolute position) and have to work backwards from that. I worked for some years in a project for a client that was full of edge cases, because whenever the input PDF (from a government agency) would have a slight layout change the parser would break. It took multiple iterations to make it robust enough.