Show HN: ChatGPT and Document Parser = Ghost • Hacker News Board

Re: Show HN: ChatGPT and Document Parser = Ghost

#51

Post by ElFitz » Thu, Mar 09, 2023, 2:41 PM UTC

gigel82 wrote: post #27

Projects like these (using embeddings) are great, but what I'm looking for is something that can ingest an entire book (let's say a fiction book) then answer questions about the entire content (and not just by effectively doing a text search over your input, but actually "understanding" the entire contents of the book); I presume such a thing is not possible with ChatGPT (without fine-tuning), correct?

Could using GPT3 (davinci-003) to generate embeddings, then searching your vector database for relevant excerpts, then providing the results as context for the prompt lead to something close enough?

Re: Show HN: ChatGPT and Document Parser = Ghost

#52

Post by gigel82 » Thu, Mar 09, 2023, 9:55 PM UTC

ElFitz wrote: post #51

gigel82 wrote: post #27

Projects like these (using embeddings) are great, but what I'm looking for is something that can ingest an entire book (let's say a fiction book) then answer questions about the entire content (and not just by effectively doing a text search over your input, but actually "understanding" the entire contents of the book); I presume such a thing is not possible with ChatGPT (without fine-tuning), correct?

Could using GPT3 (davinci-003) to generate embeddings, then searching your vector database for relevant excerpts, then providing the results as context for the prompt lead to something close enough?

No. That works for documentation where you do text search and extract paragraphs around the results for "context".

I want it to understand a complete fiction book and tell me about how a character grows throughout their journey from chapter 1 to chapter 12 over 350 pages.

Re: Show HN: ChatGPT and Document Parser = Ghost

#53

Post by ElFitz » Fri, Mar 10, 2023, 11:26 AM UTC

gigel82 wrote: post #52

ElFitz wrote: post #51

Earlier quoted context omitted.

Could using GPT3 (davinci-003) to generate embeddings, then searching your vector database for relevant excerpts, then providing the results as context for the prompt lead to something close enough?

No. That works for documentation where you do text search and extract paragraphs around the results for "context". I want it to understand a complete fiction book and tell me about how a character grows throughout their journey from chapter 1 to chapter 12 over 350 pages.

Depending on the book you could use that to extract all excerpts where the character appears.

Then each excerpt could be fed to the LLM asking it how this part relates to the question you’d want an answer to.

Then ask for each what it shows about the character and it’s personality, weaknesses, etc.

And finally recursively summarise them, asking for the summary to show how the character has grown through the summarised content.

Basically ending up with a map-reduce.

Bigger sources, or lots of content related to the character, would lead to less accuracy, and increase the likelihood of hitting the window’s limit.

It would also be highly specific and quite brittle, although one could probably turn it into a more generic process / pipeline (ie what dust.tt enables).

I might have completely missed your point or overlooked some glaring flaw though, in which case please do let me know what you think.