[dead]
Show HN: ChatGPT and Document Parser = Ghost
21–30 of 53 posts
Re: Show HN: ChatGPT and Document Parser = Ghost
#225 pages fits in the context window. How exactly do you plan on expanding the limit? Without explanation we have to assume you haven't completely solved your core technical challenges. In my testing the biggest challenges with using for example OpenAI embeddings with cosine similarity or something are A) figuring out the section breaks or right chunk size so that information stays in context and B) retrieving enough c…
From the website, it seems as though they are retrieving five chunks. Also looks like they split documents by paragraph sections, unless the paragraphs are small enough- then they put a couple of them together.
Re: Show HN: ChatGPT and Document Parser = Ghost
#23(This critique is unrelated to this project. It works as expected, OP, and looks good.) How could one ever trust the output of ChatGPT? This feels to me a bit like non-L5 autonomous driving: If I have to assist at all, it'd be easier to do it myself. In the same vein, for this project (and ChatGPT generally): Can I actually trust that the output from ChatGPT in answering my question about the document is factually co…
If it's using a search then it is possible to identify the paragraphs with a number in the database along with the embeddings. Then once the similar chunks are retrieved, part of the prompt could be to return the paragraph, line numbers or exact quote(s) used to answer the question.
Yours is not a good example though because "10 minutes late" is never going to be in a document like that.
Re: Show HN: ChatGPT and Document Parser = Ghost
#24A better way to do this might be to use the embedding API. That allows you to upload a text corpus and to then get vectors. You can then calculate the cosign similarity for a search string on those to get relevant results of clustered text from the uploaded corpus.
Re: Show HN: ChatGPT and Document Parser = Ghost
#25I wanted to try it, but the pdf i tried just got an error with no info so ¯\_(ツ)_/¯
Re: Show HN: ChatGPT and Document Parser = Ghost
#26Ghost is a well known blogging platform so you might want to change the name. This seems similar to ChatPDF.com (with a 200 page limit though, instead of the 5 page limit that you have, it seems) which I suppose we'll see a lot more competitors for as the ChatGPT API expands.
There is also Ghostscript which is a postscript/pdf library, and since the site is operating on PDF content, my initial thought were that they were somehow related.
Re: Show HN: ChatGPT and Document Parser = Ghost
#27Projects like these (using embeddings) are great, but what I'm looking for is something that can ingest an entire book (let's say a fiction book) then answer questions about the entire content (and not just by effectively doing a text search over your input, but actually "understanding" the entire contents of the book); I presume such a thing is not possible with ChatGPT (without fine-tuning), correct?
Re: Show HN: ChatGPT and Document Parser = Ghost
#28A better way to do this might be to use the embedding API. That allows you to upload a text corpus and to then get vectors. You can then calculate the cosign similarity for a search string on those to get relevant results of clustered text from the uploaded corpus.
Would you mind explaining this and maybe dumbing it down? Sounds useful
Re: Show HN: ChatGPT and Document Parser = Ghost
#295 pages fits in the context window. How exactly do you plan on expanding the limit? Without explanation we have to assume you haven't completely solved your core technical challenges. In my testing the biggest challenges with using for example OpenAI embeddings with cosine similarity or something are A) figuring out the section breaks or right chunk size so that information stays in context and B) retrieving enough c…
Same, the is lies in the details. You basically need a good semantic search in front of GPT to feed it the best context given the question.
Any code or pseudo-code you could share that does something like that?
Re: Show HN: ChatGPT and Document Parser = Ghost
#30Projects like these (using embeddings) are great, but what I'm looking for is something that can ingest an entire book (let's say a fiction book) then answer questions about the entire content (and not just by effectively doing a text search over your input, but actually "understanding" the entire contents of the book); I presume such a thing is not possible with ChatGPT (without fine-tuning), correct?
Right now that’s not a use case supported out of the box by ChatGPT.
It also seems to be one of the most important limitations of ChatGPT, and a lot of people/teams are looking for solutions.