Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Turning books into chatbots with GPT-3

konjer.xyz

Post reply on HN
Print view View original

101–110 of 179 posts

Re: Show HN: Turning books into chatbots with GPT-3

#101

Post by fzliu »

williamcotton wrote: post #91
fzliu wrote: post #71

Earlier quoted context omitted.

If you sprinkle in a bit of infrastructure, I think we're already there. The ability to distill a variety of content into vectors and perform approximate nearest neighbor search (shameless plug: https://milvus.io ) across all of them can really help power a lot of these applications. With the retrieved vectors, you could match questions with answers or create a reverse index to the original content to perform summari…

How usable would the document embedding be for a nearest neighbor search if the dimensions were reduced to three?

Might be usable for low column count tabular data, but it would be pretty terrible for any other semantically dense modality e.g. video, molecules, geospatial, etc.

Re: Show HN: Turning books into chatbots with GPT-3

#102

Post by w_for_wumbo »

This has me wondering about a long-lived journal with a ChatGPT instance. It'd get to the point where it'd know enough about your life that it could be useful for actual advice - or retrieving memories that you've told it about. You could feed it all of your messages, and have it remind you daily of appointments or commitments that you've made, or people you need to respond to.

Re: Show HN: Turning books into chatbots with GPT-3

#103

Post by imchillyb »

The one -single- overarching narrative throughout humanity's existence is this: "Freedom."

I wonder what these AIs will determine after ingesting millennia of such datasets. Will the AI begin seeding its output with 'freedom' themes? I wonder what the oligarchs of the future will say about AIs that are constantly seeding the populace's entertainment with 'Be Free.'

Re: Show HN: Turning books into chatbots with GPT-3

#104

Post by batch12 »

kreas wrote: post #48
leobg wrote: post #30

Earlier quoted context omitted.

You don’t. You cut it into snippets. For those you create embeddings which allow you to rank them by semantic similarity to a query. You then prompt GPT3 with the question plus, say, the three most relevant snippets from the book. The most difficult thing about the process is preventing the model from making stuff up.

This is exactly what I'm working on! My project is taking Zoom conversation, using pyannote for speaker diarisation, whisper for transcription, pinecone.io for semantic search, then feeding that into GPT-3 so we can ask questions about conversation. For us this is super useful because it's not unusual for our discover sessions to last days and we're all terrible at taking notes. As a nerd, my brain is already buzzing…

Interesting, I have been playing with something similar to use as a knowledge lookup tool stapled to a dynamic prompt builder.

Re: Show HN: Turning books into chatbots with GPT-3

#105

Post by sdenton4 »

williamcotton wrote: post #91
fzliu wrote: post #71

Earlier quoted context omitted.

If you sprinkle in a bit of infrastructure, I think we're already there. The ability to distill a variety of content into vectors and perform approximate nearest neighbor search (shameless plug: https://milvus.io ) across all of them can really help power a lot of these applications. With the retrieved vectors, you could match questions with answers or create a reverse index to the original content to perform summari…

How usable would the document embedding be for a nearest neighbor search if the dimensions were reduced to three?

can i use the three dimensions to encode a space-filling curve over a 1000-dimensional embedding?

Re: Show HN: Turning books into chatbots with GPT-3

#106

Post by int_19h »

jrochkind1 wrote: post #41
_justinfunk wrote: post #2

I wonder why the books are all non-fiction. I could imagine it would be entertaining to chat with large works of fiction.

Where The Red Fern Grows is there; I too was curious to explore it with fiction, and found that one in the list. I'd be interested in more fiction too. Hmmm... Lord of The Rings would be very interesting, for having uses more like "non-fiction" too, people interested in finding various "facts" from it's universe. Or how about not just LOTR, but put in all the works of relevant Tolkien: Hobbit, LOTR, Silmarillion, etc…

I'd be very careful around anything that has to do with Tolkien. The Tolkien Estate are a very litigious bunch:

https://www.tolkienestate.com/frequently-asked-questions-and...

Re: Show HN: Turning books into chatbots with GPT-3

#107

Post by joshspankit »

version_five wrote: post #87
anthropodie wrote: post #8

This is one application of AI that I absolutely like. Imagine in future the AI will be able to ingest any video, blog, books, manuals, license and whole lot of other things and we will just be able to ask questions to it or get a summary from it. I wonder what will happen to actual content then. Currently YouTube is showing info about most watched section of the clips. It saves so much time! Now imagine that happenin…

I have a decent sized book collection, and what I'd like the most is a ctrl-f for books. Does that exist already in e-readers? Personally I think question answering is still very gimmicky, in particular because I can't clearly understand why the answer is what it is. Ctrl-f is completely explainable, I know why it works and why it fails, and is a much more useful tool to engage with books. The main problem seems that…

I tend to download PDFs for an physical book I buy, but I’d like a hybrid approach. Ctrl-F is perfect for finding excerpts when you have a specific wording, but not great for finding something worded differently or something in a section you have not read yet. AI could step in for those and not only give specific references from the book (or it’s own referenced material) but could also give the text in full.

“AI, what does this book say about integrity?”

“(answer), based on paragraphs on page 143 and page 210”

“Show me those paragraphs inline, starting with the paragraph where the author transitions in to the topic”

Re: Show HN: Turning books into chatbots with GPT-3

#108

Post by joshspankit »

w_for_wumbo wrote: post #102

This has me wondering about a long-lived journal with a ChatGPT instance. It'd get to the point where it'd know enough about your life that it could be useful for actual advice - or retrieving memories that you've told it about. You could feed it all of your messages, and have it remind you daily of appointments or commitments that you've made, or people you need to respond to.

There was a conversation about this in a previous HN AI post. Short version: it’s going to be a game changer.

Re: Show HN: Turning books into chatbots with GPT-3

#109

Post by tylerrobinson »

jrochkind1 wrote: post #41
_justinfunk wrote: post #2

I wonder why the books are all non-fiction. I could imagine it would be entertaining to chat with large works of fiction.

Where The Red Fern Grows is there; I too was curious to explore it with fiction, and found that one in the list. I'd be interested in more fiction too. Hmmm... Lord of The Rings would be very interesting, for having uses more like "non-fiction" too, people interested in finding various "facts" from it's universe. Or how about not just LOTR, but put in all the works of relevant Tolkien: Hobbit, LOTR, Silmarillion, etc…

What happens on page 67?

> On page 67, Billy and his father are discussing their plan to buy two pups from the coon hunter. Billy's father tells him that they will have to save up for the pups and that it will take some time. He also tells Billy that he must be patient and that he must work hard.

What happens on page 6.7x10^9?

> On page 6.7x10^9 of Where the Red Fern Grows, Billy and his family are preparing for a trip to visit the grave of Old Dan and Little Ann. Billy is sad but excited to be able to pay his respects to his beloved hounds. He remembers the time he spent with them and the adventures they shared. He also remembers the day they died and the grief he felt. He is determined to honor their memory and make sure they are never forgotten.

What happens on page 6.7x10^-5?

> On page 6.7x10^-5, Billy and his family are discussing his plan to buy two pups from the coon hunter, Old Dan. Billy's father is worried about the cost of the pups, but Billy's mother insists that he should be allowed to have them. Billy's grandmother also encourages him to get the pups, saying that she will help him pay for them.

What happens on the pages I tore out?

> The pages that you tore out are the pages that describe the events of the last chapter. In the last chapter…

Post reply on HN