What's the legality of this? Redistributing content for paid books for sale? Edit: can this data be sold?
Precedent would be Blinkist, which AFAIK just uses an army of editors. Though, they probably _have_ contracts with the publishers
Show HN: Turning books into chatbots with GPT-3
21–30 of 179 posts
Re: Show HN: Turning books into chatbots with GPT-3
#22Very interesting project. Nice work!
I'm curious if you have copyright concerns since GPT-3 may presumably quote portions of the book back to you in some instances. I still don't know if that would be a problem, but I was just curious if you did any research regarding legal/copyright issues and what conclusions you came to.
In any case, I hope you keep going with it. Echoing others here, I also think more fiction works would be a great addition. Standard Ebooks would be an option for getting source material.
Re: Show HN: Turning books into chatbots with GPT-3
#23I wonder why the books are all non-fiction. I could imagine it would be entertaining to chat with large works of fiction.
agreed -- slowly integrating fiction. if you're interested there are two fiction books up there right now. The Alchemist and Where the Red Fern Grows https://www.konjer.xyz/the-alchemist https://www.konjer.xyz/where-the-red-fern-grows
There's a whole library of copyright-free classic fiction at https://www.gutenberg.org/ :-)
Re: Show HN: Turning books into chatbots with GPT-3
#24Could you share your ML architecture? Curious if you did it via fine-tuning or other tricks. It is amazing.
Re: Show HN: Turning books into chatbots with GPT-3
#25What's the legality of this? Redistributing content for paid books for sale? Edit: can this data be sold?
Is it redistributing content? I don't believe that to be the case.
Selling the "books" which are training on all text, wouldn't that require some sort of license?
Can I legally cut up a book, spit out certain sections and resell it?
Re: Show HN: Turning books into chatbots with GPT-3
#26This would be great! Extra credit if someone can work out how to avoid spoilers by taking your page number and only using pages 1-N for answers
Re: Show HN: Turning books into chatbots with GPT-3
#27I wonder why the books are all non-fiction. I could imagine it would be entertaining to chat with large works of fiction.
Books that operate in a narrative/temporal style are hard to manage, because statements of fact are mutative. Consider the story: "Justin is hungry. Justin eats dinner. Justin is not hungry." You ask the chatbot "Is Justin hungry?". There is a temporal aspect to this question that is hard for simple systems that are just embedding facts into a vector DB (or similar techniques) to reconcile.
Is that how LLMs work?
Re: Show HN: Turning books into chatbots with GPT-3
#28This would be a fantastic aide for fantasy books like the Simarilian where there’s so many characters you lose track of who’s who
Re: Show HN: Turning books into chatbots with GPT-3
#29What's the legality of this? Redistributing content for paid books for sale? Edit: can this data be sold?
Why, is it quoting substantial portions from the books? Copyright covers expression, not information.
rarely seen it quote the book verbatim. when it did it was very short snippets.
it's more like talking to a person who knows the contents of the book by heart.
Re: Show HN: Turning books into chatbots with GPT-3
#30How are you getting the book text into gpt3?
You don’t. You cut it into snippets. For those you create embeddings which allow you to rank them by semantic similarity to a query. You then prompt GPT3 with the question plus, say, the three most relevant snippets from the book.
The most difficult thing about the process is preventing the model from making stuff up.