Books in .txt format for AI training purposes • Hacker News Board

Re: Books in .txt format for AI training purposes

#11

Post by yodon » Sun, Oct 25, 2020, 3:20 PM UTC

yorwba wrote: post #10

yodon wrote: post #5

Earlier quoted context omitted.

> Are you objecting to the use of data for training, or to the compilation of training datasets? The GP is objecting to the casualness with which privacy rights and intellectual property rights are ignored by so many in the AI community, not to a choice of whether they object to one or another specific manifestation of how the community is doing so. [edit added]: Those who would argue "no, there is no casual ignoring…

The official DMCA takedown instructions are actually at http://the-eye.eu/dmca/ and look much less unusual.

You did notice those files are both hosted on the same site, I trust? I suspect the mp4 speaks much more directly to the actual state of mind and beliefs of the creator(s) of the collection. In the context of this conversation (discussing the attitudes of many in the AI community towards privacy and intellectual property rights in training data), that would make the mp4 the more official statement of policy.

Re: Books in .txt format for AI training purposes

#12

Post by [deleted] » Sun, Oct 25, 2020, 3:21 PM UTC

[deleted]

Re: Books in .txt format for AI training purposes

#13

Post by jhauris » Sun, Oct 25, 2020, 3:23 PM UTC

nix23 wrote: post #6

yodon wrote: post #5

Earlier quoted context omitted.

> Are you objecting to the use of data for training, or to the compilation of training datasets? The GP is objecting to the casualness with which privacy rights and intellectual property rights are ignored by so many in the AI community, not to a choice of whether they object to one or another specific manifestation of how the community is doing so. [edit added]: Those who would argue "no, there is no casual ignoring…

The question is..if a AI reads a book is it against copyright? Or is the trained model a derived work of those books?

IANAL: I think it's pretty clear it's a derived work, the open question afaik is whether it's fair use our not.

Re: Books in .txt format for AI training purposes

#14

Post by nix23 » Sun, Oct 25, 2020, 3:31 PM UTC

ahoka wrote: post #8

nix23 wrote: post #6

Earlier quoted context omitted.

The question is..if a AI reads a book is it against copyright? Or is the trained model a derived work of those books?

Of course it is a derived work.

That opens some interesting questions...

Re: Books in .txt format for AI training purposes

#15

Post by Spooky23 » Sun, Oct 25, 2020, 3:43 PM UTC

nix23 wrote: post #6

yodon wrote: post #5

Earlier quoted context omitted.

> Are you objecting to the use of data for training, or to the compilation of training datasets? The GP is objecting to the casualness with which privacy rights and intellectual property rights are ignored by so many in the AI community, not to a choice of whether they object to one or another specific manifestation of how the community is doing so. [edit added]: Those who would argue "no, there is no casual ignoring…

The question is..if a AI reads a book is it against copyright? Or is the trained model a derived work of those books?

If AI has human like rights to read and learn and create derived works, what other rights does it have?

Perhaps we should figure that before we start a dystopian nightmare of written material.

Re: Books in .txt format for AI training purposes

#16

Post by WClayFerguson » Sun, Oct 25, 2020, 3:43 PM UTC

Is there a torrent link? Also, I know IPFS is still new, but growing rapidly, so some day each of these books will just have a CID on IPFS, and basically to publish all the books as a list you'd simply publish a list of the CIDs.

Re: Books in .txt format for AI training purposes

#17

Post by JoeDaDude » Sun, Oct 25, 2020, 3:48 PM UTC

There is always Project Gutenberg [1]. I suppose there may be pros and cons to each, with Project Gutenberg being generally older books.

[1]. https://www.gutenberg.org/

Fun trivia: I once took an NLP class and discovered that I could predict whether a text was written by Edgar Allan Poe or H. G. Wells with ~86% accuracy based on the presence of one word: "whereupon". The language had changed enough in 60 years such that that single word was no longer used.

Re: Books in .txt format for AI training purposes

#18

Post by mdifrgechd » Sun, Oct 25, 2020, 3:50 PM UTC

nix23 wrote: post #6

yodon wrote: post #5

Earlier quoted context omitted.

> Are you objecting to the use of data for training, or to the compilation of training datasets? The GP is objecting to the casualness with which privacy rights and intellectual property rights are ignored by so many in the AI community, not to a choice of whether they object to one or another specific manifestation of how the community is doing so. [edit added]: Those who would argue "no, there is no casual ignoring…

The question is..if a AI reads a book is it against copyright? Or is the trained model a derived work of those books?

I can imagine building a model that is a derived work, but I don't see why all models trained on a corpus that contains some text can automatically be thought of as derived from that text in a creative sense. Models like GPT are using the text as a specific instance of the latent relationships that make up our language. The text is an example of english to learn from, it's not a creative work being extended.

Training a language model, I would argue, is equivalent (in terms of being derivative) to generating a list of the top ten words in a corpus of text. Not really a derived work.

Re: Books in .txt format for AI training purposes

#19

Post by wyldfire » Sun, Oct 25, 2020, 3:54 PM UTC

detaro wrote: post #2

Always impressive how the AI community, both inside large companies and not, seems to assume data just exists for them to use, copyright or personal rights be damned.

Aside: it's interesting to think about the legal ramifications of copyrights in the face of AI. If my AI model "reads" a bunch of text, is the model now a "derived work" of the copyrighted work? Perhaps. But of course humans don't infringe when they watch/listen/read a copyrighted work and it changes their state of mind. Perhaps the model can be said not to infringe if it sufficiently abstracts concepts of the art in question. Typically when tuning a model we include lots of counter-cases to avoid overfitting - to make sure that the model is indeed abstracting concepts.

Re: Books in .txt format for AI training purposes

#20

Post by mdifrgechd » Sun, Oct 25, 2020, 3:56 PM UTC

mdifrgechd wrote: post #18

nix23 wrote: post #6

Earlier quoted context omitted.

The question is..if a AI reads a book is it against copyright? Or is the trained model a derived work of those books?

I can imagine building a model that is a derived work, but I don't see why all models trained on a corpus that contains some text can automatically be thought of as derived from that text in a creative sense. Models like GPT are using the text as a specific instance of the latent relationships that make up our language. The text is an example of english to learn from, it's not a creative work being extended. Training…

One possible counterpoint- sample based explanation techniques can tell what training data was most influential in a ML model's prediction, and this has been considered in language models[0]. So you could argue that if there are training examples within a corpus that are having an outsized influence on the model output, then maybe it is derivative. This would be pretty cool to look at - are some GPT or other language model outputs relying strongly on a few sources?

[0] https://arxiv.org/abs/1810.03611