Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Battle over Books3

wired.com

Post reply on HN
Print view View original

21–30 of 135 posts

Re: The Battle over Books3

#21

Post by Dylan16807 »

gremlinsinc wrote: post #7

I lean to the side of AI progress over copyright, though I fear AI becoming stupid when incentive for original works maybe goes down, so I think society needs to figure out some social contract at least, like subsidies to keep new content flowing. For instance blogs will stop posting if chatbots in search grab all the answers directly from their sites bypassing all ad revenue.

The blog posts I see that are based around some small specific answer that could be extracted all seen to be spam anyway.

Re: The Battle over Books3

#22

Post by OfSanguineFire »

intrasight wrote: post #14
gremlinsinc wrote: post #7

I lean to the side of AI progress over copyright, though I fear AI becoming stupid when incentive for original works maybe goes down, so I think society needs to figure out some social contract at least, like subsidies to keep new content flowing. For instance blogs will stop posting if chatbots in search grab all the answers directly from their sites bypassing all ad revenue.

> incentive for original works maybe goes down I don't know why you would have that concern. LOTS of people write because they just want to share. Now broaden it out to include speech. LOTS of people talk - it's what we do. Progress in AI will happen because humans like to express themselves. The challenge isn't copyright. It's figuring out how to capture the vast content that just isn't getting captured. Also, this…

> LOTS of people write because they just want to share.

The blogging scene from the early millennium is now a shadow of its former self, and one of the most often stated reasons for abandoning blogging is “my site just wasn’t getting many views any more”. In a world where AI generated content abounds, there will be even fewer eyeballs on whatever one shares and therefore less feeling of reward for sharing. Moreover, the people still blogging are often loading their content with referral links, because in an economy full of glamorous influencers, even ordinary people are tempted to seek some financial reward for sharing beyond the mere pleasure of it. Less eyeballs due to AI competition means fewer people clicking those referral links.

Re: The Battle over Books3

#23

Post by pk-protect-ai »

capableweb wrote: post #10
chasing wrote: post #4

Earlier quoted context omitted.

Laws. And the lawsuits that arise when such theft is discovered.

Easy to work around. Contract someone outside of the jurisdiction to provide a dataset, then it's up to them to deliver it to you. You "weren't aware" of the data source until people outside the organization starts shouting or the police starts asking questions, but the model "doesn't contain any of the data" so you continue shipping the model/product.

You will need to train the model in the same jurisdiction too, to avoid any kind of intervention into the training process. Ideally that would be a shelter company that "sells" you "training services" in that jurisdiction.

Re: The Battle over Books3

#24

Post by rhdunn »

civilitty wrote: post #15
skilled wrote: post #2

What's next for models like GPT now that a lot of sites will outright block CCBot, GPTBot and others? How big of an impact is this going to have on the LLM itself? Isn't OpenAI in a bit of a pickle in regards to this? The problem with my question is the following: Content gets syndicated anyway, so if DigitalOcean blocks GPTBot (which it does), pretty much every single one of those tutorials will be syphoned off to o…

Is all the data on the internet from 2010 to 2025 that much more valuable than all the data on the internet from 2010 to 2022? Better data yes, but more data up to the present day? Do we really need that to keep improving AI? GPT is already capable of incredible generalized language understanding and I'd wager we've long since hit diminishing returns from raw internet data. RLHF, fine tuning, and better (and more dat…

Language shifts, so NLP models need to understand those. For example, compare the use of "gay" before ~1980 and after. Some words change in spelling, like "to-morrow" present in works around 1800 changing to "tomorrow" in current usage. Some words are also coined, like "woke" or "enshitification".

If your data or models don't account for those then it can make mistakes. For example, if a model is only trained on modern sources (and does not know about Early Modern English 2nd person pronouns "thy"/"thine"/etc.) then it can easily get confused when determining parts of speech, which then affects other down-stream processing.

Re: The Battle over Books3

#25

Post by rhdunn »

PeterisP wrote: post #16
rhdunn wrote: post #8

I wonder how this affects things like the Oxford English Dictionary, wiktionary, and other corpus-based linguistics, which rely on sample sentences of the word usage in order to determine the context. Because language is an evolving thing, it is almost certain that they have referenced sentences from copyrighted sources. E.g. I'm willing to bet that they have the sentence where Cory Doctorow introduces the term "ensh…

It shouldn't affect corpus linguistics but the other way around, because all these things have long ago (even before computers) been legally contested by authors and publishers w.r.t. what can be done to text by dictionary makers and corpora managers without the authors' permission, so unless new laws get passed, the current precedents establishing what's permissible for corpus linguistics would still be valid and al…

Interesting, thanks!

Re: The Battle over Books3

#26

Post by thechao »

capableweb wrote: post #10
chasing wrote: post #4

Earlier quoted context omitted.

Laws. And the lawsuits that arise when such theft is discovered.

Easy to work around. Contract someone outside of the jurisdiction to provide a dataset, then it's up to them to deliver it to you. You "weren't aware" of the data source until people outside the organization starts shouting or the police starts asking questions, but the model "doesn't contain any of the data" so you continue shipping the model/product.

I don't know a lot of judges, but shenanigans like that (maybe the second or third time) are a great way to get a summary judgement, and find your way to county for a few days to think about the intent of a law.

Re: The Battle over Books3

#27

Post by DanHulton »

Any time I see the phrase "democratize access," my spidey-sense starts tingling. It's almost never used to describe an action that's an unadulterated good for society. It's USUALLY used to describe something sketchy at best, or even outright evil, with the justification that only "the bad guys" have access currently, and everything would be better if EVERYONE had access.

Look, I get that unethical corporations using this pirated training data for their artist-usurpation machines is bad, right? But EVERYONE being able to dismiss the rights and wishes of current artists while they work to create artist-usurpation machines of their own? That's not any better! You don't need to "democratize access" to that!

Re: The Battle over Books3

#29

Post by banana_giraffe »

fbdab103 wrote: post #17

From a preservation angle, how big is Books3? Is it easy enough for mortals to mirror for that unlikely future where it might be possible to self-train reasonably good models from scratch if provisioned with data?

books3.tar.gz itself is ~37gb compressed. Often really the entire "The Pile" dataset (composed of both the mostly compressed archives, along with a ~450gb compressed jsonl compilation of the data) is being discussed. That's around 825gb.
Post reply on HN