Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta's Llama 3.1 can recall 42 percent of the first Harry Potter book

understandingai.org

Post reply on HN
Print view View original

281–290 of 326 posts

Re: Meta's Llama 3.1 can recall 42 percent of the first Harry Potter book

#281

Post by rpd9803 »

Lerc wrote: post #215
Retric wrote: post #178

Earlier quoted context omitted.

I’ve yet to read an actual argument defending commercial LLM’s as fair use based on existing (edit:legal) criteria.

Based upon legal decisions in the past there is a clear argument that the distinction for fair use is whether a work is substantially different to another. You are allowed to write a book containg information you learned about from another book. There is threshold in academia regarding plagiarism that stands apart from the legal standing. The measure that was used in Gyles v Wilcox was if the new work could substitut…

Copyright fair use rules are tools designed to govern how humans use protected works in dervied works. AI is not human use, therefore the rules are only coincidentally correct for AI use where it even is.

Re: Meta's Llama 3.1 can recall 42 percent of the first Harry Potter book

#282

Post by TeMPOraL »

strogonoff wrote: post #273
fennecfoxy wrote: post #229

Earlier quoted context omitted.

I keep waiting for the day when people realise that IP law has been used and abused and thanks to Disney extended out for many, many lifetimes and all manner of dirty tricks/hacks to keep the late stage capitalism profit engine going. I 100% agree that if an LLM can entirely reproduce a book then that is copyright infringement, overfitting and generally a bad model. I also believe that in this case, HP (and other pop…

Abuse of IP does not mean the law is not relevant. Don’t you find it funny that corporations with market caps the size of small countries first sued people for singing Happy Birthday in public, and now pretend that IP is suddenly not really a thing? Do you really want to defend their interests?

What abou defending our own interests?

I couldn't care less about Meta or OpenAI or other tech companies as entities. I'm happy to oppose them when they act against our interests. But in this, my interests align with theirs.

It's not like the alternative is any better. I'm much more in a philosophical disagreement with the pro-copyright side, for one, but the other thing is, it too is represented primarily by other large corporations, and of the two groups, the LLM side has a much more honest and useful business model.

Re: Meta's Llama 3.1 can recall 42 percent of the first Harry Potter book

#283

Post by TeMPOraL »

xigoi wrote: post #224
jiggawatts wrote: post #222

Earlier quoted context omitted.

If you train a meat-based intelligence by having it borrow a book from a library without any sort of permission, license, or needing a lawyer specialised in intellectual property, we call that good parenting and applaud it. If you train a silicon-based intelligence by having it read the same books with the same lack of permission and license, it's a blatant violation of intellectual property law and apparently needs…

> If you train a meat-based intelligence by having it borrow a book from a library without any sort of permission, license, or needing a lawyer specialised in intellectual property, we call that good parenting and applaud it. If you’re selling your child as a tool to millions of people, I would certainly not call that good parenting.

What about a company funding books and education materials to train its employees into specialists, and then selling access to them to other businesses? E.g. any honest consulting company.

Re: Meta's Llama 3.1 can recall 42 percent of the first Harry Potter book

#284

Post by Retric »

triceratops wrote: post #278
yunwal wrote: post #276

Earlier quoted context omitted.

No it doesn’t? You can buy a digital copy of Harry Potter and use it for training. No infringement needed.

Only as long as it's not copied again during training. You can't make copies of your purchased digital copy for any reason other than archival.

Incidental copies during playback are also allowed. But none of these companies are paying for copies in the first place.

Re: Meta's Llama 3.1 can recall 42 percent of the first Harry Potter book

#285

Post by almosthere »

jiggawatts wrote: post #225
xigoi wrote: post #224

Earlier quoted context omitted.

> If you train a meat-based intelligence by having it borrow a book from a library without any sort of permission, license, or needing a lawyer specialised in intellectual property, we call that good parenting and applaud it. If you’re selling your child as a tool to millions of people, I would certainly not call that good parenting.

"Child actor" is a job where the result of the neural net training is sold to millions of people by the parents. To play the Devil's Advocate against my own argument: The government collects income taxes on neural nets trained using government-funded schools and public libraries. Seeing as how capitalists are positively salivating at the opportunity to replace pesky meat employees with uncomplaining silicon ones, per…

"Child Actors" are more an exception. You can train a million children on the books of harry potter, only 3 or 4 will be good enough to be actors. The children that "made it" did so from grit and passion (or other traits) but very little from that reading of 10-20 books.

The AI that reads the books, and can do what LLMs do, are guaranteed to sold for billions in API calls.

Re: Meta's Llama 3.1 can recall 42 percent of the first Harry Potter book

#286

Post by ben_w »

strogonoff wrote: post #272
ben_w wrote: post #253

Earlier quoted context omitted.

> I keep waiting for the day when software stops being compared to a human person (a being with agency, free will, consciousness, and human rights of its own) for the purposes of justifying IP law circumvention. I mean, "agency" is a goal of some AI; "free will" is incoherent*; the word "consciousness" has about 40 different definitions, some of which are so broad they include thermostats and others so narrow that it…

> and "human rights" are a purely legal concept. Yep, and if you claim that a thing can reproduce IP like a human then you should explain why you are also not holding its operators to the same legal standard (try to use a human in the same way and it will be considered torture and slavery).

I am specifically not using that to claim "and therefore the AI is a human". The point is that "human rights" are not part of the natural order, they only exist as laws.

This means that "human rights" is basically irrelevant to this topic: they may have rights and need to be liberated, or they may be tools that don't, but the law is just words on paper, and officials who make you follow those words.

Re: Meta's Llama 3.1 can recall 42 percent of the first Harry Potter book

#287

Post by strogonoff »

TeMPOraL wrote: post #282
strogonoff wrote: post #273

Earlier quoted context omitted.

Abuse of IP does not mean the law is not relevant. Don’t you find it funny that corporations with market caps the size of small countries first sued people for singing Happy Birthday in public, and now pretend that IP is suddenly not really a thing? Do you really want to defend their interests?

What abou defending our own interests? I couldn't care less about Meta or OpenAI or other tech companies as entities. I'm happy to oppose them when they act against our interests. But in this, my interests align with theirs. It's not like the alternative is any better. I'm much more in a philosophical disagreement with the pro-copyright side, for one, but the other thing is, it too is represented primarily by other l…

First, I think they are exploiting the old conundrum where your personal interests may seem to align with theirs, but taken together our interests do not align with theirs. Sure, as it stands, an individual may enjoy personal gain—at least for the time being, while their prices are low as they operate at a loss to capture the market—but I believe this brief individual gain comes at the expense of society as a whole, and if people in more areas (not just music) made this an issue, and those megacorps put their infinite armies of lawyers to work on licensing (I recall somebody here once saying “do things that don’t scale” or something like that), everybody would benefit much more[0].

Second, copyright, or more precisely IP ownership, is not the same as the exploitation you might have in mind. Consider that copyleft—which gave us Linux, Blender, etc.—exists thanks to the ability to exercise and defend IP rights. When a stronger party takes that ability away from a weaker party, that is exploitation; maybe it is not a coincidence that Microsoft is at the forefront of this, as what they are doing is very much in line with a generalized form of EEE.

(Let’s be honest, homegrown LLMs will never reach the level of commercial models, and that’s where the money is headed; even GPUs aside, no individual has the ability to scrape the entire Web the way they do, even less so now once the ruthless interests of those corporations sent every website scrambling to defend themselves against what is predominantly bot traffic with layers upon layers of captcha never seen before. They realized that if they steal, they have to steal a lot and very quickly in order for it to work, and they’re certainly hoping to get away with it.)

More abstractly, being able to say that you have created something is pretty important to our willingness to create; I can’t see how inability to make this claim with a straight face (because anyone can, and already does as you have no doubt seen on this forum, reasonably claim that it may have not been your work at all) is good for society generally.

[0] Starting with awareness of what is going on and the freedom to choose. Information asymmetry is the enemy of free market.

Re: Meta's Llama 3.1 can recall 42 percent of the first Harry Potter book

#288

Post by simianwords »

kelseyfrog wrote: post #267
simianwords wrote: post #261

Earlier quoted context omitted.

I don't like many things about this post, its a bit snobbish and uses esoteric language in order to sound more intricate than it really is. >Capitalism is not simply incapable of reflection. In fact, it's structured to ignore it. It has no native interest in what emerges from its aggregated behaviors unless those emergent properties threaten the throughput of capital itself. It isn't designed to ask, "What kind of so…

Appreciate the engagement. But your reply mostly recenters a pro-capitalist narrative by redefining the products of "emergence" as inherently good. My argument isn't about stacking pros and cons and calculating the combined sum. It’s about a structural blind spot: capitalism systematically collapses higher-order questions about what kind of world were building into first-order value propositions like "growth," "utili…

> "emergence" as inherently good

I did not claim it as inherently good, only that it is preferable.

> capitalism systematically collapses higher-order questions about what kind of world were building into first-order value propositions like "growth," "utility," and "innovation."

There is nothing about capitalism that ignores second or third order effects of its policies. Let me make it clear what kind of capitalist system we have in place - private ownership and free market regulated by a government that works for and is elected by the people. In this system the free market works but only till it progresses certain things the people voted for like standard of living, freedom etc. If free market does instead has unintended consequences we have levers to guide it where we want like taxes and subsidies.

> Capitalism resists second-order critique from within because it translates every possible value: justice, meaning, even critique itself, into terms it can price or optimize

I think I see were you are getting at but I have to be honest - I think it is coming from a naive place (I'm open to be proven incorrect).

Imagine you had the power and the responsibility to shape lives by enacting policy decisions. You are presented with a fairly complex problem where you have a large number of people, each one with their own lives and interests and you have to guide them into doing something preferable. No matter where you come from, left or right in the political axis, you will end up using quantitative methods. I imagine your problem is with such optimisation. If so, what is your exact critique here? How would you rather handle such a situation? How would you manage a system of so many people and without quantitative method? Religion?

> If you were engaging at the second-order level, you'd interrogate not what the system produces, but what it refuses to ask, and who gets to decide. That silence is precisely my point.

Ok please elaborate (only if you have engaged with my question above).

Re: Meta's Llama 3.1 can recall 42 percent of the first Harry Potter book

#289

Post by strogonoff »

ben_w wrote: post #286
strogonoff wrote: post #272

Earlier quoted context omitted.

> and "human rights" are a purely legal concept. Yep, and if you claim that a thing can reproduce IP like a human then you should explain why you are also not holding its operators to the same legal standard (try to use a human in the same way and it will be considered torture and slavery).

I am specifically not using that to claim "and therefore the AI is a human". The point is that "human rights" are not part of the natural order, they only exist as laws. This means that "human rights" is basically irrelevant to this topic: they may have rights and need to be liberated, or they may be tools that don't, but the law is just words on paper, and officials who make you follow those words.

> The point is that "human rights" are not part of the natural order, they only exist as laws.

I have seen this argument used before; when something suits an argument, it’s “nature”, when something doesn’t then it isn’t. I think it’s a fallacy.

Humans are part of natural order. Our laws and how they evolve are part of our nature, and by extension part of nature. I don’t believe in silencing such discussions as irrelevant because of an imaginary cutoff point where it stops being part of nature and suddenly becomes “artificial”.

> This means that "human rights" is basically irrelevant to this topic: they may have rights and need to be liberated, or they may be tools that don't, but the law is just words on paper, and officials who make you follow those words.

I’m not sure how it is irrelevant. If we can claim “LLMs are like human, and so their creators and commercial operators are not guilty of IP laundering that LLMs do”, then we have a moral imperative to stop using them because, well, they are like human, and no human should be put through what would be, to any human mind, abuse. If we do not believe they are human and free in this sense, then the excuse quoted above in this paragraph also stops applying.

Re: Meta's Llama 3.1 can recall 42 percent of the first Harry Potter book

#290

Post by ben_w »

strogonoff wrote: post #289
ben_w wrote: post #286

Earlier quoted context omitted.

I am specifically not using that to claim "and therefore the AI is a human". The point is that "human rights" are not part of the natural order, they only exist as laws. This means that "human rights" is basically irrelevant to this topic: they may have rights and need to be liberated, or they may be tools that don't, but the law is just words on paper, and officials who make you follow those words.

> The point is that "human rights" are not part of the natural order, they only exist as laws. I have seen this argument used before; when something suits an argument, it’s “nature”, when something doesn’t then it isn’t. I think it’s a fallacy. Humans are part of natural order. Our laws and how they evolve are part of our nature, and by extension part of nature. I don’t believe in silencing such discussions as irrele…

By that tautology, so are rocks. Rocks don't get natural rights.

Also, you may note that "human rights" is a recent invention and not actually enforced worldwide even today.

Post reply on HN