Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Things are about to get worse for generative AI

garymarcus.substack.com

Post reply on HN
Print view View original

471–480 of 769 posts

Re: Things are about to get worse for generative AI

#471

Post by proaralyst »

aimor wrote: post #467

I did an interesting thing and looked at how well the Llama2 models could compress text. For example, I took the first chapter of the first Harry Potter book and recorded the index of the 'correct' predicted token. The original text, compressed with 7zip (LZMA?) to about 14kB. The Llama2 encoded indexes compressed to less than 1kB. Then, of course, I can send that 1kB file around and decode the original text. (Unless…

While I don't disagree that these models seem to contain the ability to recreate copyrighted text, I don't think your conclusion holds. How well does zstd compress Harry Potter with a dictionary based on English prose? I think you'll get some impressive ratios, and I also think there's nothing infringing in this case.

Re: Things are about to get worse for generative AI

#472

Post by ncruces »

regularfry wrote: post #438
madamelic wrote: post #333

Earlier quoted context omitted.

> Taken to its conclusion, liability is then on everyone who decides to publish anything that ChatGPT “tells” them Yes. Just like reading anything else on the internet. An LLM is no different from typing "popular cola logo" into Google search and claiming you invented it. If I type "cola logo" into DALL-E and get a replica of Coca-Cola... that doesn't mean I created that logo and can exploit it for commercial purpose…

OpenAI might not want to shield all their customers from liability, but that is exactly what GitHub have done with Copilot. It's not a hypothetical, it's being done today.

Otherwise it wouldn't get used.

I mean get this great auto complete; if you use it, your code might be AGPLed for all you know, and you're in violation, because you didn't even add a notice.

Would you pay for that?

Re: Things are about to get worse for generative AI

#473

Post by sanderjd »

peatmoss wrote: post #388
sanderjd wrote: post #67

Earlier quoted context omitted.

I'm curious: Have you seen indications that major militaries and politicians believe AGI , rather than special purpose ML for military purposes, is important for national security? I'm really not sure whether this is true, or whether military and political leaders think it's true.

I'd guess leaders are thinking more in terms of national capacity to create more advanced technologies than geopolitical adversaries. If US policy shakes out in a way that protects copyright holders at the expense of AI innovation, I think it's apparent that the end result will be that our rivals will both violate copyright and beat us to building widespread expertise.

> I'd guess leaders are thinking more in terms of national capacity to create more advanced technologies than geopolitical adversaries.

I think there's a strong argument that they should be thinking in those terms, but I'm a lot less convinced that they do usually think in that way.

Or more charitably, they have the responsibility to balance current interests against future interests. And this isn't just a tricky thing for democracies, dictators also have to strike this same balance, just with different trade offs.

But in this case, for the US, it honestly isn't clear to me that policy makers should favor the AI side of this tussle. I think culture has been among the, if not the very, most important export of the US for nearly a century, and I think favorable copyright treatment has been at least part of the story with that.

Maybe that whole landscape is different now in a way that makes that whole model obsolete, but I think it's an open question at least.

Re: Things are about to get worse for generative AI

#474

Post by jimbobimbo »

zanfr wrote: post #446
DigitallyFidget wrote: post #443

Per United States law, imagery/art/music/text/photography generated by non-human means (such as machinery, animals, or generative AI) cannot hold copyright. https://copyright.gov/comp3/chap300/ch300-copyrightable-auth... Section 306 on page 7. I'm not sure how it'll hold up in law to claim copyright violations against something that wasn't created by a person. It'll really depend on the lawyers and judge's interpreta…

hmm then it meants generative music, as in say brian eno's experiments aren't copyrighted?

Did he ever use AI to generate music? As opposed to crafting and using an algorithm, in which case the computer is just an instrument, like synthesizer is.

Re: Things are about to get worse for generative AI

#475

Post by sebzim4500 »

aimor wrote: post #467

I did an interesting thing and looked at how well the Llama2 models could compress text. For example, I took the first chapter of the first Harry Potter book and recorded the index of the 'correct' predicted token. The original text, compressed with 7zip (LZMA?) to about 14kB. The Llama2 encoded indexes compressed to less than 1kB. Then, of course, I can send that 1kB file around and decode the original text. (Unless…

Couldn't you use the same argument to reach the absurd conclusion that the 7zip source code contains the vast majority of Harry Potter?

A decent control would be to compare it to similar prose that you know for a fact is not in the training data (e.g. because it was written afterwards).

Re: Things are about to get worse for generative AI

#476

Post by CamperBob2 »

ImPleadThe5th wrote: post #387
madamelic wrote: post #329

Earlier quoted context omitted.

Copyright should be the problem of the person using the works and not the problem of the AI generating it. Unless Nintendo plans on busting down the doors of every person who tries to draw Mario or preventing little Timmy from making a parody of Coca-Cola, making it where AI cannot generated copyrighted works is insane imo. Those brands should be proud to be such a big part of the cultural fabric that it is difficult…

The problem is the AI companies monetizing the work of copyrighted materials. It's not a problem for me to draw Micky Mouse. It _is_ a problem when someone pays me to draw an animated mouse and I sell them a picture of Micky Mouse. For me, its not really about the AI at all, it's a problem of undervaluing Artists contribution to these tools. And it's not even fully about copyright it's about not asking for permission…

It's a problem that it's still a problem to do that 100+ years after Mickey Mouse's creation.

When the law doesn't respect the people, the people will not respect the law. Fix the existing copyright system, then we can talk about AI.

Re: Things are about to get worse for generative AI

#477

Post by dangus »

madamelic wrote: post #452
dangus wrote: post #391

Earlier quoted context omitted.

The AI generating it (Hosted on OpenAI-controlled servers in the case of ChatGPT and DALL-E) is the entity redistributing the work. The end user who asked for the infringing content isn't the entity that is infringing on the copyrights and trademarks. I'm perfectly free to ask people on the street for t-shirt with Mario on it, but as soon as someone who isn't Nintendo or licensed by Nintendo sells me that t-shirt the…

> OpenAI is the entity redistributing the work It's difficult to say really. If I essentially encode knowledge of something then can recall and remix at will, am I redistributing the exact work or the knowledge of it? Yes, it is capable of producing a close to exact replica, if not the exact same input image byte-for-byte, but I find it difficult to say OpenAI is willfully redistributing copyrighted work in a whole l…

This “close to exact” thing is actually the Achilles heel of this argument. The example images in this article are so close to exact that they are quite clearly infringement, trademark or copyright. We aren’t talking about Ratatouille mouse versus Mickey Mouse, we are talking about the source picture of Mario versus a slightly altered picture of Mario that every layperson would immediately recognize as Mario composed in the exact same manner as the source image.

Courts have already defined this line over decades of copyright and trademark cases, and the examples in this article definitely cross that line.

> which again would be the fault of the person inducing the creation and not the tool that allowed it to happen.

This is not really true in practice, we can see that in various legal cases against Napster or The Pirate Bay.

Re: Things are about to get worse for generative AI

#479

Post by throwuwu »

Copyright is fucked. Even if Open AI somehow loses this and has to delete GPT4 and their training data, the generative AI cat is so far out of the bag that it’s gone on to live a full life and have many grandkittens. It’s already easy to install and run generative models and it’s just going to get easier and the models will keep getting better. These lawsuits are futile and won’t matter in 2 years or less.

Re: Things are about to get worse for generative AI

#480

Post by phatfish »

Spivak wrote: post #420
j_maffe wrote: post #146

Earlier quoted context omitted.

That's a really eloquent way of saying "It's already happening, so give up on it." I'm sure it works out great for taking action and solving problems.

It's already happening and most people like having AI more than the DMCA. Selling people on the idea that ML training is piracy to people who on average pirate content with no moral quandary will go nowhere.

People liked having Napster, but it didn't stop file sharing going from a big mainstream app to underground sites run out of Russia (or other places that ignore copyright law). Sure, you can download music/movies still, but it's not like the Napster days.

"Generative AI" is obviously copyright infringement, so owners of the copyright will win in court. Either Microsoft will have to fight a mass of legal cases, some with very deep pockets themselves, or ChatGPT will be crippled for public use.

The un-crippled models will exist if you know where to look (and have the hardware), but using them for anything apart from hobby projects would be a legal risk.

Certain specific tools may be easier to deal with from a legal standpoint, like code completion maybe. Or models for a specific purpose, like training on a law firm's case history.

It looks like Adobe has the right idea with their image generation that is trained on images which they know they have the rights to use.

Post reply on HN