Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub is sued, and we may learn something about Creative Commons licensing

scholarlykitchen.sspnet.org

Post reply on HN
Print view View original

461–470 of 475 posts

Re: GitHub is sued, and we may learn something about Creative Commons licensing

#461

Post by naasking »

TeMPOraL wrote: post #419
naasking wrote: post #367

Earlier quoted context omitted.

Sure, ROT13 encoding is a derivative work because the entire original work is still there, encoded. Ditto for FFT. Large language models are not that. Sometimes parts of the original works are still encoded, which we've seen when some code is reproduced verbatim, and I'm sure that happens to people as well, ie. they see some algorithm and down the road have to write something similar and end up reproducing the exact…

> Ditto for FFT. Large language models are not that. They're not, but the "giant table of token frequencies and associative keywords" reminded me of doing FFT on images, and I wanted to communicate the idea that transformations like this can actually retain the original information, and reproduce it back through inverse transform. > by which I mean, LLMs are lossy encodings, not lossless encodings Exactly. And while…

> And while I doubt most training data is recoverable, "lossy encoding" is still a spectrum. [...] Compare e.g. with JPEG

I'll just finally note that LLMs are not lossy encodings in the same sense as JPEG. LLMs are closer to human-like learning, where learning from data enables us to create entirely new expressions of the same concepts contained in that data, rather than acting as pure functions of the source data. That's why this will be interesting to see play out in the courts.

Re: GitHub is sued, and we may learn something about Creative Commons licensing

#462

Post by TeMPOraL »

naasking wrote: post #461
TeMPOraL wrote: post #419

Earlier quoted context omitted.

> Ditto for FFT. Large language models are not that. They're not, but the "giant table of token frequencies and associative keywords" reminded me of doing FFT on images, and I wanted to communicate the idea that transformations like this can actually retain the original information, and reproduce it back through inverse transform. > by which I mean, LLMs are lossy encodings, not lossless encodings Exactly. And while…

> And while I doubt most training data is recoverable, "lossy encoding" is still a spectrum. [...] Compare e.g. with JPEG I'll just finally note that LLMs are not lossy encodings in the same sense as JPEG. LLMs are closer to human-like learning, where learning from data enables us to create entirely new expressions of the same concepts contained in that data, rather than acting as pure functions of the source data. T…

My belief is there is no fundamental difference here. That is, learning is a form of compression. Learning concepts is just a more complex form of achieving that much greater (if lossy) compression levels. If the courts will see it the same way too, things will get truly interesting.

Re: GitHub is sued, and we may learn something about Creative Commons licensing

#463

Post by jrm4 »

Dylan16807 wrote: post #432
aliqot wrote: post #207

Earlier quoted context omitted.

> acquire that goodwill and monetize it Embrace Extend Extinguish

Uh, what's the extend that has happened since microsoft bought github? Do you mean copilot? I would not classify that as extending anything. It's just a thing they made. "Monetizing goodwill" is not an extend.

Of course that's "extending." It's a very clever attack on open source generally.

Re: GitHub is sued, and we may learn something about Creative Commons licensing

#464

Post by naasking »

TeMPOraL wrote: post #462
naasking wrote: post #461

Earlier quoted context omitted.

> And while I doubt most training data is recoverable, "lossy encoding" is still a spectrum. [...] Compare e.g. with JPEG I'll just finally note that LLMs are not lossy encodings in the same sense as JPEG. LLMs are closer to human-like learning, where learning from data enables us to create entirely new expressions of the same concepts contained in that data, rather than acting as pure functions of the source data. T…

My belief is there is no fundamental difference here. That is, learning is a form of compression. Learning concepts is just a more complex form of achieving that much greater (if lossy) compression levels. If the courts will see it the same way too, things will get truly interesting.

Yes learning concepts is a form of compression, but I'm not sure that implies there's no "fundamental" difference. I see it as akin to a programming language having only first-order functions vs. having higher-order functions. Higher-order functions give you more expressive power but not any more computational power.

You could say a higher order program can "just" be transformed into a first-order program via defunctionalization, but I think the expressive difference is in and of itself meaningful. I hope the courts can tease that out in the end, and we'll see if LLMs cross that line, or if we need something even more general to qualify.

Re: GitHub is sued, and we may learn something about Creative Commons licensing

#465

Post by Dylan16807 »

jrm4 wrote: post #463
Dylan16807 wrote: post #432

Earlier quoted context omitted.

Uh, what's the extend that has happened since microsoft bought github? Do you mean copilot? I would not classify that as extending anything. It's just a thing they made. "Monetizing goodwill" is not an extend.

Of course that's "extending." It's a very clever attack on open source generally.

But it's not open source specific. Not in creation and definitely not in use.

And the idea of the extend of EEE applying to all of open source at once, the way you could apply to a product or a standard or a protocol, doesn't really make sense.

Re: GitHub is sued, and we may learn something about Creative Commons licensing

#466

Post by baby »

sfifs wrote: post #457
baby wrote: post #445

Earlier quoted context omitted.

You’re saying this like users of github copilot are out of the eq

The mafia used to steal dresses from New York garment factory delivery trucks and hawk them door to door in poorer neighbourhoods. The users (poorer households) definitely got value out of this by being able to obtain dresses they could not afford, but doesn't make a case for what the mafia was doing was right.

I feel pretty strongly that getting rid of copilot will slow down progress at a massive scale. Not only by affecting users who are getting a huge benefit from it, but by setting a precedence in how you can train AI

Re: GitHub is sued, and we may learn something about Creative Commons licensing

#467

Post by kmeisthax »

andybak wrote: post #394
b112 wrote: post #215

Earlier quoted context omitted.

You make GPL code, a make an AI that learns from GPL code, shouldn't its output be GPL licensed as well? I think, for the desired outcome to occur, you should instead ask: You write close sourced code, then a make an AI that learns from that code, shouldn't its output be licensed as well? Ask the above, and suddenly Microsoft will agree.

Depends if you think the GPL means "copyright is great!" vs "let's use their biggest weapon against them..." It's a surprisingly subtle distinction. EDIT - if I squint hard enough in exactly the right way, there's a sense in which CoPilot etc aligns perfectly with the goals of the free software movement. A world in which you can use it as a code copyright laundry might be a world where code is actually free. Is that…

If Copilot was released as FOSS with trained model weights, I don't think the Free Software movement would have "shot first" in the resulting copyright fight.

It is specifically the idea of using copyright to eat itself that is harmed by AI training. In the world where we currently live in, only source code can be trained on. If I want to train an AI on, say, the NT kernel; I have to decompile it first, and even then it's not going to be good training data because there's no comments or variable names to guide the AI. The whole point of the GPL was to force other companies to not lock down programs and withhold source code, after all.

Keep in mind too that AI is basically proprietary software's final form. Not even the creator of an AI program has anything that resembles "source code"; and a good chunk of AI safety research boils down to "here's a program you can't comprehend except through gradient descent, how do we design it to have an incentive to not do bad things".

If you like copyright licensing and just view the GPL as an exception sales vehicle, then AI is less of a threat, because it's just another thing to sell licenses for.

Re: GitHub is sued, and we may learn something about Creative Commons licensing

#468

Post by esperent »

des1nderlase wrote: post #452
mattstir wrote: post #338

Earlier quoted context omitted.

You seem to completely miss the point about using data which was freely given. I would say that most of us like the idea of Copilot what OpenAI is accomplishing. The main issue stems from violating licenses which require attribution etc. As the article noted, one can get around attribution by getting express permission from the copyright holder (or by not using their work at all). The fact of the matter is that some…

So we are going to end up with a less powerful version of Copilot, which would benefit who? Copilot competitors?

This field is moving so fast that copilot is already way behind state of the art. New tools, even with more limited data sets, are going to be more powerful, not less powerful.

Re: GitHub is sued, and we may learn something about Creative Commons licensing

#469

Post by TeMPOraL »

naasking wrote: post #464
TeMPOraL wrote: post #462

Earlier quoted context omitted.

My belief is there is no fundamental difference here. That is, learning is a form of compression. Learning concepts is just a more complex form of achieving that much greater (if lossy) compression levels. If the courts will see it the same way too, things will get truly interesting.

Yes learning concepts is a form of compression, but I'm not sure that implies there's no "fundamental" difference. I see it as akin to a programming language having only first-order functions vs. having higher-order functions. Higher-order functions give you more expressive power but not any more computational power. You could say a higher order program can "just" be transformed into a first-order program via defunct…

> I see it as akin to a programming language having only first-order functions vs. having higher-order functions.

Interesting analogy, and I think there are a couple different "levels" of looking at it. E.g. fundamentally, they're the same thing under Turing equivalence, and in practice one can be transformed into the other - but then, I agree there is a meaningful difference for humans having to read or think in those languages. Additionally, if those are typical programming languages, you can't really have the code in the "weaker" language self-upgrade to the point the upgraded language has the same expressive power as the "stronger" one. If the "weaker" one is Lisp though, you can lift it like this.

In this sense I see traditional compression algorithms - like the ones we use for archiving, images and sound - to be like those typical weaker languages. There's a fixed set of features they exploit in their compression. But human learning vs. neural network models (or sophisticated enough non-DNN ML) is to me like Lisp vs. that stronger programming language, or even Lisp vs. a better Lisp - both can arbitrarily raise their conceptual levels as needed. But it's still fundamentally compression / programming Turing machines.

Re: GitHub is sued, and we may learn something about Creative Commons licensing

#470

Post by chrismcb »

PeterisP wrote: post #421
mpoteat wrote: post #228

Earlier quoted context omitted.

If this lawsuit succeeds, I have a startup idea that I think would be effective. Create a for-profit copyright registry for code snippets that are long enough to qualify for copyright protection. You can be the canonical owner of the copyright for a given piece of code! For a premium fee, we can generate and submit a patent on your behalf as well. Once I have a large corpus (perhaps millions of entries of code, most…

Copyright grants an author of a copyrightable work the exclusive right to make more copies of it. However, if people independently come up with the same exact thing, copying has not occurred and that exclusive right was not violated (and then the court battle effectively becomes one about proving whether copying did in fact occur). In copyright law there is no such concept as "code snippets that are long enough to qu…

Length does come into play. Copyright is about creativity. If there is no creativity there is no copyright. A code snippet that is too short probably doesn't have enough creativity for it to be copyrightable. If you wrote some code and then registered it so it has a date. It will be difficult for someone who wrote the exact same code, later, to prove it wasn't copied.
Post reply on HN