Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

All public GitHub code was used in training Copilot

twitter.com

Post reply on HN
Print view View original

611–620 of 734 posts

Re: All public GitHub code was used in training Copilot

#611

Post by svaha1728 »

nonfamous wrote: post #561
svaha1728 wrote: post #474

Earlier quoted context omitted.

So if a foreign company pilfers the source code to Windows, can they add it to a training set and then 'prompt' the machine learning algorithm to spit out a new 'copyright free' Windows, just by transforming the variable names?

Well no, because only GitHub has access to the training set. But more importantly this misunderstands how Copilot even works -- even if Windows was in the training set, you couldn't get Copilot to reproduce it. It only generates a few lines of code at a time, and even then it's almost certainly entirely novel code. Now, if you knew the code you wanted Copilot to generate you could certainly type it character by chara…

GPT-3 is still Microsoft licensed, but a similar model can be put together with the freely available GPT-2 and source code -- especially if your intent is copyright transfer.

As Francois Chollet points out in this talk, ultimately deep neural network models are locally sensitive hash tables, so the examples of people pulling out source code is an inherent shortcoming of deep learning models in general. Give the right 'key' and you can 'recall' the value you are looking for.

https://www.youtube.com/watch?v=J0p_thJJnoo

Re: All public GitHub code was used in training Copilot

#612

Post by lathiat »

fleddr wrote: post #551

To me, the particular use case and whether it is fair use or not, is of minor interest. A far more pressing matter is at hand: AI centralization and monopolization. Take Google as an example, running Google Photos for free for several years. And now that this has sucked in a trillion photos, the AI job is done, and they likely have the best image recognition AI in existence. Which is of course still peanuts compared…

I've got 70,000 photos in my library, with AI search and recognition, all done on my device. Thanks Apple.

In fairness it's not quite as good, but, it's good enough for the searches I've wanted to do so far and gets better all the time. And they're adding searching text in photos this release. I'm happy to wait a little for this better implementation.

Re: All public GitHub code was used in training Copilot

#613

Post by jcheng »

oefnak wrote: post #571
jcheng wrote: post #525

Earlier quoted context omitted.

Only if you agree that copyleft licenses are also stupid; without copyright, there's no way to prevent companies from making closed-source forks of code you wrote and intended to stay open.

Yes, sure. Without copyright there's no need for copyleft left, right?

No...? Not unless that closed-source project's source code is leaked?

Re: All public GitHub code was used in training Copilot

#614

Post by mikewarot »

mceachen wrote: post #604
mikewarot wrote: post #575

Earlier quoted context omitted.

I have about 300,000 photos that haven't been scanned by AI (unless someone at Backblaze did it without permission). I'm sure there are lots of other photographers out there who miss Picassa, which Google killed off to push everyone's data to their service. (It did really well in matching faces, even across age, but the last version has a bug when there are multiple faces in a picture, sometimes it swaps the labels)…

Absolutely possible. What I'm building into PhotoStructure is typically called "transfer learning." https://en.wikipedia.org/wiki/Transfer_learning PhotoStructure is entirely self-hosted, including model training and application: the public domain base models (trained on huge datasets) are fetched and cached locally. By design, none of your data (or even metadata) leaves your server. (I expect to ship this in an upco…

I want to label all the faces in the photos I've taken since 1997, and save them in the metadata. I'll be glad to run it against my photos. Windows 10, WSL, and/or Virtual Machine with Linux of your choice.

Re: All public GitHub code was used in training Copilot

#615

Post by mikewarot »

Mehdi2277 wrote: post #598
cgio wrote: post #586

Earlier quoted context omitted.

I am not an expert on the field. But my hope was that this could be facilitated by Transfer Learning. Still don’t know how the scale economies could be achieved. Maybe just out of the sweat and network of passionate people like in the case of open source.

I work in the field. Transfer learning helps get you decent/good models, but the best models remain ones trained on large amounts of data. You may be able to get away with good performance and not great on your task. For some areas you really care a lot about long tail performance (like self driving) that you will need massive dataset. For other areas if your goal is to be the best relative to other large companies y…

If there are hundreds of people with 100,000 photos each, that collectively is a massive training database, with a lot more labels and diversity of subjects.

By keeping the training data itself private, distributed and outsourced, you might be able to get otherwise unachievable levels of performance.

Re: All public GitHub code was used in training Copilot

#616

Post by krainboltgreene »

fragmede wrote: post #568
superfrank wrote: post #511

Earlier quoted context omitted.

> Perhaps people on HN start sensing that successors of Github Copilot will take their programming job. Rightly so. I still think we're a long way from that. Copilot will help write code quicker, but it's not doing anything you couldn't do with a Google search and copy/paste. Once developers move beyond the jr. level, writing code tends to become the least of their worries. Writing the code is easy, understanding how…

Depends on your definition of "a long way". Some of the GPT3 based code generation demos (which, explicitly, are just that - demos - we aren't shown the limitations of the system during the demo) say that's closer than I think. https://analyticsindiamag.com/open-ai-gpt-3-code-generator-a... has a bunch of videos of this in action.

That's because the training set had that specific demo, not because copilot imagined up a demo.

Re: All public GitHub code was used in training Copilot

#617

Post by zarzavat »

nerdponx wrote: post #416
jrm4 wrote: post #261

Earlier quoted context omitted.

You're strongly and incorrectly implying that "Fair Use" is a clear (and relatively immutable) concept within copyright law, which couldn't be further from the truth. Even if this or that particular case sets out what appears to be solid grounds, one shouldn't take that as gospel by any means. This mostly has to do with the nature of the wishy-washy nature of the 4 part Fair Use test, which, unlike decent legal tests…

Unanswered question, as far as I know: is a trained model a derivative work? If the model accidentally retains a copy of the work, is that an unauthorized copy?

In my opinion, the model would not be an unauthorized copy given that it's primary purpose was for some other task and the inclusion of the work was merely incidental.

The unauthorized copy arises when someone gets the work out of the model.

Of course if you make a model explicitly for the purpose of evading copyright then the courts will see through that ploy.

Re: All public GitHub code was used in training Copilot

#618

Post by minxomat »

someperson wrote: post #607
fleddr wrote: post #551

To me, the particular use case and whether it is fair use or not, is of minor interest. A far more pressing matter is at hand: AI centralization and monopolization. Take Google as an example, running Google Photos for free for several years. And now that this has sucked in a trillion photos, the AI job is done, and they likely have the best image recognition AI in existence. Which is of course still peanuts compared…

On the other hand, DeepL (made by a small German company) is better than Google Translate.

Makes it sound like DeepL exists in isolation. It's good because the company behind it has the largest hyperlocal (small phrases with confirmed usages) translation data set.

Re: All public GitHub code was used in training Copilot

#619

Post by spyckie2 »

fleddr wrote: post #551

To me, the particular use case and whether it is fair use or not, is of minor interest. A far more pressing matter is at hand: AI centralization and monopolization. Take Google as an example, running Google Photos for free for several years. And now that this has sucked in a trillion photos, the AI job is done, and they likely have the best image recognition AI in existence. Which is of course still peanuts compared…

The thought that data monopolization will be a moat against competitors is actually argued against by VC firms specializing in AI companies, who claim that after a certain amount of data (which is accessible to most people) the additional data isn't going to improve the model much.

https://a16z.com/2019/05/09/data-network-effects-moats/

https://a16z.com/2020/02/16/the-new-business-of-ai-and-how-i...

Re: All public GitHub code was used in training Copilot

#620

Post by midev »

MadcapJake wrote: post #289
PieUser wrote: post #241

Earlier quoted context omitted.

Copilot just suggests code.

If I suggest whole lines of dialogue to you, the screenwriter, did I write those lines or you? If you change names in those lines of dialogue to fit your story, do you now gain credit for writing those lines? Suggesting code is generating code

> did I write those lines or you

Neither. Someone else did, and published it. Copilot copied the dialog and suggested it.

> If you change names in those lines of dialogue to fit your story, do you now gain credit for writing those lines?

It depends. Talking generalities isn't productive or interesting. Can you give an example and we can discuss specifics?

> Suggesting code is generating code

This isn't even superficially true

Post reply on HN