Ask HN: Any indications Copilot scans your local files? • Hacker News Board

Re: Ask HN: Any indications Copilot scans your local files?

#21

Post by numair » Mon, Nov 08, 2021, 4:18 PM UTC

Have you told your client that you are making use of tools that upload the intellectual property they’ve shared with you / they’ve paid you to create for them, onto third-party servers?

If I was paying for work and the contractor was uploading the end result onto some sort of shared AI training set, we would not be working together for very long.

You may have brought up an excellent point that needs to be inserted into new legal contracts — either an opt in or out regarding the use of tools of various kinds that upload data into the cloud to “help” you. Maybe other companies would be okay with stuff like Copilot if it allows them to pay less money for developers who can’t write proper code without it, or something. I don’t know. I know that I want nothing to do with these sorts of systems, and I don’t want any of my code anywhere near it. I’ll definitely try to make sure nobody with access to my private repos has any of that nonsense enabled.

Maybe the legal version of Copilot can write an appropriate contract clause for us?

Re: Ask HN: Any indications Copilot scans your local files?

#22

Post by gruez » Mon, Nov 08, 2021, 4:22 PM UTC

HWR_14 wrote: post #18

lvass wrote: post #10

Earlier quoted context omitted.

Are you using Windows? IIRC Microsoft collects a hefty amount of data from your filesystem.

IIRC that collection is optional

According to microsoft's documentation[1] it's unintentional. Presumably because it's picked up from whatever's in memory, rather than being collected intentionally.

>which may unintentionally contain user content, such as parts of a file you were using when the problem occurred

[1] https://docs.microsoft.com/en-us/windows/privacy/configure-w...

Re: Ask HN: Any indications Copilot scans your local files?

#23

Post by rustc » Mon, Nov 08, 2021, 4:27 PM UTC

numair wrote: post #21

Have you told your client that you are making use of tools that upload the intellectual property they’ve shared with you / they’ve paid you to create for them, onto third-party servers? If I was paying for work and the contractor was uploading the end result onto some sort of shared AI training set, we would not be working together for very long. You may have brought up an excellent point that needs to be inserted in…

> uploading the end result onto some sort of shared AI training set

Does Copilot use the data it collects for training their AI? From their terms [1]

> GitHub Copilot does not use these URLs, file paths, or snippets collected in your telemetry as suggestions for other users of GitHub Copilot. This information is treated as confidential information and accessed on a need-to-know basis.

[1] https://docs.github.com/en/github/copilot/github-copilot-tel...

Re: Ask HN: Any indications Copilot scans your local files?

#24

Post by iamcurious » Mon, Nov 08, 2021, 4:28 PM UTC

Do the following experiment:

1) Put in the same folder where you had that marketing blurb, a new blurb. Make sure it is unique but still looks like english. Example: now your startup allows you to fly to the moon in rockets powered by angry tweets.

2) Try to force a reload of copilot. Maybe reinstall it?

3) Recreate the conditions that suggested the first blurb, but trying to suggest the second blurb.

4) Share the results, I am curious.

If you get that suggestion, you have proof and reproductible steps for others to get proof. If you don't get that suggestion, we can't be sure, but the odds of it being just a coincidence increase.

Good luck!

Re: Ask HN: Any indications Copilot scans your local files?

#25

Post by aspenmayer » Mon, Nov 08, 2021, 4:31 PM UTC

iamcurious wrote: post #24

Do the following experiment: 1) Put in the same folder where you had that marketing blurb, a new blurb. Make sure it is unique but still looks like english. Example: now your startup allows you to fly to the moon in rockets powered by angry tweets. 2) Try to force a reload of copilot. Maybe reinstall it? 3) Recreate the conditions that suggested the first blurb, but trying to suggest the second blurb. 4) Share the re…

I’m assuming you meant “blurb” instead of “blur”?

Re: Ask HN: Any indications Copilot scans your local files?

#26

Post by rustc » Mon, Nov 08, 2021, 4:31 PM UTC

numair wrote: post #21

Have you told your client that you are making use of tools that upload the intellectual property they’ve shared with you / they’ve paid you to create for them, onto third-party servers? If I was paying for work and the contractor was uploading the end result onto some sort of shared AI training set, we would not be working together for very long. You may have brought up an excellent point that needs to be inserted in…

> Have you told your client that you are making use of tools that upload the intellectual property they’ve shared with you / they’ve paid you to create for them, onto third-party servers?

Related: Have any of the big tech companies banned their employees from using Copilot yet?

Re: Ask HN: Any indications Copilot scans your local files?

#27

Post by WithinReason » Mon, Nov 08, 2021, 4:34 PM UTC

Is it possible Copilot is using a network pretrained on a large text dataset that did contain the marketing blurb, then retrained for code prediction from GitHub source code? That might explain why it has memorized non-GitHub content (it's a bit of a reach though).

Re: Ask HN: Any indications Copilot scans your local files?

#28

Post by paulintrognon » Mon, Nov 08, 2021, 4:35 PM UTC

numair wrote: post #21

Have you told your client that you are making use of tools that upload the intellectual property they’ve shared with you / they’ve paid you to create for them, onto third-party servers? If I was paying for work and the contractor was uploading the end result onto some sort of shared AI training set, we would not be working together for very long. You may have brought up an excellent point that needs to be inserted in…

> you are making use of tools that upload the intellectual property [...] onto third-party servers?

That's the whole point: if they did, it was NOT willingly: they have the intellectual property stored in a completely separate folder, which should not be uploaded to Github.

Re: Ask HN: Any indications Copilot scans your local files?

#29

Post by iamcurious » Mon, Nov 08, 2021, 4:36 PM UTC

aspenmayer wrote: post #25

iamcurious wrote: post #24

Do the following experiment: 1) Put in the same folder where you had that marketing blurb, a new blurb. Make sure it is unique but still looks like english. Example: now your startup allows you to fly to the moon in rockets powered by angry tweets. 2) Try to force a reload of copilot. Maybe reinstall it? 3) Recreate the conditions that suggested the first blurb, but trying to suggest the second blurb. 4) Share the re…

I’m assuming you meant “blurb” instead of “blur”?

Thanks corrected. While we are at it. Is it "make the following experiment" or "do the following experiment" ?

Re: Ask HN: Any indications Copilot scans your local files?

#30

Post by rowanG077 » Mon, Nov 08, 2021, 4:38 PM UTC

akersten wrote: post #12

Have you considered that your marketing blurb is actually, not that novel after all? GPT3 is damn convincing these days :)

The entire point of GPT3 is to create novel content that is similar to existing content. So that copilot suggested it doesn't mean it's not novel as a whole.