Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Any indications Copilot scans your local files?

news.ycombinator.com

Post reply on HN
Print view View original

71–80 of 88 posts

Re: Ask HN: Any indications Copilot scans your local files?

#71

Post by moyix »

moyix wrote: post #63

Alex Graveley, the Chief Architect at GitHub on Copilot, says pretty definitively here that it does not look at anything outside your project: https://twitter.com/orph/status/1457790239796199424 So I'm thinking either it made a very good guess, or the assets got included in your project without you realizing?

And a follow-up:

> We do try to assemble prompts based on contextual understanding, e.g. if you explicitly import sources by path from some other location Copilot might include it. But not for html (yet), as is being described here.

https://twitter.com/orph/status/1457796816196431872

Re: Ask HN: Any indications Copilot scans your local files?

#72

Post by a-dub »

harshitaneja wrote: post #64
kevinsundar wrote: post #59

Earlier quoted context omitted.

Seems totally reasonable to me too. It probably has just seen the pattern ``` str = "{{SOME_ID_HERE}}-jash127hg27128h" participant.follow({twitterUserId:"{{SOME_ID_HERE}}"}) ```

This doesn't seem likely. No one would be generating it this way as access token is issued after oauth and I am unaware of any method to get the second half of the token without the first half. And given that in the same response that contains access token, user id is passed as well so there is no need to extract it from there.

hm. how many large integer literals are there in your code? it could just be learning that user ids are long strings of digits and is making a guess as to which long string of digits (based on some context, like sharing a line with "id" in it) might be the right one...

Re: Ask HN: Any indications Copilot scans your local files?

#73

Post by lvass »

forgingahead wrote: post #39

It's crazy to me that amongst the many things I need to create legal blankets over when hiring developers, I now need to worry about what IDE they are using because Copilot ostensibly has access to their (our) source code, files, and other proprietary information.

Not just which IDE, but the whole development environment. If you're worried about what Copilot (Microsoft) can do with your data, how could you trust Windows machines? Even people using Emacs on Linux may be running some Microsoft assistant like LSP servers.

Re: Ask HN: Any indications Copilot scans your local files?

#74

Post by stordoff »

throwawaygh wrote: post #17

I'll second the plausible suggestion that perhaps the name/marketing copy combo isn't as unpredictable as you might think. Corporate speak and company names are pretty formulaic. Try running the name and through GPT-3 and see what happens (or GPT-2 here: https://bellard.org/textsynth/ ) E.g., I just prompted GPT-2 with a made up company name that doesn't have any google search results and got a completion like this:…

"The choice is up to you. Will you live your life in a way that’s enjoyable and worth living? Or will you drag out the most mundane and ordinary day of your life and hope that you’ll die some day? [snip a few paragraphs]

Kenji.click is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to amazon.com."

It's slightly eerie how good that is (GPT-J 6B). I generated about ten paragraphs, and other then it being a bit rambly, there was basically nothing that would make my question whether it was written by a human. Capping it off with the Amazon Associates disclaimer somehow helped complete the illusion.

I did also seemingly walk in on a bunch of AIs arguing:

> I just don’t believe it.

> It’s gotta be them.

> There’s not enough evidence.

> There’s just not enough evidence.

> It’s gotta be them.

> It’s just too far-fetched.

> There’s just not enough evidence

Re: Ask HN: Any indications Copilot scans your local files?

#75

Post by chinathrow »

polycaster wrote: post #5
dnissley wrote: post #3

Have you tried using github search to see if it's popping up there?

Not until now, thanks! There are no matches. The claim contains my client's company name. And not even the company name yields any search results.

Any (hidden) symlink in your project towards that marketing copy?

Re: Ask HN: Any indications Copilot scans your local files?

#76

Post by harshitaneja »

a-dub wrote: post #72
harshitaneja wrote: post #64

Earlier quoted context omitted.

This doesn't seem likely. No one would be generating it this way as access token is issued after oauth and I am unaware of any method to get the second half of the token without the first half. And given that in the same response that contains access token, user id is passed as well so there is no need to extract it from there.

hm. how many large integer literals are there in your code? it could just be learning that user ids are long strings of digits and is making a guess as to which long string of digits (based on some context, like sharing a line with "id" in it) might be the right one...

But that's the thing, if it took the whole string it would be fine, it extracted the exact right substring. There for 5 others in the same file. None I would have been able to distinguish from each other without context.

Re: Ask HN: Any indications Copilot scans your local files?

#77

Post by a-dub »

harshitaneja wrote: post #76
a-dub wrote: post #72

Earlier quoted context omitted.

hm. how many large integer literals are there in your code? it could just be learning that user ids are long strings of digits and is making a guess as to which long string of digits (based on some context, like sharing a line with "id" in it) might be the right one...

But that's the thing, if it took the whole string it would be fine, it extracted the exact right substring. There for 5 others in the same file. None I would have been able to distinguish from each other without context.

think of it this way, in the entire corpus of github, how often do you think that there are numeric identifiers that appear near terms like "id" where the numeric part is then used elsewhere with terms like "id" or terms that are frequently found near terms like "id"?

don't get me wrong, it's cool, but these models operate on a character by character basis with sequence context. if they can learn things like matching pairs of parens and quotes in certain contexts, it seems they could certainly learn things like extracting long strings of digits.

now what would be cool would be if they could generate regular expressions for the rules they're learning.

Re: Ask HN: Any indications Copilot scans your local files?

#78

Post by polycaster »

chinathrow wrote: post #75
polycaster wrote: post #5

Earlier quoted context omitted.

Not until now, thanks! There are no matches. The claim contains my client's company name. And not even the company name yields any search results.

Any (hidden) symlink in your project towards that marketing copy?

Nope :/

Re: Ask HN: Any indications Copilot scans your local files?

#79

Post by brudgers »

The “git” in GitHub involves cryptographic hashing.

If a file in your on-computer repository has the same git hash as a file GitHub stores, the content of each file is (statistically) identical. That’s intrinsic to git.

Copilot does not need to scan the content of files in an on-computer repository to identify relevant identical files in a remote repository on GitHub.

Though the behavior might be surprising, there’s nothing nefarious. Comparing cryptographic hashes is how Git identifies and distributes changes to files. How it controls versions.

From a security standpoint, you already had the content that Copilot suggested. The horse was already out of the bag, the cat had already sailed and the barn doors were full open not leaking around the edge.

Re: Ask HN: Any indications Copilot scans your local files?

#80

Post by polycaster »

eduhecateaway wrote: post #19

Oof, that doesn't sound great. Just out of curiosity, did you seek permission to use Copilot from your client? I wonder how widely accepted it is in roles which handle sensitive data.

As a matter of fact I didn't ask the client for consent.

In my expectation however all I willingly injected into Copilot were some HTML drafts without meaningful content or logic.

I certainly can't use Copilot anymore until I figure out what's been happening.

Post reply on HN