Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Any indications Copilot scans your local files?

news.ycombinator.com

Post reply on HN
Print view View original

51–60 of 88 posts

Re: Ask HN: Any indications Copilot scans your local files?

#51

Post by harshitaneja »

arawde wrote: post #46
harshitaneja wrote: post #38

I noticed a very peculiar thing with copilot. I was writing a twitter api wrapper and had hardcoded access token for a test user. Copilot when testing a function that requires a user id suggested one. I searched the id and it belonged to my test user. I searched my entire codebase but couldn't find any place I had used the particular id. The only place it could have extracted it from would have been access token whic…

This is something I've seen with copilot with market data. I was creating a unit test in a Go codebase and I had dumped the JSON that I was going to be decoding at the top of the file, and when I started writing the assertions, Copilot was very quick to use the data from the JSON, with quite high accuracy, based solely on me typing which ticker I was going to assert against.

I have seen that as well and it is impressive. But this is more than that. The code contained a string variable named accessToken with a string like "218276172612672-jash127hg27128h'(random data here, not the actual id), where 218276172612672 was the user id. When testing a function that required user id it not only suggested 218276172612672 but also did it with the full context participant.follow({twitterUserId:"218276172612672"})

Re: Ask HN: Any indications Copilot scans your local files?

#52

Post by Viliam1234 »

thecupisblue wrote: post #15

Well this could be a huge security issue. Can lead to potentially Copilot-surfing for company secrets in a new form, since Copilot is already leaking secret API keys and copyrighted code. The dangers of just regurgitating what has been read are unreal, since with good enough targeting you can read the data someone else wrote and expected to be anonymized. It's like huge global RAM of code, you just need to figure out…

Hacking has never been easier. Just type "username: thecupisblue, password:" and wait for autocomplete. :D

Re: Ask HN: Any indications Copilot scans your local files?

#53

Post by gwbas1c »

Assuming you're on Windows, you can see all of a process's IO using a tool like Process Monitor: https://docs.microsoft.com/en-us/sysinternals/downloads/proc...

FYI: It's a firehose, but you should be able to filter it down to copilot and a path prefix. Then you'll know if it's being scanned.

Re: Ask HN: Any indications Copilot scans your local files?

#54

Post by HWR_14 »

gruez wrote: post #22
HWR_14 wrote: post #18

Earlier quoted context omitted.

IIRC that collection is optional

According to microsoft's documentation[1] it's unintentional. Presumably because it's picked up from whatever's in memory, rather than being collected intentionally. >which may unintentionally contain user content, such as parts of a file you were using when the problem occurred [1] https://docs.microsoft.com/en-us/windows/privacy/configure-w...

Another reason why turning off automatic bug reporting is a good idea.

Re: Ask HN: Any indications Copilot scans your local files?

#55

Post by woah »

harshitaneja wrote: post #51
arawde wrote: post #46

Earlier quoted context omitted.

This is something I've seen with copilot with market data. I was creating a unit test in a Go codebase and I had dumped the JSON that I was going to be decoding at the top of the file, and when I started writing the assertions, Copilot was very quick to use the data from the JSON, with quite high accuracy, based solely on me typing which ticker I was going to assert against.

I have seen that as well and it is impressive. But this is more than that. The code contained a string variable named accessToken with a string like "218276172612672-jash127hg27128h'(random data here, not the actual id), where 218276172612672 was the user id. When testing a function that required user id it not only suggested 218276172612672 but also did it with the full context participant.follow({twitterUserId:"218…

Seems reasonable. What's your question? GPT-3 is just that good.

Re: Ask HN: Any indications Copilot scans your local files?

#56

Post by kall »

harshitaneja wrote: post #38

I noticed a very peculiar thing with copilot. I was writing a twitter api wrapper and had hardcoded access token for a test user. Copilot when testing a function that requires a user id suggested one. I searched the id and it belonged to my test user. I searched my entire codebase but couldn't find any place I had used the particular id. The only place it could have extracted it from would have been access token whic…

If it‘s always in the same location inside the api key, it seems very reasonable that it would pick up that pattern, as most projects that include a hardcoded key would include it. API key variables are probably named almost the same everywhere.

Re: Ask HN: Any indications Copilot scans your local files?

#57

Post by yumraj »

iamcurious wrote: post #29
aspenmayer wrote: post #25

Earlier quoted context omitted.

I’m assuming you meant “blurb” instead of “blur”?

Thanks corrected. While we are at it. Is it "make the following experiment" or "do the following experiment" ?

I'll say, run the following experiment.

Re: Ask HN: Any indications Copilot scans your local files?

#58

Post by polycaster »

minimaxir wrote: post #32
akersten wrote: post #12

Have you considered that your marketing blurb is actually, not that novel after all? GPT3 is damn convincing these days :)

This is a joke, but it may be the correct answer: if the company name is self-demonstrating, it's possible Codex could recognize that.

I don't think so. The phrase is 272 characters long and contains very specific terms which cannot be derived from the company's name. Also, it's a 1:1 match with the unpublished marketing material.

Re: Ask HN: Any indications Copilot scans your local files?

#59

Post by kevinsundar »

harshitaneja wrote: post #51
arawde wrote: post #46

Earlier quoted context omitted.

This is something I've seen with copilot with market data. I was creating a unit test in a Go codebase and I had dumped the JSON that I was going to be decoding at the top of the file, and when I started writing the assertions, Copilot was very quick to use the data from the JSON, with quite high accuracy, based solely on me typing which ticker I was going to assert against.

I have seen that as well and it is impressive. But this is more than that. The code contained a string variable named accessToken with a string like "218276172612672-jash127hg27128h'(random data here, not the actual id), where 218276172612672 was the user id. When testing a function that required user id it not only suggested 218276172612672 but also did it with the full context participant.follow({twitterUserId:"218…

Seems totally reasonable to me too. It probably has just seen the pattern

``` str = "{{SOME_ID_HERE}}-jash127hg27128h"

participant.follow({twitterUserId:"{{SOME_ID_HERE}}"}) ```

Re: Ask HN: Any indications Copilot scans your local files?

#60

Post by a-dub »

harshitaneja wrote: post #51
arawde wrote: post #46

Earlier quoted context omitted.

This is something I've seen with copilot with market data. I was creating a unit test in a Go codebase and I had dumped the JSON that I was going to be decoding at the top of the file, and when I started writing the assertions, Copilot was very quick to use the data from the JSON, with quite high accuracy, based solely on me typing which ticker I was going to assert against.

I have seen that as well and it is impressive. But this is more than that. The code contained a string variable named accessToken with a string like "218276172612672-jash127hg27128h'(random data here, not the actual id), where 218276172612672 was the user id. When testing a function that required user id it not only suggested 218276172612672 but also did it with the full context participant.follow({twitterUserId:"218…

are you sure that "big number that starts with 2" wasn't just the greatest 32 bit 2s complement signed integer, which is often used for sentinel/testing values?
Post reply on HN