Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When coding style survives compilation: De-anonymizing programmers from binaries

freedom-to-tinker.com

Post reply on HN
Print view View original

61–70 of 74 posts

Re: When coding style survives compilation: De-anonymizing programmers from binaries

#61

Post by kazinator »

Tinyyy wrote: post #32

This is an interesting article, but there's one part that I don't really understand: 'After scaling up the approach by increasing the dataset size, we de-anonymize 600 programmers with 52% accuracy.'. Isn't 52% close to a random guess? I don't get how this ties in with the rest of the paragraph either.

52% is close to a random guess, yes: between two alternative outcomes, such as heads or tails in a coin toss.

If the question is, which of these 20 programmers wrote this code", then a random guess has only a 5% chance (1/20) of being right! So 52% is more than 10X better than random.

Re: When coding style survives compilation: De-anonymizing programmers from binaries

#62

Post by dllthomas »

nothrabannosir wrote: post #57
Lawtonfogle wrote: post #36

Earlier quoted context omitted.

If they were saying that in 52% of the cases, they could guess the correct choice out of 600, then random chance alone would have been If they were saying that in 52% of cases, they could guess if some code did or did not belong to a given programmer, then it is basically a random guess.

> If they were saying that in 52% of cases, they could guess if some code did or did not belong to a given programmer, then it is basically a random guess. That entirely depends on the distribution of the original sample.

Does it? If you're asked "does this code belong to this programmer?" and flip a coin, you'll approximate 50% regardless of distribution of programmers.

Of course, "const No" might do quite a bit better than 50%, depending on the distribution of the original sample (for that matter, so might "const Yes", but those distributions seem less likely).

Re: When coding style survives compilation: De-anonymizing programmers from binaries

#63

Post by SoftwareMaven »

x1798DE wrote: post #7
motoboi wrote: post #2

Maybe we can use this method to find Satoshi, if he uses github.

Leaving aside that finding Satoshi isn't a particularly laudable goal, isn't the original Bitcoin client open source? It's almost certainly easier to detect programming style from source code than from binaries.

The bitcoin client would be the training set.

Re: When coding style survives compilation: De-anonymizing programmers from binaries

#64

Post by nightpool »

tshadwell wrote: post #13
cjslep wrote: post #5

Really neat, especially since the source could be from different languages and compilers. I'd be interested at the deanonymization accuracy within the Go language where there is a widespread adoption of the code formatting tool and seeing if that has any impact.

I write almost exclusively Go these days, and I think most of the people I work with could tell you if I wrote a Go program

From only the binary code? That's quite a feet for the unassisted human :P (mind sharing what kind of people you work with?)

Re: When coding style survives compilation: De-anonymizing programmers from binaries

#65

Post by michaelt »

dllthomas wrote: post #62
nothrabannosir wrote: post #57

Earlier quoted context omitted.

> If they were saying that in 52% of cases, they could guess if some code did or did not belong to a given programmer, then it is basically a random guess. That entirely depends on the distribution of the original sample.

Does it? If you're asked "does this code belong to this programmer?" and flip a coin, you'll approximate 50% regardless of distribution of programmers. Of course, "const No" might do quite a bit better than 50%, depending on the distribution of the original sample (for that matter, so might "const Yes", but those distributions seem less likely).

Well, if there were 600 programmers, one of whom had written 649 programs and 599 of whom had written 1 program each, you could achieve 52% accuracy by always guessing the same guy.

Of course, the paper explicitly says "For this experiment, we use 600 contestants from [Google Code Jam] with 9 files" so I think in this case the distribution was probably fairly even?

Re: When coding style survives compilation: De-anonymizing programmers from binaries

#66

Post by dllthomas »

michaelt wrote: post #65
dllthomas wrote: post #62

Earlier quoted context omitted.

Does it? If you're asked "does this code belong to this programmer?" and flip a coin, you'll approximate 50% regardless of distribution of programmers. Of course, "const No" might do quite a bit better than 50%, depending on the distribution of the original sample (for that matter, so might "const Yes", but those distributions seem less likely).

Well, if there were 600 programmers, one of whom had written 649 programs and 599 of whom had written 1 program each, you could achieve 52% accuracy by always guessing the same guy. Of course, the paper explicitly says "For this experiment, we use 600 contestants from [Google Code Jam] with 9 files" so I think in this case the distribution was probably fairly even?

I think you missed some context. Up-thread, someone raised the point that it's unclear whether the question was "Of these 600 programmers, which wrote this code?" or "Did this programmer write this code?", and this subthread is discussing the latter case.

Re: When coding style survives compilation: De-anonymizing programmers from binaries

#67

Post by worldsayshi »

placeybordeaux wrote: post #55
worldsayshi wrote: post #53

Earlier quoted context omitted.

I thought about looking at the bitcoin implementation for finding Satoshi.

We have the commit history for the source code of bitcoin to profile Satoshi. No need to try and do it based off of the binary.

Sure. I thought of the general method.

I saw that this has been tried to some extent: https://en.wikipedia.org/wiki/Satoshi_Nakamoto#Nick_Szabo

Re: When coding style survives compilation: De-anonymizing programmers from binaries

#68

Post by bambax »

golergka wrote: post #47
bambax wrote: post #16

This statement seems debatable: " Since all the contestants implement the same functionality, the main difference between their samples is their coding style. " All the (winning) contestants implement the same functionality, yes, but with possibly wildly different approaches/algorithms, so the main difference between code samples is not just "style" but what could be called "general thinking in and around the problem…

Why don't you consider different approaches and algorithms a part of definition of style?

It doesn't seem to be the primary meaning of the word; style is considered superficial, a coating over substance.

Of course that's a matter of interpretation, but I find the summary of the paper suffers from the ambiguity.

There is also the fact, that I alluded to, that winning entries in a programming contest should be more alike than non-winning entries, in that they all actually solve the problem; if non-winning entries were included, it could explain some of the discrepancy in identification success.

Re: When coding style survives compilation: De-anonymizing programmers from binaries

#69

Post by EGreg »

GFK_of_xmaspast wrote: post #49
EGreg wrote: post #4

Well, this puts a damper on my plan to create a secret identity on the internet under which I release software the way Banksy releases art. Unless I stop publicly writing software for a few years.

> I release software the way Banksy releases art. It's pretty easy to vandalize a wiki tho.

Wha

Re: When coding style survives compilation: De-anonymizing programmers from binaries

#70

Post by anonaoeu »

runn1ng wrote: post #39
kyrre wrote: post #23

Earlier quoted context omitted.

you can use another language

One of the points of the talk/article is that style survives compilation, obfuscation and optimization. I think (but the talk did not support that directly) that it would survive another language, at least to a certain degree.

I doubt that their system would be able to recognize programmers if their training samples in C and their test samples are in Haskell.
Post reply on HN