Live data from Hacker News

Entropy, a CLI that scans files to find high entropy lines (might be secrets)

github.com

51–60 of 141 posts

Re: Entropy, a CLI that scans files to find high entropy lines (might be secrets)

#52

I transcend this problem by making all my database passwords 'abcd'

Reminds me of https://xkcd.com/936/ I think "correct horse battery staple" has a low entropy, since it is just ordinary looking words (strings).

Re: Entropy, a CLI that scans files to find high entropy lines (might be secrets)

#53
post #51

I transcend this problem by making all my database passwords 'abcd'

The tool found "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz1234567890" in our codebase as a high entropy line :)

Well, it is...

Re: Entropy, a CLI that scans files to find high entropy lines (might be secrets)

#55
Interesting. If I had to do this, I would have done something like

    perl -lne 'next unless $_; $z = qx(echo "$_" | gzip | wc -c); printf "%5.2f    %s\n", $z/length($_), $_'
on the principle that high entropy means it compresses badly. However, that uses each line as the dictionary, rather than the entire file, so it has a little trouble with very short lines which compress badly.

It did react to this line

    return map { $_ > 1 ? 1 : ($_ 
which is valid code but indeed seems kind of high in entropy. I was also able to fool it to not detect a high-entropy line by adding a comment of natural English to it.

I'm on the go but it would be interesting to see comparisons between the Perl command and this tool. The benefit of the Perl command is that it would run out of the box on any non-Windows machine so it might not need to be as powerful to gain adoption.

Re: Entropy, a CLI that scans files to find high entropy lines (might be secrets)

#56
post #30

Earlier quoted context omitted.

People have been buying ineffective ads since the invention of ads

zero clicks is a little different

Bots do click in real ad fraud, so your moved goalpost isn't all that solid

Re: Entropy, a CLI that scans files to find high entropy lines (might be secrets)

#58

Why would I need to install go to run this tool? I thought one advantage of go was that devs could just distribute a single binary file that works...

Because it's a security tool so trusting a binary upfront defeats the purpose. With source you at least have the option to inspect what it really does.

Re: Entropy, a CLI that scans files to find high entropy lines (might be secrets)

#59
post #30

Earlier quoted context omitted.

People have been buying ineffective ads since the invention of ads

Not really, advertising is really the only field of human endeavour that is both data-driven and results-oriented. (Doesn't still stop smart people from committing fraud, but that is a different story.)

Unfortunately I beg to differ. I worked for several companies where we the management clearly saw that the results were very poor (for Facebook ads, for example) but continued to invest because there is a defined budget for it and so on. It was like this last year and 20 years ago.

Re: Entropy, a CLI that scans files to find high entropy lines (might be secrets)

#60

Why would I need to install go to run this tool? I thought one advantage of go was that devs could just distribute a single binary file that works...

Because it's a security tool so trusting a binary upfront defeats the purpose. With source you at least have the option to inspect what it really does.

does the stated purpose of the tool influence whether or not you can trust it?
Post reply on HN