Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Entropy, a CLI that scans files to find high entropy lines (might be secrets)

github.com

Post reply on HN
Print view View original

61–70 of 141 posts

Re: Entropy, a CLI that scans files to find high entropy lines (might be secrets)

#61

Post by eru »

baq wrote: post #49
dumbo-octopus wrote: post #39

Earlier quoted context omitted.

4 diceware words is hardly a good password. It's ~51 bits of entropy, about the same as 8 random ascii symbols. It could be trivially cracked in less than an hour. Your average variable name assigned to the result of an object name with a method name called with a couple parameter names has much more entropy.

So you do random capital words, random punctuation and add a number somewhere and you’re at 60. Add more for whatever threat model you’re trying to be secure against. https://beta.xkpasswd.net/

The random punctuation sort-of defeats the point, doesn't it?

Otherwise, I agree.

Re: Entropy, a CLI that scans files to find high entropy lines (might be secrets)

#62

Post by spoonjim »

menacingly wrote: post #60
benterix wrote: post #58

Earlier quoted context omitted.

Because it's a security tool so trusting a binary upfront defeats the purpose. With source you at least have the option to inspect what it really does.

does the stated purpose of the tool influence whether or not you can trust it?

If you're trying to improve the security of your product by running random binaries from the Internet you're going to have a bad time

Re: Entropy, a CLI that scans files to find high entropy lines (might be secrets)

#63

Post by jazzyjackson »

hackinthebochs wrote: post #42
krick wrote: post #37

Is there any good posts about the use of entropy for tasks like that? I am wondering for quite some time of how do people actually use it and if it is any effective, but never actually got to investigating the problem myself. First of all, how to define "entropy" for text is a bit unclear in the first place. Here it's as simple as `-Sum(x log(x))` where x = countOccurences(char) / len(text). And that raises a lot of…

Entropy is a measure of complexity or disorder of a signal. The interesting part is that the disorder is with respect to the proper basis or dictionary. Something can look complex in one encoding but be low entropy in the right encoding. You need to know the right basis, or figure it out from the context, to accurately determine the entropy of a signal. A much stronger way of building a tool like the OP is to have a…

bookmarking to think about later... does this hold for representing numbers as one base compared to another?

Regarding a prime as having higher entropy / less structure than say a perfect square or highly divisible number

a prime is a prime in any base, but the number of divisors will differ in non-primes, if the number is divisible by the base then it may appear to have more structure (smaller function necessary to derive, kolmogorov style),

does prime factorization have anything to do with this? i can almost imagine choosing a large non-prime whose divisibity is only obvious with a particular base such that the base becomes the secret key - base of a number is basically specifying your dictionary, no?

Re: Entropy, a CLI that scans files to find high entropy lines (might be secrets)

#64

Post by jazzyjackson »

benterix wrote: post #59
otabdeveloper4 wrote: post #48

Earlier quoted context omitted.

Not really, advertising is really the only field of human endeavour that is both data-driven and results-oriented. (Doesn't still stop smart people from committing fraud, but that is a different story.)

Unfortunately I beg to differ. I worked for several companies where we the management clearly saw that the results were very poor (for Facebook ads, for example) but continued to invest because there is a defined budget for it and so on. It was like this last year and 20 years ago.

these companies should be outcompeted by firms that don't blow a million dollars a month paying out to click fraudsters but alas the market is not perfectly competitive

is it a cargo cult? it works for coca cola so maybe if we just spend a little more we'll see returns...

Re: Entropy, a CLI that scans files to find high entropy lines (might be secrets)

#65

Post by alias_neo »

menacingly wrote: post #60
benterix wrote: post #58

Earlier quoted context omitted.

Because it's a security tool so trusting a binary upfront defeats the purpose. With source you at least have the option to inspect what it really does.

does the stated purpose of the tool influence whether or not you can trust it?

I think that question is a little backwards.

Certain tools are more likely to be used by people working in spaces where they should/must be less trusting.

If there was a tool (there is) to scan my platform deployment against some NCSC/NSA guidance for platform security, and I wanted to use it, I'm likely operating in a space that should consider being cautious about running random tools I find on the internet.

Re: Entropy, a CLI that scans files to find high entropy lines (might be secrets)

#66

Post by saagarjha »

spoonjim wrote: post #62
menacingly wrote: post #60

Earlier quoted context omitted.

does the stated purpose of the tool influence whether or not you can trust it?

If you're trying to improve the security of your product by running random binaries from the Internet you're going to have a bad time

That's how most people run compilers

Re: Entropy, a CLI that scans files to find high entropy lines (might be secrets)

#67

Post by hamasho »

itemize wrote: post #28
hamasho wrote: post #27

I didn't know what entropy means in software, so here's the definition[0]: ---- Software entropy is a measure of the disorder or complexity of a software system. It is a natural tendency for software entropy to increase over time, as new features are added and the codebase becomes more complex. High entropy in software development means that the code is difficult to understand, maintain, and extend. It is often chara…

thanks for the search. this is textual entropy however, I am not sure if definition is applicable

Oh, thanks for pointing out.
Post reply on HN