Ask HN: Why are there no online databases of whitelisted ubuntu binaries?

#1

Post by HD142245b » Mon, Oct 17, 2016, 10:10 PM UTC

I'm looking for a powerful tool to scan for illegitimate ubuntu binaries. The existing tools (tripwire, rkhunter, chkrootkit) seem to require you to maintain your own brittle whitelist of acceptable binaries. Why is there no online database to do this?

My definition of a whitelist is: the set of SHA256 hashes of every binary found in every release of ubuntu.

This is a trivially monetizable idea as well - you can keep the whitelist on your servers and make the users upload their live binary SHA256 hashes to the server before checking them, so that your whitelist remains a commercial asset (similar to the VirusTotal model).

I'd be willing to pay good money for this. Can someone please build this?

Re: Ask HN: Why are there no online databases of whitelisted ubuntu binaries?

#2

Post by stevekemp » Tue, Oct 18, 2016, 1:34 AM UTC

If you look beneath /var/lib/dpkg/info/ you'll find a set of checksums for binaries in, for example, `zip.md5sums`.

It wouldn't take much effort to extract all the packages from a release-archive and make that searchable.

I once processed the Debian archive in such a manner, but the problem is security updates will release every now and again and cause you to need to update things. Not a problem to store a set of hashes for binaries, with a floating "most recent" value.

But given that the information is already available to you I'd suspect not many people would pay for the service, or use it.

Re: Ask HN: Why are there no online databases of whitelisted ubuntu binaries?

#3

Post by HD142245b » Tue, Oct 18, 2016, 2:01 AM UTC

stevekemp wrote: post #2

If you look beneath /var/lib/dpkg/info/ you'll find a set of checksums for binaries in, for example, `zip.md5sums`. It wouldn't take much effort to extract all the packages from a release-archive and make that searchable. I once processed the Debian archive in such a manner, but the problem is security updates will release every now and again and cause you to need to update things. Not a problem to store a set of has…

This goes back to the problem of maintaining your own whitelist, though. The problem is that maintaining your own whitelist is a lot of work. For example, if you update your system, but don't update your rkhunter database, your next report is totally inaccurate.

I just want a solution where I can boot up any system from a live CD, and inspect an existing installation, based on an external verified database.

Re: Ask HN: Why are there no online databases of whitelisted ubuntu binaries?

#4

Post by stevekemp » Tue, Oct 18, 2016, 5:49 AM UTC

HD142245b wrote: post #3

stevekemp wrote: post #2

If you look beneath /var/lib/dpkg/info/ you'll find a set of checksums for binaries in, for example, `zip.md5sums`. It wouldn't take much effort to extract all the packages from a release-archive and make that searchable. I once processed the Debian archive in such a manner, but the problem is security updates will release every now and again and cause you to need to update things. Not a problem to store a set of has…

This goes back to the problem of maintaining your own whitelist, though. The problem is that maintaining your own whitelist is a lot of work. For example, if you update your system, but don't update your rkhunter database, your next report is totally inaccurate. I just want a solution where I can boot up any system from a live CD, and inspect an existing installation, based on an external verified database.

Even if you paid for access to a list of known-good binaries you would still need to have your own local list.

e.g. Malicious exploit drops in /etc/cron.d/botnet.

You might also feel safe if you have "good" copy of bash, but your version matched a checksum from pre-shellshock.

Really making the list is easy, you could do it yourself, or you could pay somebody to give you one, but it doesn't solve the problem because you'll still have local changes you need to flag/whitelist/update.

Re: Ask HN: Why are there no online databases of whitelisted ubuntu binaries?

#5

Post by HD142245b » Tue, Oct 18, 2016, 4:15 PM UTC

stevekemp wrote: post #4

HD142245b wrote: post #3

Earlier quoted context omitted.

This goes back to the problem of maintaining your own whitelist, though. The problem is that maintaining your own whitelist is a lot of work. For example, if you update your system, but don't update your rkhunter database, your next report is totally inaccurate. I just want a solution where I can boot up any system from a live CD, and inspect an existing installation, based on an external verified database.

Even if you paid for access to a list of known-good binaries you would still need to have your own local list. e.g. Malicious exploit drops in /etc/cron.d/botnet. You might also feel safe if you have "good" copy of bash, but your version matched a checksum from pre-shellshock. Really making the list is easy, you could do it yourself, or you could pay somebody to give you one, but it doesn't solve the problem because…

Well, I'm not denying that config-file trojans exist, but whitelisting all executables massively reduces the haystack.

Thx for the tips.