Security Incident on FreeBSD Infrastructure

Could you please elaborate on that? How do you do "trusted" live forensics on systems with possibly infected kernels and stuff? Assuming these servers were normal COTS and nothing fancy (thinking of CPU-bypassing memory access...)

Lacking cryptographic hashes used by most DVCS software, SVN repositories need some external mechanism to be pretty sure the source repository hasn't been compromised. For instance, one might have snapshots of the SVN tree taken every week.

It's much easier with git or hg, even in the single-master-repo model, even if all developers are using the CVS gateway so they don't have local copies of commit history. All such a project would need in order to protect against compromise (ignoring physical redundancy) is a backup git or hg repo, kept offline except for periodic pulls. As long as there are no warnings about upstream rebases, and you trust all visible commits, then everything's fine[1].

With SVN, as long as there's no built-in cryptographically secure way to connect one snapshot to the next when backing up SVN repos, determining the existence of a backdoor requires comparing entire backup SVN repos against the live main repo. Even if sufficient backups exist, the process is slow and requires taking the main SVN repo offline if there's any chance of a root compromise.

[1] Caveat 1: in any RCS, there could be backdoors masquerading as legitimate-looking commits if the attacker had commit access. Caveat 2: The security is subject to the cryptographic security of SHA-1.

In this FreeBSD case, if they're certain the svn repo wasn't compromised, all they have to do is validate the checkout on the two compromised machines against the svn repo.

Take note:

"We unfortunately cannot guarantee the integrity of any packages available for installation between 19th September 2012 and 11th November 2012, or of any ports compiled from trees obtained via any means other than through svn.freebsd.org or one of its mirrors. Although we have no evidence to suggest any tampering took place and believe such interference is unlikely, we have to recommend you consider reinstalling any machine from scratch, using trusted sources."

Scary stuff.

Please use passwords for your keys and allow key access only to a small set of known IP addresses.

Also do share other security techniques you're using besides the ones above.

Firewire is awesome for the attacker, unfortunately few servers have it, especially not externally exposed ports. Also, smart OSes use some of the newer Intel features (VT-d) to lock down DMA while the OS is running, which usually protects from rogue firewire, and can theoretically help against rogue PCIe, although usually badly implemented in chipset and OS.

Another option is a reboot onto a custom OS which is designed specifically to preserve memory (you get a safe few seconds of holdover). LiveKd is pretty cool (sysinternals)

There are PCIe cards which do processor/network and let you explore main memory -- WindowsSCOPE CaptureGUARD for PCIe or ExpressCard. Probably enough time to pop the case open and throw one in before memory degrades.

Countermeasures are numerous -- everything from doing memory encryption inside the CPU die (putting code in the cache, like TRESOR) and doing hypervisor tricks ("TresorVisor") (http://www1.informatik.uni-erlangen.de/tresor) to using Hardware Security Modules (like the SafeNet or Thales nCipher) to just keeping your servers physically secured from intruders who might memory-analyze them (although a software bootloader and remote-reboot could still be applied).

Forensics as a field seems to be a lot more interested in attacking mobile phones (which is one of the things I'm talking about at RSA 2013), but desktops and servers are still interesting targets.