Live data from Hacker News

Central Logging with Open Source Software

divisionbyzero.net

1–10 of 51 posts

Central Logging with Open Source Software

#1
I'm attempting to implement a Splunk-like setup with open source components. This blog entry is the first in many to be a brain dump of how I'm using this setup, what I get from it, and why I've arrived at each of these components.

Central Logging with Open Source Software
divisionbyzero.net

Re: Central Logging with Open Source Software

#2
This reminds me of something I've been wondering about since the Bitcoinica heist: how do people usually set up secure offline backups which can't be erased using the credentials on the backed-up server? I would probably do something with ssh authorized_keys if I had to make it from scratch, but are there obscure security/reliability risks, and tools which have already mitigated these risks for you?

Re: Central Logging with Open Source Software

#3
post #2

This reminds me of something I've been wondering about since the Bitcoinica heist: how do people usually set up secure offline backups which can't be erased using the credentials on the backed-up server? I would probably do something with ssh authorized_keys if I had to make it from scratch, but are there obscure security/reliability risks, and tools which have already mitigated these risks for you?

The standard way used to be to use write-only media. For example, if you log to a server which writes the logs incrementally to a DVD writer, you can be fairly certain that the logs won't be erased...

Re: Central Logging with Open Source Software

#4
post #3
post #2

This reminds me of something I've been wondering about since the Bitcoinica heist: how do people usually set up secure offline backups which can't be erased using the credentials on the backed-up server? I would probably do something with ssh authorized_keys if I had to make it from scratch, but are there obscure security/reliability risks, and tools which have already mitigated these risks for you?

The standard way used to be to use write-only media. For example, if you log to a server which writes the logs incrementally to a DVD writer, you can be fairly certain that the logs won't be erased...

Not softwarily, anyway.

Re: Central Logging with Open Source Software

#5
post #2

This reminds me of something I've been wondering about since the Bitcoinica heist: how do people usually set up secure offline backups which can't be erased using the credentials on the backed-up server? I would probably do something with ssh authorized_keys if I had to make it from scratch, but are there obscure security/reliability risks, and tools which have already mitigated these risks for you?

One of the follow-up posts to this is going to be on using OSSEC-HIDS which will give you logfile chained checksums. It's not perfect, but again it's about achieving the most value for the least amount of effort.

Re: Central Logging with Open Source Software

#6
post #2

This reminds me of something I've been wondering about since the Bitcoinica heist: how do people usually set up secure offline backups which can't be erased using the credentials on the backed-up server? I would probably do something with ssh authorized_keys if I had to make it from scratch, but are there obscure security/reliability risks, and tools which have already mitigated these risks for you?

You pull rather than push.

The webserver has no credentials for accessing the backup server. Instead the backup server accesses the webserver.

This strategy places higher trust on the backup server, but the backup server is easier to defend -- it only needs connectivity to a small number of other IPs.

Re: Central Logging with Open Source Software

#7
post #2

This reminds me of something I've been wondering about since the Bitcoinica heist: how do people usually set up secure offline backups which can't be erased using the credentials on the backed-up server? I would probably do something with ssh authorized_keys if I had to make it from scratch, but are there obscure security/reliability risks, and tools which have already mitigated these risks for you?

My quick and dirty way to it is to run a cron on the backup server that chowns incoming files to another user (with a few refinements, like preventing exec, etc). But I'd definitely like something more solid.

Re: Central Logging with Open Source Software

#8
post #7
post #2

This reminds me of something I've been wondering about since the Bitcoinica heist: how do people usually set up secure offline backups which can't be erased using the credentials on the backed-up server? I would probably do something with ssh authorized_keys if I had to make it from scratch, but are there obscure security/reliability risks, and tools which have already mitigated these risks for you?

My quick and dirty way to it is to run a cron on the backup server that chowns incoming files to another user (with a few refinements, like preventing exec, etc). But I'd definitely like something more solid.

You should look into a tool that stores meta information on the backup files, such as rdiff-backup. Manually restoring ownership/permissions from a backup is probably tiring.

Re: Central Logging with Open Source Software

#10
post #8
post #7

Earlier quoted context omitted.

My quick and dirty way to it is to run a cron on the backup server that chowns incoming files to another user (with a few refinements, like preventing exec, etc). But I'd definitely like something more solid.

You should look into a tool that stores meta information on the backup files, such as rdiff-backup. Manually restoring ownership/permissions from a backup is probably tiring.

I backup (and chown) archives, not directly the files, so restoring the permissions isn't much of an issue. Sorry, was unclear :s
Post reply on HN