Holy balls, talk about going out of your way to avoid syslog.
Another Redis case: Centralized logging
11–20 of 47 posts
Re: Another Redis case: Centralized logging
#12Holy balls, talk about going out of your way to avoid syslog.
Syslog is a pile of shit, Ted. It's a relic. You clearly happen to love that relic, and I think you should find a way to place it just-so in a nicely lit alcove in your apartment. The rest of us should move on from it. I don't think less of you for admiring it. I have useless old things on display in my house too.
* Freeform text is a terrible way to track system events.
* Periodically rotated flat files are not a great way to store log information.
* Goofy little UDP messages are not a good way to convey system events
* The syslog PRI field dates back to when we exchanged messages with UUCP.
I could keep going, but since you're just going to reply with "lolwut umad?", I'll leave it at that.
Re: Another Redis case: Centralized logging
#13Depends on how much you care about latency, right? The easy solution is just, y'know, write the log to a file and scp it back to some central place every so often. But then you have to either (a) keep track of how much of a file you've copied, which is a pain; or (b) only grab files that you're no longer actively writing to (as determined by naming scheme or something), but that introduces some latency, depending on…
I prefer sending the logs to some other computer over UDP.
Why do you want to send logs over UDP?
I get that you were making a snarky allusion to syslog, but what part of the syslog UDP protocol do you feel beats Redis' TCP protocol?
Re: Another Redis case: Centralized logging
#14My only question: how is this better than syslog?
It's queryable.
It's trivially capped and rolled.
It's centralized.
It provides a common logging interface across platforms.
It's extremely simple (the script that pipes syslog output directly into Redis is probably just a couple lines long).
I have to believe that the people who really seem to like syslog have never worked in organizations that had to deploy things like Splunk or (worse) LogLogic and ArcSight just to make sense of the giant morass of useless text gunk they generate.
Have you noticed how none of the cool kids postprocess http log files anymore?
Re: Another Redis case: Centralized logging
#15Holy balls, talk about going out of your way to avoid syslog.
Syslog is a pile of shit, Ted. It's a relic. You clearly happen to love that relic, and I think you should find a way to place it just-so in a nicely lit alcove in your apartment. The rest of us should move on from it. I don't think less of you for admiring it. I have useless old things on display in my house too. * Freeform text is a terrible way to track system events. * Periodically rotated flat files are not a gr…
We use syslog where I work, and I've always felt the same way, but never heard any suggestions for better options with as wide adoption, support, and background as syslog.
Out of pure curiosity, what do you see as the tool most likely to displace syslog in the future? Is there any alternative available that fixes most of these problems without rolling your own from pieces and parts?
Re: Another Redis case: Centralized logging
#16Earlier quoted context omitted.
I prefer sending the logs to some other computer over UDP.
Why do you want to send logs over UDP? I get that you were making a snarky allusion to syslog, but what part of the syslog UDP protocol do you feel beats Redis' TCP protocol?
I don't, really (you can use syslog pretty successfully, I guess, if you don't mind it), but UDP has certain advantages over TCP, namely that your code keeps running even if the server you're sending to goes down or is unable to respond to messages, it's faster, etc.
Ideally, I'd use a function that sent things to a small server over UDP, which would then put them in Redis. This assumes you don't mind losing a few lines, of course.
Re: Another Redis case: Centralized logging
#17Holy balls, talk about going out of your way to avoid syslog.
Syslog is a pile of shit, Ted. It's a relic. You clearly happen to love that relic, and I think you should find a way to place it just-so in a nicely lit alcove in your apartment. The rest of us should move on from it. I don't think less of you for admiring it. I have useless old things on display in my house too. * Freeform text is a terrible way to track system events. * Periodically rotated flat files are not a gr…
OK Thomas. First of all: u mad bro?
Second of all: maybe on the day when you build something substantive, useful, and valuable, you can lecture me. Until then, take a knee and lend your ear.
You know what else is a "relic of the past"? The internal combustion engine. It's wildly inefficient, but somehow, people keep on using it because it solves the problem. Maybe not optimally, but satisfactory.
More to the point:
* freeform text is an _excellent_ way to track system events. Know why? It's human readable. Anything more is a micro-optimization that is perpetrated by people who have never had to debug a serious production system.
* Periodic rotation is fine, if you are storing a lot of "chatter" type logs. If you have critical events that need more careful tracking, then wowie zowie, syslog can do that too. It's almost like those Unix guys knew what they were doing.
* Modern syslog implementations can use TCP, if that's your thing. Dumb if you ask me, but whatever.
* So? What problem does that create?
If you want to chest-thump about technical masturbation, then you're in the right place. Hacker News Wantrepreneurs love that kind of stuff. However, the rest of us who have work to do will use the tools available that solve the problem. Nothing is perfect, but I've got better ways to spend my time than worry about making my own centralized logging system.
I ain't even mad.
Re: Another Redis case: Centralized logging
#18Take a look at logstash [1], it's on GitHub [2]. I think it could replace or integrate with RedisLogHandler.
logstash takes logs from various inputs (syslog, files, Redis, HTTP), filters/normalized the formats into JSON, and outputs various formats (ElasticSearch, Redis, MongoDB, Graylog2). There is a WebUI with search and graphs. It's designed to scale-out and run on multiple machines.
[1] http://logstash.net/ [2] https://github.com/logstash/logstash
Re: Another Redis case: Centralized logging
#19Holy balls, talk about going out of your way to avoid syslog.
Syslog is a pile of shit, Ted. It's a relic. You clearly happen to love that relic, and I think you should find a way to place it just-so in a nicely lit alcove in your apartment. The rest of us should move on from it. I don't think less of you for admiring it. I have useless old things on display in my house too. * Freeform text is a terrible way to track system events. * Periodically rotated flat files are not a gr…
There's syslog the protocol, syslog the API, and syslog the daemon. There are syslog daemon and protocol replacements that are much better, but retain the API for wide-spread compatibility.
Re: Another Redis case: Centralized logging
#20My only question: how is this better than syslog?
I agree, it is nothing different than syslog if all you want to do is just dump all logs in one file. But one advantage of this approach in our case was that we wanted to show last 1000 critical logs on a web interface and with logs stored in redis, it was pretty easy to do that. And as redis was part of our stack, it was very easy to hook it up for this specific task.
Ahem.
tail -n 1000