11–20 of 58 posts
Which is what syslog can't do.
with reference to the name: "and I do live in South Berkeley / North Oakland…" I had a feeling. We may hope your code is hella tight...
Earlier quoted context omitted.
Interesting so its more of a ease of use then a performance issue ? The numbers you quotes for performance seemed impressive.
We needed performance as well as simplicity. We started by extending logstash, but our needs were more "we need a router" and logstash isn't meant to be a router. Statically linking the world isn't trivial. For our existing Python code bases - how are you going to deal with third party libraries from PyPI? Come by on #heka on irc.mozilla.org, we're kicking around in there.
Depends on what you want?
You could freeze the pip-requires to always install the same version and use a virtualenv per application. This is basically the same as bundling everything together, it has all the benefits with the least amount of work.
You could use distribution packages for security, correctness and stability or even roll your own repository inside your infrastructure to absolutely control everything.
Finally you could just bundle everything manually by fooling around with the PYTHONPATH and putting all the dependencies in a single directory. This is kind of like improvising your own virtualenv, it's very hacky, but it can work.
with reference to the name: "and I do live in South Berkeley / North Oakland…" I had a feeling. We may hope your code is hella tight...
I get it, no jokes on HN. This was a quote from the developer, who has, I suppose, a better sense of humor than y'all.
Off the top of my head this is a reimplementation of the following * SNMP * CollectD * Carbon * JMX * WMI * CMIP And a whole host of other proprietary transports. So its cool and looks awesome, but what does it give me that the entirety of other monitoring protocols doesn't
There's a bunch of things going on on your boxes (logs, jmx, syslog, etc), and you want to get them out in a useful unified format. You have to do some ugly things (e.g. parse rails logs for latencies), and then emit the data, preferably in some structured format that knows that render=17ms is a duration so that you can graph it.
They chose their own transport to speak between heka nodes, because it maps perfectly to their internal representation, but it looks like they are willing to speak any of those protocols you mentioned to the outside world. It's useful to do a limited amount of munging inside the hekasystem before sending the data to logstash, graphite, etc, so it looks like they spent quite a bit of time building a framework for that initial work, so you can move it as close to the edges as you'd like.
To me, the transport and/or protocol isn't interesting, it's that you have a flexible, lightweight agent that's also capable of doing pre-processing and rollups.
Seems similar to Riemann: http://riemann.io/
If you're not familiar with collectd's capbilities, you can get a quick overview of the official plugins at http://git.verplant.org/?p=collectd.git;a=blob;hb=master;f=R...
Earlier quoted context omitted.
One of the driving motivations was simplicity for developers and get a reasonable out-of-the-box experience. This comes from a couple things. Go compiles to a single static library so you don't have to worry about having dozens of "the right" library installed on your machine. Grab the heka binary and run with it. This greatly eases our operations work as we have fewer dependency conflicts to deal with when we push t…
That doesn't make a lot of sense. You don't have to write a monitoring software from scratch just because you want statically compiled bundled libraries. You can do that with any programming language.
Earlier quoted context omitted.
We needed performance as well as simplicity. We started by extending logstash, but our needs were more "we need a router" and logstash isn't meant to be a router. Statically linking the world isn't trivial. For our existing Python code bases - how are you going to deal with third party libraries from PyPI? Come by on #heka on irc.mozilla.org, we're kicking around in there.
> Statically linking the world isn't trivial. For our existing Python code bases - how are you going to deal with third party libraries from PyPI? Depends on what you want? You could freeze the pip-requires to always install the same version and use a virtualenv per application. This is basically the same as bundling everything together, it has all the benefits with the least amount of work. You could use distributio…