Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

systemd's syslogd replacement supports JSON output

freedesktop.org

Post reply on HN
Print view View original

21–30 of 54 posts

Re: systemd's syslogd replacement supports JSON output

#21

Post by mordae »

duck_typer wrote: post #3

Is this a good time to define Poettering's Law - any simple and useful Unix service will eventually get rewritten into a monstrous uber service that requires a supercomputer just to run, what should be, a simple service.

No, what happened was init scripts. Something completely alien to the original init tab concept. It was an extremely ugly hack that worked only because of extensive documentation and effort by much larger amount of people than systemd has now. Both systemd and journal are an actual solution. Try journalctl on your system, it actually makes sense! The silly thing even has bash completion of filtering criteria! And they are both lightning fast.

Re: systemd's syslogd replacement supports JSON output

#22

Post by dman »

DASD wrote: post #13
FooBarWidget wrote: post #12

Earlier quoted context omitted.

And how exactly does Xorg allow you to use parts of it? More importantly, what parts of systemd do you not want to use or do you want to replace, and why? EDIT: what you're saying appears to be untrue. According to http://0pointer.de/blog/projects/the-biggest-myths.html , they provide configure switches for enabling or disabling many things, not unlike how the Linux kernel allows you to enable or disable features. Yo…

I'd like to not use udev. I have a custom Linux that runs both on standardized(internally defined) servers and also virtualized(known hardware) servers. As such, I'm well under 900ms boot times that systemd mentions by using SysV init. But the systemd project does not accept patches ( http://freedesktop.org/wiki/Software/systemd/MinimalBuilds/ ) to disable what is considered a core component.

Could you elaborate a bit more on your hardware setup? How are your hardware devices getting initialized that quick?

Re: systemd's syslogd replacement supports JSON output

#23

Post by spc476 »

FooBarWidget wrote: post #8
duck_typer wrote: post #3

Is this a good time to define Poettering's Law - any simple and useful Unix service will eventually get rewritten into a monstrous uber service that requires a supercomputer just to run, what should be, a simple service.

Which is easy for you to state, were it not for the fact that the "simple service" is broken in various ways. Let's start with the implementation-specific brokenness. I configured a log host with rsyslog. Rsyslog's configuration format, as well as documentation, is completely abysmal. And once in a while, rsyslog would use 100% CPU for no reason. A quick strace reveals that it keeps reading or writing to a file descr…

systemd can log the PID, UID and GID of a message because its using a Linux-specific feature of datagram based local sockets; such a feature cannot be supported on other Unix systems. But there are other reasons to use syslogd/rsyslogd/syslog-ng besides logging to local files, and that's to forward the logs to a centralized logging service. Of course, there are issues with that, as the default syslog protocol also allows any process, anywhere, to claim it is any other process (and if root, even fake its own source address) but it's being addressed (I know rsyslogd is, I'm not sure about other syslogd replacements).

My issue with systemd is that forwarding logs to a central server is an afterthought, if even that. And the problem with that is that it won't be as secure or work as well if it's part of the design. And another issue with the "afterthought support for remote logging" is that I still need to run syslogd/rsyslogd/syslog-ng because I have a ton of other network equipment (routers) that can forward their logs via the syslog protocol (which I have used on networks I've managed---by doing that, I've been able to receive notifications of OSPF routing changes, for instance). And if I have syslogd/rsyslogd/syslog-ng already running, what are the benefits of systemd logging?

Re: systemd's syslogd replacement supports JSON output

#24

Post by zanny »

FooBarWidget wrote: post #17
Tobu wrote: post #14

Earlier quoted context omitted.

I can't disable the journal, for example. If I run something more full-featured, I'll still be streaming things through the journal. It's not at all like the kernel does it. With Linux, there are multiple competing implementations of most features, pushed by people with competing interests. There is a lot more room for debate and experimentation.

According to 'man journald.conf', you can set "Storage=none". If by "monolithic" you mean the lack of multiple competing implementations, then lots and lots of things are monolithic. Apache and Nginx would be monolithic by your definition.

I love systemd, but from a lot of experience with it over 3 years now, that doesn't disable the journal daemon, it only prevents output piping to its binary logs. You can still poll the journal daemon for events and output it elsewhere (ex, syslog).

Re: systemd's syslogd replacement supports JSON output

#25

Post by zanny »

Tobu wrote: post #9
FooBarWidget wrote: post #6

Earlier quoted context omitted.

A quick look into systemd reveals that it's comprised of multiple components. For example the journal service is a separate daemon. What part of this does not align with your vision? EDIT: looks like the "monolithic" myth has already been busted: http://0pointer.de/blog/projects/the-biggest-myths.html I would say the distinction between systemd and older systems is that systemd follows the "better is better" mantra o…

It's monolithic because you can't pick and choose the components. You have to use all of systemd, or none of it. You can't replace bits of it, or use just parts of it.

Ubuntu just started using logind in 13.04, but haven't adopted the rest of systemd. Some parts (journal, systemd core, udev) can't be detached, but a lot of the peripheral software outside the init -> maintain services -> log events process are optional.

Re: systemd's syslogd replacement supports JSON output

#26

Post by zanny »

mordae wrote: post #21
duck_typer wrote: post #3

Is this a good time to define Poettering's Law - any simple and useful Unix service will eventually get rewritten into a monstrous uber service that requires a supercomputer just to run, what should be, a simple service.

No, what happened was init scripts. Something completely alien to the original init tab concept. It was an extremely ugly hack that worked only because of extensive documentation and effort by much larger amount of people than systemd has now. Both systemd and journal are an actual solution. Try journalctl on your system, it actually makes sense! The silly thing even has bash completion of filtering criteria! And the…

I do want to mention binary logs piss me off, but I just set up journald to output to syslog and everything is fine. I just ignore journalctl. I personally find it to be quite slow, but I can just set it to not log its binary blobs.

Re: systemd's syslogd replacement supports JSON output

#27

Post by FooBarWidget »

zanny wrote: post #24
FooBarWidget wrote: post #17

Earlier quoted context omitted.

According to 'man journald.conf', you can set "Storage=none". If by "monolithic" you mean the lack of multiple competing implementations, then lots and lots of things are monolithic. Apache and Nginx would be monolithic by your definition.

I love systemd, but from a lot of experience with it over 3 years now, that doesn't disable the journal daemon, it only prevents output piping to its binary logs. You can still poll the journal daemon for events and output it elsewhere (ex, syslog).

Does journald impose a sufficient performance penalty that you want to remove it entirely?

Re: systemd's syslogd replacement supports JSON output

#28

Post by zanny »

duck_typer wrote: post #3

Is this a good time to define Poettering's Law - any simple and useful Unix service will eventually get rewritten into a monstrous uber service that requires a supercomputer just to run, what should be, a simple service.

I use systemd and find it to be significantly faster boot-wise than Upstart and the sysv scripts. One doesn't do socket initialization and the other is a bunch of slow ass shell scripts. It doesn't matter if systemd is beefy if the actual task of starting services is lightning fast.

Just for comparison, last year I was triple booting Debian Testing, Ubuntu 12.04, and Arch. I use KDE on all 3 (I do active development on KDE in the Arch one) and logged my boot times over 6 months. I misplaced those records but the effective boot times were around 15 seconds for Debian, 25 for Ubuntu, and 5 for Arch. Mainly because tasks in Ubuntu like NetworkManager or tmpfs mounting were taking 5 - 10 seconds and nothing else was running.

Re: systemd's syslogd replacement supports JSON output

#29

Post by asveikau »

FooBarWidget wrote: post #8
duck_typer wrote: post #3

Is this a good time to define Poettering's Law - any simple and useful Unix service will eventually get rewritten into a monstrous uber service that requires a supercomputer just to run, what should be, a simple service.

Which is easy for you to state, were it not for the fact that the "simple service" is broken in various ways. Let's start with the implementation-specific brokenness. I configured a log host with rsyslog. Rsyslog's configuration format, as well as documentation, is completely abysmal. And once in a while, rsyslog would use 100% CPU for no reason. A quick strace reveals that it keeps reading or writing to a file descr…

Reading this I thought: how does one authenticate a pid or uid in a daemon? I am assuming Unix domain sockets, which don't provide you with the caller's pid or uid.

Doing some googling I found there are indeed nonportable interfaces for this. First I found getpeerucred which is a Sun thing. Then I found Linux has a SO_PEERCRED socket option. [Edit: and there seems to exist getpeereid and LOCAL_CRED on some BSDs]

Still seems like a weird or otherwise shaky thing to me. It feels like it breaks the abstraction of a socket.

Re: systemd's syslogd replacement supports JSON output

#30

Post by FooBarWidget »

spc476 wrote: post #23
FooBarWidget wrote: post #8

Earlier quoted context omitted.

Which is easy for you to state, were it not for the fact that the "simple service" is broken in various ways. Let's start with the implementation-specific brokenness. I configured a log host with rsyslog. Rsyslog's configuration format, as well as documentation, is completely abysmal. And once in a while, rsyslog would use 100% CPU for no reason. A quick strace reveals that it keeps reading or writing to a file descr…

systemd can log the PID, UID and GID of a message because its using a Linux-specific feature of datagram based local sockets; such a feature cannot be supported on other Unix systems. But there are other reasons to use syslogd/rsyslogd/syslog-ng besides logging to local files, and that's to forward the logs to a centralized logging service. Of course, there are issues with that, as the default syslog protocol also al…

Regarding PID/UID/GID logging: if it has to use a Linux-specific feature to be less broken in that regard, then I'm all for it. In my opinion, authentication is such an important security feature that other Unices should just implement similar mechanisms (or, preferably, the same mechanism). Right now I've only seen the BSD people complaining that systemd uses Linux-specific features, but doing nothing to solve the problem.

Regarding central logging: indeed, that is a problem. Has Lennart ever spoken about this issue?

Post reply on HN