Live data from Hacker News

Systemd, ten years later: a historical and technical retrospective

blog.darknedgy.net

391–400 of 458 posts

Re: Systemd, ten years later: a historical and technical retrospective

#391
post #376

Earlier quoted context omitted.

You have to, since systemd implemented the feature incorrectly. You're missing the forest for the trees.

I've already been in a forest of bash scripts and I would not go back there again. I have no comment on systemd's implementation but the implementation you're talking about is also incorrect. It has never been safe to kill random processes using a bash script running in the background, on most Unixes (and Linux) it is 100% impossible to do that without race conditions due to the limitations of procfs. Doing "ps | gre…

>I've already been in a forest of bash scripts and I would not go back there again. I have no comment on systemd's implementation but the implementation you're talking about is also incorrect.

In certain cases I don't disagree, but systemd does not implement this feature correctly, so using functionality that's easily reviewable from decades past makes sense. If systemd could properly implement the feature, there would be no need for the scripting.

>It has never been safe to kill random processes using a bash script running in the background, on most Unixes (and Linux) it is 100% impossible to do that without race conditions due to the limitations of procfs.

This is hilariously wrong. Lots of UNIX OSes don't even implement the ps suite of tools by using procfs.

Even then, there were no race conditions in this use case anyway.

>You would need to implement this in C for it to have a chance of being safe at all, and even then, you would need to rely on system-specific functionality because there is no portable way in POSIX to actually do this.

It's already implemented this way on plenty of UNIXen. ps and kill is the POSIX portable way to approach this, so you're wrong about that as well.

Almost everything you said above is incorrect, or misunderstanding basic UNIX.

Re: Systemd, ten years later: a historical and technical retrospective

#392
post #371

Earlier quoted context omitted.

Can you reference that poll? There was a GR, which is entirely different than a poll. The outcome of that was also pretty clear, Debian as a whole is ok to depend a bit more on systemd. I don't get why you use it as an example of the opposite. See https://lwn.net/Articles/808217/ for the outcome of the GR.

It is indeed this GR. I might be misinterpreting the result, but "Systemd but we support exploring alternatives" means, to me, that there is some unhappiness here. To begin with, I believe the GR would not have happened if there was no issue to be discussed?

The GR happened because it wasn't clear what maintainers should do regarding alternatives init systems. Were they required to support them? Should they just focus on Systemd? The result of the vote was maintainers are required to support Systemd, but should accept patches to support other units.

Re: Systemd, ten years later: a historical and technical retrospective

#393
post #351
post #234

Earlier quoted context omitted.

Not sure if joking (since, sysvinit is very old and was in need of replacement) so I'll answer in good faith. > You made a typo in init file, so that daemon does not start. The "start" command succeeds, but there is no output anywhere, and process does not start. How do you debug it in the sysvinit world? "oh my process failed; let's add `-x` to the invocation." > You made a typo in init file, so that daemon does not…

Puzzle level 1) ini file: you mean, "add -x to the invocation, get back a few screenful of text, wade through that looking for "export" lines and copy them to your terminal, find "start-stop-daemon" line, copy the part of the line and remove irrelevant parameters, and paste into your terminal as well? I would not call this "easy" or "solved". Possible, yes. Pleasant, no. Puzzle level 2) inherited env -- I think you m…

> you mean, "add -x to the invocation, get back a few screenful of text, wade through that looking for "export" lines and copy them to your terminal, find "start-stop-daemon" line, copy the part of the line and remove irrelevant parameters, and paste into your terminal as well?

I mean, yes? Granted I'm not a developer but printing out exactly what's happening line by line with all variables expanded and all logical conditions printed is awesome, it's something we try to reverse engineer into programs with tools like bpftrace and strace.

Conversely an opaque C application which fails has much less tooling in the way of debugging failures. Especially if your package maintainers have stripped symbols.

> Surprise! Neither dmesg or /var/log/messages or /var/log/dmesg usually contain startup problems! Sometimes the nice daemon might log stuff there, but it is no way universal or required.

Actually, that's the opposite. A "bad" daemon might not print anything to stdout when invoked (which goes to dmesg) but regardless, this whole comment just reeks of someone who hasn't actually dealt with systems before 2014.

> Are you running any servers? If so, you might want to connect to the monitor, reboot and see if any service has error messages. Or you can use console scrollback after the server is booted, but you might need to recompile your kernel to increase is size, it is pretty small. Or just install systemd, it has full boot logging fully integrated with primary logging. And even a special command which lists all daemons which failed to start.

Console scrollback is "nice to have", personally I'm very thankful that it does not use memory keeping things there because there is the odd regression when you configure something (like when you tell IPtables to log deny's: that also gets printed to console.)

re----gardless. I think we're talking passed each other. I had no real issue as a sysadmin with a fully fledged bash, because I can debug it. I get that there's issues with that and most people do not like it- I'm not advocating for it, just saying that what you presume to be "impossible" was somehow solved for decades, and quite competently. It may not be pleasant to you but the alternative is not pleasant for me.

The larger issue I have is that "the alternative" is embedded and entangled itself to everything- there can never be anything better produced.

With old init's, it was a small process with well defined functionality so replacing it was easy. (as evidenced by the many init's among unices.).

Anyway, I'm getting ahead of myself.

If you want to see what systemd could have been, I invite you to look at Solaris' SMF. Please ignore the XML-ness of it, it's "very old" in comparison to systemd. :)

Re: Systemd, ten years later: a historical and technical retrospective

#394
post #391

Earlier quoted context omitted.

I've already been in a forest of bash scripts and I would not go back there again. I have no comment on systemd's implementation but the implementation you're talking about is also incorrect. It has never been safe to kill random processes using a bash script running in the background, on most Unixes (and Linux) it is 100% impossible to do that without race conditions due to the limitations of procfs. Doing "ps | gre…

>I've already been in a forest of bash scripts and I would not go back there again. I have no comment on systemd's implementation but the implementation you're talking about is also incorrect. In certain cases I don't disagree, but systemd does not implement this feature correctly, so using functionality that's easily reviewable from decades past makes sense. If systemd could properly implement the feature, there wou…

If you really want systemd to add this then I'm sure they will look at your feature request or PR.

I would urge you to actually check with your system instead of blindly dismissing me as wrong just because your bash script happened to work without error. In my experience BSD-based Unixes get it right and don't use procfs for ps or pkill. They don't have the problem because they use special syscalls for this. But SysV-based systems have historically used procfs to implement ps. Linux also still does. Try unmounting proc and running ps or kill and see what happens. If you can't do it then your system suffers from the problem, which is that you can't safely send a signal to the process after reading it because there is no guarantee that the actual PID will persist in between calls to read() and kill(). POSIX says nothing about this because it doesn't specify procfs, or how pkill should actually be implemented. This is all fair game as far as compatibility is concerned.

There is also the other more obvious race condition in your bash script which can also be pre-empted in between the calls to ps and kill. This can happen on any Unix and is not some big mystery either. PID reuse has been a known problem for decades and Linux finally got a solution to it a couple years ago with pidfd_send_signal. There is also the matter of cgroups but I am not going to get into that because I doubt you will want to hear about it.

Re: Systemd, ten years later: a historical and technical retrospective

#395
post #379
post #372

Earlier quoted context omitted.

The functionality of implementing it yourself? But that’s such a useless definition because then all software “provides nothing” since it’s implementation proves that you could have written it yourself. logind is far more robust than any of the janky shell scripts I’ve seen over the years to accomplish this.

The functionality to gracefully end long running background processes has been a part of standard killall, pkill, etc. for quite some time. Even then, yes, the functionality of using ps and kill together has existed for 20+ years. Those tools are already implemented, providing the functionality for decades. In no universe is logind considered robust, and basic scripts from middling UNIX admins have provided this func…

See my comment below. These tools are fundamentally broken for this use case and have never provided the functionality needed. Bash scripts are not a process manager and it is incredibly wrong to try and make it work like one.

Re: Systemd, ten years later: a historical and technical retrospective

#396

Earlier quoted context omitted.

As a person for whom Linux and BSDs have been a hobby since 1996, I have no idea what you're talking about.

One small example that i've struggled with is logging. It used to be I could reuse my editor knowledge to search and read log files. Now every time I want to look at a log I have to re-read the man pages to figure out the proper incantation.

Learning new tricks gets harder when we get older. Is that the software’s fault?

Re: Systemd, ten years later: a historical and technical retrospective

#397
post #371

Earlier quoted context omitted.

Can you reference that poll? There was a GR, which is entirely different than a poll. The outcome of that was also pretty clear, Debian as a whole is ok to depend a bit more on systemd. I don't get why you use it as an example of the opposite. See https://lwn.net/Articles/808217/ for the outcome of the GR.

It is indeed this GR. I might be misinterpreting the result, but "Systemd but we support exploring alternatives" means, to me, that there is some unhappiness here. To begin with, I believe the GR would not have happened if there was no issue to be discussed?

The unhappiness was that the existing status quo made people afraid to actually use features of systemd, lest they invite flames from people about not supporting alternatives (which don't have those features). The GR changes the status quo to allow hard dependencies on systemd to actually use its features. The "but we support exploring alternatives" ensures that the GR does not preclude the possibility that people might create something better than systemd in the future.

Re: Systemd, ten years later: a historical and technical retrospective

#398
post #63

Earlier quoted context omitted.

To answer your question: remote login, start screen, ctl-A d (detach), log out, log back in, screen -r (resume) - there is no screen session because it was killed by systemd immediately after log out. (syslog entry: May 3 09:01:25 $HOSTNAME systemd[1]: session-6.scope: Killing process 3290 (screen) with signal SIGTERM. ) But that's hardly the point, the point is the toxic "you're doing it wrong" mentality that infest…

Based on one of the sibling comments, if you modify screen to add a runtime dependency on systemd, the problem will go away. I agree this is totally unacceptable, and that the inevitable tone deaf response to such complaints is also unacceptable.

Is it that weird that if you want to run a job in the background outside your session that ask your service manager to spawn the job?

I mean all of this is just a difference of opinion between, "I want a process supervisor to manage all my background jobs" and "I just want to double fork and throw processes at init."

Re: Systemd, ten years later: a historical and technical retrospective

#399
post #304
post #259

Earlier quoted context omitted.

I agree it would be nice to fix this in the kernel instead, but as the article said, it might not be possible due to compatibility constraints. The fact that the logind developers came to a different solution than you, after spending much more time thinking about it and actually implemented it, doesn't exactly imply logind developers are the ones with an attention deficit, or that logind is bad. ("CADT" apparently me…

>The fact that the logind developers came to a different solution than you, after spending much more time thinking about it and actually implemented it, doesn't exactly imply logind developers are the ones with an attention deficit, or that logind is bad. ("CADT" apparently means "Cascade of Attention-Deficit Teenagers") The logind developers went against the established wisdom of experience (that big rewrites are ge…

>We should be less "inclusive" of people who want to rewrite everything. They should feel like impostors. You can't produce good quality if you're not willing to call out bad quality; Linux succeeded (for a time) because Torvalds had high standards and was willing to maintain them.

I'm making a humble request, please do not bring this attitude in open source projects. Really I mean it. It's not helpful and it only makes people angry. You are also misinterpreting the behavior of Mr. Torvalds and confusing things. The kernel developers have actually been some of the most adamant about rewriting major parts of the kernel and breaking internal APIs over and over again (not syscalls) because it's known that the only way to thoroughly improve on the code is to aggressively iterate on it like this. This is actually a major strength of open source: anyone who wants to try to rewrite something can pick up the code and just do it. If it's bad then you throw it away and forget about. If it's good then you keep it. This is precisely how the "high standard" even gets maintained.

Re: Systemd, ten years later: a historical and technical retrospective

#400
post #302

Earlier quoted context omitted.

Fedora does not have this feature on: https://src.fedoraproject.org/rpms/systemd/blob/265d91aff516...

That's most interesting, given that both systemd and Fedora are Red Hat products. If there's one distro that would use the systemd defaults, I would expect it to be Fedora. Who exactly are these defaults for, then?

I've mainly seen it switched on for appliances and things that don't really have a legacy of this behavior.

It's not switched on in RHEL or Fedora. It's likely to never be so.

Post reply on HN