Systemd, ten years later: a historical and technical retrospective • Hacker News Board

Re: Systemd, ten years later: a historical and technical retrospective

#171

Post by lmm » Mon, May 04, 2020, 6:43 AM UTC

sneak wrote: post #118

nine_k wrote: post #86

My beef with systemd is not its reinventing things. That part may actually be good. One problem is that a number of reinventions were poorly made. E.g. the log format. Yes, unstructured logs have a ton of drawbacks. Can we take some ridiculously well-tested, reliable embedded database or serialization format (like sqlite) and use it as log storage? Alas. Another problem is the "I know better" attitude. Are user proce…

It's astounding to me the issues people have with this. systemd's existence doesn't preclude anyone from using the old, pre-systemd ways of doing things. There are thousands of linux distributions, and afaik there are at least a dozen which are primarily defined as never-using-systemd. Face it: the linux userspace has always sucked. Linus punted on it way-back-when (to be fair, he was a solo dev working on a kernel,…

> Face it: the linux userspace has always sucked. Linus punted on it way-back-when (to be fair, he was a solo dev working on a kernel, he had his hands full) and opted for the gnu tools, but those (other than the compiler and libc and a few other spots like grep) have never been best-in-class.

Which userspace are you claiming is better then? Linux used to offer the best userland experience in the world, because you had control over every part of it, and since things were loosely coupled you could mix and match at will (e.g. prefer BSD grep or Solaris tar? No problem, install them).

> Somehow, this ended in a holy war and corresponding schism. It makes no sense to me. Are linux users upset that someone dared question their decades of suffering under a poorly-designed userspace? Agree or disagree with systemd's decisions, it's hard to fault someone for making and releasing software and offering it as an alternative, even if it sucks.

They didn't build a better alternative and let users choose to adopt it organically. They made a wilfully incompatible, tightly coupled thing, and then pressured other projects to hard-depend on it (e.g. Gnome).

This is quite literally the antithesis of everything that Linux and Unix stands for. The whole reason people were able to use the GNU tools with the Linux kernel is because they are loosely coupled and communicate through standardised interfaces. Systemd by contrast doesn't run anywhere but linux, and you need closely matched versions of everything: your linux has to match your udev has to match your systemd has to match your dbus has to match your gnome has to match your...

If systemd had been around when GNU and Linux were starting out, the whole project would have been literally impossible. It's a betrayal of everything those projects stand for.

Re: Systemd, ten years later: a historical and technical retrospective

#172

Post by enriquto » Mon, May 04, 2020, 6:43 AM UTC

JChase2 wrote: post #105

pknopf wrote: post #74

Earlier quoted context omitted.

I'm not seeing a lot of hope here. There is an incredible investment in systemd by distributions, there is no turning back. Help me..

Slackware doesn't use it, and Patrick Volkerding recently got a Patreon account set up so Slackware 15.0 or 14.3 will hopefully be out soonish with updated packages. I'm writing this on 14.2 and have been running it since it came out with basically no issues. Also have it on a backup server, runs great.

I love slack.

There's something extremely beautiful in launching htop and seeing less than 10 processes (including those of root).

Re: Systemd, ten years later: a historical and technical retrospective

#173

Post by AnthonyMouse » Mon, May 04, 2020, 6:44 AM UTC

cycloptic wrote: post #170

AnthonyMouse wrote: post #157

Earlier quoted context omitted.

They didn't used to break if you wanted to switch from one init system to another. What you could do is to reduce unnecessary interdependencies. And where there is inherently interaction between separate packages, use a stable interface and publish the standard so that alternative implementations can replace a single package in a standard way without breaking several others.

Yes, they did. It they didn't break it was because those inits were compatible with sysvinit scripts. You lose that once you start porting your init scripts over to rc or whatever it is you replaced your init with, and now your system will break if you try to switch to something else.

Your init scripts would have to be replaced, that's to be expected, and when multiple ordinary init systems are popular, packages could easily support more than one since an init script is generally only a few lines.

I'm talking about all the things that now break not related to the normal function of init at all.

Re: Systemd, ten years later: a historical and technical retrospective

#174

Post by xfs » Mon, May 04, 2020, 6:46 AM UTC

cycloptic wrote: post #141

AnthonyMouse wrote: post #138

Earlier quoted context omitted.

What you are missing is that it is possible for two things to be true at once. It is possible for sysvinit to suck and for systemd to suck in entirely different and potentially much worse ways. It's as though someone is complaining about having to live under the USSR and how bad things are, and then someone responds that the German rocket program has been dismantled and the British never really had much of one and th…

Sysvinit is still around, you can always go back to it if you really want. The package is even still in debian.

This is the kind of false argument being thrown around often in OSS discourse that ignores the structural power differential.

Don't like it? Write your own/Leave.

The fact is you can't go back to it as an individual, because the system has changed and as an individual you're powerless to change the situation at all, especially against an army of developers paid full-time. The latest news from Debian is sysvinit support is no longer guaranteed.

Re: Systemd, ten years later: a historical and technical retrospective

#175

Post by ChuckMcM » Mon, May 04, 2020, 6:49 AM UTC

gorgoiler wrote: post #119

When writing software, it’s comparatively easy to come up with grand new ideas and turn them into lines of code and files filled with modules. What’s much harder is (1) to expand ones code without coupling everything together (2) present it in a persuasive way that naturally builds a user base. SystemD feels like the new grad hire who decided the first thing the would do after joining — the opening power play — is to…

The other component here is "where you came from." When I joined Sun there were two very distinct programmer/hacker cultures; the PC and its spawn and "real" computers (mini and mainframe computers) and their OSes.

The cultures were very different, I straddled them with my hobby working being CP/M->PC->Amiga, and my professional work being RSX-11M->TENEX->TOPS20->UNIX.

Linux (and Minix) started out very much being a PC version of UNIX and PC-BSD jumped in there too. But Linus ceded userland to the community and as the community's membership has grown the people who wanted Linux to be the perfect UNIX lost out to the people who wanted Linux to be an open source version of Microsoft Windows.

It appeared to me that the folks who liked UNIX semantics fled to MacOS, although the complaints on that front are getting louder as well. So here we are.

Re: Systemd, ten years later: a historical and technical retrospective

#176

Post by lmm » Mon, May 04, 2020, 6:49 AM UTC

thristian wrote: post #91

zozbot234 wrote: post #29

Earlier quoted context omitted.

This is yet another example of pointless incidental complexity in systemd. The whole point of "nohup"-based tools like tmux and screen is to cleanly separate the management of user sessions from the incidental mechanism of whether a remote connection is being closed (the 'HUP' in nohup is short for "hang up" i.e. close a [possibly remote] connection). Systemd should simply acknowledge this fact and keep the user sess…

The trouble is that 'nohup' is not a particular state that specifically marks processes that want to survive logout. Instead, the rule is that processes running inside a particular terminal are killed when that terminal closes, and processes running outside any terminal run as they please. If you SSH to a server, or telnet, or login on the text console, or via a serial port, you get a terminal and everything you run…

> Since the kernel's idea of "a login session" is wrong (it only includes text-terminals), and it can't easily be changed for compatibility reasons

This is a CADT attitude. Improving existing interfaces while maintaining compatibility is hard; it's also what makes the difference between a serious software professional and an incompetent vandal.

Re: Systemd, ten years later: a historical and technical retrospective

#177

Post by cycloptic » Mon, May 04, 2020, 6:55 AM UTC

lmm wrote: post #164

cycloptic wrote: post #153

Earlier quoted context omitted.

I don't see how that is the problem. It's exactly what you asked for. If you're replacing a low-level component with something else that hasn't been updated in years and doesn't implement features that other packages do then yeah, you have to be prepared to accept that some newer things will break, and you'll have to revert a lot of other packages to older versions too. What else can we do? This isn't even a systemd-…

Systemd deliberately refused to conform to standardised interfaces and got other packages to be changed to explicitly depend on it instead (e.g. RedHat made changes to Gnome to make it break under non-systemd). This was not necessary to achieve the technical things that systemd wanted to achieve - competing alternatives (e.g. runit) didn't need that. (Though it probably was necessary to force adoption).

This is not true in any way. Gnome still works on non-systemd systems but you have to install elogind. And, there were valid technical reasons to have gnome depend on a login manager, in particular supporting multi-seat securely is very difficult without one. I should also mention that the previous "standardised interface" for this was ConsoleKit which was also written by the same developers as gnome and systemd. So this is not a matter of some developers going around and raiding projects and "forcing adoption", it's all stuff that happened under the same umbrella anyway.

If the "competing alternatives" don't need it then I would advise those projects to put their money where their mouth is and either start making improvements to elogind, or develop a new login manager that better fulfills the needs of gnome. In particular the solution used by elogind to deal with cgroups is to just disable them entirely, or at least it was last time I checked. I don't know what runit does for cgroups but a good step would be to make elogind compatible with that.

Re: Systemd, ten years later: a historical and technical retrospective

#178

Post by vezzy-fnord » Mon, May 04, 2020, 6:57 AM UTC

theamk wrote: post #127

gorgoiler wrote: post #119

When writing software, it’s comparatively easy to come up with grand new ideas and turn them into lines of code and files filled with modules. What’s much harder is (1) to expand ones code without coupling everything together (2) present it in a persuasive way that naturally builds a user base. SystemD feels like the new grad hire who decided the first thing the would do after joining — the opening power play — is to…

I think you underestimate how badly the old sysvinit sucked in the world with many good distribution-provided packages. It was pretty good in the good, old days, where a sysadmin could list every daemon the system runs by heart, but it was breaking apart when you had tons of random packages: Package X uses "start-stop-daemon" in ini file, so there is no way to get error messages if config file is incorrect. Package B…

The problem with this straightforward thesis is twofold: firstly, distribution developers did not have any coherent position on the disadvantages of sysvinit or initscripts; secondly, the problem was mostly an iatrogenic one, as the very attempts to 'solve' or 'reform' it made things even less sustainable.

When it comes to Upstart, it was almost never used except as a wrapper around initscripts. Effectively you booted from Upstart which greedily synthesized all of its 'events' on startup that in turn triggered initscripts, which brought some degree of parallelism and that was about it. The reverse-dependency 'start/stop on' model was too esoteric, and Upstart's state machine brittle. Hilariously enough, the only system to actually use Upstart to its fullest (still does) is ChromeOS.

Prior to systemd and Upstart, as I discuss in chapter 1, distro developers either simply added various preprocessors for initscripts to introduce dependency headers, or they had these vague hunches of 'waiting for Godot' that we could either replace init entirely with D-Bus, put D-Bus interfaces in initscripts, or some other roundabout way that involved grafting D-Bus. Much of this was owing to the 'irrational exuberance' of Linux developers wanting to hotplug the world after 2003-4, but it always remains an on-and-off effort with the people behind it never certain what they want.

Indeed, besides grunt work of optimizing initscripts, I point out that as late as 2009 distros like Debian were still very concerned about LSB-compliance, and that even when they were willing to adopt a new fancy event-driven init like Upstart, it was conditional on reusing tools like insserv to parse LSB headers, and on incorporating initscripts for service management. Also the entire bizarre diversion that the Debian group had of using a Perl script to generate initscripts.

So far as I know, the daemontools approach was never influential. The available dichotomy was between putting lipstick on a pig, and utopian dreaming.

The paradox remains: how can we trust that the same people who did the Wrong Thing so persistently then suddenly turned out and did the Right Thing like they hit jackpot? If anything, 'jackpot' is the right analogy -- it was a fluke. And the same integration failures that were had with sysvinit have now been shifted into wrestling systemd's job model.

"Unification" was a folly, as the distro is still the place that gets its finger pointed at if upstream can't deliver, and upstream insists it's "just" offering a bunch of "modular tools," which ironically was supposed to be the problem to be addressed! Besides, there is still a whole world of Linux devices out there running Busybox, musl, OpenWrt, etc. that are outside the systemd bubble. Or think of all the container images running Alpine (which is also the basis of postmarketOS). Linux is still the same bazaar it always logically must be, but with a new class distinction added to the mix.

Re: Systemd, ten years later: a historical and technical retrospective

#179

Post by terrywang » Mon, May 04, 2020, 6:59 AM UTC

ArchD wrote: post #40

throw0101a wrote: post #28

Earlier quoted context omitted.

> An init system is harder than it might seem at first […] systemd-as-init-system is not the problem. systemd-as-kitchen-sink is the problem. It's the tight coupling that annoys many people. Does udevd really need to be in the same repo? While there may be some nice things about journald, does it really have to be in the same source package? (And why can't it support remote logging with the industry standard syslog p…

Yeah, don't get me started on the fragile glass test tube called systemd-resolved that fails in mysterious ways when I use wicd on Ubuntu and the WiFi gets momentarily disconnected. I don't see the sense behind treating name resolution as something so special that it has to be part of an init system. I like the overall theoretical concept of systemd but it has ugly implementation details like this.

Funny you mentioned systemd-resolved, it generated so many coredumps across the machines I use at home (Arch, Manjaro, Fedora), its stability / reliability improved over time but still not up to standard.

Unfortunately, currently systemd-resolved is the only OS level DNS-Over-TLS solution that works for my use cases: roaming laptops and desktops running Linux. At home it's fine because the core router runs AsusWRT-Merlin which does DNS over TLS for all clients using it as resolver.

I don't have a strong opinion against systemd (initially I disliked it moving from Arch's BSD style rc.d). Trend is that systemd is taking over more and more *ctl, seem like its goal is the entire user-space?. Most distros choose to accept and adopt it, and overall (at some point) it eliminated distro differences and lowered the barrier for new Linux users (desktop) and sysadmins.

Personally I started systemd when it rolled into Arch Linux, after which point, rolling upgrade has never gone broke (workstation use case - Desktop Environment / WM), could be skill levelled up after gaining experience though lol...

BTW: I watched `reinventing home directories` by Lennart Poettering (recording @ All System Go! 2019), interesting one. Personally I like to see reinventing wheels to solve legacy problems (people get used to them, sometimes confused, question but don't know how to solve) others don't even consider problems. These helped open my mind and look at things from a different PoV.

[1]: notes on systemd-homed https://sites.google.com/site/imterry/aboutme/profiles/skill...

Re: Systemd, ten years later: a historical and technical retrospective

#180

Post by [deleted] » Mon, May 04, 2020, 7:07 AM UTC

jacquesm wrote: post #169

aganame wrote: post #123

Earlier quoted context omitted.

As a person for whom Linux and BSDs have been a hobby since 1996, I have no idea what you're talking about.

A hobbyist making hiring decisions and maintaining a blacklist of professionals who bitch about a change they had no control over? You're not helping.

[deleted]