Live data from Hacker News

Why systemd is a problem for embedded Linux

kevinboone.me

131–140 of 308 posts

Re: Why systemd is a problem for embedded Linux

#131
post #10
post #8

Earlier quoted context omitted.

it's also why corporate software is so shit - it's shiny in all the ways it needs to be shiny for management to make the decision, but it's a pain for the actual users. the trick in both cases is to know your customers!

Turns out, many people use systemd after what, 15? years of doomsday predictions :-)

I'm not a systemd hater, but I use it mainly only because my distro of choice (Debian) switched to it. If Debian chose something else, I'd be fine with that too. I'm a user out of necessity (or perhaps inertia), not out of choice.

Re: Why systemd is a problem for embedded Linux

#132

Earlier quoted context omitted.

Huh? Been a while since I was in the digital signage space but a lot of the equipment runs of the shelf RK3288 plugged into commercial displays. 2GB of RAM was pretty common. IIRC's though LG's WebOS TV's have minimum 2GB of RAM in the digital signage space directly built into the units themselves. I believe Samsung Tizen based units has similar RAM. My router has 1GB of RAM in it. But even my cheapest routers have 1…

A decade ago, I had to settle for 512MB of ram for my Windows XP desktop.

Three decades perhaps. In 2014 it was Windows 8, SSDs, 5th gen i7 (Haswell), 8-16GB of DDR4 ram. Even the iPhone 6 came with 1GB of ram.

Re: Why systemd is a problem for embedded Linux

#133
Try s6 or runit, and mdev from busybox. Mdev is not a "full featured" udev replacement, but that's sort of the point. It does just enough to get the job done and is extensible with scripts.

If you want to do the size of embedded where this matters, you also probably want a smaller libc like musl, and then buildroot and busybox are your friends.

Re: Why systemd is a problem for embedded Linux

#134
post #84

Earlier quoted context omitted.

Did you ever dig in to see why that changed? I'm surprised that in a steady-state (not booting or restarting services) systemd/not-systemd would have an impact.

A little bit. The tldr; is that there are a lot of periodic activities including log rotation, automatic updates, time syncing, waking up to check if such things were currently needed, .... Those are generally nice (aside from automatic updates, which are sometimes good but I think usually more harmful than whatever they're fixing), but systemd doesn't know anything about what my userspace demands on the hardware are…

> For anything quasi-realtime, I can certainly hunt down and squash everything systemd is running at the wrong time, but in practice it's much easier to install something else, ensure no processes are running other than the ones I care about, and move those concerns to userspace.

Should these issues be fixed by CPU/IO schedulers? Are there any systemd realtime tasks involved?

Re: Why systemd is a problem for embedded Linux

#135
post #83

I think what he's running into is "the unix philosophy". It is basically about small tools, limited in scope, that solve a problem surgically and comprehensively. They work together by mixing and matching to solve the problem. systemd is sort of like if microsoft word took on system init. In the beginning it was small, but now it is web based and does videoconferencing. I don't know, it is sort of like busybox, repla…

> systemd is sort of like if microsoft word took on system init. In the beginning it was small, but now it is web based and does videoconferencing.

It's actually a suite of tools that do solve individual problems comprehensively. Not as small as the programs in coreutils, but still not a huge monolith.

Re: Why systemd is a problem for embedded Linux

#136

I couldn't disagree more: I've worked with lots of embedded devices running systemd, and it solves many more problems than it introduces. The community is also quite responsive and helpful in my experience. I won't pretend there aren't occasional weird problems... but there's always a solution, here's a recent example: https://github.com/systemd/systemd/issues/34683 Memory use is irrelevant to me: every embedded Linu…

> every embedded Linux device I've been paid to work on in the past five years had over 1GB of RAM. If I'm on a tiny machine where I care about 8MB RSS, I'm not running Linux, I'm running Zypher or FreeRTOS The gap between “over 1GB of RAM” and 8MB RSS contains the vast majority of embedded Linux devices. I, too, enjoy when the RAM budget is over 1GB. The majority of cost constrained products don’t allow that, though…

> The gap between “over 1GB of RAM” and 8MB RSS contains the vast majority of embedded Linux devices

The gap between 16MB RAM and 64MB RAM doesn't exist, though. Literally doesn't, the components have the same cost down to a cent in the BOM.

And if you can have 64MB, then there's systemd's own true memory use (around 3-4MB) is completely immaterial.

Re: Why systemd is a problem for embedded Linux

#137

Earlier quoted context omitted.

> OpenWRT should just stop being stubborn and integrate it. Why? If it fails to provide benefits commensurate to the effort required to integrate it, then there's no reason to do the work. OpenWRT is for routers and access points... devices which usually have a fixed [hardware] configuration, and very little need for the absurdly-complex (and in my professional experience, often subtly buggy) event-driven systems tha…

OpenWRT is a mess. It started as a small distro for routers, but it stopped being that long ago. Now it can host telephony services, full-blown Docker orchestration systems, etc. So you get all the pleasures of complex systems running on antiquated infrastructure. > so you only get to just write a unit file and call it a day in the most trivial of cases. What "start/stop verification"? Give examples, please. Systemd…

> Now it can host telephony services, full-blown Docker orchestration systems, etc.

What features that systemd provides that aren't provided by openrc, runit, monit, openrc's init system, etc, etc, etc that are needed by software like Asterisk, Docker/runc, Kubernetes, and similar?

If what OpenWRT is running works fine to bring that up, then systemd doesn't have any relevant compelling features that aren't provided by another init system. Switching away from something just because it's "antiquated" is a great resume-building tactic, but it's a pretty poor strategy when your objective is to keep things that have been working working with a minimum of effort.

> What "start/stop verification"? Give examples, please.

One example (out of many) are pre-start, pre-stop, and pre-restart configuration file correctness verification so that a typo or thinko doesn't bring a service down.

A concrete example of this is Gentoo's running of 'named-checkconf' in BIND 9's service file [0]. Gentoo's BIND 9 systemd unit file [1] tries, but probably does not do the same thing... I don't know if 'ExecStartPre' runs before systemd brings down a service on a 'restart' action, but I bet you that it definitely does not run on service stop.

I just checked and Ubuntu 22 does NOT do this sort of checking for its bind9 package, which is pretty wild.

> It's a very rare case when you _need_ shell [or other such helpers] in systemd.

Then you're either running "seatbelts off" (maybe without even knowing that you are), or you rarely deal with anything nontrivial. This need comes up a LOT in my day-to-day.

[0] https://gitweb.gentoo.org/repo/gentoo.git/tree/net-dns/bind/...> [2]

[1] https://gitweb.gentoo.org/repo/gentoo.git/tree/net-dns/bind/...>

[2] You might look at that file and be like "wow, TL;WR", but do try to notice how much of that file is dedicated to creating useful, human-readable status messages and error messages with good corrective instructions. If you care at all about your sysadmins, that complexity has to go somewhere, and that somewhere cannot be inside a systemd unit file... systemd just doesn't provide the features required to do that (and it would be kind of insane for the devs to even try).

Re: Why systemd is a problem for embedded Linux

#138

Earlier quoted context omitted.

A decade ago, I had to settle for 512MB of ram for my Windows XP desktop.

Three decades perhaps. In 2014 it was Windows 8, SSDs, 5th gen i7 (Haswell), 8-16GB of DDR4 ram. Even the iPhone 6 came with 1GB of ram.

There's a huge chunk of people out there, not able to afford the latest.

Re: Why systemd is a problem for embedded Linux

#139
post #83

I think what he's running into is "the unix philosophy". It is basically about small tools, limited in scope, that solve a problem surgically and comprehensively. They work together by mixing and matching to solve the problem. systemd is sort of like if microsoft word took on system init. In the beginning it was small, but now it is web based and does videoconferencing. I don't know, it is sort of like busybox, repla…

> I think what he's running into is "the unix philosophy".

"Unix philosophy" is throwing crap at the problem, until it kinda-sorta works on the developer's machine.

Re: Why systemd is a problem for embedded Linux

#140

Earlier quoted context omitted.

It is legal to sell unhealthy food, ie poisonous food. Sugar is a poison, drugs are poisons. You can legally sell escolar a fish that if eaten too much is poisonous. What is poisonous food? You can sell plastic that breaks down into the environment and people are drinking it in water, eating it in fish. Is that legally sold poisonous food?

Sugar is not, in fact, a poison. As for drugs: > Alle Dinge sind Gift, und nichts ist ohne Gift; allein die Dosis macht, dass ein Ding kein Gift ist. - Paracelsus, 1538 https://en.wikipedia.org/wiki/The_dose_makes_the_poison

Found the Hackernews who hasn't yet read Lustig's research into the hepatotoxicity of sugar. It's as bad for your liver as alcohol.
Post reply on HN