Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A timesyncd failure and systemd's lack of debugability

utcc.utoronto.ca

Post reply on HN
Print view View original

41–50 of 126 posts

Re: A timesyncd failure and systemd's lack of debugability

#41

Post by LoSboccacc »

tomjakubowski wrote: post #9
insaneirish wrote: post #2

Given things like this: "The default configuration is defined during compilation" [1] and the fact that you'd think 'timedatectl status' would tell you the NTP server you're supposedly trying to sync to (spoiler alert: it doesn't), I'm unsurprised of any further atrocities committed by systemd. [1]: https://www.freedesktop.org/software/systemd/man/timesyncd.c...

> Given things like this: "The default configuration is defined during compilation [so a configuration file is only needed when it is necessary to deviate from those defaults....]" I don't get it. What's so bad about that?

Once you could just maintain some file from etc in a version control and setup a machine predictably with a script three lines long

Then it came puppet chef and a shitload other stuff because ‘devops’ which make the task easier once and then fuck up your environment at every single upgrade changing defaults, changing dependencies and changing the environment you run

I swear what could be done repeatedly the same way for a decade now it’s a frail mess that requires yearly fiddling to keep it running and current

No wonder companies need so much devops people/time these days, they need them to keep following the frail shit that’s dropped into stable distro.

Systemd is just one more thing that you have to reconfigure at every release because changes defaults subtly.

Re: A timesyncd failure and systemd's lack of debugability

#42

Post by thwarted »

mikekchar wrote: post #40
gcb0 wrote: post #36

Earlier quoted context omitted.

That's not quite right in this case. Systemd and gnome3 were cases where the new 'version' did not need full feature parity. Both those projects were written on leisure time, with no pressure for deadline or feature. and still adopted from top down. i guess some projects forget the benevolent part of benevolent dictator for life. systemd arguments were just "boot faster" and "it's newer". Everyone knew it was going t…

You're getting down voted mainly because you are wrong. I'm actually not well versed in this, so hopefully somebody will correct my wrong version -- but at least I think it will steer you in the right direction ;-) Systemd solves real problems that have to do with virtualisation and/or containerisation. Red Hat is the company that wrote Systemd and they did it for commercial reasons (i.e. they needed these facilities…

Systemd solves real problems that have to do with virtualisation and/or containerisation.

systemd predates the industry love affair with containerization and virtualization for workload management, although work on containerization within systemd may have helped increase awareness of containerization's value. I don't remember early reasons to use systemd mentioning either; the rationales trotted out in favor of systemd in its early days were reducing boot time and replacing shell scripts used during boot, shell scripts labeled things like "old" and "fragile", despite multiple decades of shell scripts being successfully used to reliably manage the boot process on open source and proprietary Unixes, alike.

Re: A timesyncd failure and systemd's lack of debugability

#43

Post by ploxiln »

andrewstuart wrote: post #6

Before systemd, a difficult bug was a difficult bug. With systemd, a difficult bug leads to an attack on systemd itself, and questioning whether it should even exist. I like systemd, it's a pity some vocal people aren't enthusiastic about it. Just like politics and the news, when you read of something being attacked, I think "who is saying this, doing they have an underlying bias against this thing they are attacking…

With systemd, any really tricky low-level bug is systemd's fault, because systemd tries to do everything, in the most "advanced" way. And due gnome3 dependencies it's been adopted by all major distros. Before systemd, a difficult bug was not always the fault of one newfangled opinionated and unavoidable package.

I read the first blog post about systemd when it was written, and thought it was really interesting. Then I read the second blog post and thought "eh this is getting a bit complicated". And since then of course systemd has relentlessly taken over and added much more ... so of course practically any low-level system bug, randomly distributed, has a really high chance of being in systemd. And again, it does lots of stuff that was never needed before - in this case it's un-sharing a mount namespace for extra isolation of a small trusted service, which is overkill and complicated, and this struggle to figure out what the hell happened seems particularly unnecessary.

Re: A timesyncd failure and systemd's lack of debugability

#44

Post by lima »

KaiserPro wrote: post #37
andrewstuart wrote: post #6

Before systemd, a difficult bug was a difficult bug. With systemd, a difficult bug leads to an attack on systemd itself, and questioning whether it should even exist. I like systemd, it's a pity some vocal people aren't enthusiastic about it. Just like politics and the news, when you read of something being attacked, I think "who is saying this, doing they have an underlying bias against this thing they are attacking…

Before systemd the error were shell errors. before systemd the bundled logger wrote to a file, didn't direct people to journald, that lags like a champ when a log file is > 1 meg. before systemd errors in config files were on specific lines I don't have bias against systemd I have bias against it's flaws. I like its config file. I hate it's documentation. I like that it has a cron like feature, I hate that its pretty…

I'll take the rare systemd bug over the buggy hell that most sysv init scripts were.

In fact, I run both distros with and without systemd and broken init scripts are a constant headache, while I've yet to encounter a single issue caused by systemd.

Re: A timesyncd failure and systemd's lack of debugability

#45

Post by jhoechtl »

erik_seaberg wrote: post #5

systemd's job is to start daemons. It can be reasonable to decide dynamically when to start them based on which others are healthy (maybe you care more about speed than determinism on a laptop). But there's no reason to choose a uid and mkdir dynamically. You want these errors at install time, not at the box's next (probably unattended) restart.

And do the logging. My only gripe is that it logs in a binary format requiring special tools to read the logs. Otherwise systemd is a marvel.

Re: A timesyncd failure and systemd's lack of debugability

#46

Post by LukeShu »

gcb0 wrote: post #36
thaumaturgy wrote: post #34

Let's carefully step around the whole distraction that is systemd. Instead: > The real problem here is that it is impossible to diagnose or debug this situation. Simply to get this far I had to read the systemd source code (to find the code in timesyncd that printed this specific error message) and then search through 25,000 lines of strace output. And I still don't know what the problem is or how to fix it. This see…

That's not quite right in this case. Systemd and gnome3 were cases where the new 'version' did not need full feature parity. Both those projects were written on leisure time, with no pressure for deadline or feature. and still adopted from top down. i guess some projects forget the benevolent part of benevolent dictator for life. systemd arguments were just "boot faster" and "it's newer". Everyone knew it was going t…

I want to expand on mikekchar's "You're getting down voted mainly because you are wrong."

systemd has been adopted by many different distros, however this is not because of "top down" pressure (From who, Red Hat? Most distros don't answer to Red Hat), and most of them don't have a (B)DFL who could have imposed that decision on them.

There are many arguments for systemd other than "boot faster" and "it's newer". In fact, the author of the article this is a thread for has written a pretty good list of them: https://utcc.utoronto.ca/~cks/space/blog/linux/SystemdRight

The article you linked... is bad for very many reasons. In particular, it makes no acknowledgement that what it refers to as "init" (http://www.nongnu.org/sysvinit/) is only really half of an "init system" as the term is commonly used today, and all distros would be pairing it with something else on top of it. The table at the bottom is particularly ridiculous, and I take issue with almost every row; a lot of that comes down to the silliness of comparing systemd to sysvinit-by-itself, rather than sysvinit+initscripts or sysvinit+OpenRC or sysvinit+LSB-init-scripts.

Re: A timesyncd failure and systemd's lack of debugability

#47

Post by lolc »

wokwokwok wrote: post #16
LukeShu wrote: post #13

Earlier quoted context omitted.

Systemd is straight C, not C++.

spins finger in the air the difference is immaterial and irrelevant in this context.

A huge part of GNU/Linux is written in C. A small part is written in C++. I would accept the argument that C++ is less accessible than C to people maintaining Linux systems.

Based on this alone I would say that yes, the difference is material and relevant.

Re: A timesyncd failure and systemd's lack of debugability

#48

Post by onli »

LukeShu wrote: post #46
gcb0 wrote: post #36

Earlier quoted context omitted.

That's not quite right in this case. Systemd and gnome3 were cases where the new 'version' did not need full feature parity. Both those projects were written on leisure time, with no pressure for deadline or feature. and still adopted from top down. i guess some projects forget the benevolent part of benevolent dictator for life. systemd arguments were just "boot faster" and "it's newer". Everyone knew it was going t…

I want to expand on mikekchar's "You're getting down voted mainly because you are wrong." systemd has been adopted by many different distros, however this is not because of "top down" pressure (From who, Red Hat? Most distros don't answer to Red Hat), and most of them don't have a (B)DFL who could have imposed that decision on them. There are many arguments for systemd other than "boot faster" and "it's newer". In fa…

> systemd has been adopted by many different distros, however this is not because of "top down" pressure (From who, Red Hat? Most distros don't answer to Red Hat), and most of them don't have a (B)DFL who could have imposed that decision on them.

Often it was a "matter of fact" thing. If the ecosystem under the distro changes to use systemd distros will follow. There might be more advantages to it, but it being required to run Gnome 3 without additional effort, and in general other servies starting to rely on it, make the decision pro systemd the easy one for distro maintainers. You can see that point being made in the debian discussion on the issue.

Till they hit the showstopper bugs and can't solve them.

Re: A timesyncd failure and systemd's lack of debugability

#49

Post by erik_seaberg »

LukeShu wrote: post #22
erik_seaberg wrote: post #5

systemd's job is to start daemons. It can be reasonable to decide dynamically when to start them based on which others are healthy (maybe you care more about speed than determinism on a laptop). But there's no reason to choose a uid and mkdir dynamically. You want these errors at install time, not at the box's next (probably unattended) restart.

Everything that would have been "a weird problem with Foo Daemon's init script" we now call "a weird problem with systemd". So you think that systemd-timesyncd's service file shouldn't choose the uid at service-start time. That's not really the issue though. The issue is that for service isolation reasons (related to choosing the uid at service-start time), the service file wants to run systemd-timesyncd in a private…

If SELinux doesn't cut it and you actually need to jail a badly-behaved service, "set up a jail in this specific way and exec this process in it" should be a tool to be invoked by an init script or a unit file or whatever. unshare(1) looks like it's halfway there.

Re: A timesyncd failure and systemd's lack of debugability

#50

Post by LukeShu »

onli wrote: post #48
LukeShu wrote: post #46

Earlier quoted context omitted.

I want to expand on mikekchar's "You're getting down voted mainly because you are wrong." systemd has been adopted by many different distros, however this is not because of "top down" pressure (From who, Red Hat? Most distros don't answer to Red Hat), and most of them don't have a (B)DFL who could have imposed that decision on them. There are many arguments for systemd other than "boot faster" and "it's newer". In fa…

> systemd has been adopted by many different distros, however this is not because of "top down" pressure (From who, Red Hat? Most distros don't answer to Red Hat), and most of them don't have a (B)DFL who could have imposed that decision on them. Often it was a "matter of fact" thing. If the ecosystem under the distro changes to use systemd distros will follow. There might be more advantages to it, but it being requi…

sysvinit is short enough to be trivially maintainable. If you're just using it as a shim to load a framework on top of it, you can ditch even more of it cutting it down to around 500LOC (see openrc-init). So that's not a large enough part of "the ecosystem under the distro".

Debian maintained their own init-scripts framework (and still does, for kFreeBSD and other kernels), so that certainly didn't move out from under them.

Arch Linux maintained their own initscripts framework, so that wasn't moving out from under them.

I suppose there is a case to be made that distros were pushed to systemd by Gnome 3's reliance on systemd-logind's D-Bus API.

Post reply on HN