Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A timesyncd failure and systemd's lack of debugability

utcc.utoronto.ca

Post reply on HN
Print view View original

61–70 of 126 posts

Re: A timesyncd failure and systemd's lack of debugability

#61

Post by iforgotpassword »

I'm repeating myself, but the best way to use systemd is as an init system only, and then staying away from those fancy features like dynamic users (wtf) or socket activation. Especially avoid all that bolted on crap like timesyncd, resolved, etc. Yes they integrate a bit better with systemd, but no you don't need this better integration, especially since these tools are usually less configurable than their bigger brothers and are often more buggy just because they are not that battle tested yet.

> systemd will tell me to reproduce on the latest version, Ubuntu will ignore it as always

Amen to that. Distro bug reporting/handling is a pretty sad story on its own, and so far dealing with Ubuntu has been the worst. The most insane instance was when I reported a bug in their ati x11 driver. It was about two weeks after a new non-LTS came out, I even tracked down the bug and found that it was already fixed upstream only 2 commits after the one they shipped. So they didn't need to do anything but apply that single patch. No response ever. 6 months later the next release just shipped newer drivers and the problem was gone, but hadn't I recompiled the fixed driver myself I wouldn't have been able to switch vt for half a year without crashing X. I really like open source and all and make an effort to produce usable bug reports to help improve things for everyone, but instances like this make me wonder what the f*ck is wrong with people maintaining such projects.

Re: A timesyncd failure and systemd's lack of debugability

#63

Post by zaarn »

LukeShu wrote: post #39
agf wrote: post #3

The follow up article ( https://utcc.utoronto.ca/~cks/space/blog/linux/SystemdDynami... ) explains the underlying problem. Basically, to get around root-only readable directories, a special mount namespace is created when Dynamic Users are used. That fails when there are mount points that systemd can't access, which was due to FUSE filesystems mounted inside owner-only home directories, combined with an NFS mount. Th…

> and there isn't a simple way to tell systemd to delay creating that socket. … and there is no generic way to say to start a service after the auth provider is up That's what After=nss-user-lookup.target is for; just stick that in the [Unit] section of the .socket file. See the systemd.special(7) man page for more information. ---- Edit: However, there is an interesting problem related to that: when you use socket a…

That should be doable with a single additional line, IIRC

    Before=dbus.service

Re: A timesyncd failure and systemd's lack of debugability

#65

Post by Chris2048 »

agwa wrote: post #31
kemitche wrote: post #19

Earlier quoted context omitted.

There is nothing wrong with examining the motive behind someone's argument. That's very different than an ad hominem attack.

That is literally an ad hominem attack: https://en.wikipedia.org/wiki/Ad_hominem

later parts of the page claim the ad-homs are legitimate. Most people would consider a claim of "ad-home" as implying illegitimacy though.

Re: A timesyncd failure and systemd's lack of debugability

#66

Post by zaarn »

KaiserPro wrote: post #37
andrewstuart wrote: post #6

Before systemd, a difficult bug was a difficult bug. With systemd, a difficult bug leads to an attack on systemd itself, and questioning whether it should even exist. I like systemd, it's a pity some vocal people aren't enthusiastic about it. Just like politics and the news, when you read of something being attacked, I think "who is saying this, doing they have an underlying bias against this thing they are attacking…

Before systemd the error were shell errors. before systemd the bundled logger wrote to a file, didn't direct people to journald, that lags like a champ when a log file is > 1 meg. before systemd errors in config files were on specific lines I don't have bias against systemd I have bias against it's flaws. I like its config file. I hate it's documentation. I like that it has a cron like feature, I hate that its pretty…

sysvinit has it's warts too and it can get quite ugly.

For example, atm I need to have 3 services. Two of these services need to run at the same time, they can run separately but then they are not useful. The third services can only run with both services online and is first run 15 minutes after boot (not earlier!) and then every hour but the hourly run MUST NOT run before the first run of the boot (ie the very first run may happen 15 minutes after boot, even if the hourly run would have happened in that time, it cannot)

This is not trivially achievable with cron and classical init other than resorting to complicated shell scripts in both.

With SystemD I can cut down on the complexity by declaring this dependency in the service files and the timers in systemd are sufficiently configurable to achieve what I need.

Sure, systemd isn't flawless, but it's a mile better than the average init script I'd have to hack together otherwise. Journald isn't pretty either but with the right configuration you can cut down on the lag.

And before systemd problems like the above wouldn't have errors in specific lines either, some shell script would be subtly wrong and the entire charade would collapse in an instant.

You don't have to think about systemd at all. Ansible and other modern tooling can abstract managing the machines for you (atm I managing about 200 VMs in Ansible playbooks and it works great)

Re: A timesyncd failure and systemd's lack of debugability

#67

Post by jononor »

steevdave wrote: post #56
newnewpdro wrote: post #28

TL;DR: systemd has bugs. Mountain out of a mole hill. Lack of debuggability? It's not like this is some threaded golang program you'll lose your mind attempting to make sense of in strace. systemd is a single-threaded C program, if you consider this as not debuggable it's your failing not systemd's. There are valid complaints to be made about systemd, but from what I've seen the crux of the problem is generally the h…

The lack of contributors has a lot to do with Lennart's (apologies if I misspelled his name) attitude early on in systemd's development. I have no idea if it has changed, as I gave up trying to get things fixed. The one issue I still have with systemd is the way it handles the kernel's command line. I work with Chromebooks a lot, using software that isn't ChromeOS. The cool thing is that they can use FIT images so yo…

Lack of contributors? systemd had over 300 different contributors the last 12 months. That makes it in the top 2% of open source projects (tracked by OpenHub). Which is very impressive for being such a low level component. https://www.openhub.net/p/systemd

Re: A timesyncd failure and systemd's lack of debugability

#68

Post by majewsky »

> this server drifted more than half a second out of synchronization before I found this issue entirely by coincidence

Sounds like time synchronization is not particularly important for you. We have alerts set up for high clock drift that trigger at abs(drift) >= 300ms.

Re: A timesyncd failure and systemd's lack of debugability

#69

Post by LukeShu »

zaarn wrote: post #63
LukeShu wrote: post #39

Earlier quoted context omitted.

> and there isn't a simple way to tell systemd to delay creating that socket. … and there is no generic way to say to start a service after the auth provider is up That's what After=nss-user-lookup.target is for; just stick that in the [Unit] section of the .socket file. See the systemd.special(7) man page for more information. ---- Edit: However, there is an interesting problem related to that: when you use socket a…

That should be doable with a single additional line, IIRC Before=dbus.service

By itself that would only schedule it before dbus.service if they are both queued to be started; but since we'd like to be using socket activation to lazy-start it, you'll also need to make sure it's queued when dbus.service is queued:

    [Install]
    WantedBy=dbus.service
That's what I meant by "manually ordering nslcd.service before dbus.service".

Re: A timesyncd failure and systemd's lack of debugability

#70

Post by nickez »

ploxiln wrote: post #43
andrewstuart wrote: post #6

Before systemd, a difficult bug was a difficult bug. With systemd, a difficult bug leads to an attack on systemd itself, and questioning whether it should even exist. I like systemd, it's a pity some vocal people aren't enthusiastic about it. Just like politics and the news, when you read of something being attacked, I think "who is saying this, doing they have an underlying bias against this thing they are attacking…

With systemd, any really tricky low-level bug is systemd's fault, because systemd tries to do everything, in the most "advanced" way. And due gnome3 dependencies it's been adopted by all major distros. Before systemd, a difficult bug was not always the fault of one newfangled opinionated and unavoidable package. I read the first blog post about systemd when it was written, and thought it was really interesting. Then…

timesyncd is a network connected daemon. Why would you not isolate it as much as possible?
Post reply on HN