Live data from Hacker News

Writing systemd units that stop gracefully before shutdown

psdn.io

31–40 of 57 posts

Re: Writing systemd units that stop gracefully before shutdown

#32

It shouldn't be this hard to stop a service gracefully. This is far, far more complicated than SysV init, where you just need to drop a script into /etc/init.d symlink it from the appropriate rc directories. (For shutdown/reboot, you'd create symlinks in rc5.d and rc6.d named KNNwhatever, where NN is an integer that specifies the order the script will be run in. The "K" stands for "kill".) Edit: Note that my example…

> It shouldn't be this hard to stop a service gracefully.

It's not.

> you just need to drop a script into /etc/init.d symlink it from the appropriate rc directories.

You just need to drop a unit file into /etc/systemd/system/ and symlink it from the appropriate /etc/systemd/system/${target}.wants/ directories.

Don't tell me that "shutdown.target.wants" and "reboot.target.wants" are harder than "rc0.d" and "rc6.d".

A lot of the article is about ordering of dependencies (don't stop a dependency until after the dependent has stopped). Don't tell me that adding `Before=` and `After=` lines in the unit file is harder than having to remember all of the dependencies and manually figure out the correct "NN" for it all to work correctly.

A lot of the article is about either having your daemon handle SIGTERM, or coming up with the appropriate `ExecStop=` command. The same command you'd be writing in your rc script (the "handle SIGTERM" stuff being for if your rc script simply says `kill $PID`).

That is: The complex parts of the article are things that were complex with sysvinit too.

Re: Writing systemd units that stop gracefully before shutdown

#33

I fundamentally disagree with the idea that software should require or even expect a graceful shutdown. You can never stop the user from yanking the power cord out of the socket, which is what they will do if you force a bunch of housekeeping to happen before shutdown. You have to deal with crash/power failure recovery anyway. So do your housekeeping on startup. Shutdown should be a quick and simple termination.

I'm with you.

It's easier said than done, of course, but crash-only software is a worthwhile goal IMO.

Re: Writing systemd units that stop gracefully before shutdown

#34
post #8

Earlier quoted context omitted.

I tend to push the metaphor of software being meant to be read and only incidentally to be run by a computer as far as I can. We are telling a story to future us or our successors. Stories have rules and it's jarring when you violate them. There's an idea in software that's a bit like the corollary of Chekhov's gun. Chekhov's gun is about not presaging jarring story elements that will never come to pass. But it's nea…

Quoted post unavailable.

If it is, then call me a sucker because I thought it was an entertaining perspective.

Re: Writing systemd units that stop gracefully before shutdown

#35
post #17

Earlier quoted context omitted.

I have the following unit file saved for that purpose: [Unit] # https://stackoverflow.com/questions/36729207/trigger-event-on-aws-ec2-instance-stop-terminate Description=unlink agent from remote server Before=shutdown.target [Service] Type=oneshot EnvironmentFile=-/etc/environment KillMode=none ExecStart=/bin/true ExecStop=/opt/service-name/shutdown-unlink RemainAfterExit=yes User=root [Install] WantedBy=multi-user.t…

You unit races with the network being brought down, since it isn't listed as a "Before" target, FYI.

They need After=network.target, not Before=network.target.

The shutdown order is the reverse of startup order, and they execute the payload on shutdown (ExecStop).

Re: Writing systemd units that stop gracefully before shutdown

#36
post #8

Earlier quoted context omitted.

I tend to push the metaphor of software being meant to be read and only incidentally to be run by a computer as far as I can. We are telling a story to future us or our successors. Stories have rules and it's jarring when you violate them. There's an idea in software that's a bit like the corollary of Chekhov's gun. Chekhov's gun is about not presaging jarring story elements that will never come to pass. But it's nea…

Quoted post unavailable.

This is exactly the type of novel analogy that ML can not produce because the words don’t tend to appear next to each other in any corpus.

Re: Writing systemd units that stop gracefully before shutdown

#37
post #17

A much more challenging task is writing a systemd unit that starts gracefully before shutdown. I wanted to write a unit that could issue an API call to delete the instance rather than doing a normal power off. Putting it at a reasonable place in the sequence took a lot of trial and error! The trick is actually to have the unit be started at some point in normal boot up (e.g. “armed”) and then do the actual task when…

I have the following unit file saved for that purpose: [Unit] # https://stackoverflow.com/questions/36729207/trigger-event-on-aws-ec2-instance-stop-terminate Description=unlink agent from remote server Before=shutdown.target [Service] Type=oneshot EnvironmentFile=-/etc/environment KillMode=none ExecStart=/bin/true ExecStop=/opt/service-name/shutdown-unlink RemainAfterExit=yes User=root [Install] WantedBy=multi-user.t…

The "Before=shutdown.target" is superfluous, that is a default dependency (as regular services obviously need to be shut down before system shutdown).

"KillMode=none" shouldn't be necessary, unless your command leaves processes running that need to stay running (and in that case you have other problems, since those processes may not actually finish before the system is shut down). Systemd waits for your service to stop before reaching shutdown.target (per the default Before= and Conflicts= dependencies).

Since your command probably needs networking, you also want "After=network.target" to ensure your command runs before network is torn down, as covered in sibling comments.

Re: Writing systemd units that stop gracefully before shutdown

#38
post #8

Earlier quoted context omitted.

I tend to push the metaphor of software being meant to be read and only incidentally to be run by a computer as far as I can. We are telling a story to future us or our successors. Stories have rules and it's jarring when you violate them. There's an idea in software that's a bit like the corollary of Chekhov's gun. Chekhov's gun is about not presaging jarring story elements that will never come to pass. But it's nea…

Quoted post unavailable.

Can we not make this the new low-effort way of saying "this doesn't make sense to me"? I've seen it a lot lately and it's at least as insulting as a regular old ad hominem attack.

FWIW it made perfect sense to me. It's a fun and thought-provoking analogy.

Re: Writing systemd units that stop gracefully before shutdown

#39

Earlier quoted context omitted.

While I agree it's a bit of a weird take, for example -- there may be performance tradeoffs made in any given workload to make the disk consistent, inconsistently The 'most' there is doing some effort It is actually quite a common practice for those being audited for disaster recovery to do exactly that -- yank cables. More realistically, flip some switches We do it once a year, set aside a region and time... then te…

Right, there are failure modes that have to be tested and accounted for, and one of them is the state being inconsistent after a shutdown. The previous poster seemed to advocate for not thinking of this as a failure mode at all but rather normal operation, which I just don’t see as true.

Totally, it's certifiably untrue!

Take the InnoDB storage engine in MySQL/MariaDB for example.

For performance (and likely other) reasons, this file only grows. It never shrinks... it will only go to 0 or grow.

The DB (or individual tables, depending on config) have to be truncated/emptied to reclaim those blocks.

Stop it uncleanly and there's a good chance you'll have to sacrifice a considerable amount of the data just to get the engine to start

This and countless other things have to make consistency trade-offs. While everything could be written to only operate atomically, it will also slow to a crawl.

Re: Writing systemd units that stop gracefully before shutdown

#40
post #8

A much more challenging task is writing a systemd unit that starts gracefully before shutdown. I wanted to write a unit that could issue an API call to delete the instance rather than doing a normal power off. Putting it at a reasonable place in the sequence took a lot of trial and error! The trick is actually to have the unit be started at some point in normal boot up (e.g. “armed”) and then do the actual task when…

I tend to push the metaphor of software being meant to be read and only incidentally to be run by a computer as far as I can. We are telling a story to future us or our successors. Stories have rules and it's jarring when you violate them. There's an idea in software that's a bit like the corollary of Chekhov's gun. Chekhov's gun is about not presaging jarring story elements that will never come to pass. But it's nea…

If software is meant to be read, and only incidentally run, then having it in the config to only run at shutdown is still introducing it early on. As a matter of fact, I think it would be like introducing the the reaper character in the very beginning and keeping them around in every scene sitting silently in the background until it was time for their one line at the very end. Many people might not notice, but to someone who does, it would be very odd indeed.
Post reply on HN