Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Writing systemd units that stop gracefully before shutdown

psdn.io

Post reply on HN
Print view View original

51–57 of 57 posts

Re: Writing systemd units that stop gracefully before shutdown

#51

Post by aequitas »

empthought wrote: post #13
SoftTalker wrote: post #11

I fundamentally disagree with the idea that software should require or even expect a graceful shutdown. You can never stop the user from yanking the power cord out of the socket, which is what they will do if you force a bunch of housekeeping to happen before shutdown. You have to deal with crash/power failure recovery anyway. So do your housekeeping on startup. Shutdown should be a quick and simple termination.

This is a weird take; most systems in data centers don’t have people walking from rack to rack yanking power cords, and most consumer systems don’t even have a power cord to yank.

They do. Reminds me of a previous job long ago where a datacenter tech was checking where a network cable went by tugging on it, resulting in a network switch blade being yanked out of a chassis, bringing down half of the production environment.

These thing did happen, can happen and will happen.

Even in modern cloud environments. AWS might consider the hardware your EC2 VM is running on unstable, prompting you to replace/move the VM within 24 hours (if it has not already brought down by hardware failure).

Re: Writing systemd units that stop gracefully before shutdown

#52

Post by MarkusWandel »

Quoting from the article:

   TimeoutStopSec=0
That's cost me more than one hard power button powerdon (on a desktop machine with the system partition on SSD - unnerving).

One of the innumerable things that systemd stops on shutdown gets stuck - permanently - and the machine goes into a state out of which, to my knowledge, is only a powerdown or reset.

I ended up searching for the above and replacing them with a reasonable timeout (several minutes).

Re: Writing systemd units that stop gracefully before shutdown

#53

Post by dghubble »

MarkusWandel wrote: post #52

Quoting from the article: TimeoutStopSec=0 That's cost me more than one hard power button powerdon (on a desktop machine with the system partition on SSD - unnerving). One of the innumerable things that systemd stops on shutdown gets stuck - permanently - and the machine goes into a state out of which, to my knowledge, is only a powerdown or reset. I ended up searching for the above and replacing them with a reasonab…

That's fair, that example would be better with a modest timeout (e.g. if podman had a bug that caused it to hang), without taking away from the main points. Updated.

Someone asked about the opposite on Twitter: https://twitter.com/DannoHung/status/1585350836074446869

Re: Writing systemd units that stop gracefully before shutdown

#54

Post by kentonv »

empthought wrote: post #44
kentonv wrote: post #42

Earlier quoted context omitted.

Weird take? It is a table stakes expectation for most servers that they will not lose data when the power goes out, or when the kernel panics, or when the server itself crashes or runs out of memory. If your software requires graceful shutdown, that seems to imply that it will lose data in all those cases. You can perhaps use graceful shutdown to perform some optimization that allows subsequent startup to go faster,…

That software should not “even expect” a graceful shutdown is the weird take. Servers can and do “lose data” all the time when they’re shut down unexpectedly. I don’t know why you’d think they don’t. If the data has been read from somewhere (a socket maybe) and not fsynced, it’ll be lost. I agree that the system needs to be designed in such a way that this is a recoverable state, but I disagree with the ideas that ap…

By "lose data" I mean losing a confirmed write. That is, the server got a request to modify some state, and it responded to the request indicating success, but the change is later lost. Generally it's expected that databases will not lose confirmed writes, unless the application has explicitly made the decision that this is acceptable and opted into possible data loss to improve performance.

Re: Writing systemd units that stop gracefully before shutdown

#55

Post by vngzs »

AnssiH wrote: post #37
vngzs wrote: post #17

Earlier quoted context omitted.

I have the following unit file saved for that purpose: [Unit] # https://stackoverflow.com/questions/36729207/trigger-event-on-aws-ec2-instance-stop-terminate Description=unlink agent from remote server Before=shutdown.target [Service] Type=oneshot EnvironmentFile=-/etc/environment KillMode=none ExecStart=/bin/true ExecStop=/opt/service-name/shutdown-unlink RemainAfterExit=yes User=root [Install] WantedBy=multi-user.t…

The "Before=shutdown.target" is superfluous, that is a default dependency (as regular services obviously need to be shut down before system shutdown). "KillMode=none" shouldn't be necessary, unless your command leaves processes running that need to stay running (and in that case you have other problems, since those processes may not actually finish before the system is shut down). Systemd waits for your service to st…

Thanks. I ripped it from a colleague, and now I have some bugs to report.

Re: Writing systemd units that stop gracefully before shutdown

#56

Post by quickthrower2 »

dasil003 wrote: post #36
slivanes wrote: post #19

Earlier quoted context omitted.

Quoted post unavailable.

This is exactly the type of novel analogy that ML can not produce because the words don’t tend to appear next to each other in any corpus.

Until now!

Re: Writing systemd units that stop gracefully before shutdown

#57

Post by dasil003 »

quickthrower2 wrote: post #56
dasil003 wrote: post #36

Earlier quoted context omitted.

This is exactly the type of novel analogy that ML can not produce because the words don’t tend to appear next to each other in any corpus.

Until now!

Indeed! Every brilliant idea anyone ever posted online can now appear in ML-spew with all the context and wit stripped away.
Post reply on HN