Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Systemd.timer, an Alternative to Cron

andrewpillar.com

Post reply on HN
Print view View original

131–140 of 155 posts

Re: Systemd.timer, an Alternative to Cron

#131

Post by egberts1 »

mariusor wrote: post #106
egberts1 wrote: post #100

Earlier quoted context omitted.

I think a link prominently placed on their website would be helpful. Also systemd documentation isn't compliance with: * Tutorial * Explaination * How To * Reference For instance, I want to know which part of systemd uses `SYSTEMD_DEBUG`. Hold on ... I am digging thru the Git repo. Screw the systemd manual: Here's my environment variable reference guide for debugging systemd (that is NOT documented by systemd) https:…

> I think a link prominently placed on their website would be helpful. It's such a weird disconnect seeing people that can't go outside of the mindset that everything is a web application. Systemd ships their documentation (most of what you asked for, actually) with the package, and their environment variables description can be found in /usr/share/doc/systemd/ENVIRONMENT.md (at least on my machine). When you're deal…

Documentation and Systemd will remain a weird disconnect.

until it address the 4-ways Divio documentation methods.

https://documentation.divio.com/

Re: Systemd.timer, an Alternative to Cron

#132

Post by ilyt »

marbu wrote: post #127
pikrzyszto wrote: post #90

Earlier quoted context omitted.

Why would it care about stracing? It was never too slow for me. A "bad" performance might be caused by ex. consistency checking. Benchmarks like these don't tell much - you can't reduce software performance to one number. > It's DB implementation is utter garbage. It doesn't even organize files by time or really anything useful. That sounds interesting, can you please elaborate on the internal structure of journald f…

I don't have a negative experience with journald file format. That said, the format is documented here: https://systemd.io/JOURNAL_FILE_FORMAT/ On the other hand I agree that journald seems to call open syscall more often than I would originally expect, which can be a problem for some edge cases[1], but I don't consider this to be a real problem. [1] https://blog.marbu.eu/posts/2022-11-20-ebpf-journald/

The problem is:

If cache is hot, you have hundreds of MBs of systemd logs littering your buffer cache, while it could be used for something more useful

If cache is cold simple operations like systemctl status take up to tens of seconds, especially on loaded servers. Example:

    # time systemctl status influxdb >/dev/null

    real 0m11.913s
    user 0m0.013s
    sys 0m0.531s
It doesn't even keep index of "last file where app wrote logs" which causes above.

Re: Systemd.timer, an Alternative to Cron

#133

Post by ilyt »

jeroenhd wrote: post #74
ilyt wrote: post #34

Earlier quoted context omitted.

Right but you basically never run the service ran by timer manually, so on average it's just adding noise to equation. Having option for separate timer is all and well (after all you might want to use it to periodically start service, for example start some sync service after midnight) but for cron-like behaviour it should really have all in one option, perhaps by just adding [Timer] block in .service or vice versa

I can't count the times I've had to debug cron scripts by copying the cron command line and running it through sudo because the script ran into an edge case, only to find out I didn't have the exact cron environment so manual runs didn't trigger the bug. I much prefer the systemd solution. As an added bonus, systemd services allow for easy configuration of things like sandboxes and resource limits that would need to…

And I said "add option to have it in one file" (instead of "only have it in single file") for exactly that reason

Re: Systemd.timer, an Alternative to Cron

#134

Post by egberts1 »

mariusor wrote: post #116
egberts1 wrote: post #108

Earlier quoted context omitted.

Right. But still, I had to do some digging and this time it only came from its source code for description, references, and still no How To.

> still no How To. How to what? systemd umbrella contains a lot of functionality, just like the detractors keep on telling us. If you found a way to improve the systemd documentation you should probably reach out to the devs: https://github.com/systemd/systemd/tree/main/docs

Still no debugging HOWTO, much less (but still poorly documented) DEBUG-related environment variables?

If you want people to troubleshoot your stuff (which is arguably the #1 documentation to start with), gotta start someplace (which is for most of us is agrep'ing the code, or save yourself a step and look at my link).

Or the RedHat/IBM systemd development team can quit doing that terse but cryptic error code output.

Or the documentation team can start covering these error codes.

But having to do any form of "agrep" is a sign of poor ... everything, unless formenting job security is what the ultimate goal is.

Re: Systemd.timer, an Alternative to Cron

#135

Post by jerf »

lazide wrote: post #88
ciupicri wrote: post #66

Earlier quoted context omitted.

/etc/cron.{hourly,daily,weekly,monthly} are just some convenience shortcuts. No one is forcing you to use them.

When a bunch of important system functionality is done using them, I’m required to know they exist. /etc/crontab is no different. It’s not that much different than systemd in that sense. Especially since the /etc/cron.whatever special rules like files with dots in them get ignored (ugh)

Another fun one I've gotten bitten by a lot is that the /etc/cron.* directories are run sequentially, so if someone who doesn't know this put some long running process in it, ideally with a name that sorts first, it blocks everything else in that directory. (I personally know this, obviously, since I'm saying it now, but I've been bitten by the other people who do not realize this and do not think about how putting a process that may run for three hours at the beginning of cron.hourly means that the other processes may not run for that long.)

Re: Systemd.timer, an Alternative to Cron

#137

Post by mike_hearn »

luckycharms810 wrote: post #37

While I agree Systemd timers aren't as ergonomic to set up - there are some benefits that I would basically never give up at this point. * Running the service on command ( this is the best way to troubleshoot issues with your slightly different environments / cron's version of shell. I've spent many minutes over my career - setting the crontab one minute and the future and waiting for it to run / fail ) * Easily insp…

Speaking on OnFailure, one of the unfortunate aspects of systemd timers (which are otherwise quite nice) is you need to do extra work to get proper emails when something fails. Here's how I do this:

Define email-unit-status@.service

    [Unit]
    Description=Email status for %i to root
    After=network.target

    [Service]
    Type=oneshot
    ExecStart=email-unit-status %i
Define email-unit-status (a script)

    #!/usr/bin/env bash
    set -eu
     
    dest="admin@whatever.com"
     
    userflag=
    if [[ $HOME == /home/* ]]; then
      userflag=--user
    fi
     
    html=$(SYSTEMD_COLORS=1 systemctl $userflag status --full "$1" | ansi2html -w -c)
     
    /usr/sbin/sendmail -t 
    Subject: SystemD unit failure alert: $1
    Content-Transfer-Encoding: 8bit
    Content-Type: text/html; charset=UTF-8
    Mime-Version: 1.0
     
    $html
    ERRMAIL
Then you can add to a service:

    OnFailure=email-unit-status@%n
You will need to install the ansi2html tool. This stuff should come with systemd really.

Re: Systemd.timer, an Alternative to Cron

#138

Post by exabrial »

kaba0 wrote: post #122
exabrial wrote: post #112

> Job output will automatically be written to systemd-journald This is a bad thing, not something I’d boast about. I actually really enjoy using systemd. Being able to start a process with complete isolation without heavyweight docker downloading mystery meat off the internet is a huge boon. However, one thing systemd is logging. Jounalctl sucks. Grep, cat, etc, work infinitesimally better.

You can redirect all your logs to traditional string-based ones if you want though.

Precisely. That's exactly what we end up doing

Re: Systemd.timer, an Alternative to Cron

#140

Post by egberts1 »

kaba0 wrote: post #121
egberts1 wrote: post #98

Earlier quoted context omitted.

Failsafe? I'm still experiencing file corruption regularly with journald on various distros. Cannot recall corruption nor unreadability with syslog.

Then you have some corrupted hardware. Also, syslog wasn’t capable of logging during the initial stages of boot, which is pretty important.

That's what dmesg is for.
Post reply on HN