systemd Auto-restart a crashed service in systemd
Mattias Geniar, January 13, 2020
Follow me on Twitter as @mattiasgeniar
Systemd allows you to configure a service so that it automatically restarts in case it’s crashed.
Take a typical unit file that looks like this.
$ cat /etc/systemd/system/yourdaemon.service [Unit] Description=Your Daemon After=network-online.target Wants=network-online.target systemd-networkd-wait-online.service [Service] ExecStart=/path/to/daemon [Install] WantedBy=multi-user.target
Most unit files are longer, but this gives you the gist of it. In the above example, if your daemon would crash or be killed, systemd would leave it alone.
You can however let systemd auto-restart it in case it fails or is accidentally killed. To do so, you can add the
Restart option to the
$ cat /etc/systemd/system/yourdaemon.service [Unit] Description=Your Daemon After=network-online.target Wants=network-online.target systemd-networkd-wait-online.service StartLimitIntervalSec=500 StartLimitBurst=5 [Service] Restart=on-failure RestartSec=5s ExecStart=/path/to/daemon [Install] WantedBy=multi-user.target
The above will react to anything that stops your daemon: a code exception, someone that does
kill -9 <pid>, … as soon as your daemon stops, systemd will restart it in 5 seconds.
In this example, there are also
StartLimitBurst directives in the
[Unit] section. This prevents a failing service from being restarted every 5 seconds. This will give it 5 attempts, if it still fails, systemd will stop trying to start the service.
(Note: if you change your systemd unit file, make sure to run
systemctl daemon-reload to reload the changes.)
If you ask for the status of your daemon after it’s been killed, systemd will show
$ systemctl status yourdaemon ● yourdaemon.service - Your Daemon Loaded: loaded (/etc/systemd/system/yourdaemon.service; enabled; vendor preset: enabled) Active: activating (auto-restart) (Result: signal) since Mon 2020-01-13 09:07:41 UTC; 4s ago Process: 27165 ExecStart=/path/to/daemon (code=killed) Main PID: 27165 (code=killed, signal=KILL)
Give it a few seconds, and you’ll see the daemon was automatically restarted by systemd.
$ systemctl status yourdaemon ● yourdaemon.service - Your Daemon Loaded: loaded (/etc/systemd/system/yourdaemon.service; enabled; vendor preset: enabled) Active: active (running) since Mon 2020-01-13 09:07:46 UTC; 6min ago
Pretty useful if you have a buggy service that’s safe to just restart on failure!