systemd Auto-restart a crashed service in systemd

Image of Mattias Geniar

Mattias Geniar, January 13, 2020

Follow me on Twitter as @mattiasgeniar

Systemd allows you to configure a service so that it automatically restarts in case it’s crashed.

Take a typical unit file that looks like this.

$ cat /etc/systemd/system/yourdaemon.service
[Unit]
Description=Your Daemon
After=network-online.target
Wants=network-online.target systemd-networkd-wait-online.service

[Service]
ExecStart=/path/to/daemon

[Install]
WantedBy=multi-user.target

Most unit files are longer, but this gives you the gist of it. In the above example, if your daemon would crash or be killed, systemd would leave it alone.

You can however let systemd auto-restart it in case it fails or is accidentally killed. To do so, you can add the Restart option to the [Service] stanza.

$ cat /etc/systemd/system/yourdaemon.service
[Unit]
Description=Your Daemon
After=network-online.target
Wants=network-online.target systemd-networkd-wait-online.service

StartLimitIntervalSec=500
StartLimitBurst=5

[Service]
Restart=on-failure
RestartSec=5s

ExecStart=/path/to/daemon

[Install]
WantedBy=multi-user.target

The above will react to anything that stops your daemon: a code exception, someone that does kill -9 <pid>, … as soon as your daemon stops, systemd will restart it in 5 seconds.

In this example, there are also StartLimitIntervalSec and StartLimitBurst directives in the [Unit] section. This prevents a failing service from being restarted every 5 seconds. This will give it 5 attempts, if it still fails, systemd will stop trying to start the service.

(Note: if you change your systemd unit file, make sure to run systemctl daemon-reload to reload the changes.)

If you ask for the status of your daemon after it’s been killed, systemd will show activating (auto-restart).

$ systemctl status yourdaemon
● yourdaemon.service - Your Daemon
   Loaded: loaded (/etc/systemd/system/yourdaemon.service; enabled; vendor preset: enabled)
   Active: activating (auto-restart) (Result: signal) since Mon 2020-01-13 09:07:41 UTC; 4s ago
  Process: 27165 ExecStart=/path/to/daemon (code=killed)
  Main PID: 27165 (code=killed, signal=KILL)

Give it a few seconds, and you’ll see the daemon was automatically restarted by systemd.

$ systemctl status yourdaemon
● yourdaemon.service - Your Daemon
   Loaded: loaded (/etc/systemd/system/yourdaemon.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2020-01-13 09:07:46 UTC; 6min ago

Pretty useful if you have a buggy service that’s safe to just restart on failure!

0 (0)
Article Rating (No Votes)
Rate this article
Attachments
There are no attachments for this article.
Comments
There are no comments for this article. Be the first to post a comment.
Full Name
Email Address
Security Code Security Code
Related Articles RSS Feed
Use Fail2ban to Secure Your Server
Viewed 11092 times since Fri, Jul 5, 2019
Logowanie za pomocą kluczy Secure Shell
Viewed 2028 times since Thu, May 24, 2018
zabbix linux How to solve apache error No space left on device: Cannot create SSLMutex
Viewed 771 times since Wed, Nov 11, 2020
watchdog How to restart a process out of crontab on a Linux/Unix
Viewed 4508 times since Tue, Jul 31, 2018
RHCS6: Mirror/unmirror a GFS2 volume
Viewed 4302 times since Sun, Jun 3, 2018
Modifying the inode count for an ext2/ext3/ext4 file system
Viewed 10687 times since Fri, Sep 18, 2020
RHEL: Resize/disable /dev/shm filesystem
Viewed 11006 times since Sun, May 27, 2018
Learn Linux System Auditing with Auditd Tool on CentOS/RHEL
Viewed 2869 times since Fri, Apr 5, 2019
Enabling or disabling a repository using Red Hat Subscription Management
Viewed 3615 times since Mon, Oct 29, 2018
Linux - How to monitor CPU usage
Viewed 5134 times since Fri, Jun 8, 2018