Lsyncd: live file syncronization across multiple Linux servers

Lsyncd is a tool I discovered a few weeks ago. It is a synchronization server based primarily on Rsync. It is a server daemon that runs on the “master” server, and it can sync / mirror any file or directory changes within seconds into your “slaves” servers, you can have as many slave servers as you want. Lsyncd is constantly watching a local directory and monitoring file system changes using inotify / fsevents.

By default lsyncd uses rsync to send the data over the slave machines, however there are other ways to do it. It does not require you to build new filesystems or blockdevices, and does not harm your server I/O performance.

With lsyncd, you can have your web servers constantly syncronized with the same information on each one if you decide to load balancing using Nginx for example. Now I will show you How to install & configure Lysncd on CentOS & Ubuntu Linux.

Hardware and Software Requirements

  • 100Mbit private network (for normal websites)
  • 1Gbit private network (big storage / busy servers)

Software requirements for CentOS

yum -y install lua lua-devel pkgconfig gcc asciidoc

Software requirements for Ubuntu

  • Run all the installation process as root
apt-get update && apt-get install -y lua5.1 liblua5.1-dev pkg-config rsync asciidoc

Install lsyncd on CentOS

yum install lsyncd
service lsyncd start
chkconfig lsyncd on
mkdir /var/log/lsyncd

Install lsyncd on Ubuntu

apt-get install lsyncd
mkdir /var/log/lsyncd
mkdir /etc/lsyncd

Configure Lsyncd service

On CentOS

nano -w /etc/lsyncd.conf

On Ubuntu

nano -w /etc/lsyncd/lsyncd.conf.lua

On both CentOS and Ubuntu, paste this content inside the lsyncd.comf file:

settings {
logfile = "/var/log/lsyncd/lsyncd.log",
statusFile = "/var/log/lsyncd/lsyncd-status.log",
statusInterval = 10
}

-- Slave server configuration

sync {
default.rsync,
source="/var/www/",
target="IP:/var/www/",

rsync = {
compress = true,
acls = true,
verbose = true,
owner = true,
group = true,
perms = true,
rsh = "/usr/bin/ssh -p 22 -o StrictHostKeyChecking=no"
}
}

Important: you have to replace the variables ‘source’ with your real source path, and target with the real server IP and destination path. You may also need to tweak the rsh variable if you use another port different than 22.

Generate the SSH public keys

ssh-keygen -t rsa

Once it has completed, you will have 2 new files in ~/.ssh/ id_rsa and id_rsa.pub

cat ~/.ssh/ id_rsa.pub

Then copy the text content into your slave server and paste it inside this file:

nano -w ~/.ssh/authorized_keys

Test SSH connectivity

From the master server, ssh into each slave server, it may ask you to accept the key, just say “yes” and it will allow you to ssh into each slave without asking for root password.

Testing Lsyncd

Start the lsyncd service by running:

/etc/init.d/lsyncd start

Check that is is runnig ok:

/etc/init.d/lsyncd status

Add some files into /var/www directory and then check on the other server, as well as watching the log with tail:

tail -f /var/log/lsyncd/lsyncd.log

Other way to test it:

Open two ssh sessions, one for the master an the other for each slave, on all the slaves run this command:

watch ls -alh /var/www

On the master server, try adding a big file into /var/www and at the same time keep an eye on the slave servers. If sync is working fine, within seconds you will notice new file changes with watch command on each slave.

Read more information about Lsyncd

 

Popular search terms:

  • https://www scalescale com/tips/nginx/lsyncd-live-file-syncronization-linux/
  • lsyncd
  • lsyncd vs rsync
  • lsyncd config example ssh
0 (0)
Article Rating (No Votes)
Rate this article
Attachments
There are no attachments for this article.
Comments
There are no comments for this article. Be the first to post a comment.
Full Name
Email Address
Security Code Security Code
Related Articles RSS Feed
LVM: Recovering Physical Volume Metadata
Viewed 12709 times since Sat, Jun 2, 2018
Install Security Patches or Updates Automatically on CentOS and RHEL
Viewed 1397 times since Fri, Oct 26, 2018
Check a Website Availability from the Linux Command Line
Viewed 5965 times since Mon, Feb 18, 2019
Linux PAM configuration that allows or deny login via the sshd server
Viewed 1449 times since Wed, Oct 3, 2018
stunnel Securing telnet connections with stunnel
Viewed 1107 times since Sun, Dec 6, 2020
Top 25 Best Linux Performance Monitoring and Debugging Tools
Viewed 6480 times since Sun, Sep 30, 2018
RHEL7: How to get started with Firewalld.
Viewed 11287 times since Wed, May 22, 2019
Find All Large Files On A Linux System
Viewed 1716 times since Mon, Oct 29, 2018
Df command in Linux not updating actual diskspace, wrong data
Viewed 2432 times since Wed, May 30, 2018
ZFS: Verify/change properties of a zfs filesystem
Viewed 2196 times since Sun, Jun 3, 2018