Lsyncd: live file syncronization across multiple Linux servers -

Lsyncd is a tool I discovered a few weeks ago. It is a synchronization server based primarily on Rsync. It is a server daemon that runs on the “master” server, and it can sync / mirror any file or directory changes within seconds into your “slaves” servers, you can have as many slave servers as you want. Lsyncd is constantly watching a local directory and monitoring file system changes using inotify / fsevents.

By default lsyncd uses rsync to send the data over the slave machines, however there are other ways to do it. It does not require you to build new filesystems or blockdevices, and does not harm your server I/O performance.

With lsyncd, you can have your web servers constantly syncronized with the same information on each one if you decide to load balancing using Nginx for example. Now I will show you How to install & configure Lysncd on CentOS & Ubuntu Linux.

Hardware and Software Requirements

100Mbit private network (for normal websites)
1Gbit private network (big storage / busy servers)

Software requirements for CentOS

Run all the installation process as root
Install EPEL repo for CentOS

yum -y install lua lua-devel pkgconfig gcc asciidoc

Software requirements for Ubuntu

Run all the installation process as root

apt-get update && apt-get install -y lua5.1 liblua5.1-dev pkg-config rsync asciidoc

Install lsyncd on CentOS

yum install lsyncd
service lsyncd start
chkconfig lsyncd on
mkdir /var/log/lsyncd

Install lsyncd on Ubuntu

apt-get install lsyncd
mkdir /var/log/lsyncd
mkdir /etc/lsyncd

Configure Lsyncd service

On CentOS

nano -w /etc/lsyncd.conf

On Ubuntu

nano -w /etc/lsyncd/lsyncd.conf.lua

On both CentOS and Ubuntu, paste this content inside the lsyncd.comf file:

settings {
logfile = "/var/log/lsyncd/lsyncd.log",
statusFile = "/var/log/lsyncd/lsyncd-status.log",
statusInterval = 10
}

-- Slave server configuration

sync {
default.rsync,
source="/var/www/",
target="IP:/var/www/",

rsync = {
compress = true,
acls = true,
verbose = true,
owner = true,
group = true,
perms = true,
rsh = "/usr/bin/ssh -p 22 -o StrictHostKeyChecking=no"
}
}

Important: you have to replace the variables ‘source’ with your real source path, and target with the real server IP and destination path. You may also need to tweak the rsh variable if you use another port different than 22.

Generate the SSH public keys

ssh-keygen -t rsa

Once it has completed, you will have 2 new files in ~/.ssh/ id_rsa and id_rsa.pub

cat ~/.ssh/ id_rsa.pub

Then copy the text content into your slave server and paste it inside this file:

nano -w ~/.ssh/authorized_keys

Test SSH connectivity

From the master server, ssh into each slave server, it may ask you to accept the key, just say “yes” and it will allow you to ssh into each slave without asking for root password.

Testing Lsyncd

Start the lsyncd service by running:

/etc/init.d/lsyncd start

Check that is is runnig ok:

/etc/init.d/lsyncd status

Add some files into /var/www directory and then check on the other server, as well as watching the log with tail:

tail -f /var/log/lsyncd/lsyncd.log

Other way to test it:

Open two ssh sessions, one for the master an the other for each slave, on all the slaves run this command:

watch ls -alh /var/www

On the master server, try adding a big file into /var/www and at the same time keep an eye on the slave servers. If sync is working fine, within seconds you will notice new file changes with watch command on each slave.

Popular search terms:

https://www scalescale com/tips/nginx/lsyncd-live-file-syncronization-linux/
lsyncd
lsyncd vs rsync
lsyncd config example ssh