Lsyncd: live file syncronization across multiple Linux servers

Lsyncd is a tool I discovered a few weeks ago. It is a synchronization server based primarily on Rsync. It is a server daemon that runs on the “master” server, and it can sync / mirror any file or directory changes within seconds into your “slaves” servers, you can have as many slave servers as you want. Lsyncd is constantly watching a local directory and monitoring file system changes using inotify / fsevents.

By default lsyncd uses rsync to send the data over the slave machines, however there are other ways to do it. It does not require you to build new filesystems or blockdevices, and does not harm your server I/O performance.

With lsyncd, you can have your web servers constantly syncronized with the same information on each one if you decide to load balancing using Nginx for example. Now I will show you How to install & configure Lysncd on CentOS & Ubuntu Linux.

Hardware and Software Requirements

  • 100Mbit private network (for normal websites)
  • 1Gbit private network (big storage / busy servers)

Software requirements for CentOS

yum -y install lua lua-devel pkgconfig gcc asciidoc

Software requirements for Ubuntu

  • Run all the installation process as root
apt-get update && apt-get install -y lua5.1 liblua5.1-dev pkg-config rsync asciidoc

Install lsyncd on CentOS

yum install lsyncd
service lsyncd start
chkconfig lsyncd on
mkdir /var/log/lsyncd

Install lsyncd on Ubuntu

apt-get install lsyncd
mkdir /var/log/lsyncd
mkdir /etc/lsyncd

Configure Lsyncd service

On CentOS

nano -w /etc/lsyncd.conf

On Ubuntu

nano -w /etc/lsyncd/lsyncd.conf.lua

On both CentOS and Ubuntu, paste this content inside the lsyncd.comf file:

settings {
logfile = "/var/log/lsyncd/lsyncd.log",
statusFile = "/var/log/lsyncd/lsyncd-status.log",
statusInterval = 10
}

-- Slave server configuration

sync {
default.rsync,
source="/var/www/",
target="IP:/var/www/",

rsync = {
compress = true,
acls = true,
verbose = true,
owner = true,
group = true,
perms = true,
rsh = "/usr/bin/ssh -p 22 -o StrictHostKeyChecking=no"
}
}

Important: you have to replace the variables ‘source’ with your real source path, and target with the real server IP and destination path. You may also need to tweak the rsh variable if you use another port different than 22.

Generate the SSH public keys

ssh-keygen -t rsa

Once it has completed, you will have 2 new files in ~/.ssh/ id_rsa and id_rsa.pub

cat ~/.ssh/ id_rsa.pub

Then copy the text content into your slave server and paste it inside this file:

nano -w ~/.ssh/authorized_keys

Test SSH connectivity

From the master server, ssh into each slave server, it may ask you to accept the key, just say “yes” and it will allow you to ssh into each slave without asking for root password.

Testing Lsyncd

Start the lsyncd service by running:

/etc/init.d/lsyncd start

Check that is is runnig ok:

/etc/init.d/lsyncd status

Add some files into /var/www directory and then check on the other server, as well as watching the log with tail:

tail -f /var/log/lsyncd/lsyncd.log

Other way to test it:

Open two ssh sessions, one for the master an the other for each slave, on all the slaves run this command:

watch ls -alh /var/www

On the master server, try adding a big file into /var/www and at the same time keep an eye on the slave servers. If sync is working fine, within seconds you will notice new file changes with watch command on each slave.

Read more information about Lsyncd

 

Popular search terms:

  • https://www scalescale com/tips/nginx/lsyncd-live-file-syncronization-linux/
  • lsyncd
  • lsyncd vs rsync
  • lsyncd config example ssh
0 (0)
Article Rating (No Votes)
Rate this article
Attachments
There are no attachments for this article.
Comments
There are no comments for this article. Be the first to post a comment.
Full Name
Email Address
Security Code Security Code
Related Articles RSS Feed
RHEL: Rename a network interface on RHEL 7
Viewed 6878 times since Sat, Jun 2, 2018
How To Add Swap Space on Ubuntu 16.04
Viewed 1120 times since Fri, Jun 8, 2018
RHEL: Handling SCSI disks
Viewed 10090 times since Sun, May 27, 2018
linux aix Killing a process and all of its descendants
Viewed 1421 times since Tue, May 5, 2020
Get UUID of Hard Disks [Update]
Viewed 1179 times since Tue, Jul 17, 2018
List usernames instead of uids with the ps command for long usernames
Viewed 985 times since Wed, Jul 25, 2018
Automatic YUM Updates with Yum-cron
Viewed 9526 times since Fri, Oct 26, 2018
ubuntu How to Reset Forgotten Passwords in Ubuntu 16.04
Viewed 545 times since Tue, Dec 8, 2020
ZPOOL: Grow a zpool by adding new device(s)
Viewed 4044 times since Sun, Jun 3, 2018
RHEL: How to change a USER/GROUP UID/GID and all owned files
Viewed 12542 times since Sat, Jun 2, 2018