Lsyncd: live file syncronization across multiple Linux servers

Lsyncd is a tool I discovered a few weeks ago. It is a synchronization server based primarily on Rsync. It is a server daemon that runs on the “master” server, and it can sync / mirror any file or directory changes within seconds into your “slaves” servers, you can have as many slave servers as you want. Lsyncd is constantly watching a local directory and monitoring file system changes using inotify / fsevents.

By default lsyncd uses rsync to send the data over the slave machines, however there are other ways to do it. It does not require you to build new filesystems or blockdevices, and does not harm your server I/O performance.

With lsyncd, you can have your web servers constantly syncronized with the same information on each one if you decide to load balancing using Nginx for example. Now I will show you How to install & configure Lysncd on CentOS & Ubuntu Linux.

Hardware and Software Requirements

  • 100Mbit private network (for normal websites)
  • 1Gbit private network (big storage / busy servers)

Software requirements for CentOS

yum -y install lua lua-devel pkgconfig gcc asciidoc

Software requirements for Ubuntu

  • Run all the installation process as root
apt-get update && apt-get install -y lua5.1 liblua5.1-dev pkg-config rsync asciidoc

Install lsyncd on CentOS

yum install lsyncd
service lsyncd start
chkconfig lsyncd on
mkdir /var/log/lsyncd

Install lsyncd on Ubuntu

apt-get install lsyncd
mkdir /var/log/lsyncd
mkdir /etc/lsyncd

Configure Lsyncd service

On CentOS

nano -w /etc/lsyncd.conf

On Ubuntu

nano -w /etc/lsyncd/lsyncd.conf.lua

On both CentOS and Ubuntu, paste this content inside the lsyncd.comf file:

settings {
logfile = "/var/log/lsyncd/lsyncd.log",
statusFile = "/var/log/lsyncd/lsyncd-status.log",
statusInterval = 10
}

-- Slave server configuration

sync {
default.rsync,
source="/var/www/",
target="IP:/var/www/",

rsync = {
compress = true,
acls = true,
verbose = true,
owner = true,
group = true,
perms = true,
rsh = "/usr/bin/ssh -p 22 -o StrictHostKeyChecking=no"
}
}

Important: you have to replace the variables ‘source’ with your real source path, and target with the real server IP and destination path. You may also need to tweak the rsh variable if you use another port different than 22.

Generate the SSH public keys

ssh-keygen -t rsa

Once it has completed, you will have 2 new files in ~/.ssh/ id_rsa and id_rsa.pub

cat ~/.ssh/ id_rsa.pub

Then copy the text content into your slave server and paste it inside this file:

nano -w ~/.ssh/authorized_keys

Test SSH connectivity

From the master server, ssh into each slave server, it may ask you to accept the key, just say “yes” and it will allow you to ssh into each slave without asking for root password.

Testing Lsyncd

Start the lsyncd service by running:

/etc/init.d/lsyncd start

Check that is is runnig ok:

/etc/init.d/lsyncd status

Add some files into /var/www directory and then check on the other server, as well as watching the log with tail:

tail -f /var/log/lsyncd/lsyncd.log

Other way to test it:

Open two ssh sessions, one for the master an the other for each slave, on all the slaves run this command:

watch ls -alh /var/www

On the master server, try adding a big file into /var/www and at the same time keep an eye on the slave servers. If sync is working fine, within seconds you will notice new file changes with watch command on each slave.

Read more information about Lsyncd

 

Popular search terms:

  • https://www scalescale com/tips/nginx/lsyncd-live-file-syncronization-linux/
  • lsyncd
  • lsyncd vs rsync
  • lsyncd config example ssh
0 (0)
Article Rating (No Votes)
Rate this article
Attachments
There are no attachments for this article.
Comments
There are no comments for this article. Be the first to post a comment.
Full Name
Email Address
Security Code Security Code
Related Articles RSS Feed
Transform XML to CSV Format | Unix String Pattern Manipulation The Ugly Way
Viewed 5804 times since Sun, Jan 9, 2022
Fedora 32: Simple Local File-Sharing with Samba CIFS Linux
Viewed 8930 times since Sun, Dec 6, 2020
Epoch & Unix Timestamp Conversion Tools
Viewed 63083 times since Fri, Jun 22, 2018
RHCS: Install a two-node basic cluster
Viewed 10066 times since Sun, Jun 3, 2018
How to retrieve and change partition’s UUID Universally Unique Identifier on linux
Viewed 3178 times since Tue, Jul 17, 2018
RHEL: Displaying/setting kernel parameters - ’sysctl’
Viewed 3130 times since Sat, Jun 2, 2018
ZFS: Grow/Shrink an existing zfs filesystem
Viewed 6446 times since Sun, Jun 3, 2018
LVM: Reduce an existing Logical Volume / Filesystem
Viewed 3653 times since Sat, Jun 2, 2018
SSH ProxyCommand example: Going through one host to reach another server
Viewed 13781 times since Tue, Aug 6, 2019
RHEL: Extending a multipath LUN
Viewed 5173 times since Sun, May 27, 2018