Lsyncd: live file syncronization across multiple Linux servers

Lsyncd is a tool I discovered a few weeks ago. It is a synchronization server based primarily on Rsync. It is a server daemon that runs on the “master” server, and it can sync / mirror any file or directory changes within seconds into your “slaves” servers, you can have as many slave servers as you want. Lsyncd is constantly watching a local directory and monitoring file system changes using inotify / fsevents.

By default lsyncd uses rsync to send the data over the slave machines, however there are other ways to do it. It does not require you to build new filesystems or blockdevices, and does not harm your server I/O performance.

With lsyncd, you can have your web servers constantly syncronized with the same information on each one if you decide to load balancing using Nginx for example. Now I will show you How to install & configure Lysncd on CentOS & Ubuntu Linux.

Hardware and Software Requirements

  • 100Mbit private network (for normal websites)
  • 1Gbit private network (big storage / busy servers)

Software requirements for CentOS

yum -y install lua lua-devel pkgconfig gcc asciidoc

Software requirements for Ubuntu

  • Run all the installation process as root
apt-get update && apt-get install -y lua5.1 liblua5.1-dev pkg-config rsync asciidoc

Install lsyncd on CentOS

yum install lsyncd
service lsyncd start
chkconfig lsyncd on
mkdir /var/log/lsyncd

Install lsyncd on Ubuntu

apt-get install lsyncd
mkdir /var/log/lsyncd
mkdir /etc/lsyncd

Configure Lsyncd service

On CentOS

nano -w /etc/lsyncd.conf

On Ubuntu

nano -w /etc/lsyncd/lsyncd.conf.lua

On both CentOS and Ubuntu, paste this content inside the lsyncd.comf file:

settings {
logfile = "/var/log/lsyncd/lsyncd.log",
statusFile = "/var/log/lsyncd/lsyncd-status.log",
statusInterval = 10
}

-- Slave server configuration

sync {
default.rsync,
source="/var/www/",
target="IP:/var/www/",

rsync = {
compress = true,
acls = true,
verbose = true,
owner = true,
group = true,
perms = true,
rsh = "/usr/bin/ssh -p 22 -o StrictHostKeyChecking=no"
}
}

Important: you have to replace the variables ‘source’ with your real source path, and target with the real server IP and destination path. You may also need to tweak the rsh variable if you use another port different than 22.

Generate the SSH public keys

ssh-keygen -t rsa

Once it has completed, you will have 2 new files in ~/.ssh/ id_rsa and id_rsa.pub

cat ~/.ssh/ id_rsa.pub

Then copy the text content into your slave server and paste it inside this file:

nano -w ~/.ssh/authorized_keys

Test SSH connectivity

From the master server, ssh into each slave server, it may ask you to accept the key, just say “yes” and it will allow you to ssh into each slave without asking for root password.

Testing Lsyncd

Start the lsyncd service by running:

/etc/init.d/lsyncd start

Check that is is runnig ok:

/etc/init.d/lsyncd status

Add some files into /var/www directory and then check on the other server, as well as watching the log with tail:

tail -f /var/log/lsyncd/lsyncd.log

Other way to test it:

Open two ssh sessions, one for the master an the other for each slave, on all the slaves run this command:

watch ls -alh /var/www

On the master server, try adding a big file into /var/www and at the same time keep an eye on the slave servers. If sync is working fine, within seconds you will notice new file changes with watch command on each slave.

Read more information about Lsyncd

 

Popular search terms:

  • https://www scalescale com/tips/nginx/lsyncd-live-file-syncronization-linux/
  • lsyncd
  • lsyncd vs rsync
  • lsyncd config example ssh
0 (0)
Article Rating (No Votes)
Rate this article
Attachments
There are no attachments for this article.
Comments
There are no comments for this article. Be the first to post a comment.
Full Name
Email Address
Security Code Security Code
Related Articles RSS Feed
Przekazywanie portów TCP rinetd
Viewed 41543 times since Thu, May 24, 2018
RHEL: Reserved space on a ext2/ext3/ext4 filesystem
Viewed 3537 times since Sun, May 27, 2018
RHEL: Extending a multipath LUN
Viewed 3357 times since Sun, May 27, 2018
Learn Linux System Auditing with Auditd Tool on CentOS/RHEL
Viewed 3173 times since Fri, Apr 5, 2019
Linux - How to shutdown or reboot
Viewed 1476 times since Fri, Jun 8, 2018
RHCS6: Basic operations on clustered services
Viewed 2052 times since Sun, Jun 3, 2018
Kernel sysctl configuration file for Linux
Viewed 3227 times since Fri, Aug 3, 2018
RHEL: Crash kernel dumps configuration and analysis on RHEL 7
Viewed 6263 times since Sat, Jun 2, 2018
Manage SSH Key File With Passphrase
Viewed 1559 times since Tue, Mar 5, 2019
Modifying the inode count for an ext2/ext3/ext4 file system
Viewed 11557 times since Fri, Sep 18, 2020