How to create an RSYNC script with configuration in PERL

RSYNC is really simple, you can do an RSYNC by a single command line on your console. so, probably now your thinking.., what's the difference of my script? the difference is that I was able to create a script that has configuration files for you to re-use it every time you have different directories and servers to SYNC on.

First, you have to know the basic command line to do RSYNC. please see below command line.

rsync -crtz --bwlimit=56 --stats -e "ssh -l [USERNAME] -p [PORT]" [SOURCE DIRECTORY] [DESTINATION IP]:[DESTINATION DIRECTORY]

[USERNAME] is your account username to the destination server. your username should be set in passwordless to make it run in background.

[PORT] is the port that you will be using to connect to destination server.

[SOURCE DIRECTORY] is the source directory of the files you want to SYNC on.

[DESTINATION IP] is the server ip of the destination server.

[DESTINATION DIRECTORY] is the destination directory of the files you want to SYNC on.

Once you know how the RSYNC works in background, next is to know how we want our configuration to work. we should have a script that reads configuration in a flat file.

In creating configuration file, you should consider the comment syntax. this is to easily comment the lines you wanted to skip on or disregard in the configuration file.

Usually in linux, commented lines are preceded by pound sign "#" and that should be one of your condition to disregard. please see below for the sample reading of file then check for pound sign "#" using REGEX, then disregard.

open FHFILE, "< $strConf";
@arrData = <FHFILE>;
close FHFILE;

foreach $strData (@arrData)
{
      chomp $strData;
      if ($strData !~ /^\s*\#/)
      {
              print "okay here";
      }
}

The simple REGEX above checks if you have pound sign "#" on each lines and print okay if you don't.

The last thing you should know is how to pass the configuration file in command line. SYNTAX will be <YOUR SCRIPT> <space> <YOUR CONFIG FILE>. If you are new to LINUX, all parameters you passed on after your PERL SCRIPT will be catched by the ARRAY variable $ARGV. print that on your script with zero "0" as index will give you the first parameter you passed on, then succeeding index will the next parameter and so on and so forth.

Hope you were able to follow. Now we're ready to do the actual code.

Please see the recommended configuration FORMAT below that works on my RSYNC script. You will notice that I have a header which is setup to be the [DESTINATION IP] and [DESTINATION PORT]. This is for me to have it organize in a per SERVER configuration, then succeeding lines will be the actual SOURCE and DESTINATION directories.

- conf.txt as configuration file

[DESTINATION IP]:[DESTINATION PORT]
[SOURCE DIRECTORY 1]:[DESTINATION DIRECTORY 1]
[SOURCE DIRECTORY 2]:[DESTINATION DIRECTORY 2]
[SOURCE DIRECTORY n..]:[DESTINATION DIRECTORY n..]

- rsync.pl as your perl script

#!/usr/bin/perl

### reading conf files
$strConf = $ARGV[0];
if (-e $strConf)
{
    &do_rsync($strConf);
}
else
{
    print "conf: $strConf does not exist!!\n";
}
###

sub do_rsync
{
  my ($strConf) = @_;
  my ($ip, $port, $cmd, $strData, $strIncomingDir, $strOutGoingDir, $strReturn);
  my @arrData;

  open FHFILE, "< $strConf";
  @arrData = <FHFILE>;
  close FHFILE;

  $ip_port = shift @arrData;
  chomp $ip_port;

  # reading the ip and port as header
  while ($ip_port =~ /^\s*\#/)
  {
    $ip_port = shift @arrData;
    chomp $ip_port;
  }
  # end

  ($ip, $port) = split /\:/, $ip_port;

  if ($ip =~ /[^0-9|.]/)
  {
    print "ip not valid..\n\n";
  }
  elsif ($port =~ /[^0-9]/)
  {
    print "port not valid..\n\n";
  }
  else
  {
    # reading each line after the header
    foreach $strData (@arrData)
    {
      chomp $strData;

      if ($strData !~ /^\s*\#/)
      {
        ($strIncomingDir, $strOutGoingDir) = split /\:/, $strData;

        $cmd = "rsync -crtz --bwlimit=56 --stats -e \"ssh -l contents -p $port\" $strIncomingDir $ip:$strOutGoingDir";

        $strReturn = `$cmd`;
      }
    }
    # end
  }
}



1;

To execute, please see command line below. you can also put this in CRON, provided that the USERNAME is set to be PASSWORDLESS.

<path to the script>/rsync.pl <path to the config file>/conf.txt


Hope you like it!! Please leave a comment if you find this helpful. Thanks!!