Difference between revisions of "Rsync - synchronizes files and directories from one location to another"

From NAS-Central Buffalo - The Linkstation Wiki
Jump to: navigation, search
Line 33: Line 33:
 
From the Yahoo! Linkstation General Group <ref> From the Yahoo! Linkstation General Group: [http://tech.groups.yahoo.com/group/LinkStation_General/message/5376 rsync binaries anyone / SSH without password as root INTO LinkStation]</ref>
 
From the Yahoo! Linkstation General Group <ref> From the Yahoo! Linkstation General Group: [http://tech.groups.yahoo.com/group/LinkStation_General/message/5376 rsync binaries anyone / SSH without password as root INTO LinkStation]</ref>
 
  wget http://ls.jcedata.net/rsync
 
  wget http://ls.jcedata.net/rsync
  chmod a x rsync
+
  chmod a+x rsync
 
  cp rsync /usr/bin
 
  cp rsync /usr/bin
 
==FreeLink==
 
==FreeLink==
Line 52: Line 52:
 
=Examples=
 
=Examples=
 
*Create a backup named with todays date (formatted yyyy-mm-dd)
 
*Create a backup named with todays date (formatted yyyy-mm-dd)
  rsync -a /SOURCE/ /DEST/`date %Y-%m-%d`/
+
  rsync -a /SOURCE/ /DEST/`date +%Y-%m-%d`/
  
 
*A script to create a backup, named by date, which will save space by creating hard links to files which are already backed up.  It requires a symbolic link to the most recently created backup dir (similar to the last line of the script)
 
*A script to create a backup, named by date, which will save space by creating hard links to files which are already backed up.  It requires a symbolic link to the most recently created backup dir (similar to the last line of the script)
 
  #!/bin/sh
 
  #!/bin/sh
  BACKUP_DATE=`date %Y-%m-%d`
+
  BACKUP_DATE=`date +%Y-%m-%d`
 
  rsync -a --delete --link-dest=/DEST/most-recent-backup /SOURCE/ /DEST/$BACKUP_DATE/
 
  rsync -a --delete --link-dest=/DEST/most-recent-backup /SOURCE/ /DEST/$BACKUP_DATE/
 
  rm /BACKUP/most-recent-backup
 
  rm /BACKUP/most-recent-backup
Line 73: Line 73:
  
  
*Here is another one-liner which, using cron, gives me a weekly backup of my LS-HG's data partition hda3 onto a USB hard drive
+
*Here is another one-liner which, using cron, gives me a weekly backup of my LS-HG's data partition hda3 onto a USB hard drive & logs everything.  Use it either as a one-liner, or edit into separate lines and use as a bash script.  It does the following:
 +
**prints a header with a timestamp
 +
**executes the backup & shows the rsync log, along with a record of what files are updated, and the average transfer speed of the backup
 +
**prints a footer with timestamp, and then a blank line
 +
**all the output is appended, so that the log is cumulative (I also have logrotation enabled)
 +
echo ========FreeLink /dev/hda3 JFS backup started $(date) ====== >> /var/log/weeklybackuplog.txt  ;
 +
rsync -av /mnt/share/shareddirs/ /mnt/share/usb0/backups/shareddirs >> /var/log/weeklybackuplog.txt ;
 +
echo  ======end of backup run at $(date) =======  >> /var/log/weeklybackuplog.txt ;
 +
echo >>  /var/log/weeklybackuplog.txt
 +
 
 +
= References =
 +
<references/>
 +
 
 +
{{Stubs}}

Revision as of 05:46, 29 April 2007

Newrsynclogo.jpg

Contents

Background

rsync[1] is a free software computer program for Unix systems which synchronizes files and directories from one location to another while minimizing data transfer using delta encoding when appropriate. An important feature of rsync not found in most similar programs/protocols is that the mirroring takes place with only one transmission in each direction.[2] [3]

rsync can copy or display directory contents and copy files, optionally using compression and recursion.

rsyncd, the rsync protocol daemon, uses the default TCP port of 873. rsync can also be used to synchronize local directories, or via a remote shell such as RSH or SSH. In the latter case, the rsync client executable must be installed on the near as well as the far host (the computer running the remote shell daemon). There also exists a utility called rdiff[4], which can be used for incremental backups.

The Mac OS X filesystem has a special version, rsyncX[5][6], which allow transferring resource forks. To run rsync on Microsoft Windows, the Cygwin package is necessary[7] to provide the expected system interfaces. A package combination is available that include rsync, cygwin, and an installer, making it easier and more familiar to Windows users[8].

There are several well written tutorials on using rsync[9][10][11][12]

If you are interested in fully automated, ssh-secured backups from one machine to another, then see the 3rd example below, and read this easy Debian Admin HOW-TO on setting up ssh/rsa keys so that you computer can securely back itself up to your Linkstation while you are not there.

Installation

Compile from source

On any distribution FreeLink or OpenLink

  1. Make sure you have installed the Precompiled C development environment, running on the LS first.
  2. Get the source, make and install.
wget http://samba.org/ftp/rsync/rsync-2.6.9.tar.gz 
tar xfzv rsync-2.6.9.tar.gz 
cd rsync-2.6.9
./configure 
make 
su root 
make install

Raw Binaries

PowerPC

From the Yahoo! Linkstation General Group [13]

wget http://ls.jcedata.net/rsync
chmod a+x rsync
cp rsync /usr/bin

FreeLink

Use apt-get to install rsync

apt-get install rsync

OpenLink (Ipkg)

PowerPC

ipkg install rsync

MIPSel

Alexander Skwar has created a fairly extensive selection of Ipkg packages for the MIPSel (LS2) LinkStation, Install Ipkg and enable his feed:

ipkg install rsync

Examples

  • Create a backup named with todays date (formatted yyyy-mm-dd)
rsync -a /SOURCE/ /DEST/`date +%Y-%m-%d`/
  • A script to create a backup, named by date, which will save space by creating hard links to files which are already backed up. It requires a symbolic link to the most recently created backup dir (similar to the last line of the script)
#!/bin/sh
BACKUP_DATE=`date +%Y-%m-%d`
rsync -a --delete --link-dest=/DEST/most-recent-backup /SOURCE/ /DEST/$BACKUP_DATE/
rm /BACKUP/most-recent-backup
ln -s /DEST/$BACKUP_DATE /DEST/most-recent-backup


  • This one-liner is incorporated into my crontab, giving me a daily backup of my KuroboxHG's mp3's as it rips them, and a log of the backup. Use it either as a one-liner, or edit into separate lines and use as a bash script. It does the following:
    • prints a header with a timestamp
    • executes a secure, ssh-keyed machine-to-machine transfer, gives an rsync log, along with a record of what files are updated, and the average transfer speed of the backup
    • prints a footer with timestamp, and then a blank line
    • all the output is appended, so that the log is cumulative (I also have logrotation enabled)
echo ========kurohg mediarippertunes backup started $(date) ====== >> /var/log/musicbackuplog.txt  ;
rsync -av --rsh=ssh /mnt/mediarippertunes/ 10.0.1.10:/mnt/share/mediarippertunes >> /var/log/musicbackuplog.txt ;
echo  ======== end of backup run at $(date) ========  >> /var/log/musicbackuplog.txt ;
echo    >>  /var/log/musicbackuplog.txt     


  • Here is another one-liner which, using cron, gives me a weekly backup of my LS-HG's data partition hda3 onto a USB hard drive & logs everything. Use it either as a one-liner, or edit into separate lines and use as a bash script. It does the following:
    • prints a header with a timestamp
    • executes the backup & shows the rsync log, along with a record of what files are updated, and the average transfer speed of the backup
    • prints a footer with timestamp, and then a blank line
    • all the output is appended, so that the log is cumulative (I also have logrotation enabled)
echo ========FreeLink /dev/hda3 JFS backup started $(date) ====== >> /var/log/weeklybackuplog.txt  ;
rsync -av /mnt/share/shareddirs/ /mnt/share/usb0/backups/shareddirs >> /var/log/weeklybackuplog.txt ;
echo  ======end of backup run at $(date) =======  >> /var/log/weeklybackuplog.txt ;
echo >>  /var/log/weeklybackuplog.txt

References

  1. Wikipedia: rsync
  2. rsync homepage
  3. rsync algorithm
  4. rdiff-backup homepage
  5. RsyncX - Frontend for rsync under Mac OS X
  6. Fixing rsync on MacOS X 10.4 (Tiger) - http://www.onthenet.com.au/~q/rsync/
  7. Rsync for Windows - using Cygwin
  8. NasBackup rsync Windows GUI
  9. Tutorial: Using rsync
  10. Tutorial: Mirroring with rsync
  11. Tutorial: Backing up files with rsync
  12. Tutorials (with screenshots) for setup of Rsync on Windows, Rsync on Linux/Unix/BSD and Rsync on Mac OS X
  13. From the Yahoo! Linkstation General Group: rsync binaries anyone / SSH without password as root INTO LinkStation
Link.png This article is currently a stub. You can help this Wiki by expanding it

. This template will categorize articles that include it into Category:Stubs.