Difference between revisions of "Create a perfect backup of the LinkStation filesystem"

From NAS-Central Buffalo - The Linkstation Wiki
Jump to: navigation, search
m (Unix-style Backup)
m (Improved Regular Shared Data Backup)
 
(6 intermediate revisions by 6 users not shown)
Line 7: Line 7:
 
== Backing up the Linkstation ==
 
== Backing up the Linkstation ==
  
== Very Basic Backup ==
+
There are many different ways to back up a LinkStation. They not only vary in complexity, but also in completeness and safety of the resulting backup. The following section lists a few. In general the naive definition of the term ''backup'' is used: A copy of some data, which might or might not be complete and consistent and which might or might not work in case of emergency. A real backup, on the other hand, is a process, in which consistently and completely copying data is just one building block, while e.g. organizational issues are equally important.
  
If you don't want to read the contents of hda1 and simply want to backup and later restore it you don't need to fix_ext2_magic because dd copies low level without interpreting the data. Use the command:
+
The ''perfect backup'' in the title of this article is therefore misleading. You will not get perfect backups with the following procedures. You will "just" get copies of some data.
  
: '''dd if=/dev/hda1 of=/mnt/whatever/hda1.bin'''
 
  
No compression, saved on the data partition, you can copy it to any other PC later. This command should work even when the system is running and files are locked. If you want to make the image better compressable you could do:
+
== Very basic Backup via Dumping the Partition ==
  
: '''dd if=/dev/zero of=/fillmeup'''
+
{{Postit|Inconsistent Backups|Inconsistent backups are a common reason why a backup is unusable and restore fails or partly fails. A copy of a partition takes some time. At some point in time management information from the file system (i-nodes, directories, etc.) is written to the copy. Later (or earlier) in time the related data is written. Now, if the data (or management information) is changed between this two points in time on the copied versions of them don't match any more on the backup. The file system in the backup is inconsistent and corrupt and there is a good chance that such a backup will not or only partly work after it is restored. So much for ''a perfect backup''...}}
 +
One way to duplicate the data on a LinkStation partition is to dump the data of the whole partition into a file, e.g. by using '''dd'''(1). When doing so it is rather clever to not have any process writing to the partition or deleting something on the partition which should be copied. Because this easily leads to inconsistent copies. It would be best to unmount the partition first, or at least to mount it read-only. However, there are vital directories on a Linux system which must be available for writing or the system doesn't run. 
  
and delete this file later on. This has the advantage that unused parts become filled with zeroes which allows better compression.
+
Anyhow, if you don't want to read the contents of hda1 file-by-file and want a simple copy of the partition you can try to copy the whole partition. One tool to do this is '''dd'''(1). If you don't want "to look into" the copied data you don't need to run [[fix_ext2_magic]] because '''dd''' copies low level without interpreting the data. Use the command:
  
Be careful what you type ! Specifying the wrong "of" parameter can easily wipe out data you would like to keep
+
dd if=/dev/hda1 of=/mnt/whatever/hda1.bin
 +
 
 +
The command can run even when the system is running and files are locked - however, the resulting copy ''hda1.bin'' has a high probability of being inconsistent and unusable.
 +
 
 +
The above command does not apply any compression. But that can be easily added, e.g. by piping the data through '''compress'''(1), '''zcat'''(1), or whatever.
 +
If you want to make the image better compressible you could add a file with all-zeros to the partition first. Doing so replaces the random data on the previously unused disk blocks with zeros. Run
 +
 
 +
dd if=/dev/zero of=/fillmeup
 +
 
 +
and delete the file ''/fillmeup'' afterwards, before copying the partition to the backup. This has the advantage that unused parts become filled with zeroes which allows better compression.
 +
 
 +
Be careful what you type! Specifying the wrong "of" parameter can easily wipe out data you would like to keep.
  
 
== USB or External Hard Drive Backup ==
 
== USB or External Hard Drive Backup ==
Line 25: Line 36:
 
As soon as you connect an external memory device to the LinkStation it get's mounted as /mnt/usbdisk1 or /mnt/usbdisk2. You could now copy the image you created above to the device:
 
As soon as you connect an external memory device to the LinkStation it get's mounted as /mnt/usbdisk1 or /mnt/usbdisk2. You could now copy the image you created above to the device:
  
: '''cp /mnt/whatever/hda1.bin /mnt/usbdisk1'''
+
cp /mnt/whatever/hda1.bin /mnt/usbdisk1
  
 
In case you use a quite small usb stick you can compress the data using:
 
In case you use a quite small usb stick you can compress the data using:
  
: '''gzip -9 /mnt/whatever/hda1.bin'''
+
gzip -9 /mnt/whatever/hda1.bin
  
 
This commands shrinks the image of my system partition to 95MB which easily fit's on a 128MB usb stick. If you gzip you have to add a ".gz" to the copy command above.
 
This commands shrinks the image of my system partition to 95MB which easily fit's on a 128MB usb stick. If you gzip you have to add a ".gz" to the copy command above.
Line 35: Line 46:
 
Do:
 
Do:
  
: '''umount /mnt/usbdisk1'''
+
umount /mnt/usbdisk1
  
 
after you are finished.
 
after you are finished.
Line 48: Line 59:
  
 
You want a true incremental backup, i.e. after one full backup only daily changes shall be transferred to the LS? Install [[rsync]] and use it in the LS backup script:
 
You want a true incremental backup, i.e. after one full backup only daily changes shall be transferred to the LS? Install [[rsync]] and use it in the LS backup script:
* Binary for the PPC LS see [http://forum.linkstationwiki.net/forum/index.php?action=vthread&forum=2&topic=114 this forum thread]
+
* Binary for the PPC LS see {{forumpost|2|114|this forum thread}}
* Binary for the MIPS LS see [http://forum.linkstationwiki.net/forum/index.php?action=vthread&forum=3&topic=149 this forum thread]
+
* Binary for the MIPS LS see {{forumpost|3|149|this forum thread}}
* Replace the existing file ''/www/cgi-bin/do-backup.pl'' with the modified script posted by weikai on [http://forum.linkstationwiki.net/forum/index.php?action=vthread&forum=2&topic=114 this forum thread]
+
* Replace the existing file ''/www/cgi-bin/do-backup.pl'' with the modified script posted by weikai on [http://forum.nas-central.org/index.php?action=vthread&forum=2&topic=114 this forum thread]
  
 
You want an incremental backup with history, i.e. keep replaced and deleted files? For information on these more advanced [[rsync]] uses (like creating snapshots using hard links and [[w:cron|cron]]) read [http://www.mikerubel.org/computers/rsync_snapshots/ rsync snapshots]
 
You want an incremental backup with history, i.e. keep replaced and deleted files? For information on these more advanced [[rsync]] uses (like creating snapshots using hard links and [[w:cron|cron]]) read [http://www.mikerubel.org/computers/rsync_snapshots/ rsync snapshots]

Latest revision as of 17:32, 11 September 2007

This article Based on work by haberschnasel, nix, and frontalot. Originally by frontalot. at Linkstationwiki.org

Contents

Backing up the Linkstation

There are many different ways to back up a LinkStation. They not only vary in complexity, but also in completeness and safety of the resulting backup. The following section lists a few. In general the naive definition of the term backup is used: A copy of some data, which might or might not be complete and consistent and which might or might not work in case of emergency. A real backup, on the other hand, is a process, in which consistently and completely copying data is just one building block, while e.g. organizational issues are equally important.

The perfect backup in the title of this article is therefore misleading. You will not get perfect backups with the following procedures. You will "just" get copies of some data.


Very basic Backup via Dumping the Partition

Inconsistent Backups
Bar.png
Inconsistent backups are a common reason why a backup is unusable and restore fails or partly fails. A copy of a partition takes some time. At some point in time management information from the file system (i-nodes, directories, etc.) is written to the copy. Later (or earlier) in time the related data is written. Now, if the data (or management information) is changed between this two points in time on the copied versions of them don't match any more on the backup. The file system in the backup is inconsistent and corrupt and there is a good chance that such a backup will not or only partly work after it is restored. So much for a perfect backup...


One way to duplicate the data on a LinkStation partition is to dump the data of the whole partition into a file, e.g. by using dd(1). When doing so it is rather clever to not have any process writing to the partition or deleting something on the partition which should be copied. Because this easily leads to inconsistent copies. It would be best to unmount the partition first, or at least to mount it read-only. However, there are vital directories on a Linux system which must be available for writing or the system doesn't run.

Anyhow, if you don't want to read the contents of hda1 file-by-file and want a simple copy of the partition you can try to copy the whole partition. One tool to do this is dd(1). If you don't want "to look into" the copied data you don't need to run fix_ext2_magic because dd copies low level without interpreting the data. Use the command:

dd if=/dev/hda1 of=/mnt/whatever/hda1.bin

The command can run even when the system is running and files are locked - however, the resulting copy hda1.bin has a high probability of being inconsistent and unusable.

The above command does not apply any compression. But that can be easily added, e.g. by piping the data through compress(1), zcat(1), or whatever. If you want to make the image better compressible you could add a file with all-zeros to the partition first. Doing so replaces the random data on the previously unused disk blocks with zeros. Run

dd if=/dev/zero of=/fillmeup

and delete the file /fillmeup afterwards, before copying the partition to the backup. This has the advantage that unused parts become filled with zeroes which allows better compression.

Be careful what you type! Specifying the wrong "of" parameter can easily wipe out data you would like to keep.

USB or External Hard Drive Backup

As soon as you connect an external memory device to the LinkStation it get's mounted as /mnt/usbdisk1 or /mnt/usbdisk2. You could now copy the image you created above to the device:

cp /mnt/whatever/hda1.bin /mnt/usbdisk1

In case you use a quite small usb stick you can compress the data using:

gzip -9 /mnt/whatever/hda1.bin

This commands shrinks the image of my system partition to 95MB which easily fit's on a 128MB usb stick. If you gzip you have to add a ".gz" to the copy command above.

Do:

umount /mnt/usbdisk1

after you are finished.

Regular Shared Data Backup

The LinkStation comes with a simple backup system for performing regular backups of the data in the shared directories to an USB disk. To configure it use the LS web interface. Goto Maintenance -> Disk Backup and configure the backup as you desire.

The LS' own backup system is rather slow, so you might want to consider the following methods.

Improved Regular Shared Data Backup

You want a true incremental backup, i.e. after one full backup only daily changes shall be transferred to the LS? Install rsync and use it in the LS backup script:

You want an incremental backup with history, i.e. keep replaced and deleted files? For information on these more advanced rsync uses (like creating snapshots using hard links and cron) read rsync snapshots

Unix-style Backup

You find rsync not suitable or risky? Install cpio. Write / setup a cron job. In this job use find to identify files which need to be backed up. E.g., for an incremental backup search for all files newer than the last backup. Pipe the output of find through cpio. Let cpio. write its output to an external USB disk or some other mounted network partition on a physically separate device.

Use a backup rotation schema like GFS (grandfather-father-son). Store backups off-site.