Create a perfect backup of the LinkStation filesystem - Revision history

Ramuk: /* Improved Regular Shared Data Backup */

Ramuk — Tue, 11 Sep 2007 17:32:06 GMT

Improved Regular Shared Data Backup

← Older revision		Revision as of 17:32, 11 September 2007
Line 59:		Line 59:

	You want a true incremental backup, i.e. after one full backup only daily changes shall be transferred to the LS? Install [[rsync]] and use it in the LS backup script:		You want a true incremental backup, i.e. after one full backup only daily changes shall be transferred to the LS? Install [[rsync]] and use it in the LS backup script:
-	* Binary for the PPC LS see ~~[http://forum.nas-central.org/forum/index.php?action=vthread&forum=~~2~~&topic=~~114 this forum thread]	+	* Binary for the PPC LS see {{forumpost\|2\|114\|this forum thread}}
-	* Binary for the MIPS LS see ~~[http://forum.nas-central.org/forum/index.php?action=vthread&forum=~~3~~&topic=~~149 this forum thread]	+	* Binary for the MIPS LS see {{forumpost\|3\|149\|this forum thread}}
	* Replace the existing file ''/www/cgi-bin/do-backup.pl'' with the modified script posted by weikai on [http://forum.nas-central.org/index.php?action=vthread&forum=2&topic=114 this forum thread]		* Replace the existing file ''/www/cgi-bin/do-backup.pl'' with the modified script posted by weikai on [http://forum.nas-central.org/index.php?action=vthread&forum=2&topic=114 this forum thread]

Jonli447 at 02:41, 11 September 2007

Jonli447 — Tue, 11 Sep 2007 02:41:59 GMT

← Older revision		Revision as of 02:41, 11 September 2007
Line 59:		Line 59:

	You want a true incremental backup, i.e. after one full backup only daily changes shall be transferred to the LS? Install [[rsync]] and use it in the LS backup script:		You want a true incremental backup, i.e. after one full backup only daily changes shall be transferred to the LS? Install [[rsync]] and use it in the LS backup script:
-	* Binary for the PPC LS see [http://forum.~~linkstationwiki~~.~~net~~/forum/index.php?action=vthread&forum=2&topic=114 this forum thread]	+	* Binary for the PPC LS see [http://forum.nas-central.org/forum/index.php?action=vthread&forum=2&topic=114 this forum thread]
-	* Binary for the MIPS LS see [http://forum.~~linkstationwiki~~.~~net~~/forum/index.php?action=vthread&forum=3&topic=149 this forum thread]	+	* Binary for the MIPS LS see [http://forum.nas-central.org/forum/index.php?action=vthread&forum=3&topic=149 this forum thread]
-	* Replace the existing file ''/www/cgi-bin/do-backup.pl'' with the modified script posted by weikai on [http://forum.~~linkstationwiki~~.~~net~~/index.php?action=vthread&forum=2&topic=114 this forum thread]	+	* Replace the existing file ''/www/cgi-bin/do-backup.pl'' with the modified script posted by weikai on [http://forum.nas-central.org/index.php?action=vthread&forum=2&topic=114 this forum thread]

	You want an incremental backup with history, i.e. keep replaced and deleted files? For information on these more advanced [[rsync]] uses (like creating snapshots using hard links and [[w:cron\|cron]]) read [http://www.mikerubel.org/computers/rsync_snapshots/ rsync snapshots]		You want an incremental backup with history, i.e. keep replaced and deleted files? For information on these more advanced [[rsync]] uses (like creating snapshots using hard links and [[w:cron\|cron]]) read [http://www.mikerubel.org/computers/rsync_snapshots/ rsync snapshots]

77.128.184.6: /* Very basic Backup via Dumping the Partition */

77.128.184.6 — Sat, 03 Mar 2007 22:59:48 GMT

Very basic Backup via Dumping the Partition

← Older revision		Revision as of 22:59, 3 March 2007
Line 14:		Line 14:
	== Very basic Backup via Dumping the Partition ==		== Very basic Backup via Dumping the Partition ==

-	~~[[Image:Bar.png~~\|~~frame\| '''~~Inconsistent Backups~~'''<br>~~Inconsistent backups are a common reason why a backup is unusable and restore fails or partly fails. A copy of a partition takes some time. At some point in time management information from the file system (i-nodes, directories, etc.) is written to the copy. Later (or earlier) in time the related data is written. Now, if the data (or management information) is changed between this two points in time on the copied versions of them don't match any more on the backup. The file system in the backup is inconsistent and corrupt and there is a good chance that such a backup will not or only partly work after it is restored. So much for ''a perfect backup''... ]]	+	{{Postit\|Inconsistent Backups\|Inconsistent backups are a common reason why a backup is unusable and restore fails or partly fails. A copy of a partition takes some time. At some point in time management information from the file system (i-nodes, directories, etc.) is written to the copy. Later (or earlier) in time the related data is written. Now, if the data (or management information) is changed between this two points in time on the copied versions of them don't match any more on the backup. The file system in the backup is inconsistent and corrupt and there is a good chance that such a backup will not or only partly work after it is restored. So much for ''a perfect backup''...}}
	One way to duplicate the data on a LinkStation partition is to dump the data of the whole partition into a file, e.g. by using '''dd'''(1). When doing so it is rather clever to not have any process writing to the partition or deleting something on the partition which should be copied. Because this easily leads to inconsistent copies. It would be best to unmount the partition first, or at least to mount it read-only. However, there are vital directories on a Linux system which must be available for writing or the system doesn't run.		One way to duplicate the data on a LinkStation partition is to dump the data of the whole partition into a file, e.g. by using '''dd'''(1). When doing so it is rather clever to not have any process writing to the partition or deleting something on the partition which should be copied. Because this easily leads to inconsistent copies. It would be best to unmount the partition first, or at least to mount it read-only. However, there are vital directories on a Linux system which must be available for writing or the system doesn't run.

82.44.211.224: fixed url typo

82.44.211.224 — Tue, 27 Feb 2007 23:07:21 GMT

fixed url typo

← Older revision		Revision as of 23:07, 27 February 2007
Line 61:		Line 61:
	* Binary for the PPC LS see [http://forum.linkstationwiki.net/forum/index.php?action=vthread&forum=2&topic=114 this forum thread]		* Binary for the PPC LS see [http://forum.linkstationwiki.net/forum/index.php?action=vthread&forum=2&topic=114 this forum thread]
	* Binary for the MIPS LS see [http://forum.linkstationwiki.net/forum/index.php?action=vthread&forum=3&topic=149 this forum thread]		* Binary for the MIPS LS see [http://forum.linkstationwiki.net/forum/index.php?action=vthread&forum=3&topic=149 this forum thread]
-	* Replace the existing file ''/www/cgi-bin/do-backup.pl'' with the modified script posted by weikai on [http://forum.linkstationwiki.net~~/forum~~/index.php?action=vthread&forum=2&topic=114 this forum thread]	+	* Replace the existing file ''/www/cgi-bin/do-backup.pl'' with the modified script posted by weikai on [http://forum.linkstationwiki.net/index.php?action=vthread&forum=2&topic=114 this forum thread]

	You want an incremental backup with history, i.e. keep replaced and deleted files? For information on these more advanced [[rsync]] uses (like creating snapshots using hard links and [[w:cron\|cron]]) read [http://www.mikerubel.org/computers/rsync_snapshots/ rsync snapshots]		You want an incremental backup with history, i.e. keep replaced and deleted files? For information on these more advanced [[rsync]] uses (like creating snapshots using hard links and [[w:cron\|cron]]) read [http://www.mikerubel.org/computers/rsync_snapshots/ rsync snapshots]

208.124.57.221: /* Very basic Backup via Dumping the Partition */

208.124.57.221 — Tue, 20 Feb 2007 16:45:11 GMT

Very basic Backup via Dumping the Partition

← Older revision		Revision as of 16:45, 20 February 2007
Line 15:		Line 15:

	[[Image:Bar.png\|frame\| '''Inconsistent Backups'''<br>Inconsistent backups are a common reason why a backup is unusable and restore fails or partly fails. A copy of a partition takes some time. At some point in time management information from the file system (i-nodes, directories, etc.) is written to the copy. Later (or earlier) in time the related data is written. Now, if the data (or management information) is changed between this two points in time on the copied versions of them don't match any more on the backup. The file system in the backup is inconsistent and corrupt and there is a good chance that such a backup will not or only partly work after it is restored. So much for ''a perfect backup''... ]]		[[Image:Bar.png\|frame\| '''Inconsistent Backups'''<br>Inconsistent backups are a common reason why a backup is unusable and restore fails or partly fails. A copy of a partition takes some time. At some point in time management information from the file system (i-nodes, directories, etc.) is written to the copy. Later (or earlier) in time the related data is written. Now, if the data (or management information) is changed between this two points in time on the copied versions of them don't match any more on the backup. The file system in the backup is inconsistent and corrupt and there is a good chance that such a backup will not or only partly work after it is restored. So much for ''a perfect backup''... ]]
-	One way to duplicate the data on a LinkStation partition is to dump the data of the whole partition into a file, e.g. by using '''dd'''(1). When doing so it is rather clever to not have any process writing to he partition or deleting something on the partition which should be copied. Because this easily leads to inconsistent copies. It would be best to unmount the partition first, or at least to mount it read-only. However, there are vital directories on a Linux system which must be available for writing or the system doesn't run.	+	One way to duplicate the data on a LinkStation partition is to dump the data of the whole partition into a file, e.g. by using '''dd'''(1). When doing so it is rather clever to not have any process writing to the partition or deleting something on the partition which should be copied. Because this easily leads to inconsistent copies. It would be best to unmount the partition first, or at least to mount it read-only. However, there are vital directories on a Linux system which must be available for writing or the system doesn't run.

	Anyhow, if you don't want to read the contents of hda1 file-by-file and want a simple copy of the partition you can try to copy the whole partition. One tool to do this is '''dd'''(1). If you don't want "to look into" the copied data you don't need to run [[fix_ext2_magic]] because '''dd''' copies low level without interpreting the data. Use the command:		Anyhow, if you don't want to read the contents of hda1 file-by-file and want a simple copy of the partition you can try to copy the whole partition. One tool to do this is '''dd'''(1). If you don't want "to look into" the copied data you don't need to run [[fix_ext2_magic]] because '''dd''' copies low level without interpreting the data. Use the command:

89.52.209.173 at 21:51, 6 November 2006

89.52.209.173 — Mon, 06 Nov 2006 21:51:39 GMT

← Older revision		Revision as of 21:51, 6 November 2006
Line 7:		Line 7:
	== Backing up the Linkstation ==		== Backing up the Linkstation ==

-	~~== Very Basic Backup ==~~	+	There are many different ways to back up a LinkStation. They not only vary in complexity, but also in completeness and safety of the resulting backup. The following section lists a few. In general the naive definition of the term ''backup'' is used: A copy of some data, which might or might not be complete and consistent and which might or might not work in case of emergency. A real backup, on the other hand, is a process, in which consistently and completely copying data is just one building block, while e.g. organizational issues are equally important.

-	~~If you don~~'~~t want to read the contents of hda1 and simply want to~~ backup ~~and later restore it you don~~'~~t need to [[fix_ext2_magic]] because dd copies low level without interpreting~~ the ~~data~~. ~~Use~~ the ~~command:~~	+	The ''perfect backup'' in the title of this article is therefore misleading. You will not get perfect backups with the following procedures. You will "just" get copies of some data.

-	~~: '''dd if=/dev/hda1 of=/mnt/whatever/hda1.bin'''~~

-	~~No compression, saved on~~ the ~~data partition, you can copy it to any other PC later. This command should work even when the system is running and files are locked. If you want to make the image better compressable you could do:~~	+	== Very basic Backup via Dumping the Partition ==

-	: '''dd if~~=/dev/zero~~ of~~=/fillmeup~~'''	+	[[Image:Bar.png\|frame\| '''Inconsistent Backups'''<br>Inconsistent backups are a common reason why a backup is unusable and restore fails or partly fails. A copy of a partition takes some time. At some point in time management information from the file system (i-nodes, directories, etc.) is written to the copy. Later (or earlier) in time the related data is written. Now, if the data (or management information) is changed between this two points in time on the copied versions of them don't match any more on the backup. The file system in the backup is inconsistent and corrupt and there is a good chance that such a backup will not or only partly work after it is restored. So much for ''a perfect backup''... ]]
		+	One way to duplicate the data on a LinkStation partition is to dump the data of the whole partition into a file, e.g. by using '''dd'''(1). When doing so it is rather clever to not have any process writing to he partition or deleting something on the partition which should be copied. Because this easily leads to inconsistent copies. It would be best to unmount the partition first, or at least to mount it read-only. However, there are vital directories on a Linux system which must be available for writing or the system doesn't run.

-	and ~~delete~~ this ~~file later on~~. ~~This has~~ the ~~advantage that unused parts become filled with zeroes which allows better compression~~.	+	Anyhow, if you don't want to read the contents of hda1 file-by-file and want a simple copy of the partition you can try to copy the whole partition. One tool to do this is '''dd'''(1). If you don't want "to look into" the copied data you don't need to run [[fix_ext2_magic]] because '''dd''' copies low level without interpreting the data. Use the command:

-	Be careful what you type ! Specifying the wrong "of" parameter can easily wipe out data you would like to keep	+	dd if=/dev/hda1 of=/mnt/whatever/hda1.bin
		+
		+	The command can run even when the system is running and files are locked - however, the resulting copy ''hda1.bin'' has a high probability of being inconsistent and unusable.
		+
		+	The above command does not apply any compression. But that can be easily added, e.g. by piping the data through '''compress'''(1), '''zcat'''(1), or whatever.
		+	If you want to make the image better compressible you could add a file with all-zeros to the partition first. Doing so replaces the random data on the previously unused disk blocks with zeros. Run
		+
		+	dd if=/dev/zero of=/fillmeup
		+
		+	and delete the file ''/fillmeup'' afterwards, before copying the partition to the backup. This has the advantage that unused parts become filled with zeroes which allows better compression.
		+
		+	Be careful what you type! Specifying the wrong "of" parameter can easily wipe out data you would like to keep.

	== USB or External Hard Drive Backup ==		== USB or External Hard Drive Backup ==
Line 25:		Line 36:
	As soon as you connect an external memory device to the LinkStation it get's mounted as /mnt/usbdisk1 or /mnt/usbdisk2. You could now copy the image you created above to the device:		As soon as you connect an external memory device to the LinkStation it get's mounted as /mnt/usbdisk1 or /mnt/usbdisk2. You could now copy the image you created above to the device:

-	~~: '''~~cp /mnt/whatever/hda1.bin /mnt/usbdisk1~~'''~~	+	cp /mnt/whatever/hda1.bin /mnt/usbdisk1

	In case you use a quite small usb stick you can compress the data using:		In case you use a quite small usb stick you can compress the data using:

-	~~: '''~~gzip -9 /mnt/whatever/hda1.bin~~'''~~	+	gzip -9 /mnt/whatever/hda1.bin

	This commands shrinks the image of my system partition to 95MB which easily fit's on a 128MB usb stick. If you gzip you have to add a ".gz" to the copy command above.		This commands shrinks the image of my system partition to 95MB which easily fit's on a 128MB usb stick. If you gzip you have to add a ".gz" to the copy command above.
Line 35:		Line 46:
	Do:		Do:

-	~~: '''~~umount /mnt/usbdisk1~~'''~~	+	umount /mnt/usbdisk1

	after you are finished.		after you are finished.

89.52.220.74 at 12:46, 5 November 2006

89.52.220.74 — Sun, 05 Nov 2006 12:46:13 GMT

← Older revision		Revision as of 12:46, 5 November 2006
Line 9:		Line 9:
	== Very Basic Backup ==		== Very Basic Backup ==

-	If you don't want to read the contents of hda1 and simply want to backup and later restore it you don't need to fix_ext2_magic because dd copies low level without interpreting the data. Use the command:	+	If you don't want to read the contents of hda1 and simply want to backup and later restore it you don't need to [[fix_ext2_magic]] because dd copies low level without interpreting the data. Use the command:

	: '''dd if=/dev/hda1 of=/mnt/whatever/hda1.bin'''		: '''dd if=/dev/hda1 of=/mnt/whatever/hda1.bin'''

Ramuk: /* Unix-style Backup */

Ramuk — Tue, 19 Sep 2006 23:59:36 GMT

Unix-style Backup

← Older revision		Revision as of 23:59, 19 September 2006
Line 56:		Line 56:
	== Unix-style Backup ==		== Unix-style Backup ==

-	You find [[rsync]] not suitable or risky? Install @@cpio@@. Write / setup a cron job. In this job use @@find@@ to identify files which need to be backed up. E.g., for an incremental backup search for all files newer than the last backup. Pipe the output of @@find@@ through [[w:cpio\|cpio]]. Let cpio write its output to an external USB disk or some other mounted network partition on a '''physically separate''' device.	+	You find [[rsync]] not suitable or risky? Install [[w:cpio\|cpio]]. Write / setup a cron job. In this job use [[w:find\|find]] to identify files which need to be backed up. E.g., for an incremental backup search for all files newer than the last backup. Pipe the output of [[w:find\|find]] through [[w:cpio\|cpio]]. Let [[w:cpio\|cpio]]. write its output to an external USB disk or some other mounted network partition on a '''physically separate''' device.

	Use a backup rotation schema like GFS (grandfather-father-son). Store backups off-site.		Use a backup rotation schema like GFS (grandfather-father-son). Store backups off-site.

Ramuk at 23:58, 19 September 2006

Ramuk — Tue, 19 Sep 2006 23:58:07 GMT

← Older revision		Revision as of 23:58, 19 September 2006
Line 47:		Line 47:
	== Improved Regular Shared Data Backup ==		== Improved Regular Shared Data Backup ==

-	You want a true incremental backup, i.e. after one full backup only daily changes shall be transferred to the LS? Install @@rsync@@ and use it in the LS backup script:	+	You want a true incremental backup, i.e. after one full backup only daily changes shall be transferred to the LS? Install [[rsync]] and use it in the LS backup script:
	* Binary for the PPC LS see [http://forum.linkstationwiki.net/forum/index.php?action=vthread&forum=2&topic=114 this forum thread]		* Binary for the PPC LS see [http://forum.linkstationwiki.net/forum/index.php?action=vthread&forum=2&topic=114 this forum thread]
	* Binary for the MIPS LS see [http://forum.linkstationwiki.net/forum/index.php?action=vthread&forum=3&topic=149 this forum thread]		* Binary for the MIPS LS see [http://forum.linkstationwiki.net/forum/index.php?action=vthread&forum=3&topic=149 this forum thread]
	* Replace the existing file ''/www/cgi-bin/do-backup.pl'' with the modified script posted by weikai on [http://forum.linkstationwiki.net/forum/index.php?action=vthread&forum=2&topic=114 this forum thread]		* Replace the existing file ''/www/cgi-bin/do-backup.pl'' with the modified script posted by weikai on [http://forum.linkstationwiki.net/forum/index.php?action=vthread&forum=2&topic=114 this forum thread]

-	You want an incremental backup with history, i.e. keep replaced and deleted files? For information on these more advanced @@rsync@@ uses (like creating snapshots using hard links and @@cron@@) read [http://www.mikerubel.org/computers/rsync_snapshots/ rsync snapshots]	+	You want an incremental backup with history, i.e. keep replaced and deleted files? For information on these more advanced [[rsync]] uses (like creating snapshots using hard links and [[w:cron\|cron]]) read [http://www.mikerubel.org/computers/rsync_snapshots/ rsync snapshots]

	== Unix-style Backup ==		== Unix-style Backup ==

-	You find @@rsync@@ not suitable or risky? Install @@cpio@@. Write / setup a cron job. In this job use @@find@@ to identify files which need to be backed up. E.g., for an incremental backup search for all files newer than the last backup. Pipe the output of @@find@@ through @@cpio@@. Let @@cpio@@ write its output to an external USB disk or some other mounted network partition on a '''physically separate''' device.	+	You find [[rsync]] not suitable or risky? Install @@cpio@@. Write / setup a cron job. In this job use @@find@@ to identify files which need to be backed up. E.g., for an incremental backup search for all files newer than the last backup. Pipe the output of @@find@@ through [[w:cpio\|cpio]]. Let cpio write its output to an external USB disk or some other mounted network partition on a '''physically separate''' device.

	Use a backup rotation schema like GFS (grandfather-father-son). Store backups off-site.		Use a backup rotation schema like GFS (grandfather-father-son). Store backups off-site.

Haberschnasel: /* Improved Regular Shared Data Backup */

Haberschnasel — Sat, 12 Aug 2006 13:42:17 GMT

Improved Regular Shared Data Backup

← Older revision		Revision as of 13:42, 12 August 2006
Line 48:		Line 48:

	You want a true incremental backup, i.e. after one full backup only daily changes shall be transferred to the LS? Install @@rsync@@ and use it in the LS backup script:		You want a true incremental backup, i.e. after one full backup only daily changes shall be transferred to the LS? Install @@rsync@@ and use it in the LS backup script:
-	* Binary for the PPC LS see [http://linkstationwiki.~~org~~/forum/index.php?action=vthread&forum=2&topic=114 this forum thread]	+	* Binary for the PPC LS see [http://forum.linkstationwiki.net/forum/index.php?action=vthread&forum=2&topic=114 this forum thread]
-	* Binary for the MIPS LS see [http://linkstationwiki.~~org~~/forum/index.php?action=vthread&forum=3&topic=149 this forum thread]	+	* Binary for the MIPS LS see [http://forum.linkstationwiki.net/forum/index.php?action=vthread&forum=3&topic=149 this forum thread]
-	* Replace the existing file ''/www/cgi-bin/do-backup.pl'' with the modified script posted by weikai on [http://linkstationwiki.~~org~~/forum/index.php?action=vthread&forum=2&topic=114 this forum thread]	+	* Replace the existing file ''/www/cgi-bin/do-backup.pl'' with the modified script posted by weikai on [http://forum.linkstationwiki.net/forum/index.php?action=vthread&forum=2&topic=114 this forum thread]

	You want an incremental backup with history, i.e. keep replaced and deleted files? For information on these more advanced @@rsync@@ uses (like creating snapshots using hard links and @@cron@@) read [http://www.mikerubel.org/computers/rsync_snapshots/ rsync snapshots]		You want an incremental backup with history, i.e. keep replaced and deleted files? For information on these more advanced @@rsync@@ uses (like creating snapshots using hard links and @@cron@@) read [http://www.mikerubel.org/computers/rsync_snapshots/ rsync snapshots]