Difference between revisions of "Using XFS instead of ext3 (network performance boost)"
(→Why?) |
(→Why Not?) |
||
| Line 57: | Line 57: | ||
==Why Not?== | ==Why Not?== | ||
| − | ext3 has much better error recovery than xfs: ext3 is slower but safer. You've got to decide for yourself which one is right for you. As far as safety goes, it is important to remember that ext2, ext3, XFS, JFS andReiserFS are all supported by some sort of Debian utility/tool package that can be used to at least attempt a fix of your hard drive if something goes wrong. It is a good idea to get to know the pertinent packages below | + | ext3 has much better error recovery than xfs: ext3 is slower but safer. You've got to decide for yourself which one is right for you. As far as safety goes, it is important to remember that ext2, ext3, XFS, JFS andReiserFS are all supported by some sort of Debian utility/tool package that can be used to at least attempt a fix of your hard drive if something goes wrong. It is a good idea to get to know the pertinent packages below and their parent utilities before delving into things. |
*e2fsck | *e2fsck | ||
Revision as of 02:44, 11 April 2007
Contents
Why?
Because it improves the network performance. Some of us have found that moving to XFS from ext3 seems to remove a bottleneck on the transfers between the computer desktop and the Linkstation. The amount of improvement may vary but is significant and quite beneficial. Transfers for one user showed a tranfer rate about 2 to 2.5 times the original stock rate (100%-150% improvement) on an HG, upgraded to FreeLink, a 2.6 kernel and XFS.
| Model | ext3 | XFS | Difference (MB/s) | Difference (%) |
| LS1: | ? | ? | ? | ? |
| HG(stock->FreeLink): | 5MB/s | 12.5MB/s | 7.5 | 150 |
| HG(FreeLink ext3->xfs): | 10MB/s | 12.5MB/s | 2.5 | 25 |
| KuroHG(Debian ext3->xfs): | 9.5MB/s | 11.7MB/s | 2.2 | 23 |
| HS: | ? | ? | ? | ? |
Chart notes: In the first two HG comparisons, HG(stock->FreeLink) and HG(FreeLink ext3->xfs), the increase is artificially understated, as this particular unit had a minor, preexisting hard drive problem, and its udma setting had been throttled back to udma5 using hdparm. The performance gain should be much higher in a unit that has udma6 (ata133) enabled. The KuroHG's hard drive also was restricted to udma5.
Why Not?
ext3 has much better error recovery than xfs: ext3 is slower but safer. You've got to decide for yourself which one is right for you. As far as safety goes, it is important to remember that ext2, ext3, XFS, JFS andReiserFS are all supported by some sort of Debian utility/tool package that can be used to at least attempt a fix of your hard drive if something goes wrong. It is a good idea to get to know the pertinent packages below and their parent utilities before delving into things.
- e2fsck
- xfs_repair
- fsck.jfs
- reiserfsck
Of course, a valid and reliable backup strategy should be in place before proceeding. In critical situations, one may want to use an UPS to add another layer of safety.
Prerequisites
Custom Kernel
You will need a custom kernel running in your LS1 or HG. The latest 2.6 kernels and modules available at this site feature such support for XFS, JFS and a variety of other formats. Both UBoot and bootloader.o are good options for making your LS1 or HG play nicely with the 2.6 kernels. (Currently, there is no support for this modification on the MIPsel-based LS2.)
Knowing the risks
IF something goes wrong then it is quite sure that you will have to connect the hdd to a workstation running Linux as there is no XFS-Support in EM Mode.
Mindbender is working on this:
The Linkstation Community Forum / General Development / Development of flash ramdisk from scratch - linux knowledge wanted!
The Linkstation Community Forum / General Development / (flash-development) telnet + ftp enabled flash images ready for testing(LS1, HG, HS)
Time and Hard Drive Space
If you are going use convertfs to convert your partition, you will need time and hard drive space to do it.
- Time: The relocation of blocks seems to take much longer when the partition has gigabytes of data on it.
- Space: The convertfs system used here creates temporary partitions and some potentially very large files to store and convert the contents of your hda3. For this reason, your partition should not be very full when you attempt this conversion of filesystem types. One estimate says that it should be no more than 1/3 full.
Ideally, the process works best (fastest) when the partition is relatively empty.
Installation
Depending on which distro you have installed on your LinkStation, choose the FreeLink or OpenLink option below and install using the specified commands. The FreeLink options are from the stable branch.
FreeLink
From Debian stable:
apt-get install xfsprogs gettext uuid-dev
Then, either temporarily enable the unstable branch and:
apt-get update apt-get install convertfs
or download by:
wget http://downloads.linkstationwiki.net/packages/ppc/convertfs-13jan2005_ppc.tar.gz tar -C ~ -xzvf convertfs-13jan2005_ppc.tar.gz
OpenLink
cd /tmp wget http://downloads.linkstationwiki.net/packages/ppc/xfsprogs-2.8.16_ppc.tar.gz tar -C / -xzvf xfsprogs-2.8.16_ppc.tar.gz wget http://downloads.linkstationwiki.net/packages/ppc/gettext-0.16_ppc.tar.gz tar -C / -xzvf gettext-0.16_ppc.tar.gz wget http://downloads.linkstationwiki.net/packages/ppc/convertfs-13jan2005_ppc.tar.gz tar -C ~ -xzvf convertfs-13jan2005_ppc.tar.gz
Loading the XFS-Kernelmodule
modprobe xfs
The preceding simple command both invokes and test if XFS is supported by your kernel. If it executes without error, you can then use the command "lsmod" gives an overview over all loaded modules. If XFS is shown there everything is alright, and you should be good to go on to the next step. If modprobe xfs returns an error such as this:
modprobe: QM_MODULES: Function not implemented modprobe: Can't locate module xfs
then you may not have your module-init-tools installed. Get them from Debian stable branch, using
apt-get install module-init-tools
and attempt the modprobe xfs again.
Conversion to XFS
The actual conversion of the data-partition (/mnt probably) from ext3 to XFS is done by the command below. One way to do this is to cd to the convertfs directory that you untar after downloading it, and invoke the command from within that directory. In the example below, it is assumed that it was untarred at /. The directory is named convertfs, and several required executables and directories are found within it. The main executable is also called convertfs (which may be a bit confusing at first to some).
cd /convertfs ./contrib/convertfs /dev/hda3 ext3 xfs
This process may take from minutes to days, depending on how much data you have on your partition. The output for converting ext3 to xfs may appear similar to this:
./contrib/convertfs /dev/hda3 ext3 xfs == Creating clone of `ext3' filesystem that's on `/dev/hda3'. == ===== Creating destination `xfs' filesystem. ===== meta-data=/dev/loop7 isize=256 agcount=16, agsize=3725322 blks = sectsz=512 data = bsize=4096 blocks=59605152, imaxpct=25 = sunit=0 swidth=0 blks, unwritten=1 naming =version 2 bsize=4096 log =internal log bsize=4096 blocks=29104, version=1 = sectsz=512 sunit=0 blks realtime =none extsz=65536 blocks=0, rtextents=0 ============== Copying files ============== total 0 drwxrwxrwx 4 root root 104 Nov 20 21:55 share Filesystem Size Used Avail Use% Mounted on /dev/loop7 228G 236M 228G 1% /tmp/convertfs/fs2root === Preparing info for block relocation === 0+1 records in 0+1 records out 204 bytes transferred in 0.001923 seconds (106080 bytes/sec) ============ Relocating blocks ============ Loading indexblocks... done. Relocating block group at 1... done. Relocating block group at 3726312... done. Relocating block group at 3727333... done. . . . ### repetitious lines omitted for brevity... . . Relocating block group at 29830203... done. Relocating block group at 29831224... done. And now the block0... === Filesystem conversion accomplished! === NOTE: If you want to boot the new filesystem on `/dev/hda3' you should edit `/etc/fstab' and add something like this. /dev/hda3 / xfs defaults 1 1
The last lines give a good hint of your last step, which follows the the fstab modification step below.
CAUTION: Do not assume that your system has frozen if you see the "Relocating block" progress slow down or seemingly halt. The blocks with more data take a long(er) time to complete their relocation. Do not disturb or halt this process.
How long does it take?Converting a 239GB partition with 240MB of data took only about 20 minutes. On the other hand, some users have reported that hundreds of gigabytes of data can take days to convert. Different amounts of data and different hard drive capacities will make this time variable, as data has to be copied, and partitions must be relocated.
If you want to check the progress of things during the conversion, you can use
df -Th
to see this kind of information (although at the relocation phase, your converted partition may not be mounted and would thus be unlisted):
df -hT Filesystem Type Size Used Avail Use% Mounted on /dev/hda1 ext3 5.0G 952M 3.8G 20% / tmpfs tmpfs 62M 4.0K 62M 1% /dev/shm tmpfs tmpfs 10M 2.5M 7.6M 25% /dev /dev/hda3 xfs 228G 197G 31G 87% /tmp/convertfs/fs1root /dev/loop7 jfs 228G 71G 157G 32% /tmp/convertfs/fs2root
Note that in the window showing the df -Th output above, the conversion is actually from xfs to jfs. Conversion from ext3 to xfs is handled in an entirely analogous manner.
Modification of /etc/fstab
At the end of convertfs you are told that you need to modify /etc/fstab because of the different file system. Change the entry for /dev/hda3 from "ext3" to "xfs"
before:
/dev/hda3 /mnt ext3 defaults,noatime 0 0
after:
/dev/hda3 /mnt xfs defaults,noatime 0 0
Double check your edits above and reboot. If you are successful then you should be able to use df -Th and see
df -hT Filesystem Type Size Used Avail Use% Mounted on /dev/hda1 ext3 5.0G 952M 3.8G 20% / tmpfs tmpfs 62M 4.0K 62M 1% /dev/shm /dev/hda3 jfs 228G 9.7G 218G 5% /mnt tmpfs tmpfs 10M 2.5M 7.6M 25% /dev
meaning your new filesystem format is up and running.
References
A nice HOW TO at the GentooWiki about Convertfs - http://gentoo-wiki.com/HOWTO_Convert_Filesystems
Where convertfs started - its home page - http://tzukanov.narod.ru/convertfs/
How convertfs works - http://members.optusnet.com.au/clausen/ideas/convertfs.txt
Debian Unstable package for convertfs - http://packages.debian.org/unstable/admin/convertfs
Linkstationwiki Forum Discussion "How to Get XFS running on an LS or HG" - http://forum.linkstationwiki.net/index.php?action=vthread&forum=6&topic=1567
Google Search for "convertfs slow" - http://www.google.com/search?&rls=en&q=convertfs+slow&ie=UTF-8&oe=UTF-8
