Difference between revisions of "Using XFS instead of ext3 (network performance boost)"

From NAS-Central Buffalo - The Linkstation Wiki
Jump to: navigation, search
(Why?)
(Knowing the risks)
Line 65: Line 65:
  
 
===Knowing the risks===
 
===Knowing the risks===
 +
 +
This is best attempted with a partition that is not nearly full.
 +
 
IF something goes wrong then it is quite sure that you will have to connect the hdd to a workstation running Linux as there is no XFS-Support in EM Mode.
 
IF something goes wrong then it is quite sure that you will have to connect the hdd to a workstation running Linux as there is no XFS-Support in EM Mode.
  
Line 70: Line 73:
 
[http://forum.linkstationwiki.net/index.php?action=vthread&forum=7&topic=1656 The Linkstation Community Forum / General Development / Development of flash ramdisk from scratch - linux knowledge wanted!]<br>
 
[http://forum.linkstationwiki.net/index.php?action=vthread&forum=7&topic=1656 The Linkstation Community Forum / General Development / Development of flash ramdisk from scratch - linux knowledge wanted!]<br>
 
[http://forum.linkstationwiki.net/index.php?action=vthread&forum=7&topic=1354&page=0#msg12292 The Linkstation Community Forum / General Development / (flash-development) telnet + ftp enabled flash images ready for testing(LS1, HG, HS)]
 
[http://forum.linkstationwiki.net/index.php?action=vthread&forum=7&topic=1354&page=0#msg12292 The Linkstation Community Forum / General Development / (flash-development) telnet + ftp enabled flash images ready for testing(LS1, HG, HS)]
 
  
 
==Installation==
 
==Installation==

Revision as of 06:21, 4 December 2006


Link.png This article is currently a stub. You can help this Wiki by expanding it

. This template will categorize articles that include it into Category:Stubs.

Contents

Why?

Because it improves the network performance. Some of us have found that moving to XFS from ext3 seems to remove a bottleneck on the transfers between the computer desktop and the Linkstation. The amount of improvement may vary but is significant and quite beneficial. Transfers for one user showed a tranfer rate about 2 to 2.5 (100%-150% gain) time the original stock rate on an HG, upgraded to FreeLink, a 2.6 kernel and XFS.

Model ext3 XFS Difference (MB/s) Difference (%)
LS1:  ?  ?  ?  ?
HG(stock->FreeLink): 5MB/s 12.5MB/s 7.5 150
HG(FreeLink ext3->xfs): 10MB/s 12.5MB/s 2.5 25
HS:  ?  ?  ?  ?


Note: In the first two HG comparisons, HG(stock->FreeLink) and HG(FreeLink ext3->xfs), the increase is artificially understated, as this particular unit had a minor, preexisting hard drive problem, and its dma setting had been throttled back to dma5 using hdparm. The performance gain should be much higher in a unit that doesn't have hardware problems.

Why Not?

ext3 has much better error recovery than xfs: ext3 is slower but safer. You've got to decide for yourself which one is right for you. As far as safety goes, it is important to remember that ext2, ext3, XFS, JFS andReiserFS are all supported by some sort of Debian utility/tool package that can be used to at least attempt a fix of your hard drive if something goes wrong. It is a good idea to get to know the pertinent packages below and and their parent utilities before delving in to things.

  • e2fsck
  • xfs_repair
  • fsck.jfs
  • reiserfsck

Of course, a valid and reliable backup strategy should be in place before proceeding. In critical situations, one may want to use an UPS to add another layer of safety.

Prerequisites

Custom Kernel

You will need a custom kernel running in your LS1 or HG. The latest 2.6 kernels and modules available at this site feature such support for XFS, JFS and a variety of other formats. Both UBoot and bootloader.o are good options for making your LS1 or HG play nicely with the 2.6 kernels. (Currently, there is no support for this modification on the MIPsel-based LS2.)

Knowing the risks

This is best attempted with a partition that is not nearly full.

IF something goes wrong then it is quite sure that you will have to connect the hdd to a workstation running Linux as there is no XFS-Support in EM Mode.

Mindbender is working on this:
The Linkstation Community Forum / General Development / Development of flash ramdisk from scratch - linux knowledge wanted!
The Linkstation Community Forum / General Development / (flash-development) telnet + ftp enabled flash images ready for testing(LS1, HG, HS)

Installation

Depending on which distro you have installed on your LinkStation, choose the FreeLink or OpenLink option below and install using the specified commands. The FreeLink options are from the stable branch.

FreeLink

From Debian stable:

 apt-get install xfsprogs gettext uuid-dev

Then, either temporarily enable the unstable branch and:

apt-get update
apt-get install convertfs

or download by:

wget http://downloads.linkstationwiki.net/packages/ppc/convertfs-13jan2005_ppc.tar.gz
tar -C ~ -xzvf convertfs-13jan2005_ppc.tar.gz

OpenLink

cd /tmp
wget http://downloads.linkstationwiki.net/packages/ppc/xfsprogs-2.8.16_ppc.tar.gz
tar -C / -xzvf xfsprogs-2.8.16_ppc.tar.gz
wget http://downloads.linkstationwiki.net/packages/ppc/gettext-0.16_ppc.tar.gz
tar -C / -xzvf gettext-0.16_ppc.tar.gz
wget http://downloads.linkstationwiki.net/packages/ppc/convertfs-13jan2005_ppc.tar.gz
tar -C ~ -xzvf convertfs-13jan2005_ppc.tar.gz

Loading the XFS-Kernelmodule

modprobe xfs

The preceding simple command both invokes and test if XFS is supported by your kernel. If it executes without error, you can then use the command "lsmod" gives an overview over all loaded modules. If XFS is shown there everything is alright, and you should be good to go on to the next step. If modprobe xfs returns an error such as this:

modprobe: QM_MODULES: Function not implemented 

modprobe: Can't locate module xfs


then you may not have your module-init-tools installed. Get them from Debian stable branch, using

apt-get install module-init-tools

and attempt the modprobe xfs again.

Conversion to XFS

The actual conversion of the data-partition (/mnt probably) from ext3 to XFS is done by the command below. One way to do this is to cd to the convertfs directory that you untar after downloading it, and invoke the command from within that directory. In the example below, it is assumed that it was untarred at /. The directory is named convertfs, and several required executables and directories are found within it. The main executable is also called convertfs (which may be a bit confusing at first to some).

cd /convertfs
./contrib/convertfs /dev/hda3 ext3 xfs

This process may take from minutes to hours, depending on how much data you have on your partition. The output for converting ext3 to xfs may appear similar to this:

./contrib/convertfs /dev/hda3 ext3 xfs 
== Creating clone of `ext3' filesystem that's on `/dev/hda3'. == 
===== Creating destination `xfs' filesystem. ===== 
meta-data=/dev/loop7             isize=256    agcount=16, agsize=3725322 blks 
=                       sectsz=512 
data     =                       bsize=4096   blocks=59605152, imaxpct=25 
=                       sunit=0      swidth=0 blks, unwritten=1 
naming   =version 2              bsize=4096 
log      =internal log           bsize=4096   blocks=29104, version=1 
=                       sectsz=512   sunit=0 blks 
realtime =none                   extsz=65536  blocks=0, rtextents=0 
============== Copying files ============== 
total 0 
drwxrwxrwx  4 root root 104 Nov 20 21:55 share 
Filesystem            Size  Used Avail Use% Mounted on 
/dev/loop7            228G  236M  228G   1% /tmp/convertfs/fs2root 
=== Preparing info for block relocation === 
0+1 records in 
0+1 records out 
204 bytes transferred in 0.001923 seconds (106080 bytes/sec) 
============ Relocating blocks ============ 
Loading indexblocks... done. 
Relocating block group at 1... done. 
Relocating block group at 3726312... done. 
Relocating block group at 3727333... done. 
. 
. 
.  ### repetitious lines omitted for brevity...
. 
. 
Relocating block group at 29830203... done. 
Relocating block group at 29831224... done. 
And now the block0... 
=== Filesystem conversion accomplished! === 
NOTE: If you want to boot the new filesystem on `/dev/hda3' you 
should edit `/etc/fstab' and add something like this. 
/dev/hda3   /   xfs   defaults   1   1

The last lines give a good hint of your last step, which follows below.

If you want to check the progress of things during the conversion, you can use

df -Th

to see this kind of information:

df -hT
Filesystem    Type    Size  Used Avail Use% Mounted on
/dev/hda1     ext3    5.0G  952M  3.8G  20% /
tmpfs        tmpfs     62M  4.0K   62M   1% /dev/shm
tmpfs        tmpfs     10M  2.5M  7.6M  25% /dev
/dev/hda3      xfs    228G  197G   31G  87% /tmp/convertfs/fs1root
/dev/loop7     jfs    228G   71G  157G  32% /tmp/convertfs/fs2root

About 120GB on a 230GB required about 3 hours for a conversion. Different amount of data and different hard drive capacities will make this time variable. Note that in the window showing the df -Th output above, the conversion is actually from xfs to jfs. Conversion from ext3 to xfs is handled in an entirely analogous manner.

Modification of /etc/fstab

At the end of convertfs you are told that you need to modify /etc/fstab because of the different file system. Change the entry for /dev/hda3 from "ext3" to "xfs"

before:

/dev/hda3       /mnt            ext3    defaults,noatime                0 0

after:

/dev/hda3       /mnt            xfs     defaults,noatime                0 0

Double check your edits above and reboot. If you are successful then you should be able use

df -Th

to verify your new filesystem format. Your hda3 should now be showing under a new format/filesystem.

References

http://gentoo-wiki.com/HOWTO_Convert_Filesystems

http://tzukanov.narod.ru/convertfs/

http://packages.debian.org/unstable/admin/convertfs

http://forum.linkstationwiki.net/index.php?action=vthread&forum=6&topic=1567

Link.png This article is currently a stub. You can help this Wiki by expanding it

. This template will categorize articles that include it into Category:Stubs.