Difference between revisions of "Terastation NFS"
(refactored from Hacking page)
(explain where page came from)
|Line 1:||Line 1:|
= Kernel-mode vs. user-mode =
= Kernel-mode vs. user-mode =
Revision as of 12:27, 1 July 2007
(moved here from Hacking)
Kernel-mode vs. user-mode
There are two choices for NFS support: kernel-mode and user-mode. Kernel-mode has better performance than user-mode, but user-mode is potentially more stable, in that if something goes badly wrong, the damage is limited to the userspace daemons. If something goes wrong via the nfsd kernel module, the machine could easily hang.
However, see also this post by Olaf Kirch which points out a number of serious deficiencies with the user-mode approach.
OK, this link contains the kernel modules, the utilities and the startup files for kernel NFS. It's built against the 2.4.20_mvl31-ppc_linkstation kernel, so won't work on a standard Terastation.
Then, because the symlink that is /var gets overwritten:
cp -a /var/lib /mnt/ram/var/lib mv /var /var.old ln -s /mnt/ram/var /var
Edit /etc/exports to match your setup, then reboot or run the init files by hand:
depmod -a /etc/init.d/portmap start /etc/init.d/nfs-common start /etc/init.d/nfs-kernel-server start
Problems with NFS may arise, please see the discussion page for details.
Having just spent several days trying to get NFS working I can give a few tips. If you can write to the NFS share, but have problems reading from it (transfers stall or terastation randomly reboots) try disabling jumbo packets. If the jumbo packet size on all your machines doesn't match than UDP transfers such as NFS wreak havoc with the terastation. I now have the 2.03 firmware and user mode NFS (from debian PPC) working smoothly. An alternative solution would be to use TCP rather than UDP for NFS, but not all NFS implementations support this.
Missing rpcinfo? After installing nfsd_terastation.tgz "tar -C / -xzf nfsd_terastation.tgz" the initialization script "/etc/init.d/nfsd-kernel-server" references "rpcinfo" and fails to start rpc.mountd because rpcinfo is missing and the parameters for rpc.mountd are wrong.
Edit the nfsd-kernel-server file and set the option on line 22 to:
RPCMOUNTDOPTS="--no-nfs-version" or RPCMOUNTDOPTS="--nfs-version"
Note line 39-40. It uses RPCMOUNTDOPTS to build an options line for rpc.mountd. Simply put on line 22 RPCMOUNTDOPTS="--nfs-version" and when you start the server, it will run "rpc.mountd --nfs-version 3".
If you get a "Permission Denied" when trying to mount an export, check to see that the reverse DNS entry resolves correctly for the server trying to mount. On the terastation, update your /etc/hosts file (optional) and create a script to reset this file every time the server boots. /etc/rc.d/init.d/networking overwrites this file.
I put in "--nfs-version 3" as I only want v3 support. I am still testing this. -- Kimbotha 10:45, 26 October 2006 (CEST)
--status: Please note that I have personally had a HUGE problem with the NFS CLIENTS when using this server. There seems to be a large degree of "retransmissions" and I/O problems as a result of communicating with the nfs server. (Run "nfsstat" on the client). If there has been any updates to the server, I'd wish they would be published! Perhaps I'll just fall back to using "smbmount" Also, this server appears to have only UDP support. NFS Clients complain with "nfs over UDP causes data corruption" (ick). -- 12:00, November 24 2006 (SGT)
However this package supports NFS v2 only, and hence has the caveats standard with v2 - in particular it won't support files >2GB.
It is possible to get an NFS v3 user-mode daemon running on the Terastation, since entropy built unofficial firmware for TSP which includes unfs3:
- Download the 1.01-2 image (the others might work, I didn't try)
- Rename it to have a .zip extension
- Unzip it to a temporary directory somewhere (see here for p/w details)
- Cherry-pick the relevant files from the root filesystem image.
However, see above for the caveats about user-mode NFS in general.
--Aspiers 14:09, 1 July 2007 (CEST)
--note: can anyone translate japanese? This might be some pertinent information. I am also trying to get NFS working well. http://blog.livedoor.jp/nano_j/archives/50498734.html --Brent 04:23, 3 December 2006 (CET)
-- Regarding Brent's reference, the Japanese page describes how to implement NFS as a user-level driver (as opposed to kernel-level). It is, however, incomplete in its description of what to do (you have modify the init.d scripts, etc.). Still, using it as a springboard, I have gotten the user-level driver working on my TS, and I've been stress-testing it days without a hiccup. However, I've been only getting 2.2MBytes/sec writes using it (where I can get 4.2Mbytes/sec writes using the rsync daemon).
Kudo's to Brent for pointing out the link. Bunshichi 10 March 2007
Alternative to NFS
After following in the foot steps of other NFS experimenters, I tried compiling my own NFSD module with TCP support. I was using the 2.4.20 kernel which comes with the OpenTera 2.12 firmware hack.
After finally getting the toolchains and code compiled, and after a battle with symbol names (the TS uses the "versioned" symbols suffixes), the NFSD w/ TCP had horrible performance and would stall out quickly. I had to reboot it once which threw the TS into it's nightmarish 12 hour raid consistency check.
My motivation for adding NFS support was so that I could perform rdiffbackup's to back up a "live" linux machine. I didn't want to loose file attributes and permissions through SMB/CIFS shortcomings.
Another alternative, while not the highest performance, but /will/ achieve the same results is to create an EXT2/3 or XFS binary image on the TS. Mount the location containing this image via smb, then use loopback to mount the FS image file itself.
I did the following: Set up a new folder for sharing called "foo", then telnet to the TS
cd /mnt/array1/foo dd if=/dev/zero of=foo.ext2.bin bs=1M count=100000 mke2fs foo.ext2.bin
Switch to your linux "client" machine:
mount -t smbfs -o yadda,yadda //ts/foo /mnt/foo.smb mount -t ext2 -o loop=/dev/loop0 /mnt/foo.smb/foo.ext2.bin /mnt/foo
do your work on /mnt/foo. When you're done:
sync umount /mnt/foo umount /mnt/foo.smb
I ran into one small problem, however. Performing dd as 'myroot' on the TS left me with incorrect owner and permission bits. Come to find out the default coreutils don't support large files. I had to install midnight commander to do the 'chmod' and 'chown' --AxMstrLP 06:55, 11 February 2007 (CET)