Terastation NFS

From NAS-Central Buffalo - The Linkstation Wiki
Jump to: navigation, search

(moved here from Hacking)

N.B. There is also a lot of potentially useful information in the discussion page corresponding to this article, which needs refactoring and moving here.

Kernel-mode vs. user-mode

There are two choices for NFS support: kernel-mode and user-mode. Kernel-mode has better performance than user-mode, but user-mode is potentially more stable, in that if something goes badly wrong, the damage is limited to the userspace daemons. If something goes wrong via the nfsd kernel module, the machine could easily hang.

However, see also this post by Olaf Kirch which points out a number of serious deficiencies with the user-mode approach.

Kernel-Mode NFS

OK, this link contains the kernel modules, the utilities and the startup files for kernel NFS. It's built against the 2.4.20_mvl31-ppc_linkstation kernel, so won't work on a standard Terastation.

I believe the correct stuff for Terastation is here --Neilfred 04:46, 14 January 2007 (CET)

I have just completed trying Neilfreds kernel modules on my 2.4.20_mvl31-ppc_terastation. I received the following error when issuing

depmod -a 
depmod: *** Unresolved symbols in /lib/modules/2.4.20_mvl31-ppc_terastation/kernel/net/sunrpc/sunrpc.o

As well as errors when trying to start the NFS daemon

# /etc/init.d/nfs-common 

start/lib/modules/2.4.20_mvl31-ppc_terastation/kernel/net/sunrpc/sunrpc.o: kernel-module version mismatch /lib/modules/2.4.20_mvl31-ppc_terastation/kernel/net/sunrpc/sunrpc.o was compiled for kernel version 2.4.20_mvl31-ppc_linkstation while this kernel is version 2.4.20_mvl31-ppc_terastation.

Very important step post-installation! During tar extraction, the /var -> /mnt/ram/var symlink will probably get overwritten, so you need to restore it:

cp -a /var/lib /mnt/ram/var/lib
mv /var /var.old
ln -s /mnt/ram/var /var

Edit /etc/exports to match your setup, then reboot or run the init files by hand:

depmod -a
/etc/init.d/portmap start
/etc/init.d/nfs-common start
/etc/init.d/nfs-kernel-server start


Problems with NFS may arise, please see the discussion page for details.

Having just spent several days trying to get NFS working I can give a few tips. If you can write to the NFS share, but have problems reading from it (transfers stall or terastation randomly reboots) try disabling jumbo packets. If the jumbo packet size on all your machines doesn't match than UDP transfers such as NFS wreak havoc with the terastation. I now have the 2.03 firmware and user mode NFS (from debian PPC) working smoothly. An alternative solution would be to use TCP rather than UDP for NFS, but not all NFS implementations support this.


Missing rpcinfo? After installing nfsd_terastation.tgz "tar -C / -xzf nfsd_terastation.tgz" the initialization script "/etc/init.d/nfsd-kernel-server" references "rpcinfo" and fails to start rpc.mountd because rpcinfo is missing and the parameters for rpc.mountd are wrong.

Edit the nfsd-kernel-server file and set the option on line 22 to:

RPCMOUNTDOPTS="--no-nfs-version" or RPCMOUNTDOPTS="--nfs-version"

Note line 39-40. It uses RPCMOUNTDOPTS to build an options line for rpc.mountd. Simply put on line 22 RPCMOUNTDOPTS="--nfs-version" and when you start the server, it will run "rpc.mountd --nfs-version 3".

If you get a "Permission Denied" when trying to mount an export, check to see that the reverse DNS entry resolves correctly for the server trying to mount. On the terastation, update your /etc/hosts file (optional) and create a script to reset this file every time the server boots. /etc/rc.d/init.d/networking overwrites this file.

I put in "--nfs-version 3" as I only want v3 support. I am still testing this. -- Kimbotha 10:45, 26 October 2006 (CEST)

--status: Please note that I have personally had a HUGE problem with the NFS CLIENTS when using this server. There seems to be a large degree of "retransmissions" and I/O problems as a result of communicating with the nfs server. (Run "nfsstat" on the client). If there has been any updates to the server, I'd wish they would be published! Perhaps I'll just fall back to using "smbmount" Also, this server appears to have only UDP support. NFS Clients complain with "nfs over UDP causes data corruption" (ick). -- 12:00, November 24 2006 (SGT)

Same here - I get a constant stream of

 Jul  1 19:24:54 atlantic kernel: nfs: server tera not responding, still trying 
 Jul  1 19:24:54 atlantic kernel: nfs: server tera OK 
 Jul  1 19:25:09 atlantic kernel: nfs: server tera not responding, still trying 
 Jul  1 19:25:11 atlantic kernel: nfs: server tera OK 

when I try to copy a large file. Sad to say that the user-mode server is more useful at this point :-( --Aspiers 20:26, 1 July 2007 (CEST)

User-Mode NFS

A User-mode NFS package is now available on Dave's OpenTera page. Bunshichi 10 March 2007

However this package supports NFS v2 only, and hence has the caveats standard with v2 - in particular it won't support files >2GB.

It is possible to get an NFS v3 user-mode daemon running on the Terastation, since entropy built unofficial firmware for TSP which includes unfs3:

  • Download the 1.01-2 image (the others might work, I didn't try)
  • Rename it to have a .zip extension
  • Unzip it to a temporary directory somewhere (see here for p/w details)
  • Cherry-pick the relevant files from the root filesystem image.
tar xvzf tmpimage.tgz
mv ./usr/sbin/unfsd /usr/sbin/
mv ./etc/init.d/nfs-user-server /etc/init.d/
mv ./usr/sbin/portmap /usr/sbin
mv ./usr/sbin/rpc.mountd /usr/sbin
mv ./usr/sbin/rpc.nfsd /usr/sbin
ln -s /etc/exports /etc/exports.unfsd
echo "/mnt/array1,root_squash)" > /etc/exports
/etc/init.d/nfs-user-server restart

However, see above for the caveats about user-mode NFS in general.

--Aspiers 14:09, 1 July 2007 (CEST)

--note: can anyone translate japanese? This might be some pertinent information. I am also trying to get NFS working well. http://blog.livedoor.jp/nano_j/archives/50498734.html --Brent 04:23, 3 December 2006 (CET)

-- Regarding Brent's reference, the Japanese page describes how to implement NFS as a user-level driver (as opposed to kernel-level). It is, however, incomplete in its description of what to do (you have modify the init.d scripts, etc.). Still, using it as a springboard, I have gotten the user-level driver working on my TS, and I've been stress-testing it days without a hiccup. However, I've been only getting 2.2MBytes/sec writes using it (where I can get 4.2Mbytes/sec writes using the rsync daemon).

Kudo's to Brent for pointing out the link. Bunshichi 10 March 2007

Alternative to NFS

After following in the foot steps of other NFS experimenters, I tried compiling my own NFSD module with TCP support. I was using the 2.4.20 kernel which comes with the OpenTera 2.12 firmware hack.

After finally getting the toolchains and code compiled, and after a battle with symbol names (the TS uses the "versioned" symbols suffixes), the NFSD w/ TCP had horrible performance and would stall out quickly. I had to reboot it once which threw the TS into it's nightmarish 12 hour raid consistency check.

My motivation for adding NFS support was so that I could perform rdiffbackup's to back up a "live" linux machine. I didn't want to loose file attributes and permissions through SMB/CIFS shortcomings.

Another alternative, while not the highest performance, but /will/ achieve the same results is to create an EXT2/3 or XFS binary image on the TS. Mount the location containing this image via smb, then use loopback to mount the FS image file itself.

I did the following: Set up a new folder for sharing called "foo", then telnet to the TS

cd /mnt/array1/foo
dd if=/dev/zero of=foo.ext2.bin bs=1M count=100000
mke2fs foo.ext2.bin

Switch to your linux "client" machine:

mount -t smbfs -o yadda,yadda //ts/foo /mnt/foo.smb
mount -t ext2 -o loop=/dev/loop0 /mnt/foo.smb/foo.ext2.bin /mnt/foo

do your work on /mnt/foo. When you're done:

umount /mnt/foo
umount /mnt/foo.smb

I ran into one small problem, however. Performing dd as 'myroot' on the TS left me with incorrect owner and permission bits. Come to find out the default coreutils don't support large files. I had to install midnight commander to do the 'chmod' and 'chown' --AxMstrLP 06:55, 11 February 2007 (CET)