Difference between revisions of "Enabling Rsync on a Linkstation Mini"

From NAS-Central Buffalo - The Linkstation Wiki
Jump to: navigation, search
(Do You Really Want Rsync?)
m (ACP)
 
(30 intermediate revisions by one user not shown)
Line 8: Line 8:
 
== Do You Really Want Rsync? ==
 
== Do You Really Want Rsync? ==
  
I measured the Mini transfer speed as 1.2 megabytes per second doing scp/rsync, which is not exactly blazing, but it did 11 megabytes per second doing ftp of a large file(and this is the limit of my old router at 100Mbits/sec), and 30 megabytes per second cp'ing from disk to /dev/null.  I suspect the reason for the slow scp/rsync is that the CPU is slow at encrypting.
+
I measured the Mini transfer speed as 1.2 megabytes per second doing scp/rsync, which is not exactly blazing, but it did 11 megabytes per second doing ftp of a large file (this is the limit of my old router at 100Mbits/sec so with a faster router the speed may be faster: the Mini did 30 megabytes per second cp'ing from disk to /dev/null).  I suspect the reason for the slow scp/rsync is that the CPU is slow at encrypting.
  
 
If you just want to transfer single large files, ftp would be better.  However, if you want to transfer complex directory structures, there are some reasons to prefer rsync.  The closest thing to rsync in the ftp world that I know is the lftp mirror command, over which rsync has the following advantages:
 
If you just want to transfer single large files, ftp would be better.  However, if you want to transfer complex directory structures, there are some reasons to prefer rsync.  The closest thing to rsync in the ftp world that I know is the lftp mirror command, over which rsync has the following advantages:
  
 
* Rsync preserves file and directory modification times, whereas ftp cannot set modification times on the remote system and does not preserve timestamps.
 
* Rsync preserves file and directory modification times, whereas ftp cannot set modification times on the remote system and does not preserve timestamps.
 +
 +
* Rsync preserves file permissions, whereas ftp cannot set permissions on the remote system and does not preserve them.
  
 
* Rsync preserves symbolic links, whereas ftp cannot create symbolic links on the remote system and does not preserve them.
 
* Rsync preserves symbolic links, whereas ftp cannot create symbolic links on the remote system and does not preserve them.
Line 18: Line 20:
 
* Rsync has a -vnc option that allows you to check whether your backup matches your original bit-for-bit by computing checksums of the files.
 
* Rsync has a -vnc option that allows you to check whether your backup matches your original bit-for-bit by computing checksums of the files.
  
Also, when lftp mirror retrieves a large complex directory system stored by rsync, lftp achieved only a 2.4 megabyte per second transfer rate, and not the 11 megabyte per second rate for large files.  My guess is that the Mini is very slow at accessing lots of directory structure: I've seen this in other situations.  Perhaps because the xfs file system has a large block size and the Mini only has 128MB RAM and can therefore only cache a few blocks.
+
Also, when I used lftp mirror to retrieve a large complex directory system stored by rsync, lftp achieved only a 2.4 megabyte per second transfer rate, and not the 11 megabyte per second rate for large files.  My guess is that the Mini is very slow at accessing lots of directory structure: I've seen this in other situations.  Perhaps the xfs file system has a large block size and as the Mini only has 128MB RAM it caches too few blocks.
  
Lastly you should always consider getting flash memory stick drives for backup before buying any backup disk.  You can do rsync to a flash drive very easily.  (Hints: your will likely want to `dd if=/dev/zero of=/dev/YOURFLASH' and `mkfs -type ext3 /dev/YOURFLASH' to get a good file system on a flash stick drive.)
+
Lastly you should always consider getting flash memory stick drives for backup before buying any backup disk.  You can do rsync to a flash drive very easily.  (Hints: you will likely want to `dd if=/dev/zero of=/dev/YOURFLASH count=1000' and then `mkfs -type ext3 /dev/YOURFLASH' to get a good file system on a flash stick drive.)
  
 
== LinkStation Mini Description ==
 
== LinkStation Mini Description ==
Line 27: Line 29:
 
The LinkStation Mini is a very small computer running a fairly full Linux system, in so far as backup is concerned.  The manufacturer supports FTP and SAMBA servers (and also LPD for printers), but not rsync or ssh, probably because as RedHat discovered, any attempt to support the latter would result in too many calls to techsupport.  However, all the software to use ssh and rsync is already installed, and only a few configuration files need to be edited to make these work.
 
The LinkStation Mini is a very small computer running a fairly full Linux system, in so far as backup is concerned.  The manufacturer supports FTP and SAMBA servers (and also LPD for printers), but not rsync or ssh, probably because as RedHat discovered, any attempt to support the latter would result in too many calls to techsupport.  However, all the software to use ssh and rsync is already installed, and only a few configuration files need to be edited to make these work.
  
I measured the Mini transfer speed as 1.2 megabytes per second doing scp/rsync, which is not exactly blazing, but it did 11 megabytes per second doing ftp (and this is the limit of my old router at 100Mbits/sec), and 30 megabytes per second cp'ing from disk to /dev/null.  I suspect the reason for the slow scp/rsync is that the CPU is slow at encrypting.
+
See the previous section for some timing info.
  
 
I put a power meter on the Mini and measured 6-7 watts when it was not transferring data and 7-8 watts when it was transferring data; most excellent.
 
I put a power meter on the Mini and measured 6-7 watts when it was not transferring data and 7-8 watts when it was transferring data; most excellent.
Line 33: Line 35:
 
The LinkStation Mini hardware is:
 
The LinkStation Mini hardware is:
  
     ARM9 CPU
+
     ARM9 CPU (266 BogoMIPS: 15 times slower than a 2Gz Intel) [http://www.marvell.com/files/products/media/88F5182_Data_Sheet.pdf Datasheet]
     2 hard drives, either 250GB or 500GB each
+
     2 hard drives, either 250GB or 500GB each (30 Mbyte/sec transfer to /dev/null)
 
     128 MB RAM
 
     128 MB RAM
     one ethernet port
+
     one ethernet port (can run at 1 Gb/sec)
     controllers for 2 USB ports (only one connector provided - hub may allow 2 ports to be used)
+
     controllers for 2 USB ports (only one connector provided - hub might allow 2 ports)
     controller for one serial port - no extenal connector provided
+
     controller for several serial ports - no external or internal connector provided
     256KB flash memory - contains uBoot bootstrap loader - not used by Linux or during normal operation
+
     256KB flash memory - contains uBoot bootstrap loader
 +
                      - not used by Linux or during normal operation
  
 
(More details at [[Disassemble the LS MINI]].)
 
(More details at [[Disassemble the LS MINI]].)
Line 46: Line 49:
  
 
     Mounted On        Format        RAID Device        Hardware Devices          Size
 
     Mounted On        Format        RAID Device        Hardware Devices          Size
 
+
   
 
     /boot              ext3          /dev/md0            /dev/sd{a,b}1              1GB
 
     /boot              ext3          /dev/md0            /dev/sd{a,b}1              1GB
 
     /                  xfs          /dev/md1            /dev/sd{a,b}2              5GB
 
     /                  xfs          /dev/md1            /dev/sd{a,b}2              5GB
Line 52: Line 55:
 
     /mnt/array1        xfs          /dev/md2            /dev/sd{a,b}6              236GB
 
     /mnt/array1        xfs          /dev/md2            /dev/sd{a,b}6              236GB
  
In addition, 15MB of RAM is used as a RAM disk for /tmp.  The flash memory which is reported to be only 256KB is not used by Linux.  The /boot partition has all the firmware and possibly can be used to restore the machine to its delivered state, possibly by using the web Maintainance/Initialization page `Reset LinkStation Configuration to Factory Defaults'.
+
In addition, 15MB of RAM is used as a RAM disk for /tmp.  The flash memory is only 256KB and is not used by Linux (it contains a bootstrap program).  The /boot partition has all the firmware and possibly can be used to restore the machine to its delivered state, possibly by using the web Maintainance/Initialization page `Reset LinkStation Configuration to Factory Defaults'.
  
 
/boot also contains a save of the `current configuration', which includes files such as /etc/passwd, /etc/shadow, etc in
 
/boot also contains a save of the `current configuration', which includes files such as /etc/passwd, /etc/shadow, etc in
Line 71: Line 74:
 
and try it out.
 
and try it out.
  
First, you need to set up your Mini and connected to the ethernet and establish a fixed IP address for it (using your router).  Then you open the Mini's web page using the IP address, set the Mini hostname using the Basic subpage, and if you like, tell your own computer about the hostname you have choosen, say by editing your /etc/hosts file or equivalent.  I gave my Mini the name linkstation1.walton (walton is my router's domain name) and will use this name hereafter in place of the Mini's IP address.
+
First, you need to set up your Mini by connecting it to the ethernet and establishing a fixed IP address for it (using your router).  Then you open the Mini's web page using the IP address, set the Mini hostname using the Basic subpage, and if you like, tell your own computer about the hostname you have choosen, say by editing your /etc/hosts file or equivalent.  I gave my Mini the name linkstation1.walton (walton is my router's domain name) and will use this name hereafter in place of the Mini's IP address.
  
 
IMPORTANT: The Mini comes without RAID turned on, so the full 500GB or 1TB is available.  If you want to use RAID1 so the two disks back up each other, you should change to RAID1 NOW, before you start changing other Mini configuration.  See the Mini Disk Management/RAID Setup subpage and click on the tool icon.
 
IMPORTANT: The Mini comes without RAID turned on, so the full 500GB or 1TB is available.  If you want to use RAID1 so the two disks back up each other, you should change to RAID1 NOW, before you start changing other Mini configuration.  See the Mini Disk Management/RAID Setup subpage and click on the tool icon.
Line 89: Line 92:
 
     java -jar acp_commander.jar -t linkstation1.walton -o
 
     java -jar acp_commander.jar -t linkstation1.walton -o
  
at which point your manufacturer's warranty is gonzo, at least a little bit.  This zeros the root password so you can log in as root, and also executes `telnetd' which starts the telnet daemon, so you can now telnet into the Mini.
+
at which point your manufacturer's warranty is gonzo, at least a little bit.  This zeros the root password so you can log in as root without giving a password, and also executes `telnetd' which starts the telnet daemon, so you can now telnet into the Mini.
  
 
You can now turn your firewall back on.  You are done with ACP.
 
You can now turn your firewall back on.  You are done with ACP.
  
However, if you want to play before you change anything, you can do
+
However, if you want to play before you change anything more, you can do
  
 
     java -jar acp_commander.jar -t linkstation1.walton -s
 
     java -jar acp_commander.jar -t linkstation1.walton -s
  
which allows you to type shell commands at the Mini.  However, each command is executed in a separate instance of the shell, and interactive commands will hang the Mini ACP interface (you can recover by using the Mini web page to reboot).
+
which allows you to type shell commands at the Mini.  However, each command is executed in a separate instance of the shell, and interactive commands will hang the Mini ACP interface (you can recover by using the Mini web page or the power switch to reboot).
  
 
Some more info is in the ACP Commander README;
 
Some more info is in the ACP Commander README;
Line 128: Line 131:
 
     (8) create the /var/log/lastlog file
 
     (8) create the /var/log/lastlog file
  
(Note: all this comes from [[Open Stock Firmware]]: many thanks to the authors of this page. Also: my Mini firmware is 1.04 which appears to be a later version than the 1.11a firmware mentioned in this reference, which is for an older LinkStation model.)
+
(Note: all this comes from [[Open Stock Firmware]]: many thanks to the authors of this page. Also: my Mini firmware version is 1.04 which appears to be a later version than the 1.11a firmware mentioned in this reference, which is for an older LinkStation model.)
  
 
The sshd.sh file contents are
 
The sshd.sh file contents are
Line 244: Line 247:
 
First, what do the Mini web pages do?
 
First, what do the Mini web pages do?
  
Their configuration information base is kept in /etc/melco/* files.  So when you make changes using the `Shared Folders Setup', `Group Management', or `User Management' pages you are modifying these files.
+
Their configuration information base is kept in ASCII /etc/melco/* files.  So when you make changes using the `Shared Folders Setup', `Group Management', or `User Management' pages you are modifying these files.
  
 
In addition to modifying the above information base:
 
In addition to modifying the above information base:
Line 258: Line 261:
 
# Using the `User Management' page add `ruser' as a user.
 
# Using the `User Management' page add `ruser' as a user.
 
# Using the `Group Management' page add `ruser' as a group and put the user `ruser' in this group (normally as its only member).
 
# Using the `Group Management' page add `ruser' as a group and put the user `ruser' in this group (normally as its only member).
# Using the `Shared Folders/Shared Folders Setup' page add `ruser' as a shared folder, enable it for FTP and read/write, enable access restrictions, and give the user `ruser' read/write access.  WARNING: do NOT give any group `read only' access if the user `ruser' is in that group (see below).
+
# Using the `Shared Folders/Shared Folders Setup' page add `ruser' as a shared folder, enable it for FTP and read/write, enable access restrictions, and give the user `ruser' read/write access.  WARNING: do NOT give any group `read only' access if the user `ruser' is in that group (see below).  I also disable `Recycle Bin', which if enabled causes files that are deleted by FTP to be moved to the trashbox directory inside the `ruser' shared folder, instead of really being deleted.
# Edit the `ruser' line in /etc/passwds to replace `/home'/ by `/mnt/array1/ruser' and to replace the hdusers group number (which is 100 on my Mini) by the `ruser' group's group number (the group number is the second of the two numbers in the line, and the `rusers' group number is in the /etc/group file).
+
# Edit the `ruser' line in /etc/passwds to replace `/home' by `/mnt/array1/ruser' and to replace the hdusers group number (which is 100 on my Mini) by the `ruser' group's group number (the group number is the second of the two numbers in the line, and the `rusers' group number is in the /etc/group file).
 
# Execute
 
# Execute
 
     cd /mnt/array1
 
     cd /mnt/array1
Line 265: Line 268:
 
     chmod 750 ruser
 
     chmod 750 ruser
  
Given this ssh ruser@linkstation1.walton will land in the home directory /home/ruser running a shell with with user ruser and group ruser.  FTP (if the service is enabled via the `Shared Folders/Service Setup' page) will land in the same home directory running an ftp process also with user ruser and group ruser.  So all files created in the Mini by ruser will have user ruser and group ruser.
+
Given this ssh ruser@linkstation1.walton will land in the home directory /mnt/array1/ruser running a shell with user ruser and group ruser.  FTP (if the service is enabled via the `Shared Folders/Service Setup' page) will land in the same home directory running an ftp process also with user ruser and group ruser.  So all files created in the Mini by ruser will have user ruser and group ruser.
  
 
The mini FTP seems to have the strange property that if you use `Shared Folders/Shared Folders Setup' to make a directory both read-write to you using your user name and also read-only to you using one of your group names, the latter takes precedence for FTP, and your FTP access will be read-only.  Also mysterious is that the FTP server on the Mini seems to be ignoring /etc/proftpd/proftpd.conf in this matter.  So you must NOT make your home directory read-only to your group using the Mini web page, if you ever want to use FTP to write in your home directory.  This seems to be a property of the vanilla Mini, and is not related to anything done above.
 
The mini FTP seems to have the strange property that if you use `Shared Folders/Shared Folders Setup' to make a directory both read-write to you using your user name and also read-only to you using one of your group names, the latter takes precedence for FTP, and your FTP access will be read-only.  Also mysterious is that the FTP server on the Mini seems to be ignoring /etc/proftpd/proftpd.conf in this matter.  So you must NOT make your home directory read-only to your group using the Mini web page, if you ever want to use FTP to write in your home directory.  This seems to be a property of the vanilla Mini, and is not related to anything done above.
Line 298: Line 301:
 
== Power Failure ==
 
== Power Failure ==
  
Power fails in my neighborhood several times a year, so I wanted some assurance the Mini could recover from a power failure.  So I unplugged it, waited 10 seconds, and plugged it back in.  After it finished restarting, I did a `Disk Management/Disk Check' which runs fsck on /mnt/array1, and `Raid Setup/Begin Immediate RAID Scan' (which takes over an hour, during which time you can use the Mini normally), with no errors detected.  -:)
+
Power fails in my neighborhood several times a year, so I wanted some assurance the Mini could recover from a power failure.  So I unplugged it, waited 10 seconds, and plugged it back in.  After it finished restarting, I did a `Disk Management/Disk Check' which runs fsck on /mnt/array1, and `Raid Setup/Begin Immediate RAID Scan' (which takes over an hour, during which time you can use the Mini normally), with no errors detected.  :-)
  
 
== How Boot Works ==
 
== How Boot Works ==
Line 304: Line 307:
 
This section currently just has some information tidbits that hopefully will eventually lead to being able to boot off something other than the Mini hard disk.
 
This section currently just has some information tidbits that hopefully will eventually lead to being able to boot off something other than the Mini hard disk.
  
Using tcpdump and nmap I determined that the Mini never sends ethernet packets or listens for such until the Linux kernel has been booted from disk (with 99% certainty), no matter what.
+
Hitting the top button during boot does nothing.  However, holding it down when turning power on causes the Mini to go into a special state where it is rapidly blinking just its topmost blue LED.  If you release and repress the top button, it goes into a different state in which the topmost blue LED and the LinkStation Logo blue LED blink rapidly, and then after a while it goes into its normal boot state where
 +
just the Logo LED blinks rapidly.
  
Hitting the top button during boot does nothing.  However, holding it down when turning power on causes the Mini to go into a special state where it is rapidly blinking just its topmost blue LED.  If you release and repress the top button, it goes into a different state in which the topmost blue LED and the LinkStation Logo blue LED blink rapidly, and then after a while just the Logo LED blinks rapidly, which is the normal boot state, and the Mini reboots normally.  During this time no packets are sent on the ethernet and the Mini does not listen on sockets.
+
The only thing the above sequence seems to do differently from the standard boot sequence is that it resets the `admin' user password to `password'.
  
So if the flash boot pays any attention to the world outside the Mini, it must be the USB port.
+
Below the contents of the proported flash boot contain lines suggesting that if the Mini has ip address 192.168.11.150 and your computer has ip address 192.168.11.1 then the flash boot has a TFTP mode for booting over the internet.  My computer did not come with a TFTP server, but I was able to obtain one named `tftpd' via `yum install tftp-server'.  I set the ip addresses as indicated, turned my firewall off, and used tcpdump to monitor packets to my computer.  I did not see any coming from the Mini during the above non-normal boot sequence.
 +
 
 +
I connected up a USB to serial cable and during all the boot phases its lights stayed off, suggesting that the flash boot ignores the USB.  At the end of boot one of the lights comes on, and looking in /proc/bus/usb/devices I find the serial cable is listed and has been recognized by Linux.
 +
 
 +
Its clear from [[Add a Serial port to the LS MINI]] that the Mini is not designed to use the serial port, as there is no easy way to connect to it even after opening the box up, although the flash boot loader does communicate with the serial port.
 +
 
 +
Disappointment so far.  Some other method of figuring out if and when TFTP mode is triggered is needed.
  
 
References:
 
References:
  
 
     [[U-boot bootloader]]
 
     [[U-boot bootloader]]
 +
    [[LS Mini: Serial Port Output - Boot-Log]]
 +
    [[LS Mini: Serial Port Output - U-Boot]]
 +
    [[Add a Serial port to the LS MINI]]
  
 
Result of fdisk -l /dev/sda:
 
Result of fdisk -l /dev/sda:
Line 327: Line 340:
 
     /dev/sda6            874      30295  236332183+  83  Linux
 
     /dev/sda6            874      30295  236332183+  83  Linux
  
Result of rsyncing /boot to my CentOS Linux computer and doing file * on the contents (file on the Mini is too ignorant):
+
Result of rsyncing /boot to my CentOS Linux computer and doing file * and ls -o on the contents (file on the Mini is too ignorant):
  
 
     boot/conf_save.tgz:                  gzip compressed data, from Unix, max compression
 
     boot/conf_save.tgz:                  gzip compressed data, from Unix, max compression
Line 336: Line 349:
 
     boot/uImage.buffalo:                PPCBoot image
 
     boot/uImage.buffalo:                PPCBoot image
  
== To Do ==
+
    -rw-r--r-- 1 walton    24101 Feb 12 20:14 boot/conf_save.tgz
 +
    -rw-r--r-- 1 walton 104485766 Jun 30  2008 boot/hddrootfs.buffalo.updated.done
 +
    -rw-r--r-- 1 walton  5637647 Jun 30  2008 boot/initrd.buffalo
 +
    -rw-r--r-- 1 walton    245776 Jun 30  2008 boot/u-boot.buffalo
 +
    -rw-r--r-- 1 walton  1757604 Jun 30  2008 boot/uImage.buffalo
 +
 
 +
Note the u-boot just fits in 256KB, and is the only likely candidate for the contents of FLASH.
 +
 
 +
Result with deletings (indicated by . . . . .) of `strings u-boot.buffalo':
  
Investigate how to connect to the bootstrap program uBoot (I could not get nc to work the first time I tried), how to load new firmware using Linux (no Window's system available), how to unbrick a Mini with blank disks. Also adding USB flash drives, and possibly using them as removable disks that substitute for / and /boot.
+
    buffalo_change_power_status
 +
    buffalo_IsStopAtUbootStatus
 +
    . . . . .
 +
    power sw is pushed
 +
    power sw is released
 +
    init
 +
    init sw is pushed
 +
    init sw is released
 +
    auto
 +
    auto sw is pushed
 +
    auto sw is released
 +
    func
 +
    func sw is pushed
 +
    func sw is released
 +
    . . . . .
 +
    func switch pushed, and exited loop
 +
    wait until func switch released.
 +
    . . . . .
 +
    Present led on off status=0x%04x
 +
    Present led blink status=0x%04x
 +
    Present led control status=0x%04x
 +
    . . . . .
 +
    missing or unknown FLASH type
 +
    . . . . .
 +
    === BUFFALO LS-GL U-Boot. ===
 +
    === BUFFALO LS-GL High Speed U-Boot. ===
 +
    === BUFFALO TS-HTGL R5 U-Boot. ===
 +
    === BUFFALO TS-HTGL R1 U-Boot. ===
 +
    === BUFFALO LS-GL R1 U-Boot. ===
 +
    === BUFFALO HS-WDHTGL/R1 U-Boot. ===
 +
    === BUFFALO LS-WTGL/R1 U-Boot. ===
 +
    === BUFFALO LS-WGL U-Boot. ===
 +
    === BUFFALO LS-WSGL/R1 U-Boot. ===
 +
    === BUFFALO LS-HGL U-Boot. ===
 +
    === BUFFALO Unknown U-Boot. ===
 +
    . . . . .
 +
    USB 0: host mode
 +
    USB 1: host mode
 +
    . . . . .
 +
    BOOTVER=1.13
 +
    build_time
 +
    18:48:49
 +
    . . . . .
 +
    disaMvPnp
 +
    overEthAddr
 +
    usb0Mode
 +
    host
 +
    usb1Mode
 +
    force_tftp
 +
    Switched to func-boot mode.
 +
    Now waiting for func switch release.
 +
    Funcka switch is release.
 +
    Now waiting for func switch pushed or timeout.
 +
    The power switch was moved to "off" position.
 +
    Func switch pushed.
 +
    Go on func-boot mode.
 +
    func-boot mode, timed out.
 +
    Switch to normal boot mode.
 +
    func=1
 +
    bootargs_func
 +
    func=0
 +
    both power and init switch pushed. change to force tftp mode
 +
    Eht phy is not found
 +
    . . . . .
 +
    Please Press HDD power button to continue ...
 +
    Magic key condition is ups shutdown.
 +
    Indicate ups error and wait re power active.
 +
    HDD is not found
 +
    . . . . .
 +
    BOOTP broadcast %d
 +
    RARP broadcast %d
 +
    . . . . .
 +
    hit any key to switch tftp boot.
 +
    Hit any key to stop autoboot: %2d
 +
    . . . . .
 +
    switched to TFTP boot.
 +
    . . . . .
 +
    rarpboot- boot image via network using RARP/TFTP protocol
 +
    . . . . .
 +
    tftpboot- boot image via network using TFTP protocol
 +
    . . . . .
 +
    bootp  - boot image via network using BootP/TFTP protocol
 +
    . . . . .
 +
    ethaddr=00:00:00:00:51:81
 +
    bootargs=$(bootargs_base) $(bootargs_root)
 +
    bootcmd=ide reset; ext2load ide 1:1 0x00100000 /$(kernel);ext2load ide 1:1 0x00800000 /$(initrd);setenv bootargs $(bootargs_base)
 +
        $(bootargs_root) $(buffalo_ver);bootm 0x00100000 0x00800000
 +
    bootdelay=3
 +
    baudrate=115200
 +
    loads_echo=0
 +
    ipaddr=192.168.11.150
 +
    serverip=192.168.11.1
 +
    . . . . .

Latest revision as of 14:22, 17 February 2009

Contents

Purpose

This page tells how to set up the LinkStation Mini to do backups using rsync and ssh. The intended audience are readers who know Linux or some other UNIX, at the root level, but do not know about LinkStations or single chip computers. Note that setting up the Mini for ssh and rsync will void the warranty, and that no warranty is implied or liability accepted by the authors of this page.

[The other purpose of this page is to `learn by writing'. I've already got my Mini set up for rsync, but writing this should close the loose ends. -User:Walton]

Do You Really Want Rsync?

I measured the Mini transfer speed as 1.2 megabytes per second doing scp/rsync, which is not exactly blazing, but it did 11 megabytes per second doing ftp of a large file (this is the limit of my old router at 100Mbits/sec so with a faster router the speed may be faster: the Mini did 30 megabytes per second cp'ing from disk to /dev/null). I suspect the reason for the slow scp/rsync is that the CPU is slow at encrypting.

If you just want to transfer single large files, ftp would be better. However, if you want to transfer complex directory structures, there are some reasons to prefer rsync. The closest thing to rsync in the ftp world that I know is the lftp mirror command, over which rsync has the following advantages:

  • Rsync preserves file and directory modification times, whereas ftp cannot set modification times on the remote system and does not preserve timestamps.
  • Rsync preserves file permissions, whereas ftp cannot set permissions on the remote system and does not preserve them.
  • Rsync preserves symbolic links, whereas ftp cannot create symbolic links on the remote system and does not preserve them.
  • Rsync has a -vnc option that allows you to check whether your backup matches your original bit-for-bit by computing checksums of the files.

Also, when I used lftp mirror to retrieve a large complex directory system stored by rsync, lftp achieved only a 2.4 megabyte per second transfer rate, and not the 11 megabyte per second rate for large files. My guess is that the Mini is very slow at accessing lots of directory structure: I've seen this in other situations. Perhaps the xfs file system has a large block size and as the Mini only has 128MB RAM it caches too few blocks.

Lastly you should always consider getting flash memory stick drives for backup before buying any backup disk. You can do rsync to a flash drive very easily. (Hints: you will likely want to `dd if=/dev/zero of=/dev/YOURFLASH count=1000' and then `mkfs -type ext3 /dev/YOURFLASH' to get a good file system on a flash stick drive.)

LinkStation Mini Description

The LinkStation Mini is a very small computer running a fairly full Linux system, in so far as backup is concerned. The manufacturer supports FTP and SAMBA servers (and also LPD for printers), but not rsync or ssh, probably because as RedHat discovered, any attempt to support the latter would result in too many calls to techsupport. However, all the software to use ssh and rsync is already installed, and only a few configuration files need to be edited to make these work.

See the previous section for some timing info.

I put a power meter on the Mini and measured 6-7 watts when it was not transferring data and 7-8 watts when it was transferring data; most excellent.

The LinkStation Mini hardware is:

   ARM9 CPU (266 BogoMIPS: 15 times slower than a 2Gz Intel) Datasheet
   2 hard drives, either 250GB or 500GB each (30 Mbyte/sec transfer to /dev/null)
   128 MB RAM
   one ethernet port (can run at 1 Gb/sec)
   controllers for 2 USB ports (only one connector provided - hub might allow 2 ports)
   controller for several serial ports - no external or internal connector provided
   256KB flash memory - contains uBoot bootstrap loader
                      - not used by Linux or during normal operation

(More details at Disassemble the LS MINI.)

As delivered and then reconfigured to use RAID1, a 250GB disk Mini has the following disk partitions:

   Mounted On         Format        RAID Device         Hardware Devices           Size
   
   /boot              ext3          /dev/md0            /dev/sd{a,b}1              1GB
   /                  xfs           /dev/md1            /dev/sd{a,b}2              5GB
   swap area          swap          /dev/md10           /dev/sd{a,b}5              1GB
   /mnt/array1        xfs           /dev/md2            /dev/sd{a,b}6              236GB

In addition, 15MB of RAM is used as a RAM disk for /tmp. The flash memory is only 256KB and is not used by Linux (it contains a bootstrap program). The /boot partition has all the firmware and possibly can be used to restore the machine to its delivered state, possibly by using the web Maintainance/Initialization page `Reset LinkStation Configuration to Factory Defaults'.

/boot also contains a save of the `current configuration', which includes files such as /etc/passwd, /etc/shadow, etc in the file /boot/config_save.tgz.

The XFS file system format is a full featured Linux file system originated by SGI that is particularly well suited to file transfer operations.

As delivered the Mini has 2 passworded users: admin, whose password you can change using the web page, and root, whose password is presumably known only to the manufacturer.

ACP

The Mini responds to a protocol called ACP that allows you to send commands to the Mini and get back responses. It appears that the Mini always responds to this protocol, except while it is booting up, so this is the root protocol, as it were.

Fortunately there is a program called acp_commander that permits ACP commands to be sent from your Linux computer's command line. To use it, download

   http://downloads.nas-central.org/TOOLS/ALL_LS_KB_ARM9/ACP_COMMANDER/acp_commander.jar

and try it out.

First, you need to set up your Mini by connecting it to the ethernet and establishing a fixed IP address for it (using your router). Then you open the Mini's web page using the IP address, set the Mini hostname using the Basic subpage, and if you like, tell your own computer about the hostname you have choosen, say by editing your /etc/hosts file or equivalent. I gave my Mini the name linkstation1.walton (walton is my router's domain name) and will use this name hereafter in place of the Mini's IP address.

IMPORTANT: The Mini comes without RAID turned on, so the full 500GB or 1TB is available. If you want to use RAID1 so the two disks back up each other, you should change to RAID1 NOW, before you start changing other Mini configuration. See the Mini Disk Management/RAID Setup subpage and click on the tool icon.

Next you need to turn off your own computer's firewall (temporarily). I tried punching a hole in my firewall but it did not work. If you do not turn off your firewall you will get a message containing:

   A SocketTimeoutException usually indicates bad firewall settings

Now you can execute on your own computer

   java -jar acp_commander.jar -t linkstation1.walton -c 'ls -l'

This will execute in the Mini the Linux `ls -l' command with root as the current user and / as current directory. Just to test.

If this works you can execute

   java -jar acp_commander.jar -t linkstation1.walton -o

at which point your manufacturer's warranty is gonzo, at least a little bit. This zeros the root password so you can log in as root without giving a password, and also executes `telnetd' which starts the telnet daemon, so you can now telnet into the Mini.

You can now turn your firewall back on. You are done with ACP.

However, if you want to play before you change anything more, you can do

   java -jar acp_commander.jar -t linkstation1.walton -s

which allows you to type shell commands at the Mini. However, each command is executed in a separate instance of the shell, and interactive commands will hang the Mini ACP interface (you can recover by using the Mini web page or the power switch to reboot).

Some more info is in the ACP Commander README;

   http://downloads.buffalo.nas-central.org/TOOLS/ALL_LS_KB_ARM9/ACP_COMMANDER/README

Many thanks to Georg for writting acp_commander.

Configuring the Mini for SSH and RSYNC

At this point you can log in as root using

   telnet linkstation1.walton

(remember, linkstation1.walton is my abbreviation for the Mini's IP address).

Look around. Get comfortable. Decide you really want to go through with this.

Also, use the `passwd' command to set a secure root password. Be careful what you choose: this is a backup computer and you may not want to have your own computer's passwords flying around the ether, so you may want to use a special password for the Mini.

To enable ssh you need to

   (1) create an /etc/init.d/sshd.sh file
   (2) test running sshd
   (3) edit the /etc/init.d/rcS file so the above file is executed when the Mini reboots
   (4) test rebooting
   (5) edit the /etc/daemonwatch.list file so sshd will be restarted if it dies
   (6) test killing sshd
   (7) change permissions on /etc/profile
   (8) create the /var/log/lastlog file

(Note: all this comes from Open Stock Firmware: many thanks to the authors of this page. Also: my Mini firmware version is 1.04 which appears to be a later version than the 1.11a firmware mentioned in this reference, which is for an older LinkStation model.)

The sshd.sh file contents are

   #! /bin/sh
   # Start/stop the SSH daemon
   #
   #
   
   test -f /usr/local/sbin/sshd || exit 0
   
   # this is used by deamonwatch
   ACTIVE_FILE=/var/run/active_sshd
   
   case "$1" in
   start)  echo -n "Start services: sshd"
           /usr/local/sbin/sshd -f /etc/sshd_config 2> /dev/null &
           touch $ACTIVE_FILE
           ;;
   stop)   echo -n "Stop services: sshd"
           killall sshd
           rm -f $ACTIVE_FILE
           ;;
   restart)
           $0 stop
           $0 start
           ;;
   *)      echo "Usage: $0 start|stop|restart"
           exit 1
           ;;
   esac
   exit 0

(Note: the firmware on my Mini is 1.04, and supports daemonwatch. Earlier versions of the firmware might not support daemonwatch, in which case see Open Stock Firmware. To test for the presence of daemonwatch execute `which daemonwatch'.)

You can create sshd.sh by using the vi editor. A bit easier is to execute

   cd /etc/init.d
   cat >sshd.sh
   <copy the above file contents here using your mouse>
   <control-D>
   vi sshd.sh
   <remove the 4 spaces beginning each line: YOU MUST DO THIS FOR FIRST LINE>

After creating sshd.sh set its mode by `chmod 755 sshd.sh'.

Now test ssh. Just

   cd /etc/init.d
   ./sshd.sh start

and try to log into your Mini by executing in another window of your computer

   ssh root@linkstation1.walton

and giving your Mini's root password. This should work. You may get an error message concerning /etc/profile which you can ignore. If things do not work, examine the section of Open Stock Firmware that discusses `Config' (or more explicitly, /etc/sshd_config).

You are not done because sshd will not yet restart on reboot (nor will telnet). To fix this you must add `sshd.sh' to the right spot in the /etc/init.d/rcS file. Do

   cd /etc/init.d
   cp -p rcS rcS.bak
   vi rcS
   /step3
   <add ` sshd.sh' to the end of the next `for' line>
   :wq

That is, rcS is executed to start the system during reboot, and when it gets to step3 it will now execute sshd.sh along with everything else it does.

Now try rebooting the Mini using the web page Maintenance/Restart `Restart Now'. Note that rebooting the Mini is a rather slow operation. Once its rebooted log in again using ssh. Telnet should no longer work (you could restart it by executing `telnetd', which if necessary you can do using acp_commander `-c telnetd').

Next you need to tell daemonwatch about sshd.sh so it will restart sshd if that should die. You need to add a line to /etc/daemonwatch.list. Do

   cd /etc
   cp -p daemonwatch.list daemonwatch.list.bak
   vi daemonwatch.list
   <add the line below to the end of the file>
   :wq
   cd /etc/init.d
   ./daemonwatch.sh restart

The line to add is:

   /var/run/sshd.pid             /var/run/active_sshd    /etc/init.d/sshd.sh restart


Now test. Do

   killall sshd

Your ssh connection should die. But after a minute or so you should be able to reconnect. If you cannot you will have to reboot as above to restart sshd.

Lastly a bit of cleanup:

   chmod 755 /etc/profile
   touch /var/log/lastlog
   chmod 644 /var/log/lastlog

This permits /etc/profile to be properly used when any user logs in via ssh, and gets rid of `lastlog does not exist' error messages in /var/log/messages.

Rsync should now work! You can test it by executing on your own computer:

 cd ~
 mkdir buffalo
 cd buffalo
 rsync -av root@linkstation1.walton:/boot/ boot
 <enter your root password>
 file boot/*

We will talk more about the contents of the /boot directory below.

Adding Users

It seems to me best to use the Mini web page to add users, home/shared directories, groups, etc. as then the standard Mini software will know about them and they will be usable via FTP, SAMBA, etc. Doing this works but you have to do a bit extra for rsync.

First, what do the Mini web pages do?

Their configuration information base is kept in ASCII /etc/melco/* files. So when you make changes using the `Shared Folders Setup', `Group Management', or `User Management' pages you are modifying these files.

In addition to modifying the above information base:

  • Users have a line in /etc/passwd and /etc/shadow. However, the home directory of all users (in /etc/passwd) is /home. Also, all users (except the builtin `guest' user) are added to the hdusers group in /etc/group and have the hdusers group number as their login group in /etc/passwd.
  • Groups have a line in /etc/group listing all the users they contain.
  • Shared files are located in /mnt/array1 (this might be different if you are not using RAID1). However, this file has owner root, group root, and protections drwxrwxrwx.

So to add an `rsync user' named `ruser' do the following:

  1. Using the `User Management' page add `ruser' as a user.
  2. Using the `Group Management' page add `ruser' as a group and put the user `ruser' in this group (normally as its only member).
  3. Using the `Shared Folders/Shared Folders Setup' page add `ruser' as a shared folder, enable it for FTP and read/write, enable access restrictions, and give the user `ruser' read/write access. WARNING: do NOT give any group `read only' access if the user `ruser' is in that group (see below). I also disable `Recycle Bin', which if enabled causes files that are deleted by FTP to be moved to the trashbox directory inside the `ruser' shared folder, instead of really being deleted.
  4. Edit the `ruser' line in /etc/passwds to replace `/home' by `/mnt/array1/ruser' and to replace the hdusers group number (which is 100 on my Mini) by the `ruser' group's group number (the group number is the second of the two numbers in the line, and the `rusers' group number is in the /etc/group file).
  5. Execute
   cd /mnt/array1
   chown ruser:ruser ruser
   chmod 750 ruser

Given this ssh ruser@linkstation1.walton will land in the home directory /mnt/array1/ruser running a shell with user ruser and group ruser. FTP (if the service is enabled via the `Shared Folders/Service Setup' page) will land in the same home directory running an ftp process also with user ruser and group ruser. So all files created in the Mini by ruser will have user ruser and group ruser.

The mini FTP seems to have the strange property that if you use `Shared Folders/Shared Folders Setup' to make a directory both read-write to you using your user name and also read-only to you using one of your group names, the latter takes precedence for FTP, and your FTP access will be read-only. Also mysterious is that the FTP server on the Mini seems to be ignoring /etc/proftpd/proftpd.conf in this matter. So you must NOT make your home directory read-only to your group using the Mini web page, if you ever want to use FTP to write in your home directory. This seems to be a property of the vanilla Mini, and is not related to anything done above.

Security Settings

There are some file permissions that should be changed if ssh is really going to be much used with your Mini. These may not be important, but for the record here is some info.

Some useful commands are

   find / -name walton3 -prune -o -type l -o -perm 666 -o -exec ls -ld {} \;
   find / -name walton3 -prune -o -type l -o -perm 777 -o -exec ls -ld {} \;
   find / -name walton3 -prune -o -group root -o -exec ls -dl {} \;

which will find the system files that have group other than root or that have the permissions 666 or 777. Here the `-name walton3 -prune -o' serves to exclude the files in my walton3 directory which would have produced a ton of spurious listings, since I was already using my Mini for backup and there were plenty of non-system files in that directory.

Based on the output I got I did the following:

   chmod 755 /home /mnt /mnt/array1
   chmod 644 /etc/pam.d/sudo/sudo.pam

Time

The best way to keep the clock at the right time is to use the network time servers. On the Mini this can be done via the `Basic' page by enabling the NTP Server. I used the `0.centos.pool.ntp.org' server because my computer runs CentOS 5.2 and uses that server, and I set the system to synchronize once a day.

Shutdown

There is no way to shut the Mini down from a web page (at least with my 1.04 firmware). You can shut it down by turning the power switch on the back to OFF. This causes the Mini to execute an orderly shutdown: nothing happens for a few seconds, and then the lights will blink for quite a few seconds before the Mini goes dark.

You should not set the power switch to `auto' unless you have the appropriate software, as the Mini may power up correctly, and then after a while power down for no obvious reason.

Power Failure

Power fails in my neighborhood several times a year, so I wanted some assurance the Mini could recover from a power failure. So I unplugged it, waited 10 seconds, and plugged it back in. After it finished restarting, I did a `Disk Management/Disk Check' which runs fsck on /mnt/array1, and `Raid Setup/Begin Immediate RAID Scan' (which takes over an hour, during which time you can use the Mini normally), with no errors detected.  :-)

How Boot Works

This section currently just has some information tidbits that hopefully will eventually lead to being able to boot off something other than the Mini hard disk.

Hitting the top button during boot does nothing. However, holding it down when turning power on causes the Mini to go into a special state where it is rapidly blinking just its topmost blue LED. If you release and repress the top button, it goes into a different state in which the topmost blue LED and the LinkStation Logo blue LED blink rapidly, and then after a while it goes into its normal boot state where just the Logo LED blinks rapidly.

The only thing the above sequence seems to do differently from the standard boot sequence is that it resets the `admin' user password to `password'.

Below the contents of the proported flash boot contain lines suggesting that if the Mini has ip address 192.168.11.150 and your computer has ip address 192.168.11.1 then the flash boot has a TFTP mode for booting over the internet. My computer did not come with a TFTP server, but I was able to obtain one named `tftpd' via `yum install tftp-server'. I set the ip addresses as indicated, turned my firewall off, and used tcpdump to monitor packets to my computer. I did not see any coming from the Mini during the above non-normal boot sequence.

I connected up a USB to serial cable and during all the boot phases its lights stayed off, suggesting that the flash boot ignores the USB. At the end of boot one of the lights comes on, and looking in /proc/bus/usb/devices I find the serial cable is listed and has been recognized by Linux.

Its clear from Add a Serial port to the LS MINI that the Mini is not designed to use the serial port, as there is no easy way to connect to it even after opening the box up, although the flash boot loader does communicate with the serial port.

Disappointment so far. Some other method of figuring out if and when TFTP mode is triggered is needed.

References:

   U-boot bootloader
   LS Mini: Serial Port Output - Boot-Log
   LS Mini: Serial Port Output - U-Boot
   Add a Serial port to the LS MINI

Result of fdisk -l /dev/sda:

   Disk /dev/sda: 250.0 GB, 250059350016 bytes
   255 heads, 63 sectors/track, 30401 cylinders
   Units = cylinders of 16065 * 512 = 8225280 bytes
   
      Device Boot      Start         End      Blocks   Id  System
   /dev/sda1               1         125     1004031   83  Linux
   /dev/sda2             126         748     5004247+  83  Linux
   /dev/sda4             749       30401   238187722+   5  Extended
   /dev/sda5             749         873     1004031   82  Linux swap
   /dev/sda6             874       30295   236332183+  83  Linux

Result of rsyncing /boot to my CentOS Linux computer and doing file * and ls -o on the contents (file on the Mini is too ignorant):

   boot/conf_save.tgz:                  gzip compressed data, from Unix, max compression
   boot/hddrootfs.buffalo.updated.done: gzip compressed data, from Unix, last modified: Mon Jun  9 00:19:12 2008
   boot/initrd.buffalo:                 PPCBoot image
   boot/lost+found:                     directory
   boot/u-boot.buffalo:                 data
   boot/uImage.buffalo:                 PPCBoot image
   -rw-r--r-- 1 walton     24101 Feb 12 20:14 boot/conf_save.tgz
   -rw-r--r-- 1 walton 104485766 Jun 30  2008 boot/hddrootfs.buffalo.updated.done
   -rw-r--r-- 1 walton   5637647 Jun 30  2008 boot/initrd.buffalo
   -rw-r--r-- 1 walton    245776 Jun 30  2008 boot/u-boot.buffalo
   -rw-r--r-- 1 walton   1757604 Jun 30  2008 boot/uImage.buffalo

Note the u-boot just fits in 256KB, and is the only likely candidate for the contents of FLASH.

Result with deletings (indicated by . . . . .) of `strings u-boot.buffalo':

   buffalo_change_power_status
   buffalo_IsStopAtUbootStatus
   . . . . .
   power sw is pushed
   power sw is released
   init
   init sw is pushed
   init sw is released
   auto
   auto sw is pushed
   auto sw is released
   func
   func sw is pushed
   func sw is released
   . . . . .
   func switch pushed, and exited loop
   wait until func switch released.
   . . . . .
   Present led on off status=0x%04x
   Present led blink status=0x%04x
   Present led control status=0x%04x
   . . . . .
   missing or unknown FLASH type
   . . . . .
   === BUFFALO LS-GL U-Boot. ===
   === BUFFALO LS-GL High Speed U-Boot. ===
   === BUFFALO TS-HTGL R5 U-Boot. ===
   === BUFFALO TS-HTGL R1 U-Boot. ===
   === BUFFALO LS-GL R1 U-Boot. ===
   === BUFFALO HS-WDHTGL/R1 U-Boot. ===
   === BUFFALO LS-WTGL/R1 U-Boot. ===
   === BUFFALO LS-WGL U-Boot. ===
   === BUFFALO LS-WSGL/R1 U-Boot. ===
   === BUFFALO LS-HGL U-Boot. ===
   === BUFFALO Unknown U-Boot. ===
   . . . . .
   USB 0: host mode
   USB 1: host mode
   . . . . .
   BOOTVER=1.13
   build_time
   18:48:49
   . . . . .
   disaMvPnp
   overEthAddr
   usb0Mode
   host
   usb1Mode
   force_tftp
   Switched to func-boot mode.
   Now waiting for func switch release.
   Funcka switch is release.
   Now waiting for func switch pushed or timeout.
   The power switch was moved to "off" position.
   Func switch pushed.
   Go on func-boot mode.
   func-boot mode, timed out.
   Switch to normal boot mode.
   func=1
   bootargs_func
   func=0
   both power and init switch pushed. change to force tftp mode
   Eht phy is not found 
   . . . . .
   Please Press HDD power button to continue ...
   Magic key condition is ups shutdown.
   Indicate ups error and wait re power active.
   HDD is not found 
   . . . . .
   BOOTP broadcast %d
   RARP broadcast %d
   . . . . .
   hit any key to switch tftp boot.
   Hit any key to stop autoboot: %2d 
   . . . . .
   switched to TFTP boot.
   . . . . .
   rarpboot- boot image via network using RARP/TFTP protocol
   . . . . .
   tftpboot- boot image via network using TFTP protocol
   . . . . .
   bootp   - boot image via network using BootP/TFTP protocol
   . . . . .
   ethaddr=00:00:00:00:51:81
   bootargs=$(bootargs_base) $(bootargs_root)
   bootcmd=ide reset; ext2load ide 1:1 0x00100000 /$(kernel);ext2load ide 1:1 0x00800000 /$(initrd);setenv bootargs $(bootargs_base)
        $(bootargs_root) $(buffalo_ver);bootm 0x00100000 0x00800000
   bootdelay=3
   baudrate=115200
   loads_echo=0
   ipaddr=192.168.11.150
   serverip=192.168.11.1
   . . . . .