Enabling Rsync on a Linkstation Mini
From NAS-Central Buffalo - The Linkstation Wiki
Contents |
Purpose
This page tells how to set up the LinkStation Mini to do backups using rsync and ssh. The intended audience are readers who know Linux or some other UNIX, at the root level, but do not know about LinkStations or single chip computers. Note that setting up the Mini for ssh and rsync will void the warranty, and that no warranty is implied or liability accepted by the authors of this page.
[The other purpose of this page is to `learn by writing'. I've already got my Mini set up for rsync, but writing this should close the loose ends. -User:Walton]
LinkStation Mini Description
The LinkStation Mini is a very small computer running a fairly full Linux system, in so far as backup is concerned. The manufacturer supports FTP and SAMBA servers (and also LPD for printers), but not rsync or ssh, probably because as RedHat discovered, any attempt to support the latter would result in too many calls to techsupport. However, all the software to use ssh and rsync is already installed, and only a few configuration files need to be edited to make these work.
I measured the Mini transfer speed as 1.2 megabytes per second doing scp/rsync, which is not exactly blazing, but it did 11 megabytes per second doing ftp (and this is the limit of my old router at 100Mbits/sec), and 30 megabytes per second cp'ing from disk to /dev/null. I suspect the reason for the slow scp/rsync is that the CPU is slow at encrypting.
I put a power meter on the Mini and measured 6-7 watts when it was not transferring data and 7-8 watts when it was transferring data; most excellent.
The LinkStation Mini hardware is:
ARM9 CPU 2 hard drives, either 250GB or 500GB each 128 MB RAM one ethernet port controllers for 2 USB ports (only one connector provided - hub may allow 2 ports to be used) controller for one serial port - no extenal connector provided 256KB flash memory - contains uBoot bootstrap loader - not used by Linux or during normal operation
(More details at Disassemble the LS MINI.)
As delivered and then reconfigured to use RAID1, a 250GB disk Mini has the following disk partitions:
Mounted On Format RAID Device Hardware Devices Size
/boot ext3 /dev/md0 /dev/sd{a,b}1 1GB
/ xfs /dev/md1 /dev/sd{a,b}2 5GB
swap area swap /dev/md10 /dev/sd{a,b}5 1GB
/mnt/array1 xfs /dev/md2 /dev/sd{a,b}6 236GB
In addition, 15MB of RAM is used as a RAM disk for /tmp. The flash memory which is reported to be only 256KB is not used by Linux. The /boot partition has all the firmware and possibly can be used to restore the machine to its delivered state, possibly by using the web Maintainance/Initialization page `Reset LinkStation Configuration to Factory Defaults'.
/boot also contains a save of the `current configuration', which includes files such as /etc/passwd, /etc/shadow, etc in the file /boot/config_save.tgz.
The XFS file system format is a full featured Linux file system originated by SGI that is particularly well suited to file transfer operations.
As delivered the Mini has 2 passworded users: admin, whose password you can change using the web page, and root, whose password is presumably known only to the manufacturer.
ACP
The Mini responds to a protocol called ACP that allows you to send commands to the Mini and get back responses. It appears that the Mini always responds to this protocol, except while it is booting up, so this is the root protocol, as it were.
Fortunately there is a program called acp_commander that permits ACP commands to be sent from your Linux computer's command line. To use it, download
http://downloads.nas-central.org/TOOLS/ALL_LS_KB_ARM9/ACP_COMMANDER/acp_commander.jar
and try it out.
First, you need to set up your Mini and connected to the ethernet and establish a fixed IP address for it (using your router). Then you open the Mini's web page using the IP address, set the Mini hostname using the Basic subpage, and if you like, tell your own computer about the hostname you have choosen, say by editing your /etc/hosts file or equivalent. I gave my Mini the name linkstation1.walton (walton is my router's domain name) and will use this name hereafter in place of the Mini's IP address.
IMPORTANT: The Mini comes without RAID turned on, so the full 500GB or 1TB is available. If you want to use RAID1 so the two disks back up each other, you should change to RAID1 NOW, before you start changing other Mini configuration. See the Mini Disk Management/RAID Setup subpage and click on the tool icon.
Next you need to turn off your own computer's firewall (temporarily). I tried punching a hole in my firewall but it did not work. If you do not turn off your firewall you will get a message containing:
A SocketTimeoutException usually indicates bad firewall settings
Now you can execute on your own computer
java -jar acp_commander.jar -t linkstation1.walton -c 'ls -l'
This will execute in the Mini the Linux `ls -l' command with root as the current user and / as current directory. Just to test.
If this works you can execute
java -jar acp_commander.jar -t linkstation1.walton -o
at which point your manufacturer's warranty is gonzo, at least a little bit. This zeros the root password so you can log in as root, and also executes `telnetd' which starts the telnet daemon, so you can now telnet into the Mini.
You can now turn your firewall back on. You are done with ACP.
However, if you want to play before you change anything, you can do
java -jar acp_commander.jar -t linkstation1.walton -s
which allows you to type shell commands at the Mini. However, each command is executed in a separate instance of the shell, and interactive commands will hang the Mini ACP interface (you can recover by using the Mini web page to reboot).
Some more info is in the ACP Commander README;
http://downloads.buffalo.nas-central.org/TOOLS/ALL_LS_KB_ARM9/ACP_COMMANDER/README
Many thanks to Georg for writting acp_commander.
Configuring the Mini for SSH and RSYNC
At this point you can log in as root using
telnet linkstation1.walton
(remember, linkstation1.walton is my abbreviation for the Mini's IP address).
Look around. Get comfortable. Decide you really want to go through with this.
Also, use the `passwd' command to set a secure root password. Be careful what you choose: this is a backup computer and you may not want to have your own computer's passwords flying around the ether, so you may want to use a special password for the Mini.
To enable ssh you need to
(1) create an /etc/init.d/sshd.sh file (2) test running sshd (3) edit the /etc/init.d/rcS file so the above file is executed when the Mini reboots (4) test rebooting (5) edit the /etc/daemonwatch.list file so sshd will be restarted if it dies (6) test killing sshd (7) change permissions on /etc/profile (8) create the /var/log/lastlog file
(Note: all this comes from Open Stock Firmware: many thanks to the authors of this page. Also: my Mini firmware is 1.04 which appears to be a later version than the 1.11a firmware mentioned in this reference, which is for an older LinkStation model.)
The sshd.sh file contents are
#! /bin/sh
# Start/stop the SSH daemon
#
#
test -f /usr/local/sbin/sshd || exit 0
# this is used by deamonwatch
ACTIVE_FILE=/var/run/active_sshd
case "$1" in
start) echo -n "Start services: sshd"
/usr/local/sbin/sshd -f /etc/sshd_config 2> /dev/null &
touch $ACTIVE_FILE
;;
stop) echo -n "Stop services: sshd"
killall sshd
rm -f $ACTIVE_FILE
;;
restart)
$0 stop
$0 start
;;
*) echo "Usage: $0 start|stop|restart"
exit 1
;;
esac
exit 0
(Note: the firmware on my Mini is 1.04, and supports daemonwatch. Earlier versions of the firmware might not support daemonwatch, in which case see Open Stock Firmware. To test for the presence of daemonwatch execute `which daemonwatch'.)
You can create sshd.sh by using the vi editor. A bit easier is to execute
cd /etc/init.d cat >sshd.sh <copy the above file contents here using your mouse> <control-D> vi sshd.sh <remove the 4 spaces beginning each line: YOU MUST DO THIS FOR FIRST LINE>
After creating sshd.sh set its mode by `chmod 755 sshd.sh'.
Now test ssh. Just
cd /etc/init.d ./sshd.sh start
and try to log into your Mini by executing in another window of your computer
ssh root@linkstation1.walton
and giving your Mini's root password. This should work. You may get an error message concerning /etc/profile which you can ignore. If things do not work, examine the section of Open Stock Firmware that discusses `Config' (or more explicitly, /etc/sshd_config).
You are not done because sshd will not yet restart on reboot (nor will telnet). To fix this you must add `sshd.sh' to the right spot in the /etc/init.d/rcS file. Do
cd /etc/init.d cp -p rcS rcS.bak vi rcS /step3 <add ` sshd.sh' to the end of the next `for' line> :wq
That is, rcS is executed to start the system during reboot, and when it gets to step3 it will now execute sshd.sh along with everything else it does.
Now try rebooting the Mini using the web page Maintenance/Restart `Restart Now'. Note that rebooting the Mini is a rather slow operation. Once its rebooted log in again using ssh. Telnet should no longer work (you could restart it by executing `telnetd', which if necessary you can do using acp_commander `-c telnetd').
Next you need to tell daemonwatch about sshd.sh so it will restart sshd if that should die. You need to add a line to /etc/daemonwatch.list. Do
cd /etc cp -p daemonwatch.list daemonwatch.list.bak vi daemonwatch.list <add the line below to the end of the file> :wq cd /etc/init.d ./daemonwatch.sh restart
The line to add is:
/var/run/sshd.pid /var/run/active_sshd /etc/init.d/sshd.sh restart
Now test. Do
killall sshd
Your ssh connection should die. But after a minute or so you should be able to reconnect. If you cannot you will have to reboot as above to restart sshd.
Lastly a bit of cleanup:
chmod 755 /etc/profile touch /var/log/lastlog chmod 644 /var/log/lastlog
This permits /etc/profile to be properly used when any user logs in via ssh, and gets rid of `lastlog does not exist' error messages in /var/log/messages.
Rsync should now work! You can test it by executing on your own computer:
cd ~ mkdir buffalo cd buffalo rsync -av root@linkstation1.walton:/boot/ boot <enter your root password> file boot/*
We will talk more about the contents of the /boot directory below.
Adding Users
It seems to me best to use the Mini web page to add users, home/shared directories, groups, etc. as then the standard Mini software will know about them and they will be usable via FTP, SAMBA, etc. Doing this works but you have to do a bit extra for rsync.
First, what do the Mini web pages do?
Their configuration information base is kept in /etc/melco/* files. So when you make changes using the `Shared Folders Setup', `Group Management', or `User Management' pages you are modifying these files.
In addition to modifying the above information base:
- Users have a line in /etc/passwd and /etc/shadow. However, the home directory of all users (in /etc/passwd) is /home. Also, all users (except the builtin `guest' user) are added to the hdusers group in /etc/group and have the hdusers group number as their login group in /etc/passwd.
- Groups have a line in /etc/group listing all the users they contain.
- Shared files are located in /mnt/array1 (this might be different if you are not using RAID1). However, this file has owner root, group root, and protections drwxrwxrwx.
So to add an `rsync user' named `ruser' do the following:
- Using the `User Management' page add `ruser' as a user.
- Using the `Group Management' page add `ruser' as a group and put the user `ruser' in this group (normally as its only member).
- Using the `Shared Folders/Shared Folders Setup' page add `ruser' as a shared folder, enable it for FTP and read/write, enable access restrictions, and give the user `ruser' read/write access. WARNING: do NOT give any group `read only' access if the user `ruser' is in that group (see below).
- Edit the `ruser' line in /etc/passwds to replace `/home'/ by `/mnt/array1/ruser' and to replace the hdusers group number (which is 100 on my Mini) by the `ruser' group's group number (the group number is the second of the two numbers in the line, and the `rusers' group number is in the /etc/group file).
- Execute
cd /mnt/array1 chown ruser:ruser ruser chmod 750 ruser
Given this ssh ruser@linkstation1.walton will land in the home directory /home/ruser running a shell with with user ruser and group ruser. FTP (if the service is enabled via the `Shared Folders/Service Setup' page) will land in the same home directory running an ftp process also with user ruser and group ruser. So all files created in the Mini by ruser will have user ruser and group ruser.
The mini FTP seems to have the strange property that if you use `Shared Folders/Shared Folders Setup' to make a directory both read-write to you using your user name and also read-only to you using one of your group names, the latter takes precedence for FTP, and your FTP access will be read-only. Also mysterious is that the FTP server on the Mini seems to be ignoring /etc/proftpd/proftpd.conf in this matter. So you must NOT make your home directory read-only to your group using the Mini web page, if you ever want to use FTP to write in your home directory. This seems to be a property of the vanilla Mini, and is not related to anything done above.
Security Settings
There are some file permissions that should be changed if ssh is really going to be much used with your Mini. These may not be important, but for the record here is some info.
Some useful commands are
find / -name walton3 -prune -o -type l -o -perm 666 -o -exec ls -ld {} \;
find / -name walton3 -prune -o -type l -o -perm 777 -o -exec ls -ld {} \;
find / -name walton3 -prune -o -group root -o -exec ls -dl {} \;
which will find the system files that have group other than root or that have the permissions 666 or 777. Here the `-name walton3 -prune -o' serves to exclude the files in my walton3 directory which would have produced a ton of spurious listings, since I was already using my Mini for backup and there were plenty of non-system files in that directory.
Based on the output I got I did the following:
chmod 755 /home /mnt /mnt/array1 chmod 644 /etc/pam.d/sudo/sudo.pam
Time
The best way to keep the clock at the right time is to use the network time servers. On the Mini this can be done via the `Basic' page by enabling the NTP Server. I used the `0.centos.pool.ntp.org' server because my computer runs CentOS 5.2 and uses that server, and I set the system to synchronize once a day.
Shutdown
There is no way to shut the Mini down from a web page (at least with my 1.04 firmware). You can shut it down by turning the power switch on the back to OFF. This causes the Mini to execute an orderly shutdown: nothing happens for a few seconds, and then the lights will blink for quite a few seconds before the Mini goes dark.
You should not set the power switch to `auto' unless you have the appropriate software, as the Mini may power up correctly, and then after a while power down for no obvious reason.
Power Failure
Power fails in my neighborhood several times a year, so I wanted some assurance the Mini could recover from a power failure. So I unplugged it, waited 10 seconds, and plugged it back in. After it finished restarting, I did a `Disk Management/Disk Check' which runs fsck on /mnt/array1, and `Raid Setup/Begin Immediate RAID Scan' (which takes over an hour, during which time you can use the Mini normally), with no errors detected. -:)
How Boot Works
This section currently just has some information tidbits that hopefully will eventually lead to being able to boot off something other than the Mini hard disk.
References:
U-boot bootloader
Result of fdisk -l /dev/sda:
Disk /dev/sda: 250.0 GB, 250059350016 bytes
255 heads, 63 sectors/track, 30401 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 1 125 1004031 83 Linux
/dev/sda2 126 748 5004247+ 83 Linux
/dev/sda4 749 30401 238187722+ 5 Extended
/dev/sda5 749 873 1004031 82 Linux swap
/dev/sda6 874 30295 236332183+ 83 Linux
Result of rsyncing /boot to my CentOS Linux computer and doing file * on the contents (file on the Mini is too ignorant):
boot/conf_save.tgz: gzip compressed data, from Unix, max compression boot/hddrootfs.buffalo.updated.done: gzip compressed data, from Unix, last modified: Mon Jun 9 00:19:12 2008 boot/initrd.buffalo: PPCBoot image boot/lost+found: directory boot/u-boot.buffalo: data boot/uImage.buffalo: PPCBoot image
To Do
Investigate how to connect to the bootstrap program uBoot (I could not get nc to work the first time I tried), how to load new firmware using Linux (no Window's system available), how to unbrick a Mini with blank disks. Also adding USB flash drives, and possibly using them as removable disks that substitute for / and /boot.
