Hacking

From NAS-Central Buffalo - The Linkstation Wiki
Revision as of 02:31, 11 March 2007 by Bunshichi (Talk)

Jump to: navigation, search

That is what this site is intended for:

Contents

Hardware Hacking

  • Adding a serial console.
  • become root: firmware version 1.04
  • moving disks: change the firmware by moving the disks to a PC.
  • Real NAS: Converting the Terastation into a NAS like server with an additional os hard disk and a fully featured operating system.
  • Extra Disks: Adding Extra disks to make a 1TB Terastation into a 2TB Terastation. With the potential to create a 6TB Terastation (Using eight 750Gb Drives).
  • Replacing all drives in 06TB : Upgrading all 4 drives.

Firmware Hacking

You can create your own firmware, it's explained here.

Already hacked firmware

Dave Walker created a hacked firmware that you can just download and use (Thank you Dave!):

 http://homepage.ntlworld.com/itimpi/buffalo.htm
 mirror: http://supportonsite.dk/files.php
 minimalistic (ssh only): http://blog.kevinashley.com/articles/2006/04/14/minimalistic-terastation-1tb-hacking-to-allow-proper-rsync-and-ssh

Success count (if you are successful, please add to this list):

  1. Bluefedora, 1.12, 2005-11-18
    • added dropbear as outlined below
  2. Jones, 1.12, 2005-10-11
    • Hacking with SSH and SAMBA 3.0.14a (Debian package).
    • Now I am shooting on the problem of led (STATUS/FULL) not working properly.
  3. yukiman, 1.12, 2005-12-03
    • Used Dave's firmware with 1.12 installed and update worked fine, logged in now via telnet.
    • Got dropbear working as well.
  4. mh123083, 1.12, 2006-01-13
    • Used Dave's firmware with dropbear and allnfs - works perfectly.
  5. alun, 1.12, 2006-01-19
    • Used Dave's firmware. Updated from (factory) 1.08. Have not installed anything else yet. Worried about possible nfs problems...
  6. SKarp 18:38, 2 Feb 2006 (CET)
    • Updated from stock 1.03 to Dave's firmware 1.12 and added allnfs. Working fine in early testing.
  7. Raphaël 22:55, 24 Mar 2006 (JST)
    • Updated from 2.03 to Dave's 2.03, works fine.
    • 2.05a didn't work and left my box flat, but an upgrade to 2.03 afterwards restored the situation.
  8. poxy bowsy 9 April 2006
    • Updated from 1.12 to Dave's 2.05a, but install failed, unit was EM'ed
    • reinitialized and tried to revert to 1.12a but unit was still EM'ed
    • reinitialized and tried to revert to stock 1.12 but unit was still EM'ed
    • reinitialized and installed 2.05a again and everything seems fine (whew)
  9. Ringgh0st, 1.12 2006-04-10
    • upgraded to Dave's hacked Firmware v1.12 on a v1.12 Terastation
    • added Midnight Commander (starts with a little codepage fault, but notthing really important)
    • didnt get the lundftpd working yet, the service doesnt start (no errorlog), if i try to start it manually with the -v string a "segmentation fault" pops up
  10. torbens
    • 27 April 2006: 1st station updated to Dave's v2.08a using Japanese firmware updater
    • 29 April 2006: 2nd station updated to Dave's v2.08a using Japanese firmware updater (had to use recovery with powerbutton on diag.)
    • 1 Juli 2006: 3rd station pro updates to Dave's v1.03 using Japanese firmware updater
  11. Risse-it 02.05.2006
    • Updated TS0.6GB from 1.12 to minimalistic (ssh only)
    • didn't reboot after installing MC and disc-tuning via hdparm (don't know which was the problem - i did both)
    • restored by mounting the first IDE-drive to my Gentoo-x86 (xfs_repair -L) and copied the imagefiles from minimalistic
    • hdparm now runs with -d1 -c1 -m16 -a64 -A1 -u1 -k1 -S241 (any suggestions?)
    • patched some .cgi and .pl-files for more statistics (anyone who's running rrdtool or any other statistics???)
    • USB-printer will only print from Windows (VMware) and not via smb from Gentoo or Ubuntu (Epson Stylus Color C82)
    • patched mail-notifier (to do smtp-auth)
  12. Entropy 2006-May-20
  13. --Luminaire 11:14, 31 May 2006 (CEST)
    • Used successfully on TS HD-H1.0TGL/R5 (1.12 firmware). Works a treat. Going to try NFS/NIS.

Used successfully on TS-1.0TGL/R5 Works but damn is this thing slow. Truly bound by pathetic CPU. Doesn't even begin to test the Gigabit network

  1. -- Kimbotha 14:57, 25 October 2006 (CEST)
    • 24 October 2006: Updated new 1TB Terastation from v1.12 to Dave's v2.10a - Failed first time, reapplied v2.10a and it nows works fine.
    • Installed dropbear following instructions successfully
    • Tried to install kernel mode nfs as per instructions but failed to load modules
    • installed user mode nfs as per Entropy's instructions successfully
    • currently looking to build a cross toolchain so I can build unfs3 as I need >2GB file support
  2. -- Kimbotha 16:37, 25 October 2006 (CEST)
    • While collecting the components to try and start build a cross compile toolchain I found an set of nfs kernel modules and the nfs kernel daemon here (http://downloads.linkstationwiki.net/terastation/kernel_modules/). These seem to load fine on my TS (with a change to the /etc/init.d/nfs-kernel-server to allow rpc.mountd to start). Will try and run some tests on >2GB files when I have some on there to work with.

Software for the TeraStation

I will add small software packages as .tgz here for you to install onto the TeraStation.

As a rule, you should be able to use most Debian PowerPC packages built for woody. Use ftp to access ftp.de.debian.org (or another mirror), go to pub/debian/pool/main/ and to the directory named with the first letter of what you are looking for, there you should find a directory for the package; download a suitable version (should have "woody" and "powerpc" in the name), unpack it on a Debian box using "dpkg-deb -x package.deb name-of-temporary-dir", remove any clutter (man pages) from your temporary directory, create a tar file and copy it over to the TeraStation.

(NB I wanted to upload a .tgz but the Wiki didn't let me, saying that .tgz was not an accepted image format. --Fred 12:05, 12 Dec 2005 (CET))

Dropbear

A small ssh server. home page

The install package (108k) already contains the proper links to have it start on boot, and it contains two keys for the server. Maybe you want to generate your own set of keys.

Just extract the .tgz as root into the root of your TeraStation:

dmin@HD-HTGL113:~$ sudo -s
Password:
root@HD-HTGL113:~# cd /
root@HD-HTGL113:/# tar -xzf /mnt/array1/share/dropbear.tgz

generate your own set of keys:

root@HD-HTGL113:/# rm /etc/dropbear/dropbear_rsa_host_key
root@HD-HTGL113:/# rm /etc/dropbear/dropbear_dss_host_key
root@HD-HTGL113:/# dropbearkey -t rsa -f /etc/dropbear/dropbear_rsa_host_key
Will output 1024 bit rsa secret key to '/etc/dropbear/dropbear_rsa_host_key'
Generating key, this may take a while...
Public key portion is:
...
root@HD-HTGL113:/# dropbearkey -t dss -f /etc/dropbear/dropbear_dss_host_key
Will output 1024 bit dss secret key to '/etc/dropbear/dropbear_dss_host_key'
Generating key, this may take a while...
Public key portion is: 
...

and reboot.


Can i setup publick key authentication on dropbear. If so can someone give me an example. I was able to upgrade the firmware succesffully on the Terastation Pro. I am planning to do a Public Key Authentication on my mac and use rsync for backup using ssh protocol. Any ideas? Thanks a gazillion, Naven

Yes you can: Create a key on your Linux/Unix workstation with: ssh-keygen -t rsa The public and private key will be automatically saved to the directory .ssh in your homedir.

Next: copy the text from id_rsa.pub and connect to your Terastation as root.

On your Terastation: - mkdir /root/.ssh - vi /root/.ssh/authorized_keys - paste the text you copied in the step before - save the file

Now you can connect to your Terastation by using public key authentication

Midnight Commander

For a description on how to install, see Midnight Commander.

Encryption, NTFS Support, and Windows Share Management

Aaron explains how he converted his TeraStation to an ETH-disk and gained Encryption, NTFS Support, and Windows Share Management by using an additional Windows system.

Samba

It is possible to modify the TeraStation's Samba install to properly map the System, Archive, Hidden, and Read-Only attributes for files (not directories). This modification will require you have Telnet access to your TeraStation as per the modifications previously mentioned.

The general approach is this. The /etc/samba/smb.conf file contains Samba's configuration settings. To map the attributes, we need to add the following to the global section or to the individual shares:

map hidden = yes
map archive = yes
map system = yes
create mode = 771
directory mode = 770

We also need to remove the "force create mode" and "force directory mode" entries. Unfortunately, just modifying the file is not enough because every boot, a script calls /sbin/teractrl -k which overwrites the configuration file. I noticed that most of the scripts also called /usr/local/bin/mkrsconf.pl afterwards to setup the rsync config for backup purposes, so I piggy back on that script to rewrite the Samba configuration.

Make a backup of your existing mkrsconf.pl file, and then replace the original with the new file content I have below. If you're working in Windows, you can copy the file to one of the shares under /mnt/array1, just be sure to use a UNIX compatible text editor if you don't want your newlines to get messed up. Then you can copy from there to the /usr/local/bin folder via Telnet to overwrite the original file.

The other thing you'll need to do when you're done is update all of the attributes on your existing shares since previously everything was probably created with 777 permisions, which will result in all of your files now showing up as hidden, system files. You can change the permissions from Telnet using chmod, or you can do it from a Windows Command Prompt using the attrib command.

Also of note, TeraStation generated files are created by "root", so with group permissions adjusted now you may not be able to modify attributes of files like backup logs. It's probably not necessary though as you can still read/write the content, but if you need to, you can either take care of those from Telnet by doing a chown to change the owner, or you can modify the script to also add:

force group = root
force user = root

I haven't tested the latter scenario, but I understand it will force all content on disk to be owned by root and have all share access performed as root. This isn't quite as bad as it sounds. Check out this link for more information: http://www.oreilly.com/catalog/samba/chapter/book/ch05_03.html

Why these extremely useful features weren't on by default I don't know. Here's a link to the modified file. Be sure to rename it to mkrsconf.pl on your side!

http://www.spicypixel.com/download/mkrsconf.pl.txt

One word of caution, for most configurations I think I'm picking up all of the required entries from the system configuration files. But, it's entirely possible you may have a configuration parameter that isn't currently read by the script. If so, please note the discrepency, let me know so I can update it, and make sure you update your copy to pick up the extra parameter if you need to use it in the mean time.

Once you've replaced the file, just type "reboot" from the Telnet prompt and when your TeraStation comes back up, you should have the ability to change file attributes. This will allow you to use real backup programs that make use of the Archive attribute for incremental backups.

--Aaron 23:58, 10 Mar 2006 (CET)


I have noticed the chown and chmod on the TS were not compiled with large file support, so if you may get something like the following if you try and change the permissions on a large file:

 # chown kimbotha:hdusers ./hdc-20060305.img.bz2
 chown: ./hdc-20060305.img.bz2: Value too large for defined data type
 # chmod g+w ./hdc-20060305.img.bz2
 chmod: ./hdc-20060305.img.bz2: Value too large for defined data type

Luckily, perl is compiled with large file support so some equivalent commands would be:

 # perl -e 'chown 1002, 100, "hdc-20060305.img.bz2";'
 # perl -e 'chmod 0775, "hdc-20060305.img.bz2";'

-- Kimbotha 11:21, 26 October 2006 (CEST)

NFS

There are two choices for NFS support: Kernel-mode and User-mode. Kernel-mode has better performance than User-mode, but User-mode is more stable.

Kernel-Mode NFS

OK, [1]. This contains the kernel modules, the utilities and the startup files for kernel NFS.

Then, because the symlink that is /var gets overwritten:

cp -a /var/lib /mnt/ram/var/lib
mv /var /var.old
ln -s /mnt/ram/var /var

Edit /etc/exports to match your setup, then reboot or run the init files by hand:

depmod -a
/etc/init.d/portmap start
/etc/init.d/nfs-common start
/etc/init.d/nfs-kernel-server start


Warning

Problems with NFS may arise, please see the discussion page for details.

Having just spent several days trying to get NFS working I can give a few tips. If you can write to the NFS share, but have problems reading from it (transfers stall or terastation randomly reboots) try disabling jumbo packets. If the jumbo packet size on all your machines doesn't match than UDP transfers such as NFS wreak havoc with the terastation. I now have the 2.03 firmware and user mode NFS (from debian PPC) working smoothly. An alternative solution would be to use TCP rather than UDP for NFS, but not all NFS implementations support this.

Alternative to NFS

After following in the foot steps of other NFS experimenters, I tried compiling my own NFSD module with TCP support. I was using the 2.4.20 kernel which comes with the OpenTera 2.12 firmware hack.

After finally getting the toolchains and code compiled, and after a battle with symbol names (the TS uses the "versioned" symbols suffixes), the NFSD w/ TCP had horrible performance and would stall out quickly. I had to reboot it once which threw the TS into it's nightmarish 12 hour raid consistency check.

My motivation for adding NFS support was so that I could perform rdiffbackup's to back up a "live" linux machine. I didn't want to loose file attributes and permissions through SMB/CIFS shortcomings.

Another alternative, while not the highest performance, but /will/ achieve the same results is to create an EXT2/3 or XFS binary image on the TS. Mount the location containing this image via smb, then use loopback to mount the FS image file itself.

I did the following: Set up a new folder for sharing called "foo", then telnet to the TS

cd /mnt/array1/foo
dd if=/dev/zero of=foo.ext2.bin bs=1M count=100000
mke2fs foo.ext2.bin

Switch to your linux "client" machine:

mount -t smbfs -o yadda,yadda //ts/foo /mnt/foo.smb
mount -t ext2 -o loop=/dev/loop0 /mnt/foo.smb/foo.ext2.bin /mnt/foo

do your work on /mnt/foo. When you're done:

sync
umount /mnt/foo
umount /mnt/foo.smb

I ran into one small problem, however. Performing dd as 'myroot' on the TS left me with incorrect owner and permission bits. Come to find out the default coreutils don't support large files. I had to install midnight commander to do the 'chmod' and 'chown' --AxMstrLP 06:55, 11 February 2007 (CET)

Notes

Missing rpcinfo? After installing nfsd_terastation.tgz "tar -C / -xzf nfsd_terastation.tgz" the initialization script "/etc/init.d/nfsd-kernel-server" references "rpcinfo" and fails to start rpc.mountd because rpcinfo is missing and the parameters for rpc.mountd are wrong.

Edit the nfsd-kernel-server file and set the option on line 22 to:

RPCMOUNTDOPTS="--no-nfs-version" or RPCMOUNTDOPTS="--nfs-version"

Note line 39-40. It uses RPCMOUNTDOPTS to build an options line for rpc.mountd. Simply put on line 22 RPCMOUNTDOPTS="--nfs-version" and when you start the server, it will run "rpc.mountd --nfs-version 3".

If you get a "Permission Denied" when trying to mount an export, check to see that the reverse DNS entry resolves correctly for the server trying to mount. On the terastation, update your /etc/hosts file (optional) and create a script to reset this file every time the server boots. /etc/rc.d/init.d/networking overwrites this file.

I put in "--nfs-version 3" as I only want v3 support. I am still testing this. -- Kimbotha 10:45, 26 October 2006 (CEST)

--status: Please note that I have personally had a HUGE problem with the NFS CLIENTS when using this server. There seems to be a large degree of "retransmissions" and I/O problems as a result of communicating with the nfs server. (Run "nfsstat" on the client). If there has been any updates to the server, I'd wish they would be published! Perhaps I'll just fall back to using "smbmount" Also, this server appears to have only UDP support. NFS Clients complain with "nfs over UDP causes data corruption" (ick). -- 12:00, November 24 2006 (SGT)

--note: can anyone translate japanese? This might be some pertinent information. I am also trying to get NFS working well. http://blog.livedoor.jp/nano_j/archives/50498734.html --Brent 04:23, 3 December 2006 (CET)


The above linked file "all-nfsd.tar.gz" seems to be for linkstation, not terastation. I believe the correct stuff is here: [2] --Neilfred 04:46, 14 January 2007 (CET)


-- Regarding Brent's reference, the Japanese page describes how to implement NFS as a user-level driver (as opposed to kernel-level). It is, however, incomplete in its description of what to do (you have modify the init.d scripts, etc.). Still, using it as a springboard, I have gotten the user-level driver working on my TS, and I've been stress-testing it days without a hiccup. However, I've been only getting 2.2MBytes/sec writes using it (where I can get 4.2Mbytes/sec writes using the rsync daemon, but the rsync daemon shipped has problems with extra-long file names that the NFS driver does not).

Kudo's to Brent for pointing out the link. Bunshichi 10 March 2007

User-Mode NFS

A User-mode NFS package is now available on Dave's OpenTera page [3]. Bunshichi 10 March 2007

Lundman page

Alternative Media Server and FTPD. home page

llink 861k.

lundftpd 1.2m.


--Scott 19:09, 18 Aug 2005 (CEST)


Minimalistic changes to firmware with SSH only (also includes step by step process)

Minimalistic ready-to use firmware update with step-by-step guide. All credits to the authors of this wiki.

[4] Blog post with image.

--kevin


Create your own Terastation cross toolchain

Note: If you want a nice structured build environment, try the ELDK instead.

  1. Get prepared with the appropriate files:
  2. Create a powerpc-603e.dat file in the crosstools directory:
    KERNELCONFIG=`pwd`/powerpc-603e.config
    TARGET=powerpc-603e-linux-gnu
    TARGET_CFLAGS="-O -mcpu=603e"
    GCC_EXTRA_CONFIG="--with-cpu=603e --enable-cxx-flags=-mcpu=603e"
  3. Create a build shell script in the crosstools directory:
    #!/bin/sh</code>
    set -ex</code>
    TARBALLS_DIR=/tmp/downloads
    RESULT_TOP=/opt/crosstool
    export TARBALLS_DIR RESULT_TOP
    #GCC_LANGUAGES="c,c++,java"
    GCC_LANGUAGES="c,c++"
    export GCC_LANGUAGES
    # Really, you should do the mkdir before running this,
    # and chown /opt/crosstool to yourself so you don't need to run as root.
    # mkdir -p $RESULT_TOP
    # Build the toolchain. Takes a couple hours and a couple gigabytes.
    eval `cat powerpc-603e.dat gcc-3.3.6-glibc-2.3.2.dat` sh all.sh --notest
    echo Done.
    A couple of things to note here. The Terastation stock gcc/glibc is 3.3.1/2.3.2. The gcc crossbuild #:of 3.3.1 requires a couple of extra patches to compile and 3.3.6 compiles cleanly. It's unlikely #:that something would specifically require 3.3.1 so I used 3.3.6.
    Also, note that the java language is not built. It does not compile without error in the crossbuild. #:It's probably trivial to get it to compile, I just quit trying because I don't currently have a use #:for it. If you get it to compile, please post the patch here.
  1. Start the Build
    First create the /opt/crosstools directory and make sure your compiling user (hopefully not root) has #:access to write files there, then:
    ./powerpc-603e.sh
    (Go have a beer, or six. This will take awhile. On my 1.4 GHz machine, it takes roughly 4 hours #:(hey, just be happy you're not building it on your wimpy TS). It will use about 2 GB of disk space #:as well.)

--Entropy 10:08, 30 November 2006 (CET)

Using a 3rd party Cross Toolchain

Recently, I've been using the Embedded Linux Development Kit toolchain from Denx Software Engineering. This is a FREE cross-platform toolchain with support for a number of target CPUs (including the PPC 603e in the Terastation). The complete system is managed by RPM, including the cross-build process. The base toolchain comes with quite a few "ready to install" RPMs for your target platform.

I'm using the 3.1.1-2005-06-07 version as it most closely matches the libraries already on the Terastation.

Package / Platform Terastation Pro ELDK 3.1.1-2005-06-07
gcc 3.3.1 3.3.3
glibc 2.3.2 2.3.1

--Entropy 10:50, 6 December 2006 (CET)

Building code on the Terastation using Gentoo

This is a quick and dirty way to get a working gcc build environment on the Terastation itself.

Due to the size of the Gentoo base layout, we don't want to do this on the root filesystem, or we'll fill it up. The easiest thing to do is create a new shared folder called (for example) gentoo under /mnt/array1.

Download the 2004.1 stage3 ppc tarball from one of the Gentoo Mirrors. You will need to look in the releases/historical/ppc/2004.1/stages/ppc/ directory for stage3-ppc-2004.1.tar.bz2. Put this in the gentoo folder.

cd /mnt/array1/gentoo
tar xpjvf stage3-ppc-2004.1.tar.bz2
mount -t proc proc ./proc

Copy some files over:

cp /etc/fstab etc
cp /etc/passwd etc
cp /etc/group etc
cp /etc/resolv.conf etc
cp /etc/hosts etc

Now chroot into the Gentoo environment:

bin/chroot . /bin/bash
source /etc/profile

At this point, you should have access to gcc, nano, ssh, and a host of other useful tools - certainly enough to download tarballs and build applications. I've succeeded in building unfs3, openssh, and a bunch of other apps this way. Note that anything you build must either be run from inside the chroot'd environment, OR you must resolve any additional library dependencies by copying new required libs into /lib on the real root filesystem.

I did try unsuccessfully to get the whole Gentoo portage system working - which would be pretty sweet, but I kept running into problems which I think are due to the 2004.1 release being so old. Unfortunately later versions of the stage3 tarball don't work because its libs are designed to work with a 2.6 kernel.

--Foobar 20:05, 20 February 2007 (CET)

Open SSL and Open SSH and use Winscp (Putty) to Access with Windows

Download WinScp and Putty Download Opensshand Open SSL and grops


  • Unrar it and transfer openssh-3.0.2p1.tar.gz and openssl-0.9.6e.tar.gz and groups.gz to the TeraStation by copying it to the share.
  • Untar both into the root directory of the TeraStation.
  • Untar groups.gz to /bin
  • Edit /etc/rc.d/init.d/ssh and remove the # from line 14 which mentions AUTOKEYGEN.
  • The following commands make it automatically start on boot and also start now:
  cd /etc/rc.d/rc3.d/
  ln -s ../init.d/ssh S30ssh
  /etc/rc.d/init.d/ssh start


If you've done all the steps correctly then OpenSSH should be running and also start each time you reboot the TeraStation. The version of OpenSSH that was compiled for the Kuro Box actually has a security flaw which was corrected in newer versions of OpenSSH. And now you can Access from Windows too your Tera with Winscp and make Tunnel with Putty. You can trick some Firewall with Putty and Tunnel :-)

--Inputsammler 00:41, 26 December 2006 (CET)


Joe 3.2 a good fast editor

Download JOE-3.2 and untar into the root directory and run it with the command

  joe

Example:

  joe /etc/rc.d/init.d/ssh

The hotkey for help ist

  STRG+K+H

--Inputsammler 00:41, 26 December 2006 (CET)