TeraStation larger disks
This article is about upgrading a PPC based TeraStation to use Hard Disks larger than 500Gb.
- 1 Background
- 2 Approach
- 3 Telnet Enabled Firmware
- 4 Setting up 2TB RAID5 Arrays
- 5 Setting up for Larger Disks
- 5.1 Preparing the Disks
- 5.2 Setting up the Shares
- 5.3 Setting up a Single Drive view
- 6 Changing/Upgrading a Single Drive
- 7 Flashing with new Firmware versions
It is now possible to get IDE disks up to 750Gb in size, and SATA disks up to 1.5Tb in size. Larger disks may become available in the future. Many users would like to be able to use such disks to upgrade their TeraStations capacity.
It is also possible to use disks larger than 750Gb on an IDE based system via a SATA/IDE converter, although you have to make sure you get one small enough to fit into the available space. The ones tested were obtained via eBay, and consisted of a small PCB that was the same size as the back of a 3.5" drive with a SATA connector on one side and an IDE socket on the other as shown below:
The board plugged into the back of the drive and simply converted the connectors from SATA to IDE and there was just enough clearance inside the case to fit the power and IDE cables. For large disks a SATA drive plus a converter is normally cheaper than the equivalent size IDE drive, so this is attractive from a price point of view.
The systems that are currently covered by this article include all the original PPC based TeraStation Models:
- Original TeraStation
- TeraStation Home Server
- TeraStation Pro v1
These all come with a Linux 2.4 Kernel. This version of Linux has a limitation where a single file system cannot exceed 2Tb. As Buffalo have assumed that normal use is to use a RAID array treating all 4 disks as a single file system the largest disks that is supported by the standard Buffalo firmware is 500Gb. This article covers techniques for using disks larger than this (albeit with some restrictions).
The following ARM based TeraStation models are not covered:
- Terastation Live
- TeraStation Pro v2
These systems come with a Linux 2.6 kernel. This does not suffer from the limit of 2Tb in a single file system. Although there is no reason to suspect that the instructions described in this article would not work they should not be required as the standard Buffalo firmware should handle larger disks without issues.
If the use of RAID5 arrays matters to you, then you are limited to a total of 2Tb (i.e. using 750Gb drives).
There are 2 approaches discussed here. One allows for disks up to 750Gb in use to be used and still maintain the RAID5 array capability. The instructions on how to set up a TeraStation to support a 2Tb array and once this has been done to be completely compatible with the standard Buffalo firmware are in the section on Setting up 2TB RAID5 Arrays.
The other approach that has been used is to forgo the use of RAID arrays for the data areas, and instead use them as single independent disks. This means that the 2Tb limit imposed by the 2.4 linux kernel applies at the individual drive level and not at the RAID array level. Using this approach means that the theoretical limit on a TeraStation is increased to 8Tb (4 x 2Tb), and that would need 2Tb drives which are not currently available.
- You can use drives larger than the 500Gb maximum supported by the standard Buffalo firmware.
- You can use mixed drive sizes, thus allowing for incremental upgrades in capacity of the system.
- A single drive failure only loses any data associated with that drive.
- Recovering the system back to a fully working state after a drive failure is simple.
- The Buffalo provided firmware continues to be used as the basis of the system
- The Buffalo browser based GUI can still be used to manage nearly all aspects of the system such as users and groups.
- You can use linux soft-links to give you a single "view" of all 4 drives. This is means that the sytem will look a bit like it is running with a single drive, although in practise all 4 drives will have their free space managed independently.
- The Buffalo supplied media servers continue to function and can deliver media from from all 4 drives.
- The software upgrades available from the itimpi website can still be used with the system.
- Buffalo firmware upgrades can still be used (although at this late date it is unlikely that they will provide new ones for these models as they have been superseded by the ARM based models).
- RAID arrays cannot be used for the Data part of the hard disks (although one is still used for the System area). This means that you are not protected against a single drive failure. You therefore need to decide on a suitable back-up strategy to take account of this. If this matters to you then the approach described in this article cannot be used.
- You have to use a telnet enabled firmware release rather than the standard Buffalo supplied ones. Many might consider this to be an advantage rather than a disadvantage!
- Manual steps are required to recover from a disk failure - you cannot use the Buffalo GUI to achieve this. However as these are the same steps as are required to set the system up in the first place this will probably not be an issue.
Telnet Enabled Firmware
TeraStations internally run the Linux operating system. Buffalo hide this from the average user providing the system "packaged" and with a browser based GUI to control and configure iy. Telnet enabled firmware allows users to log in using a telnet client to control and manipulate the system at the Linux command line level.
This allows users to do things like:
- Configure the system at a more detailed level than allowed for by the Buffalo GUI.
- Upgrade components of the underlying software to newer versions with additional feature and/or bug fixes included.
- Install new applications to extend the functionality of the Terastation.
- In the event of any problems being encountered allow for a level of access that gives more chance of recovering user data without loss.
The changes described in this article require the use of a telnet enabled release. Hopefully the instructions provided are detailed enough that users can carry out the steps provided without needing much linux knowledge.
The standard software supplied with buffalo PPC based TeraStations does not provide for telnet access. Telnet enabled releases of firmware corresponding to virtually all Buffalo firmware releases can be found at itimpi's website. These are identical in functionality to the corresponding Buffalo firmware releases - the modification to add telent functionality being trivial. This means they will have exactly the same bugs (if any) as are found in the Buffalo releases.
The itimpi firmware releases are the ones that have been used while preparing this article. Firmware from other sources should work fine as long as it is telnet enabled.
To use telnet you need a telnet client. A rudimentary one is included with Windows which you can invoked by typing a command of the following form into the Windows Run box:
A freeware one called PuTTY is recommended as a much better alternative. In addition to the standard telnet protocol PuTTY also supports the more secure SSH variant (although additional components need installing at the TeraStation end to support this).
TIP: If you already have a telnet enabled version of the firmware installed and you want to continue to use that version then you can run the Firmware updater in Debug mode and elect to not rewrite the linux kernel or boot image in flash, but merely update the hard disk. This is slightly safer as the flash chips have been known to fail.
Setting up 2TB RAID5 Arrays
For some people the use of RAID5 is important. This section covers how to set up a 2TB RAID5 array using 750Gb drives. Larger arrays are not possible due to the 2Tb limit on a single file system imposed by the 2.4 kernel.
The steps involved are:
- You need to use a telnet enabled version of the firmware.
- Start with 4 unpartitioned 750Gb drives and flash the firmware to get the basic setup as described in the section above on Starting With New Disks. This will not use a signifcant part of the available space and if a RAID5 array is set up it would be around 1.6Tb in size.
- Use the Web GUI to remove any shares and RAID arrays.
- Use the approach described above for Changing the Partition Sizes. First delete the existing partitions 3 and 4. Then create a new partition 3 on each disk that must not exceed 715, 816,238 1K blocks - you may have to experiment a bit to work out what the start and end tracks need to be. Allocate partition 4 to use the remainder of the disk. Then set the type for partitions 3 and 4 to be '
fd'. As an example on a Seagate 750Gb drive this works out as follows:
Disk /dev/hdg: 255 heads, 63 sectors, 91201 cylinders Units = cylinders of 16065 * 512 bytes Device Boot Start End Blocks Id System /dev/hdg1 1 48 385528+ fd Linux raid autodetect /dev/hdg2 49 65 136552+ 82 Linux swap /dev/hdg3 66 89180 715816237+ fd Linux raid autodetect /dev/hdg4 89181 91201 16233682+ fd Linux raid autodetect
- Format the new data partitions using the
mkfs.xfs -f /dev/hd?3
- Use the Web GUI to create the RAID5 partition and any wanted shares.
You should now be in operation with a RAID5 array with 2Tb useable space.
Setting up for Larger Disks
Preparing the Disks
Most users will tend to run with their system either in RAID5 mode or SPANNING (jbod) mode. To use larger disks in the manner described in this article the first thing is to get your system in the state where there are no RAID arrays defined in the Buffalo Web GUI, and all 4 drives are being used independently. This will destroy any existing data so ensure that you have your data backed up first. If you already have your system set up to use the four drives independently then this section can be skipped.
Note that you could do this AFTER you have carried out the step mentioned below on Perparing the Data Partitions - the order of these two steps is not critical.
Starting with a Running System
Most people will already have a running system, and in this case it is easier to get your system running in a mode where it can accept larger disks, but initially using the drives already installed. If this is the case and your TeraStation is already in a running state then:
- Ensure that you have backed up your data, if not you will lose it.
- Go into the Web GUI and select Disk Management. If this shows that you are already running with separate disks (Disk1 to Disk4) then you are already in a state where individual drives can be changed so there is nothing else to do.
- If (as is normally the case) this shows that you are running using a RAID array then click on RAID Array 1 which will take you to a screen which has a button labelled Delete RAID Array. A Dialog block is displayed asking you if are Sure You Want to continue and warning you that doing so will delete all your data. Select OK. You will now be taken to a screen where you need to enter a displayed number to confirm the delete (on many firmware releases this screen will be in Japanese). All you have to do is enter this number and then press the left-hand button to Apply the change. While the Array is being deleted (which takes several minutes) all other access to the TeraStation will be prohibited. On completion you will be returned to the Disk Management screen, and this time RAID Array 1 will show as "Not Configured".
- Most people run their teraStations with only RAID Array 1 configured. If you are running with RAID Array2 configured as well then repeat the above step for RAID Array 2.
- Now select the Home option, and you will be returned to the home page of the Web GUI. You will see that all 4 drives are now listed with their space used individually itemised.
- If you want to change any of the disks at this point then follow the instructions for Upgrading/Changing a Single Drive.
Starting with new Disks
It is perfectly feasible to start with a brand new set of disks, rather than with a running system. You are more likely to use this approach if you are considering upgrading all the disks at the same time. The alternative is to convert the system as described above, and then Upgrade a Single disk at a time as described later.
Initializing the Disks
If you are starting from scratch with a new set of hard disks then the first thing to do is to get the Buffalo software onto the system:
- Install the 4 drives you want to use. If they are IDE drives make sure the jumpers on them are set to make them the Master/Single drive. If you are going to use a mix of drive sizes then it is recommended that they are installed in ascending size (i.e. Drive 1 is the smallest). This may not be critical but that is what has been tested.
- Flash the TeraStation with the firmware release of your choice. This must be a telnet enabled variant of the firmware.
- You will be given warnings about the partition structure being wrong and the disks about to be formatted. OK these to continue. You will see that the Disk I/O lights on the TeraStation all start flashing. You will almost certainly get a message from the firmware nupdater program about a timeout waiting for a response. Quit the firmware updater at this point. The disk I/O lights will continue flashing on the TeraStation. Wait until this finishes (which could take several hours).
- Flash the TeraStation again - this time it should complete without error.
- Go into the Web GUI and delete any shares that have been set up.
- You will almost certainly find that array1 hs been setup to be a "SPANNING" array. Select this array and then the option to delete the array. You will now be given a screen in Japanese where you have to enter a displayed number for confirmation. Enter this number and then press the left-hand button. This will delete the array, and you will see your disks displayed as independent drives. Do not worry at this point if the capacities displayed are wrong.
Changing the Partition Sizes
The next thing is to get the partitions set up correctly to use all the available disk space. To do this:
- Telnet into the system. Any telnet client will do, but the recommended one if you are using Windows is PuTTY (which is freeware). Login with a user with root privileges. If using the telent software from the itimpi web site this will be the "myroot" user - initially with a blank password (it is recommended that you immediately set a password using the 'passwd' command.
- Try the 'df' command and you will see that it shows you all 4 disks but not using their full capacity.
- Unmount the 4 data partitions using command of the form
where mountname is the appropriate name from the output of the
- Start the disk partitioning software using a command of the form
mfdisk -c /dev/xxx
where xxx is one of hda/hdc/hde/hdg for the 4 disks in an Original or Home Server machine or sda/sdb/sdc/sdd for the Pro model
- You can get a menu of available commands in the mfdisk program by pressing 'm'. You may want at this point to select the 'p' option to show the current partitions that have been set up by the Buffalo firmware at the initialization stage.
- Use the 'd' command to delete partions 4 and 3.
- use the 'n' command to create new primary partitions 3 and 4. For partition 3 use the default for the start position, but for the end position type in a value one less than the value displayed as the default. For partition 4 use the defaults for start and end track. It is possible partition 4 is not needed, but the Buffalo firmware sets it up, and the above approach of giving it one track minimizes the wasted space
- Use the 't' command on partitions 1, 3 and 4 to change the type to 'fd' and partition 2 to '82'. It is not clear if this is really required, but those are the partition types the Buffalo firmware sets so it is better to be safe than sorry.
- As an example after making these changes on a 1.5TB Seagate drive and then using the 'p' command to print the current settings gives the following:
Disk /dev/sda: 255 heads, 63 sectors, 182401 cylinders Units = cylinders of 16065 * 512 bytes Device Boot Start End Blocks Id System /dev/sda1 1 48 385528+ fd Linux raid autodetect /dev/sda2 49 65 136552+ 82 Linux swap /dev/sda3 66 182400 1464605887+ fd Linux raid autodetect /dev/sda4 182401 182401 8032+ fd Linux raid autodetect
- Write out the changes using the 'w' command.
- Format the data partition using the command
mkfs.xfs -f /dev/xxx3
where xxx is one of hda/hdc/hde/hdg for the 4 disks in an Original or Home Server machine or sda/sdb/sdc/sdd for the Pro model
- Repeat this for each of the four drives varying the last part of the device name as indicated earlier for each drive
- Reboot the system
- Go into the web GUI and you should now see all 4 drives listed with their correct capacity.
It has been found that sometimes there is configuration information about shares left behind despite the fact that the drives have been changed. You therefore need to make sure that your share configuration is correct for the new mode of running.
There are some idiosyncaracies of the Buffalo software that you need to be aware of when setting up your shares:
- The system does not seem to like shares of the same name set up on more than one drive.
- Any folders that are at the root of the mounted file system will automatically be added as shares when the system next reboots, and will be given a description of
On that basis following approach is recommended
- Use the Web GUI to look at the shares and delete any existing shares that are no longer needed. Normally the steps above will have deleted any existing shares, so initially there are no shares listed.
- Set up a share on each of the 4 disks. The suggestion is to simply use share names of the form
?represents that drive number. If you do not need it, unselect the option for the Recycle Bin as this can take up a lot of space as files are deleted.
- You can optionally also set up further shares at this point. The "Single Drive View" approach mentioned in the next step does not require this as everything is accessed via the shares on Disk 1.
Setting up a Single Drive view
At the client end it is more convenient to treat the TeraStation as a single drive rather that 4 separate drives. This can be achieved using the approach given below. It is recommended that you put the commands to create these links into a separate file (e.g. CreateLinks.sh) which is set to be executable so that you can re-set them up again later using a command such as
The best place to keep this file is probably in the home folder for the 'root' user (/root) as this is where you are initially located when you log in via telnet. As this is part of the System partition which is replicated on all 4 drives, it will remain available if you change one of the drives.
If you use the commands:
chmod 777 CreateLinks.sh ln -s /root/CreateLinks.sh /mnt/disk1/share1/CreateLinks.sh
You will be able to also see this file at the base of the share. It can then be edited from Windows which most people will find easier than using the Linux level editors such as vi. Make sure you use an editor that can handle Unix style end-of-lines (e.g. TextPad). The standard Windows editors such as Notepad and Wordpad will not do.
You are likely to want to tune the commands mentioned here to suit your specific requirements.
A sample shell script incorporating these ideas is given that may be suitable with minimal modification for most users.
- Telnet into the TeraStation using a user with root priveleges (e.g. myroot)
- Issue the following command:
chmod 777 /mnt/disk?
This ensures all users will have full read/write access to each of the data partitions.
- Create a folder on the disk1 share into which you intend to put disk1 content
mkdir /mnt/disk1/share1/_disk1 #Create the content folder chmod /mnt/disk1/share1/_disk1 #Set global access permissions
- You are now going to use a set of commands to set up symbolic links between drive 1 and the other drives.
- Set up a link from Drive 1 to the others. If using the default share this would be commands of the form:
ln -s /dev/disk2/share2 /dev/disk1/share1/_disk2 ln -s /dev/disk3/share3 /dev/disk1/share1/_disk3 ln -s /dev/disk4/share4 /dev/disk1/share1/_disk4
- Open up the share from your client (e.g. your PC) and you will see the
_disk1, _disk2, _disk3, _disk4folders which can be clicked on to take you into the drive specific content in a transparent manner.
You may want to use a further level of symbolic mapping at folders within the share. As an example say you have created a folder called
TV on disk 1, and you want to have folders that at the share level (drive 1) look like this:
TV Series1 Series2 Series3
but you want some of the series of to really be on drives other than drive 1. You could create a TV folder via the _disk2, etc links set up earlier and then by setting up the following symbolic links you would get the view that you wanted:
ln -s /mnt/disk1/share1/TV/Series2 /mnt/disk2/share2/TV/Series2 ln -s /mnt/disk1/share1/TV/Series2 /mnt/disk2/share2/TV/Series2
Note that if you want to use spaces or any other special character in any of the above commands they need to be escaped (e.g. entered as '\ '. Typing in all these 'ln'commandsd can be a bit laborious and error prone, so you may prefer to isntead get the machine to do a lot of the work as shown in the sample script below
TIP: If your symbolic links do not show up in Samba then you have probably got one of the following wrong:
- The symbolic link points to a none-existent folder.
- The permissions on the file/folder do not allow Samba users to access the file
You may at times want to copy or move files between the different drives to balance out usage. If you do this, it is MUCH faster to do it via linux cp/mv commands rather than to try to do it via Windows shares.
The only downside to is that the files will end up owned by root. However the Sample script contains some chmod/chown/chgrp commands that are currently commented out that can be used for resetting the owner and access permissions. You may want to uncomment these by removing the leading '#'.
The following is a reasonably comprehensive sample script that is uses the ideas mentioned above. It is quite likely to be suitable almost unmodifed for many users. You should examine it to check that it meets your requirements.
It makes the following assumptions:
- You have used the suggested name of
CreateLinks.shfor the script file.
- You have used the default share name of 'share?' on each drive where ? is equvaelnt to the drive number
- You want all disk specific media files to appear under the _diskn style folders
- You want all symbolic links to appear in the TV folder of drive 1, and this is the only content in this folder. This is the folder that is configured for the media server to use.
- The '
TV' folder on Drive 1 and the relevant '
_diskn' folders to hold disk specific content are created if they do not already exist.
- Permissions are set on all files on the share that will allow anyone to access them.
#!/bin/sh # Script to support "Single View" mode. # (Make all drives visible from Disk 1) # First Make this file visible from the share for easy editing if [ -f /mnt/disk1/share1/CreateLinks.sh ] then rm -f /mnt/disk1/share1/CreateLinks.sh fi chmod 777 CreateLinks.sh chown nobody CreateLinks.sh chgrp nogroup CreateLinks.sh ln -s /root/CreateLinks.sh /mnt/disk1/share1/CreateLinks.sh # Now set up folders to tell apart files by disk # Disk 1 folder if ! [ -d /mnt/disk1/share1/_disk1 ] then mkdir /mnt/disk1/share1/_disk1 chmod 777 /mnt/disk1/share1/_disk1 chown nobody /mnt/disk1/share1/_disk1 chgrp nogroup /mnt/disk1/share1/_disk1 echo "/mnt/disk1/share1/_disk1 folder created" fi # Symbolic Links to other disks rm -f /mnt/disk1/share1/_disk2 rm -f /mnt/disk1/share1/_disk3 rm -f /mnt/disk1/share1/_disk4 chmod 777 /mnt/disk? ln -s /mnt/disk2/share2 /mnt/disk1/share1/_disk2 ln -s /mnt/disk3/share3 /mnt/disk1/share1/_disk3 ln -s /mnt/disk4/share4 /mnt/disk1/share1/_disk4 chmod -R 777 /mnt/disk1/share1/_disk? chown -R nobody /mnt/disk1/share1/_disk? chgrp -R nogroup /mnt/disk1/share1/_disk? # Now Individual TV Series as symbolic links within Disk 1 TV Folder # (this is then defined to PCast as the Media folder) # Create Soft-Linbks folder if does not already exist if ! [ -d /mnt/disk1/share1/TV ] then mkdir /mnt/disk1/share1/TV chmod 777 /mnt/disk1/share1/TV chown nobody /mnt/disk1/share1/TV chgrp nogroup /mnt/disk1/share1/TV echo "/mnt/disk1/share1/TV folder created" fi # Remove all existing soft links rm -fr /mnt/disk1/share1/TV/* # Recreate soft links for disk in /mnt/disk1/share1/_disk? do ls $disk | while read folder do # Include checks on folders (optional) # if [ -d "$disk/$folder" ] # then ln -s "$disk/$folder" "/mnt/disk1/share1/TV/$folder" # Change permissions (probably not really needed) # chmod 777 "$disk/$folder" # chown -R nobody "$disk/$folder" # chgrp -R nogroup "$disk/$folder" # fi done done
This script is simply re-run any time you add new folders to the _disk? folders. Only top level folders need this - sub-folders do not.
This script could also be added as a start-up script if so desired so that it is automatically run every time the system is restarted.
Changing/Upgrading a Single Drive
It is quite likely that at some point you will need to change a drive. This could be because a drive has failed or because you are intending to replace one of the existing drives with a larger drive.
The steps are as follows:
- If you are changing a drive for one of a different capacity then make sure that you have first backed up any data first. This could be to another drive on the same TeraStation as only the drive you are about to change is affected.
- Prepare the drive as for a new system making sure any jumpers are set correctly and there are no partitions already present on the drive.
- Put the new drive into the TeraStation.
- Start up the TeraStation. It should come up normally, but the TeraStation will be showing a flashing red light for the mnew drive at this point.
- Telnet into the TeraStation using a user with root privileges.
- Use the command
You will see that the disk you have just added is not shown as a mounted filesystem.
- Use the
mfdiskutility as described earlier to set up the partitions. This time you will also have to set up partions 1 and 2 as well as 3 and 4. Partitions 1 and 2 must be at least as large as these partitions on the other disk (you can safely use the
mfdisk -c /dev/xxxstyle command to look at these as long as you do not write out any changes).
- Format the data partition using a command of the form:
mkfs.xfs -f /dev/xxx3
where xxx is one of hda/hdc/hde/hdg for the 4 disks in an Original or Home Server machine or sda/sdb/sdc/sdd for the Pro model
- Prepare Partition 2 on the new disk to be used as swap space using the command
- You now need to make sure that partion 1 of this disk is added back to the RAID0 array covering all 4 disks that is used to hold system files. You can look at the details of this array using the command
mdadm --detail /dev/md0
If the disk is not reported as
faulty, removed then you may need to issue the following commands:
mdadm /dev/md0 -f /dev/xxx1
mdadm /dev/md0 -r /dev/xxx1
where xxx is one of hda/hdc/hde/hdg for the 4 disks in an Original or Home Server machine or sda/sdb/sdc/sdd for the Pro model.
- Make sure that the system partition is part of the system partition RAID0 array (
/dev/md0) by using a command of he form:
mdadm /dev/md0 -a /dev/xxx1
This command marks adds the new drive to the array and starts rebuilding its contents.. If you now look at the array status you will see it has 4 drives again, and that the new drive is marked as being "recovered". Wait for this to complete
- Mount the new data partition on the drive, the flashing red light on the drive will go out and it should be ready to use. Alternatively if you reboot the system, the new drive will be automatically mounted as part of the booting process.
Flashing with new Firmware versions
The tests done show that it should be possible to flash a TeraStation setup as described in this article without any issues. However caution should be taken as this is a none-standard setup and this cannot be guaranteed to be true in all cases.
If you attempt a flash upgrade on a system that has been setup as described in this article and the firmware updater program gives any warnings about invalid partition structure or wants to format any of the disks you should abandon the firmware update as otherwise you will almost certainly lose data.