TSM and removable USB disks

IBMs Tivoli Storage Manager is one of the best and worst backup systems I have ever used. It’s kinda of like using a chainsaw to cut butter - wrong but just soo good. The good side is that you can make TSM do just about everything. The bad side is that you can make TSM do just about everything!

A recent experiment I did was to use USB hard drives as removable media instead of traditional tapes - meaning many gigabytes or storage with fast access times for a low cost. The downside is that you need to swap media in and out manually.

In this guide, I’m not going to go into the exact details on how to configure TSM from scratch. I’ll skip the part of installing the server and get straight into the non-obvious parts of using the USB storage devices as removable media.

So first, what we have to do is set up a library for our USB HDDs to sit in:

define library usbdrives libtype=manual autolabel=yes

We’ve now defined the library and called it ‘usbdrives’. We want it to be manually operated - as I’m yet to see an autochanger for hard disks! :)

We now need to define a drive in the library for TSM to write to:

define drive usbdrives usbdrive

Now we need a path between the library and the drive:

define path SERVER1 usbdrive srctype=server desttype=drive library=usbdrives device=/mnt/usb

This tells TSM that we will mount our USB device at /mnt/usb. You can change this for what you want.

Then TSM needs a device class to know how to handle the USB drives:

define devclass usbdrives devtype=removablefile library=usbdrives mountwait=2880 mountretention=1

This is getting a little more in depth, so I’ll explain a little more. We want to define the device class to be part of the usbdrives library. The device type needs to be a removable file. The mountwait value tells TSM how long to wait for a volume to be available - I make this 2880 minutes - which is 2 days. If you don’t get the drive mounted in 2 days, TSM will abort the process. I also set mountretention to 1 minute. This means that after 1 minute of TSM finishing with the USB drive, it will automatically unmount it.

We’re almost there now as far as TSM is concerned.

We now need to configure a storage pool to group together our handywork. I assume at this point that you have 5 USB hard disks you want to use - substitute this for how many you actually have. If you don’t know, set 1 or 2 - as it can always be updated later on.

define stgpool usbdrives usbdrives maxscratch=5

TSM is now configured - but we still have to prepare the USB drives!

TSM is quite clever when it comes to using drives as storage media. Insert your USB drive and let it spin up and be detected by your OS. In my case here, I’ll use Linux as an example. If you’re using a different OS, you’ll need to substitute the following into how you’d do it on your OS. That being said, this has been untested on Windows.

Create a single partition on your new drive using fdisk. In my case, the USB HDD is sdk.

fdisk /dev/sdk

Save and write the new partition. You now should see something like this:

# fdisk -l /dev/sdk
 
Disk /dev/sdk: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
 
   Device Boot      Start         End      Blocks   Id  System
/dev/sdk1               1      121601   976760001   83  Linux

We now need to create a file system. IBM state that the file system MUST be ext2 - however I have tested this with ext3 and can verify that it works. I am yet to try it with a different file system like XFS. One thing to note here is that TSM will use the LABEL of the disk to identify it. It also requires that there are no spaces or periods in the label.

mkfs.ext3 -L USBVOL1 /dev/sdk1

If we continue to mount the new partition to /mnt/usb, and tell TSM to start a migration, you will notice that a file in /mnt/usb will appear with the same name as the file system label we created earlier.

All in all, the manual work is a little annoying - however the flexibility that you now get by being able to use cheap USB drives as bulk storage is great. If all is configured correctly, after a migration, you should get something like this:

tsm: SERVER1> q vol
 
Volume Name                  Storage         Device         Estimated       Pct      Volume
                             Pool Name       Class Name      Capacity      Util      Status
------------------------     -----------     ----------     ---------     -----     --------
/mnt/raid/tsm/disk01.dsm     DISKBACK        DISK              10.2 G     100.0     On-Line
/mnt/raid/tsm/disk02.dsm     DISKBACK        DISK              10.2 G     100.0     On-Line
/mnt/raid/tsm/disk03.dsm     DISKBACK        DISK              10.2 G      99.8     On-Line
/mnt/raid/tsm/disk04.dsm     DISKBACK        DISK              10.2 G     100.0     On-Line
5QF7TH53                     USBDRIVES       USBDRIVES        267.2 G      48.1     Filling
9QF57E67                     USBDRIVES       USBDRIVES        285.0 G       4.7     Filling