Viktor Petersson.com

The process is pretty straight forward, and I’m writing this as a ‘Note-to-self’ for future references than anything else. If anyone else find it useful, that’s great.

Identify the broken drive

Start by identifying the device as the system know it (ie. /dev/sdX or /dev/hdX). The following commands should provide you with the information:

cat /proc/mdstat

mdadm --detail /dev/mdX

cat /var/log/messages | grep -e "/dev/hd" -e "/dev/sd"

Once you’ve identified the drive, you want to know something more about this drive, as /dev/sdX doesn’t really tell us how the drive looks like. In my case, I have three identical drives, so the following command didn’t help me much, but maybe it does for you.

hdparm -i /dev/sdX

That should give you both the model, brand and in some cases even the serial number. Hence this should be plenty to identify the drive physically.

Replace the drive

Not much to be said here. I assume you already know this, but you need a drive of equal size or larger.

Partition the new drive

If your system boot up in degraded mode, then just boot up your system. If not, boot it off of a Live CD (I used Ubuntu’s LiveCD in ‘Rescue mode’).

Once you’ve made it to a console, the first thing we need to do is to partition the new hard drive. The easiest way to do this is to use sfdisk and use one of the existing disks as the template.

sfdisk -d /dev/sdY | sfdisk /dev/sdX

(where sdY is a working drive in the array, and sdX is your new drive)

Rebuilding the array

The final step is to add the new drive to the array. Doing this is surprisingly easy. Just type the following command:

mdadm /dev/mdZ -a /dev/sdX1

(assuming you want to add the partition sdX1 to the RAID array mdZ)

Of that went fine, the system will now automatically rebuild the array. You can monitor the status by running the following command:

cat /proc/mdstat

This solution is so ugly that I felt that I had to post it =).

I had a problem. Whenever I plugged in or rebooted, the a 3G modem into a Linux machine, it appeared on a different path (/dev/ttyUSBX). That creates some issues, as I’m using to connect to the internet, and Wvdial is using a hardcoded path to the modem. To fix this, I had to manually edit the file every time it changed. That’s very annoying. Now add in the fact that this is sitting on a remote machine that I have little physical access to, this is a real problem.

My initial approach was to turn to udev and write a rule for the modem that creates an automatic symlink, such as /dev/modem. Unfortunately, when you add usb_modeswitch into the mix, it breaks. For some reason, usb_modeswitch simply wouldn’t detect the modem when doing this, and hence render it useless.

Instead, I figured, if I write a Bash-script that automatically creates a symlink, that would take care of the issue. Of course, it is very ugly, but it does indeed work. Now I can simply run this script in Cron and that way know that I always have the correct path to the modem.

So how did this script look you may ask. This is how:

#!/bin/bash
MODEM=$(cat /var/log/messages |grep "GSM modem (1-port) converter now attached to" | tail -n 3 | head -n 1 | perl -pe "s/.*GSM modem \(1-port\) converter now attached to (ttyUSB.*)$/\1/g;")
CURRENT=$(ls -l /dev/modem | awk '{print $10}' | perl -pe "s/\/dev\/(ttyUSB.*)$/\1/g;")
 
if [ $CURRENT != $MODEM ];
then
	rm /dev/modem
	ln -s /dev/$MODEM /dev/modem
fi

I never said it was pretty, but it does indeed work. If you wonder what the ‘head’ and ‘tail’ part is all about, it is because the system creates three paths, but only the first one works.

Update: Turns out it is a bad idea to run this in Cron, as the logs will rotate. Instead, launch it at boot in rc.local, but make sure you insert a ‘sleep 10′ or similar to allow the modem to settle.

Update 2: Turns out there is a far more elegant solution to the problem. The system automatically generates a symlink for you. In my case, the modem is accessible via:
/dev/serial/by-id/usb-HUA_WEI_Huawei_Mobile-if00-port0
This means that you can hard-code that path instead of having to run a silly script to generate a symlink for you.