Removing Failed or Missing Paths

Print 1

 

When changing storage mapping, you’ll find yourself with failed or missing disk paths on your AIX SAN. Failed paths are commonly due to a change that’s happened. In general, paths are set to fail if they go down during normal processing. If a path is lost during a restart, it will be set to missing.

Other reasons to have a missing path include:

• Storage mapping error caused by the WWN (network address) not being in the zone on the fabric
• Changing a fibre card
• Moving to a new switch, which creates a new path and the existing path will then fail
• Storage controller goes offline
• Changing the host port on the storage within the zone

Several paths are likely to be missing in an enterprise environment. The number may depend on the number of hdisks you have and how many paths you have to an hdisk. Removing the paths one at a time from the command line is not only tedious but the repetition of the commands can lead to typos and you may remove the wrong path.

Removing Paths

You have a few different methods for removing a path. In this demonstration, I’ll use the following format where hdiskX is the hdisk number, fscsiX is the fibre adapter the path hangs off and connection is the connection string that identifies the path:

rmpath –dl <hdiskX> -p <fscsiX> -w <connection>

The connection string is taken from the lsattr and lscfg command output when run against the hdisk path to remove. The first part of the string is the ww_name and the second is part of the string is taken from the location code of the hdisk. We will look at these in more detail to follow.

It may seem easy enough to create a script that will remove the paths—and this is the method I recommend—but rather than just let the script run and remove the missing paths, you may want to pause and review the paths that will be removed before actually committing yourself. I suggest creating a script that gathers the missing paths and, using the missing path output, let it generate an output file that contains a script that does the removal. Here you can review and crosscheck the paths that are to be removed. If it looks good, then execute the script and the paths are gone.

Listing the missing paths can be achieved using the following:

# lspath |grep –i missing
Missing  hdisk20 fscsi0
Missing  hdisk20 fscsi0
Missing  hdisk20 fscsi1
Missing  hdisk20 fscsi1

The output shows we have four missing paths. For this demonstration, we will stick with only four paths, though in reality it would be higher depending on the number of disks and paths to each disk. Looking at the path locations in more detail, again using lspath but specifying for each hdisk, let’s get the name, parent connection and the status returned with:

# lspath -l hdisk20 -H -F "name:parent:connection:status" 
hdisk20:fscsi0:5001738065920143,6000000000000:Missing
hdisk20:fscsi0:5001738065920151,6000000000000:Missing
hdisk20:fscsi1:5001738065920183,6000000000000:Missing
hdisk20:fscsi1:5001738065920171,6000000000000:Missing

Note in the output the connection string is in two parts separated by a comma. This gives us all the information we need to remove a path. I could now use the following to remove the first path in the output:

rmpath –dl hdisk20 –p fscsi0 –w 5001738065920143,6000000000000

Now that we know how to remove a path individually, let’s put that information into a simple listing. The following listing contains the script:

Listing 1. rmpaths

#!/bin/sh
# rmpaths
>xrmpaths
echo "#!/bin/sh" >>xrmpaths
disks=$(lspv | awk '{print $1}')
for loop in $disks
do
lspath -l $loop -H -F "name:parent:connection:status" |grep Missing| awk -F: '{print "rmpath -dl",$1,"-p", $2, "-w", $3}'>>xrmpaths
done

This script gives us a listing of all disks on the system. Then using that list, the script loops through parsing each disk into the lspath command to output a listing of missing paths. Using awk we fill in the gaps to produce the correct syntax for the rmpath command. The output produces the file xrmpaths, which contains the paths to remove. Though I have grepped for “Missing” this could be placed with “Failed”. The resulting output file (script) xrmpaths may look like:

#!/bin/sh
rmpath –dl hdisk20 –p fscsi0 –w 5001738065920143,6000000000000
rmpath –dl hdisk20 –p fscsi0 –w 5001738065920151,6000000000000
rmpath –dl hdisk20 –p fscsi1 –w 5001738065920183,6000000000000
rmpath –dl hdisk20 –p fscsi1 –w 5001738065920171,6000000000000

Now all that is needed is to review the output and, if you’re happy, then make the script executable and run it, like so:

#./xrmpaths

That’s it! All the missing are now gone and you’ve used a much better method than removing them manually.