Topics: PowerHA / HACMP, Storage Adding a new volume group to an active PowerHA resource group

This article describes how to add a new volume group to an existing resource group of an active PowerHA cluster.

The first step is to add the storage to both of the nodes of the PowerHA cluster. In the case of SAN storage, please ensure that your storage administrator adds the storage to both nodes of the cluster. Then discover the newly added storage by running the cfgmgr command on one of the nodes:

# cfgmgr

Set a PVID on all the new disks that have been discovered. For example, for disk hdisk77, run:

# chdev -l hdisk77 -a pv=yes

Repeat this command for any of the new disks.

Next, log in to the other node, and run the cfgmgr command on that node as well, so the disks will be discovered on the other node as well. And when you run "lspv" on the second node after running cfgmgr, you'll notice that the PVID is already set for all the discovered disks (it was set on the first node).

On both nodes, make sure that the disk attributes are set correctly. Now, this may differ for the type of storage used (so please make sure to check your storage vendor's recommendations on this topic), but a good starting point is (for example, for disk hdisk4):

# chdev -l hdisk4 -a max_transfer=0x100000 -a queue_depth=32 -a reserve_policy=no_reserve -a algorithm=round_robin

For clustered nodes, it's important that the reserve_policy is set to no_reserve.

A note about the max_transfer attribute: This value should be set to the same value as the max_xfer_size attribute of the fiber adapter. By default attribute max_transfer is set to 0x40000, which is usually lower than the max_xfer_size attribute on the fiber adapter, which results in a smaller buffer size memory being used. To check the max_xfer_size attribute on the fiber adapter, run (for example for adapter fcs0):

# lsattr -El fcs0 -a max_xfer_size

Also please make sure to set the disk attributes for all the new disks on both nodes. These attributes are stored in the ODM of the AIX system locally, and don't automatically transfer over to other nodes of the cluster, so you'll have to set these attributes for all new disks on all cluster nodes.

The next step is to create the new volume group(s). The first thing that we'll need to know is a common majar number that is available on both nodes of a cluster. For that purpose, run the following command on all cluster nodes, which lists the available major numbers:

# lvlstmajor

Choose a major number available on all cluster nodes. For the purpose of this article, let's assume major number 57 is available.

For PowerHA, a new volume group should be configured as concurrent-capable (configured by the -C option of the mkvg command). Also, The quorum should be disabled (by using -Qn), and the auto-varyon should be disabled as well (by using the -n option) as PowerHA will varyon the volume group for us. Finally, the major number should be set (in the example below: -V 57).

As such, run the following mkvg command to create the volume group (Note: please adjust the volume group name, the major number and disk names according to your situation and preference):

# mkvg -v 57 -S -n -Qn -y fs01vg hdisk38 hdisk42 hdisk45

Note: run this command on only one of the cluster nodes, and continue working on this cluster node for now for the next steps.

Next, create a logical volume (adjust per your situation and preference):

# mklv -y fs01lv -t jfs2 -u 1 -x 1278 hdisk38

This command will create logical volume fs01lv for the purpose of using it for a JFS2 type file system, with an upper bound of 1 (just use 1 disk), and allocate 1278 partitions on disk hdisk38.

Then create a file system on the previously defined logical volume:

# crfs -v jfs2 -d fs01lv -a logname=INLINE -a options=noatime -m /fs01 -A no

This command will create file system /fs01 on top of logical volume fs01lv, and will use an inline log (recommended for optimal performance), will not record access times (options=noatime) to avoid unneccessary writes to the file system, and will tell AIX not to automatically mount the file system at system start (-A no), as PowerHA will mount the file system instead.

At this point, the volume group, logical volume and file system have been created. You can create additional volume groups, logical volumes and file systems as it pertains to your situation. Once done, ensure that the volume group is varied off, for example for volume group fs01vg:

# varyoffvg fs01vg

Should the varyoff command fail at this point, then please check if the file system(s) is/are still mounted, and un-mount them before running the varyoffvg command.

Now run "lspv" on both nodes of the cluster, and look at the PVIDs listed. Pick one of the disks on which you configured the volume group on the first node, and take that disk's PVID, for example 00fac78651b28b53. On the other node, run the importvg command to import the volume group (which also imports the information about any logical volumes and file systems configured in that volume group), using the PVID of one of the disks, and the major number previously defined, for example:

# importvg -n -V 57 -y fs01vg 00fac78651b28b53

Repeat this for any additional volume groups in your situation. Do make sure to select the correct PVID of one of the disks in the volume group on the first node when importing this volume group on the second node, as you'll want to make sure both nodes of a PowerHA cluster use the same volume group names.

At this time, the storage is properly configured on both nodes, and it is time to add the volume group(s) to the PowerHA resource group.

First, verify that the cluster is in a good state by running:

# smitty hacmp
  problem determination tools
    powerha verification
      verify powerha configuration

Once confirmed, allow PowerHA to discover the new disks in the cluster:

# smitty cm_discover_nw_interfaces_and_disks

Now add the volume group(s) to the resource group of the cluster:

# smitty hacmp
  cluster applications and resources
    resource groups
      change/show resources and attributes for a resource group

Select your resource group, and add the volume group(s) to the "Volume Groups" entry.

Next, sync the cluster, which should bring the volume group(s) online:

# smitty hacmp
  cluster applications and resources
    verify and synchronize cluster configuration

Once this is complete, you should be able to see the new volume group(s) online, by running:

# lsvg -o

And you should be able to see that any new file systems have been mounted by PowerHA:

# df

Not required, but good practice, is to now schedule a failover test for your PowerHA cluster, to ensure everything is still working as it should, in case of a fail-over scenario.