The standard tool for cluster monitoring is clstat, which comes along with PowerHA SystemMirror/HACMP. Clstat is rather slow with its updates, and sometimes the required clinfo deamon needs restarting in order to get it operational, so this is, well, not perfect. There's a script which is also easy to use. It is written by PowerHA/HACMP guru Alex Abderrazag. This script shows you the correct PowerHA/HACMP status, along with adapter and volume group information. It works fine on HACMP 5.2 through 7.2. You can download it here: qha. This is version 9.06. For the latest version, check www.lpar.co.uk.

This tiny but effective tool accepts the following flags:

•   -n (show network interface info)
•   -N (show interface info and active HBOD)
•   -v (show shared online volume group info)
•   -l (log to /tmp/qha.out)
•   -e (show running events if cluster is unstable)
•   -m (show status of monitor app servers if present)
•   -1 (exit after first iteration)
•   -c (CAA SAN / Disk Comms)

For example, run:

# qha -nev

It's useful to put "qha" in /usr/es/sbin/cluster/utilities, as that path is usually already defined in $PATH, and thus you can run qha from anywhere.

A description of the possible cluster states:

• ST_INIT: cluster configured and down
• ST_JOINING: node joining the cluster
• ST_VOTING: Inter-node decision state for an event
• ST_RP_RUNNING: cluster running recovery program
• ST_BARRIER: clstrmgr waiting at the barrier statement
• ST_CBARRIER: clstrmgr is exiting recovery program
• ST_UNSTABLE: cluster unstable
• NOT_CONFIGURED: HA installed but not configured
• RP_FAILED: event script failed
• ST_STABLE: cluster services are running with managed resources (stable cluster) or cluster services have been "forced" down with resource groups potentially in the UNMANAGED state (HACMP 5.4 only)