R2dump debug tool

R2dump debug tool

Troubleshooting


Problem

Customers may face a situation where a server is rebooted with no dump and nothing to explain the reason for the reboot in the error report or in the console log. Most commonly this is seen with customer using an Oracle clustering product, such as RAC or Grid, or with Veritas VCS. Customers are usually referred to these vendors to examine application logs for root cause. A product vendor may ask IBM to provide more information about the system state when the reboot command was issued. AIX development support has created a kernel extension, called r2dump. The r2dump tool loads a kernel extension that replaces the reboot system call in the AIX kernel, with code that will simply force system dump instead of reboot. It can be used on AIX 6.1 TL06 or ( later versions) and on any AIX 7.1 and 7.2 systems.

Symptom

A reboot with no dump to analyze the root cause.

Cause

In most cases, customer using an Oracle clustering product, such as RAC or Grid, or with Veritas VCS may experience a reboot with no dump.

Environment

AIX 6.1 TL06 up to 7.2

Resolving The Problem

The r2dump debug tool loads a kernel extension that replaces the reboot system call in the AIX kernel, with code that will simply force system dump instead of reboot.

This debug tool is available in epkg format following the instructions below:

To download the r2dump debug tool:

ftp://ftp.software.ibm.com/aix/tools/dumptools/

This include:

dbgr2dump.161110.epkg.Z
README.TXT

To apply the r2dump debug tool:

# emgr -X -e dbgr2dump.161110.epkg.Z (run from the directory where you saved it)


To see if debug tool is applied:

# emgr -l
To remove:
#emgr -r -L dbgr2dump
----------------------------

To enable the tool, run "/opt/AIX/r2dump/r2dump -e /opt/AIX/r2dump"

To automatically reload the extension after a system reboot, put this command in inittab:
r2dump:2:once:/opt/AIX/r2dump/r2dump -e /opt/AIX/r2dump>/dev/console 2>&1

If you want to reboot the system without collecting a dump, you can simply unload the kernel extension:

/opt/AIX/r2dump/r2dump -d /opt/AIX/r2dump


Or, you can add the above line to /etc/rc.shutdown and the kernel extension will be automatically disabled when you run a reboot or shutdown from the command line.

Conclusion

This is a useful tool to find out what is causing unexplained system reboots.

0 (0)
Article Rating (No Votes)
Rate this article
Attachments
There are no attachments for this article.
Comments
There are no comments for this article. Be the first to post a comment.
Full Name
Email Address
Security Code Security Code
Related Articles RSS Feed
Technology level update on AIX using smit_update and alt_disk_install method
Viewed 6106 times since Sun, Jun 30, 2019
How do I analyze and debug core files on AIX
Viewed 6957 times since Thu, Feb 21, 2019
Managing System Dump Devices sysdumpdev
Viewed 3812 times since Mon, Jul 9, 2018
AIX, Storage, System Admin↑ Identifying a Disk Bottleneck Using filemon
Viewed 10156 times since Fri, Apr 19, 2019
AIX 6.1: Extend VG 0516-1714 extendvg 0516-1254 extendvg
Viewed 4698 times since Fri, Jul 6, 2018
Check connection (rsh or nimsh) between NIM server and LPAR
Viewed 10998 times since Thu, Feb 21, 2019
Altering LVM Configuration When a Disk is Not in ODM Anymore
Viewed 3364 times since Mon, Jun 3, 2019
AIX smtctl The smtctl command controls the enabling and disabling of processor simultaneous multithreading mode.
Viewed 15442 times since Fri, Jan 18, 2019
AIX How to Investigate a System Reboot
Viewed 6992 times since Tue, Aug 14, 2018
AIX, System Admin↑ The chrctcp command
Viewed 3247 times since Fri, Apr 19, 2019