How do I analyze and debug core files on AIX

Sometimes, applications stop working and crash for any reason. A core file appears in scene.

First, we’re going  to set a PATH for all cores files on the system  using syscorepath command.

In order to do this, We’re going to:

1- Create and mount a new filesystem /corefiles

1.1- Create the new filesytem:

1
2
3
4
# crfs -v jfs2 -g volgrp2 -m /home/pipe -A yes -a size=1G
File system created successfully.
1048340 kilobytes total disk space.
New File System size is 2097152

1.2- Mount the filesystem:

1
# mount /corefiles

2- Using syscorepath command, we’ll set a PATH where all these cores files will be dumped..

2.1- Set the directory for all cores files:

1
# syscorepath -p /corefiles

2.2- To check if the core’s PATH was correctly set, running:

1
2
# syscorepath -g /corefiles
# /corefiles

3- Analize the core file using dbx command.

Once you get all cores in /corefiles, we’re going to analyse them using dbx command. Follow these steps:

3.1- Get the cause of the core file by running check_core command. In the example ksh93 is the victim:

1
2
# /usr/lib/ras/check_core /corefiles/core |tail -1
ksh93

3.2- Debugging with dbx (Provides an environment to debug and run programs) command:

1
2
3
4
5
6
7
# dbx /usr/bin/ksh93 /corefiles/core
Type 'help' for help.
[using memory image in /corefiles/core]
reading symbolic information ...warning: no source compiled with -g
 
Segmentation fault in strlen at 0xd0105880
0xd0105880 (strlen)    89030000         lbz   r8,0x0(r3)

3.3- Once your are in dbx shell, type ‘where’

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
(dbx) where
strlen() at 0xd0105880
varsub() at 0x100475fc
copyto(??, ??) at 0x100494c8
sh_mactrim(??, ??) at 0x1004a5a0
word_trim(??, ??) at 0x1000e340
sh_exec(??, ??) at 0x1000ff88
sh_exec(??, ??) at 0x1001043c
sh_exec(??, ??) at 0x1000ff50
sh_exec(??, ??) at 0x10010470
sh_exec(??, ??) at 0x1000fdcc
sh_exec(??, ??) at 0x1000fd94
sh_exec(??, ??) at 0x1000fe08
sh_exec(??, ??) at 0x1000fdb8
sh_exec(??, ??) at 0x1000fe08
sh_exec(??, ??) at 0x1000fdb8
sh_exec(??, ??) at 0x100106d8
sh_exec(??, ??) at 0x1000fdb8
sh_exec(??, ??) at 0x1000fe08
sh_exec(??, ??) at 0x1000fd94
b_dot_cmd(??, ??, ??) at 0x10066e20
sh_funct(??, ??, ??, ??, ??) at 0x1000e500
sh_exec(??, ??) at 0x1000ee88
sh_exec(??, ??) at 0x1000fd94
sh_exec(??, ??) at 0x10010370
sh_exec(??, ??) at 0x1000fdb8
sh_exec(??, ??) at 0x1000fe98
exfile(??, ??) at 0x10000ed8
sh_main(??, ??, ??) at 0x100015dc
pmain.main(??, ??) at 0x10000370

Or you can use others dbx’s commands. Please see man dbx.

1
2
3
4
5
6
7
</pre>
(dbx) thread
(dbx) map
(dbx) func
(dbx) dump
(dbx) quit
<pre>

Just thanks if the post was helpful

4 (3)
Article Rating (3 Votes)
Rate this article
Attachments
There are no attachments for this article.
Comments (2)
Comment By Hadar Leave - Mon, Dec 26th, 2022 1:06 PM
Very helpful. Thanks!
Comment By Ricardo Alves - Mon, Jan 24th, 2022 4:31 PM
Thanks for sharing this info! :)
Full Name
Email Address
Security Code Security Code
Related Articles RSS Feed
How to know witch process is running in a particular port on AIX using KDB
Viewed 13683 times since Thu, Nov 29, 2018
AIX alt_disk_copy
Viewed 10878 times since Sun, Jun 30, 2019
HOWTO: Implement SEA Failover with Dual VIOS
Viewed 9327 times since Tue, Jun 4, 2019
Awesome Command to show top 15 processes using memory on AIX
Viewed 24897 times since Thu, Nov 29, 2018
IBM AIX multipath I/O (MPIO) resiliency and problem determination
Viewed 15123 times since Wed, May 30, 2018
Part 2, NFS monitoring and tuning
Viewed 7540 times since Mon, Jun 4, 2018
7 Tips – Tuning Command Line History in Bash
Viewed 6802 times since Fri, Jul 5, 2019
Many Choices for AIX File Integrity
Viewed 3475 times since Wed, May 30, 2018
Using AIX VG mirroring in combination with hardware snapshots
Viewed 6094 times since Sat, May 25, 2019
AIX, Security, System Admin↑ Clearing password history
Viewed 3389 times since Fri, Apr 19, 2019