How do I analyze and debug core files on AIX

Sometimes, applications stop working and crash for any reason. A core file appears in scene.

First, we’re going  to set a PATH for all cores files on the system  using syscorepath command.

In order to do this, We’re going to:

1- Create and mount a new filesystem /corefiles

1.1- Create the new filesytem:

1
2
3
4
# crfs -v jfs2 -g volgrp2 -m /home/pipe -A yes -a size=1G
File system created successfully.
1048340 kilobytes total disk space.
New File System size is 2097152

1.2- Mount the filesystem:

1
# mount /corefiles

2- Using syscorepath command, we’ll set a PATH where all these cores files will be dumped..

2.1- Set the directory for all cores files:

1
# syscorepath -p /corefiles

2.2- To check if the core’s PATH was correctly set, running:

1
2
# syscorepath -g /corefiles
# /corefiles

3- Analize the core file using dbx command.

Once you get all cores in /corefiles, we’re going to analyse them using dbx command. Follow these steps:

3.1- Get the cause of the core file by running check_core command. In the example ksh93 is the victim:

1
2
# /usr/lib/ras/check_core /corefiles/core |tail -1
ksh93

3.2- Debugging with dbx (Provides an environment to debug and run programs) command:

1
2
3
4
5
6
7
# dbx /usr/bin/ksh93 /corefiles/core
Type 'help' for help.
[using memory image in /corefiles/core]
reading symbolic information ...warning: no source compiled with -g
 
Segmentation fault in strlen at 0xd0105880
0xd0105880 (strlen)    89030000         lbz   r8,0x0(r3)

3.3- Once your are in dbx shell, type ‘where’

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
(dbx) where
strlen() at 0xd0105880
varsub() at 0x100475fc
copyto(??, ??) at 0x100494c8
sh_mactrim(??, ??) at 0x1004a5a0
word_trim(??, ??) at 0x1000e340
sh_exec(??, ??) at 0x1000ff88
sh_exec(??, ??) at 0x1001043c
sh_exec(??, ??) at 0x1000ff50
sh_exec(??, ??) at 0x10010470
sh_exec(??, ??) at 0x1000fdcc
sh_exec(??, ??) at 0x1000fd94
sh_exec(??, ??) at 0x1000fe08
sh_exec(??, ??) at 0x1000fdb8
sh_exec(??, ??) at 0x1000fe08
sh_exec(??, ??) at 0x1000fdb8
sh_exec(??, ??) at 0x100106d8
sh_exec(??, ??) at 0x1000fdb8
sh_exec(??, ??) at 0x1000fe08
sh_exec(??, ??) at 0x1000fd94
b_dot_cmd(??, ??, ??) at 0x10066e20
sh_funct(??, ??, ??, ??, ??) at 0x1000e500
sh_exec(??, ??) at 0x1000ee88
sh_exec(??, ??) at 0x1000fd94
sh_exec(??, ??) at 0x10010370
sh_exec(??, ??) at 0x1000fdb8
sh_exec(??, ??) at 0x1000fe98
exfile(??, ??) at 0x10000ed8
sh_main(??, ??, ??) at 0x100015dc
pmain.main(??, ??) at 0x10000370

Or you can use others dbx’s commands. Please see man dbx.

1
2
3
4
5
6
7
</pre>
(dbx) thread
(dbx) map
(dbx) func
(dbx) dump
(dbx) quit
<pre>

Just thanks if the post was helpful

4 (3)
Article Rating (3 Votes)
Rate this article
Attachments
There are no attachments for this article.
Comments (2)
Comment By Hadar Leave - Mon, Dec 26th, 2022 1:06 PM
Very helpful. Thanks!
Comment By Ricardo Alves - Mon, Jan 24th, 2022 4:31 PM
Thanks for sharing this info! :)
Full Name
Email Address
Security Code Security Code
Related Articles RSS Feed
Create memory resident filesystems (RAM disk) on AIX
Viewed 2856 times since Thu, Feb 21, 2019
AIX, Monitoring, System Admin↑ NMON recordings
Viewed 3141 times since Fri, Apr 19, 2019
AIX Not all filesets for 6100-07_AIX_ML were found
Viewed 3187 times since Tue, Jul 17, 2018
AIX perf how to
Viewed 19175 times since Tue, Aug 14, 2018
Altering LVM Configuration When a Disk is Not in ODM Anymore
Viewed 3290 times since Mon, Jun 3, 2019
AIX, Storage, System Admin↑ Identifying a Disk Bottleneck Using filemon
Viewed 10057 times since Fri, Apr 19, 2019
vfcpmap on VIO Server
Viewed 9701 times since Tue, Jun 4, 2019
Topics: PowerHA / HACMP, Storage Adding a new volume group to an active PowerHA resource group
Viewed 2781 times since Mon, Jun 3, 2019
Configure log file for cron daemon on AIX
Viewed 10832 times since Thu, Feb 21, 2019
IBM AIX multipath I/O (MPIO) resiliency and problem determination
Viewed 13839 times since Wed, May 30, 2018