Many Choices for AIX File Integrity

Print 0

 

Recently, I was on a disaster recovery (DR) exercise that was going extremely well, until I had to restore a DB2 database. The restore failed due to a corrupted image. The image had been transferred from an external secure network to our DR site, so it should have been good I thought.

I checked the image size on the source and destination and both matched up OK. Next task to do was to check the integrity of the image. By that I mean compare the signature of the source and destination to confirm that the signatures match up, using the same algorithm. Signatures are sometimes referred to (though incorrectly) as checksums, digest, hashes or fingerprints. For this article, I will stick with signatures to avoid any confusion.

Plenty of data integrity tools exist, but you don’t need them to do an ad hoc check on files. The integrity tools available with AIX are:

• csum
• cksum
• sum
• openssl
• shasum

File integrity checks that there’s been no alteration—no addition or deletion of parts of the file from source to destination. It, in essence, checks that the file hasn’t been tampered with.

When producing a signature from a file, it can be considered a one-way signature, in that it looks at the input, which is the file, and produces a unique signature as its output based on the input. Two of the most popular algorithms used to generate signatures are: SHA (secure hash algorithm) and MD5. SHA is considered the best for signature uniqueness. SHA will generate a 20-byte and the MD5 will generate 16-byte signature. Clearly the longer the signature, the better chance or probability of obtaining uniqueness.

When generating a signature, you need to be pretty confident it will be unique with a high probability that it cannot be duplicated against another file. When you produce a signature on a file, say at the source location, be sure you use or inform the users comparing the file at the destination the algorithm originally used. If you don’t, it won’t be the same signature, which makes the use of comparing signatures in files rather redundant.

Let’s looks at how to use some of the AIX included utilities.

A Plethora of Tools

Using csum on a file using both algorithms returns:

#  csum -h MD5 myfile
670e251013ea807980baef53268ee264  myfile
# csum -h SHA1 myfile
550b856b4bd36229d5790d4c33cc75fb8dd0b8c4  myfile

Note in the output I specified MD5, however by default if no algorithm specified it will use MD5. Also notice the longer signature for the SHA1. Of course you can specify patterns to match many files in one hit, like:

# csum -h SHA1 myfile* 
550b856b4bd36229d5790d4c33cc75fb8dd0b8c4  myfile
c4cfa40c7076f04bc128038f4aa335ffb20e98a6  myfile2

You can also direct the output to a test file. This is handy when you’re dealing with many files and wish to compare (probably within a shell script) the signatures at the destination side. The following csum output is put in the file, out_file:

# csum -o out_file -h SHA1 myfile*

The cksum is slightly different in that it produces a checksum and the byte count of the file. In this demonstration, both files have the same content. But note though they’re the same byte count, the checksum is different. The output of the command is quite useful when doing a compare of files. One simply reads in the file from within a script and then does another cksum on the original file(s) to see if there have been any changes.

# cksum myfile*
3224161752 1058 myfile
3347010591 1058 myfile2

If you have OpenSSL installed, then you have two further options for file integrity checks, spoiling you with choices. The command shasum also ships with openssl, and you use this to generate the signature.

# openssl sha1 myfile*
SHA1(myfile)= 550b856b4bd36229d5790d4c33cc75fb8dd0b8c4
SHA1(myfile2)= c4cfa40c7076f04bc128038f4aa335ffb20e98a6

# shasum myfile*
550b856b4bd36229d5790d4c33cc75fb8dd0b8c4  myfile
c4cfa40c7076f04bc128038f4aa335ffb20e98a6  myfile2

Of course, you can redirect the output to a file and then re-issue shasum to verify the signature. It will output OK if the verification check is good:

# shasum  myfile >test
# cat test
550b856b4bd36229d5790d4c33cc75fb8dd0b8c4  myfile
# shasum  -c test
OK

Essential for Success

File checking is essential when you need to verify that the file hasn’t been tampered with. In my case, I found that the signature of the transferred file didn’t match at the DR exercise. I discovered that the DB2 image was transferred as text and not as binary mode. So it then got re-transferred correctly as binary and the signatures matched up when I checked them. The restore was successful.