Using the search_orphans script to delete expired packages in IME Server

Summary:
The search_orphans script will determine if there are any files in the <IME_INSTALL>/state/docs directory which are no longer referenced by IME. Files which are no longer referenced (by the IME database) cannot be displayed as part of a package, nor can they be cleaned up by the IME batch processing. Identifying and deleting such files can free up disk space.

This article describes how to run search_orphans and what to do with the results.

These instructions apply only to Solaris.

Detailed Information:
To run the script you must be root and know the IME database account and password:

<IME_INSTALL>/scripts/search_orphans

The script may take several hours to complete.

After running the search_orphans script you will have a file named /tmp/OUTPUT containing a list of files on the file system which no longer have any record in the database. These are candidates for deletion.

To see how many files were listed by search_orphans:

cat /tmp/OUTPUT | wc -l

IMPORTANT NOTE - If you do NOT stop IME while running search_orphans:
You must manually delete any entries for the current day from the OUTPUT file, as they may be files associated with partially created packages. You can see the date embedded in the path of the filenames, making this step possible.

If you expect that the contents referenced in the OUTPUT file will be more than 2GB, you may want to split the OUTPUT file into smaller files, as it is hard to compress and scp files over 2GB.

The following instructions are in the form of a script you may wish to save to your server. You will most likely want to run some variations of the commands below, which is fine. Be sure that you have a validated backup before you delete any files.

#!/usr/bin/sh
###################################################################

###################################################################
# To Backup the files in the referenced in the
# OUTPUT file. Run a command something like:

# Set the BACKUP_FILE variable to a useful file name where you
# have lots of space.
BACKUP_FILE=/tmp/search_orphans.backup.`date '+%Y%m%d_%H%M%S'`.tar

# make sure the file does not already exist
rm -f $BACKUP_FILE
# create a zero length file
touch $BACKUP_FILE
# back up the contents refered to in the OUTPUT file
cat /tmp/OUTPUT | xargs tar uf $BACKUP_FILE
###################################################################

###################################################################
# Be sure to sanity check the backup.
# Here's how to check first/last lines and line count of the
# backup match the OUTPUT file contents:

# First line of the OUTPUT file
head -1 /tmp/OUTPUT
# Last line of the OUTPUT file
tail -1 /tmp/OUTPUT
# Number of lines in the OUTPUT file
cat /tmp/OUTPUT | wc -l

# First file in the backup
tar tvf $BACKUP_FILE | head -1
# Last file in the backup
tar tvf $BACKUP_FILE | tail -1
# Number of files in the backup
tar tvf $BACKUP_FILE | wc -l
###################################################################


###################################################################
# To get the 100 largest files (lowest hanging fruit):
cat /tmp/OUTPUT | xargs ls -l | sed 's/ */ /g' | cut -d' ' -f5,9 | sort -n -k 1 | tail -100 | cut -d' ' -f2 > /tmp/OUTPUT.100
###################################################################



###################################################################
# To Delete the contents of the packages referenced in the OUTPUT file
# Remember the IMPORTANT NOTE detailed above.
# If you do NOT stop IME while running search_orphans:
# You must manually delete any entries for the current day
# from the OUTPUT file, as they may be files associated with
# partially created packages. You can see the date embedded in
# the path of the filenames, making this step possible.
#
# Once you have checked the backup and put it somewhere safe, to
# delete the files run the command:

cat /tmp/OUTPUT | xargs rm -f
###################################################################