Simple Solutions

Sometimes a short Bash script is just what the doctor ordered. I Needed a quick script to check a files date on a remote machine and generate an error code if it was over 24 hours old. I will paste the code and a Nagios Expect plugin I wrote to call the script and interface with Nagios.

Remote test script in Bash:
http://malcolm.baxter.googlepages.com/nohup_test.sh

Nagios check script in Expect:
http://malcolm.baxter.googlepages.com/check_applogtime.exp>

Server list creation

While I was working at International Paper I became accustomed to a server tracking file called 'masterfile'. I used it many times in scripts to parse out data about individual servers or compile lists of servers to administer. Since then I have used a similar format here at Grumman to track my boxes and run scripts against them and have put together this Perl script to do it. The syntax can use Perl regular expressions to make it simple to create your lists. You will need Term::ANSIColor module from CPAN or delete the colored output and remove the module from the code. Also your masterfile should be '::' separated with the hostname as the first field. If you would like to see a dummy masterfile email me.

Use this script at your own will I assume no responsibility.

#!/usr/bin/perl

# master.pl - script to parse masterfile
#
# M.Baxter
# Version .90
######################################
######################################

# Complain about undeclared variables
use strict;
use warnings;

# Used modules
use Getopt::Long;
use File::Basename;
use Term::ANSIColor;

# Initialise global variables
our($opt_h, $opt_l, $opt_f, $opt_v, $opt_e);
our $script = basename($0);

# Declare subroutines
sub init;
sub usage;
sub list_servers;
sub full_master;

# Get the options
Getopt::Long::Configure('bundling');
GetOptions(
'v|verbose' => \$opt_v,
'h|help' => \$opt_h,
'f|full' => \$opt_f,
'l|list=s' => \$opt_l,
'e|exclude=s' => \$opt_e,
);

# Main
($opt_h) && usage;
($opt_f) || ($opt_l) || usage;
($opt_f) && full_master;
($opt_l) && list_servers;

# Subroutines
sub usage {
print `clear`, "\n\n";
print "Usage: \n\n";
print " -h : This Help Message\n";
print " -f : full masterfile in csv format created in scripts home dir.\n";
print " -l : list to be run i.e linux|sun|aix or any regular expression to search for.\n";
print " -e : server or regular expression to exclude from list.\n";
print " -v : Verbose output to full and list options.\n\n";
print color('bold green'), "example: $script -l linux -v\n\n\n";
print color('reset');
exit;
}

sub list_servers {
open (MASTER, "/usr/global/configs/masterfile") or die "cant open masterfile: $! \n";
while () {
if (/$opt_l/i) {
if($opt_e) {
if(! $opt_v) {
my @host = split(/:/);
print color('bold green'), "$host[0]\n" if(!m/$opt_e/i and !m/^#/);
} else {
print if(!m/$opt_e/i and !m/^#/);
}
} else {
if(! $opt_v) {
my @host = split(/:/);
print color('bold green'), "$host[0]\n" if(!m/^#/);
} else {
print if(!m/^#/);
}

}
}
}
close (MASTER);
print color('reset'), "\n";
exit;
}

sub full_master {
open (MASTERCSV, ">/usr/global/configs/masterfile.csv") or die "cant open masterfile.csv: $! \n";
open (MASTER, "/usr/global/configs/masterfile") or die "cant open masterfile: $! \n";
print MASTERCSV "Hostname,MachineType,Distribution,Application,Location,serialno,MACHINEMODEL,";
print MASTERCSV "CPUno,OSLEVEL,MEMORY,Startup_Order,Console\n";
while () {
s/:/,/g;
if (!m/^#/) {
print $_ if $opt_v;
print MASTERCSV;
}
}
close (MASTERCSV);
close (MASTER);
exit;
}

Nagios Application monitoring plugin

Recently I was tasked with bringing in the open source monitoring application Nagios into our environment to monitor our servers and hardware. After a small amount of configuration I had it setup on a HP DL360 running Centos 3.4. The Centos repository from DagWiers had all the necessary packages and I was able to simply use yum to install everything I needed. Over the course of a few posts I will show some scripts and apps I have put together to bring our in house applications into the monitoring process. I will not go over the Nagios installation though as I believe it has been covered well before. My first script for monitoring a COBOL application running on a Sun E6800 is called check_runitall. I wrote it in Perl and to use it you will have to install the Net::SCP::Expect module so fire up CPAN and install it before trying this out. Also this script asumes your Nagios plugins are installed to /usr/lib/nagios/plugins.

This code may be used as you like I assume no responsibility for it. Make sure to change the paths and so forth to suit your environment.

#!/usr/bin/perl -w
############################################################
############################################################
# check_runitall
# Nagios plugin to parse runitall
# runtime log
# Version .90
# M.Baxter
# USPS NCSC
############################################################
############################################################

# Complain about undeclared variables
# and set Perl requirements
require Net::SCP::Expect;
use strict;

# Used modules
use Getopt::Long;
use lib "/usr/lib/nagios/plugins";
use utils qw(%ERRORS $TIMEOUT &print_revision);
use vars qw($opt_f $PROGNAME);

# Initialise global variables
my $opt_H = undef;
my $opt_V = undef;
my $opt_t = undef;
my $opt_f = undef;
my $finc_code = undef;
my $message_log;
sub print_usage;
sub print_help;
$PROGNAME = 'check_runitall';

# Get the options
Getopt::Long::Configure('bundling');
GetOptions(
'V|version' => \$opt_V,
'h|help' => \$opt_H,
't|time' => \$opt_t,
'f|function=s' => \$opt_f,
);

if ($opt_V) {
print_revision( $PROGNAME, '$Revision: .90 $' );
exit $ERRORS{'UNKNOWN'};
}
($opt_H) && print_help();
($opt_f) || ($opt_t) || print_usage();

# Secure copy log into local system
my $scpe = Net::SCP::Expect->new(
password => 'Password',
timeout => '25',
auto_yes => 1
);
$scpe->scp( 'User@Servername:/fast/monlogs/runitall_log',
'/home/nagios/runitall_log' );

# open log for reading
open( $message_log, "/home/nagios/runitall_log" )
or die "$PROGNAME cant open runitall_log $! \n";

# Search log for function and preform analysis
while (<$message_log>) {
if ($opt_t) {
if (/BEGIN/) {
chomp;
my @func = split(/::/);
if ( $func[1] eq "Complete" ) {
my @date = split(//, $func[3]);
printf("%-10.10s OK - Complete or waiting - Finished at %d%d\:%d%d %d%d/%d%d/%d%d\n", "Runitall", @date);
close($message_log);
exit $ERRORS{'OK'};
}
elsif ( $func[1] eq "Running" ) {
my @date = split(//, $func[2]);
printf("%-10.10s OK - Running - Start time %d%d\:%d%d %d%d/%d%d/%d%d\n", "Runitall", @date);
close($message_log);
exit $ERRORS{'OK'};
}
}
}
if ( ($opt_f) &&amp;amp; (/$opt_f/) ) {
chomp;
my @func = split(/::/);
if ( ( $func[1] eq "Complete" ) and ( $func[2] eq "Good" ) ) {
$finc_code = 0;
}
elsif ( ( $func[1] eq "Running" ) and ( $func[2] eq "Good" ) ) {
$finc_code = 1;
}
elsif ( ( $func[1] eq "Complete" ) and ( $func[2] eq "Failed" ) ) {
$finc_code = 2;
}
elsif ( ( $func[1] eq "Running" ) and ( $func[2] eq "Failed" ) ) {
$finc_code = 3;
}
}
}
close($message_log);
if ( $finc_code == 0 ) {
printf( "%-10.10s OK - Completed or waiting - Status Good\n", $opt_f);
exit $ERRORS{'OK'};
}
elsif ( $finc_code == 1 ) {
printf( "%-10.10s OK - Running - Status Good\n", $opt_f );
exit $ERRORS{'OK'};
}
elsif ( $finc_code == 2 ) {
printf( "%s Complete or Waiting In Failed State!\n", $opt_f );
exit $ERRORS{'CRITICAL'};
}
elsif ( $finc_code == 3 ) {
printf( "%s Running In Failed State!\n", $opt_f );
exit $ERRORS{'WARNING'};
}

# subroutines
sub print_help {
print "$PROGNAME\n";
print "Nagios Plugin to support log parsing from\n";
print "runitall application.\n\n";
print "Usage:\n\n";
print " -h : Help - Usage Menu\n";
print " -t : Parse Header for run times\n";
print " -f : Function name to be checked\n";
print " -V : Print Version\n\n";
print "example: $PROGNAME -f Function_name\n";
exit $ERRORS{'UNKNOWN'};
}

sub print_usage {
print "Usage: $PROGNAME [-h|-V|-t] [-f Function_name] \n";
exit $ERRORS{'UNKNOWN'};
}