Checking Status of Domains hosted on a server. Advanced Bash

This is a 2 pass system to report of domain status of domains hosted on a server.

I have a domains file I use to automatically generate DNS entries.
   Part 2 here, output reports to the web


PASS1 - Harvest list of domains, and gather whois records and DNS lookups.

#!/bin/bash

# Harvest Domains from QMAIL
sort -u /var/qmail/control/locals /var/qmail/control/plusdomain /var/qmail/control/rcpthosts > domains-from-mail


# Harvest Domains from Apache Config
grep -E "ServerName|ServerAlias" /conf/sites-enabled/*  |\
 awk '{print $NF}' |\
 awk -F "." '{print $(-1 + NF)"."$NF}' |\
 sort -u > domains-from-httpd

# Copy over other sources of Domains on the host,  may want to harvest DNS records too.


# Consolidate all domains know to the system from all domains* files in to masterlist of domains to perform checks.
cat domains* | awk '{ print $NF }' | sort -u > masterlist

# This version supports comments, change out with line above to do that.
#cat domains* | grep -v "^#" | awk '{ print $NF }' | sort -u > masterlist

# Make sure there is an out directory

# clean out bad entries in out Directory
grep -l "LIMIT EXCEEDED" out/*.whois | xargs rm


# Harvest data from local and external DNS & whois
for h in $( cat masterlist ); do

host $h > out/$h.local &
host $h 8.8.8.8  > out/$h.extern &

#ls -l out/$h.whois
if [ ! -s out/$h.whois ]; then
#echo "Running whois on out/$h.whois"
whois $h > out/$h.whois &
fi
done

sleep 2
grep -l "LIMIT EXCEEDED" out/*.whois | xargs rm

For each domain we generate 3 files, the external DNS lookup, the Internal lookup and the whois lookup.

-rw-r--r-- 1 sokol sokol 148 Oct 28 20:32 6ghz.com.extern
-rw-r--r-- 1 sokol sokol 82 Oct 28 20:32 6ghz.com.local
-rw-r--r-- 1 sokol sokol 4388 Oct 28 20:08 6ghz.com.whois
-rw-r--r-- 1 sokol sokol 148 Oct 28 20:32 8vsb.com.extern
-rw-r--r-- 1 sokol sokol 82 Oct 28 20:32 8vsb.com.local
-rw-r--r-- 1 sokol sokol 4388 Oct 28 20:08 8vsb.com.whois

Before we generate a report we need to harvest the useful information from the data we just gathered.

This is done with simple grep searches and depositing the output in to a good or bad bucket.


PASS2 - this organized the data we just harvested.

#!/bin/bash

MYDNS="72.249.144.147"
MYHOST="DNULL.COM"
MYNAME="sokol"

grep address out/*.extern | grep -v $MYDNS | sed -e 's/.*://' > extern.bad
grep address out/*.extern | grep $MYDNS | sed -e 's/.*://' | awk '{print $1}' > extern.good

grep address out/*.local | grep -v $MYDNS | sed -e 's/.*://' > local.bad
grep address out/*.local | grep $MYDNS | sed -e 's/.*://' | awk '{print $1}' > local.good

grep -L -i $MYHOST out/*.whois | sed -e 's/\.whois//' -e 's/.*\///' > whois.bad
grep -l -i $MYHOST out/*.whois | sed -e 's/\.whois//' -e 's/.*\///' > whois.good

grep -L -i $MYNAME out/*.whois | sed -e 's/\.whois//' -e 's/.*\///' > sokol.bad
grep -l -i $MYNAME out/*.whois | sed -e 's/\.whois//' -e 's/.*\///' > sokol.good

grep -l "No match for" out/*.whois | sed -e 's/\.whois//' -e 's/.*\///' > available

REPORT

#!/bin/bash
for h in $( cat masterlist ); do

out="$h: "

for i in "extern" "local" "sokol" "whois" ; do
out="$out `grep -l "^$h" $i.*`"
done

out="$out `grep -l "^$h" domains-from-httpd`"

echo $out |\
sed -e 's/domains-from-httpd/httpd/' |\
sed -e 's/\.bad/\d27[32;41m\0\d27[m/g' |\
sed -e 's/\.good/\d27[30;42m\0\d27[m/g'
done

Read the next article to see this in CGI form.