This was Intended to be a BSD only blog, but now it's about all Unix Like Free Operating Systems, Linux, FreeBSD etc.
Monday, October 31, 2011
Upcoming Kinect Development Kit Could Change In-Store Shopping
Fwd: DVR HARD DRIVES!! ORDER 1-500 PCS OF EACH
From: CCTV Imports <sales@cctvimports.com>
Date: Mon, Oct 31, 2011 at 7:41 AM
Subject: DVR HARD DRIVES!! ORDER 1-500 PCS OF EACH
To: sokol@dnull.com
Having trouble viewing this email? Click here |
CCTV Imports | 600 Deer Cross Court East | Madisonville | LA | 70447 |
Sunday, October 30, 2011
CGI's in BASH: part 1
uptime
echo Content-type: text/plain
echo
/usr/bin/uptime
Or you can expand this a little
echo Content-type: text/plain
echo
if [ -x $UPTIME ]; then
$UPTIME
else
echo Cannot find uptime command on this system.
fi
CGI's in BASH, Part 2: Conclusion of Domain report
Checking Status of Domains hosted on a server. Advanced Bash
domainreport
echo "Content-type: text/html"
echo
echo "<html><body>"
cd /data/namedb/checkdomains
echo "<table border=2 cellpadding=2 cellspacing=2>"
for h in $( cat masterlist ); do
out="<tr><td>$h </td> "
for i in "extern" "local" "sokol" "whois" ; do
out="$out <td>`grep -l "^$h" $i.*`</td>"
done
out="$out <td>`grep -l "^$h" domains-from-httpd`</td>"
out="$out <td>`grep -l "^$h" domains-from-mail`</td>"
out="$out <td>`grep -l "^$h" available`</td>"
out="$out </tr>"
echo $out |\
sed -e 's/domains-from-httpd/Httpd/' |\
sed -e 's/domains-from-mail/E-Mail/' |\
sed -e 's/\.bad/<font color="red">\0<\/font>/g' |\
sed -e 's/\.good/<font color="green">\0<\/font>/g'
done
echo "</table>"
echo "</body></html>"
This second version save the first pass output to a file, then takes a second pass to sort the report before removing the index used to sort, and then displays it.
domainreport2
echo "Content-type: text/html"
echo
echo "<html><body>"
echo "" > /tmp/domainreportout
cd /data/namedb/checkdomains
echo "<table border=1 cellpadding=2 cellspacing=2>"
for h in $( cat masterlist ); do
out="<td><a target="blank" href="http://$h">$h</a></td>"
for i in "extern" "local" "sokol" "whois" ; do
out="$out <td>`grep -l "^$h" $i.*`</td>"
done
out="$out <td>`grep -l "^$h" domains-from-httpd`</td>"
out="$out <td>`grep -l "^$h" domains-from-mail`</td>"
out="$out <td>`grep -l "^$h" available`</td>"
tally=`echo $out | tr '<' '\n' | grep -c "\.good"`
#out="<td>$tally</td>$out"
out="$tally ::$out"
out="<tr>$out</tr>"
echo $out |\
sed -e 's/domains-from-httpd/Httpd/' |\
sed -e 's/domains-from-mail/E-Mail/' |\
sed -e 's/\.bad/<font color="red">\0<\/font>/g' |\
sed -e 's/\.good/<font color="green">\0<\/font>/g' >> /tmp/domainreportout
done
sort -n -r /tmp/domainreportout | sed 's/.*:://'
echo "</table>"
echo "</body></html>"
Below is an example of the output.
6ghz.com | extern.good | local.good | sokol.good | whois.good | Httpd | ||
8vsb.com | extern.good | local.good | sokol.good | whois.good | Httpd | ||
9bnog.com | extern.bad | local.bad | sokol.good | whois.bad | |||
9bnog.net | extern.bad | local.bad | sokol.good | whois.bad | |||
9bnog.org | extern.bad | local.bad | sokol.good | whois.bad | |||
a2znetworks.net | sokol.bad | whois.bad | available |
Checking Status of Domains hosted on a server. Advanced Bash
I have a domains file I use to automatically generate DNS entries.
Part 2 here, output reports to the web
PASS1 - Harvest list of domains, and gather whois records and DNS lookups.
# Harvest Domains from QMAIL
sort -u /var/qmail/control/locals /var/qmail/control/plusdomain /var/qmail/control/rcpthosts > domains-from-mail
# Harvest Domains from Apache Config
grep -E "ServerName|ServerAlias" /conf/sites-enabled/* |\
awk '{print $NF}' |\
awk -F "." '{print $(-1 + NF)"."$NF}' |\
sort -u > domains-from-httpd
# Copy over other sources of Domains on the host, may want to harvest DNS records too.
# Consolidate all domains know to the system from all domains* files in to masterlist of domains to perform checks.
cat domains* | awk '{ print $NF }' | sort -u > masterlist
# This version supports comments, change out with line above to do that.
#cat domains* | grep -v "^#" | awk '{ print $NF }' | sort -u > masterlist
# Make sure there is an out directory
# clean out bad entries in out Directory
grep -l "LIMIT EXCEEDED" out/*.whois | xargs rm
# Harvest data from local and external DNS & whois
for h in $( cat masterlist ); do
host $h > out/$h.local &
host $h 8.8.8.8 > out/$h.extern &
#ls -l out/$h.whois
if [ ! -s out/$h.whois ]; then
#echo "Running whois on out/$h.whois"
whois $h > out/$h.whois &
fi
done
sleep 2
grep -l "LIMIT EXCEEDED" out/*.whois | xargs rm
For each domain we generate 3 files, the external DNS lookup, the Internal lookup and the whois lookup.
-rw-r--r-- 1 sokol sokol 82 Oct 28 20:32 6ghz.com.local
-rw-r--r-- 1 sokol sokol 4388 Oct 28 20:08 6ghz.com.whois
-rw-r--r-- 1 sokol sokol 148 Oct 28 20:32 8vsb.com.extern
-rw-r--r-- 1 sokol sokol 82 Oct 28 20:32 8vsb.com.local
-rw-r--r-- 1 sokol sokol 4388 Oct 28 20:08 8vsb.com.whois
Before we generate a report we need to harvest the useful information from the data we just gathered.
This is done with simple grep searches and depositing the output in to a good or bad bucket.
PASS2 - this organized the data we just harvested.
MYDNS="72.249.144.147"
MYHOST="DNULL.COM"
MYNAME="sokol"
grep address out/*.extern | grep -v $MYDNS | sed -e 's/.*://' > extern.bad
grep address out/*.extern | grep $MYDNS | sed -e 's/.*://' | awk '{print $1}' > extern.good
grep address out/*.local | grep -v $MYDNS | sed -e 's/.*://' > local.bad
grep address out/*.local | grep $MYDNS | sed -e 's/.*://' | awk '{print $1}' > local.good
grep -L -i $MYHOST out/*.whois | sed -e 's/\.whois//' -e 's/.*\///' > whois.bad
grep -l -i $MYHOST out/*.whois | sed -e 's/\.whois//' -e 's/.*\///' > whois.good
grep -L -i $MYNAME out/*.whois | sed -e 's/\.whois//' -e 's/.*\///' > sokol.bad
grep -l -i $MYNAME out/*.whois | sed -e 's/\.whois//' -e 's/.*\///' > sokol.good
grep -l "No match for" out/*.whois | sed -e 's/\.whois//' -e 's/.*\///' > available
REPORT
for h in $( cat masterlist ); do
out="$h: "
for i in "extern" "local" "sokol" "whois" ; do
out="$out `grep -l "^$h" $i.*`"
done
out="$out `grep -l "^$h" domains-from-httpd`"
echo $out |\
sed -e 's/domains-from-httpd/httpd/' |\
sed -e 's/\.bad/\d27[32;41m\0\d27[m/g' |\
sed -e 's/\.good/\d27[30;42m\0\d27[m/g'
done
Saturday, October 29, 2011
Wednesday, October 26, 2011
The Yocto Project
http://www.yoctoproject.org/
Tuesday, October 25, 2011
Fwd: [svlug] BayPIGies Meeting Thursday 10/27, 7:30 PM, "YouTube for Your Business", JJ Behrens
From: jim <jim@systemateka.com>
Date: Tue, Oct 25, 2011 at 10:50 AM
Subject: [svlug] BayPIGies Meeting Thursday 10/27, 7:30 PM, "YouTube for Your Business", JJ Behrens
To: svlug@lists.svlug.org
"YouTube for Your Business", JJ Behrens, 7:30 PM Thursday 20111027
JJ Behrens will give an introduction to YouTube and YouTube APIs with an
emphasis on how you can use them for your business. He'll also give
some information about how Python is used at YouTube.
JJ Behrens is a developer advocate at Google specializing in YouTube
APIs. He's a long-time member of BayPiggies. He blogs at
jjinux.blogspot.com
on topics such as Python, Ruby, Linux, open source software, the Web,
and lesser-known programming languages.
LOCATION
Symantec Corporation
Symantec Vcafe
350 Ellis Street
Mountain View, CA 94043
http://maps.google.com/maps/ms?oe=utf-8&client=firefox-a&ie=UTF8&fb=1&split=1&gl=us&ei=w6i_Sfr6MZmQsQOzlv0v&hl=en&t=h&msa=0&msid=116202735295394761637.00046550c09ff3d96bff1&ll=37.397693,-122.053707&spn=0.002902,0.004828&z=18
BayPIGgies meeting information is available at
http://www.baypiggies.net/
_______________________________________________
svlug mailing list
svlug@lists.svlug.org
http://lists.svlug.org/lists/listinfo/svlug
Bash & shell trick
Wiki tricks
Move heading in a level deeper
sed -e 's/\(=\+\)/=\1/g' infile > outfile
Wikipedia:Tools/Editing_tools
An HTML to Wiki syntax converter
Convert Word doc or Webpage to wiki
http://excel2wiki.net/
Sed tricks
Bash Tricks
RPM
Harvest host memory profile
gethostinfo.sh/usr/sbin/dmidecode > /tmp/DMIDECODE LLIST=`grep -n "Memory Device" /tmp/DMIDECODE | grep -v "Address" | sed 's/:.*//'` TOTSIZE=0 for i in $LLIST; do MEMSIZE=`tail -n +$i /tmp/DMIDECODE | head -13 | grep "Size:" | sed 's/No/0/' | awk '{print $2}'` TOTSIZE=$[ $TOTSIZE + $MEMSIZE ] done PNAME=`grep "Product Name" /tmp/DMIDECODE | sed -e 's/.*: //' -e 's/ /_/g' -e '1 q' ` HNAME=`hostname | sed 's/\.xyzcompany.*//'` OSINFO=`uname -n -m -r` #SPEEDINFO=`grep "Current Speed:" /tmp/DMIDECODE | sed -e 's/.*: //' -e 's/000 MHz/ GHz/' -e '1 q' ` NUMCORES=`grep "^processor" /proc/cpuinfo | sed -e 's/.*: //' | tail -1` NUMCORES=$[ $NUMCORES + 1] CPUINFO=`grep "model name" /proc/cpuinfo | sed -e 's/.*: *//' -e 's/\s\+/_/g' -e '1 q' ` RAMSIZE=`echo "$TOTSIZE MB" | sed -e 's/1024 MB/1 GB/' -e 's/2048 MB/2 GB/' -e 's/4096 MB/4 GB/' -e 's/6144 MB/6 GB/' -e 's/8192 MB/8 GB/' -e 's/12288 MB/12 GB/' -e 's/16384 MB/16 GB/' ` #echo "$OSINFO $PNAME $RAMSIZE $SPEEDINFO" echo "$OSINFO, $PNAME, $RAMSIZE, $NUMCORES Core, $CPUINFO"
TCPDUMP
sudo tcpdump -i bond0:6 -s 0 -x host 10.1.224.147 and port 8080 sudo tcpdump -i bond0:6 -s 0 sudo tcpdump -i bond0:6 -s 0 -X port 8080 -w /tmp/tdump1output file tdump1 can be read in using WireShark
rsync
Pulls files off remote and delete's then after verified transfer.sudo rsync -auvlz --remove-sent-files jsokol@xyzhost:tmp/xyx* .
search and kill processesThis was for finding and killing XYZ processes on servers, (it's assumed) they will restart themselves.
[jsokol]$ more bouncehosts
if [ -s $1 ] ; then
awk '{ print $1 }' $1 > /tmp/hostlist1
for h in $(cat /tmp/hostlist1 );do
echo ID $h
TMP1=`ps ax | grep xyz | grep Builds | grep "com_$h "`
# ps ax | grep xyz | grep Builds | grep "com_$h "
if [ "$?" -eq 1 ] ; then
TMP2=`grep "^$h " $1`
echo No XYZ Running for $TMP2
else
TMP2=`echo $TMP1 | awk '{print $1}'`
echo Killing Pid $TMP2
kill $TMP2
sleep 30
fi
# grep "$h " x2 | awk '{print "'$h' " $1}' | sed 's/XYZ=/ /' | sed 's/%2C/ /g' | sed 's/.xyzco.com/ /g'
done
else
echo "file $1 doesn't exist, or you didn't enter a filename"
fi
if [ "$UID" -ne "0" ];then echo "please sUd0 this.."; exit
ssh -q "$3"@hostdb.corp "sudo mysql -e \"select hostname,productname from host where productname like '%$1' and hostname like '%$2%' \" noc";
for i in $(cat XYZ.all); do echo "edit" | ssh sokol@$i 'sudo sed s/34/37/g -i /etc/app/configfile' ;done >> xyz.log
[jsokol@xyz.sjc ~]$ ZZ=6 [jsokol@xyz.sjc ~]$ echo $[4 + $ZZ] 10
echo xyzhost(3..6}.sjc,|sed s/", "/","/gxyzhost3.sjc,xyzhost4.sjc,xyzhost5.sjc,xyzhost6.sjc,
running average
#!/bin/bash
# example of how to get running average.
if [ ! -s "runaveragedata-last" ] ; then
echo 0 > runaveragedata-last
fi
NEWVAL=`cat newdata`
RUNAVR=`cat runaveragedata-last`
# doing work in fixpoint numbers
NEWAVR=$[ $NEWVAL * 1000 + $[ $RUNAVR * 9 ] ]
RUNAVR=$[ $NEWAVR / 10 ]
echo $RUNAVR > runaveragedata-last
echo $[ RUNAVR / 1000 ]
Bash loops
for h in 07 08 09 10 11 12 13 18; do scp host:/path /xyc$h.log .;done for h in $(cat file ); do echo $h |tee -a outputlog; rsync -auvlz host:$h $h; done rpm -qa | egrep -i 'egw|logfs' | xargs sudo rpm -e ------------ more dupesniff.sh #!/bin/bash for h in $(grep -vE ^# $1 | grep -vE ^$ | awk {'print $1'} | sort | uniq -d ) do grep -n $h $1 done ------------ [jsokol]$ cat portprobe.pl #!/usr/bin/perl -w # # tiny tool meant to mimic a device profile. It should take ports, or service names, # and mimic the open ports epected for that type of machine # use IO::Socket; use Sys::Hostname; my $proto = "udp"; my $port = "8080"; my $hostname = hostname; my $persist = 0; my $server = "localhost"; my $socket = IO::Socket::INET->new(PeerAddr => $server, PeerPort => $port, Proto => $proto, Type => SOCK_STREAM) or die "Couldn't talk to $server on $port \n\n"; print $socket "GET \n"; @answer = <$socket>; close($socket); print "@answer \n\n"; ------------ [jsokol]$ cat portlisten.pl #!/usr/bin/perl -w # # tiny tool meant to mimic a device profile. It should take ports, or service names, # and mimic the open ports epected for that type of machine # # use IO::Socket; use Sys::Hostname; my $proto = "udp"; my $port = "8080"; my $hostname = hostname; my $persist = 0; my $server = IO::Socket::INET->new( LocalPort => $port, # Type => SOCK_STREAM, Proto => $proto, # Reuse => 1, # Listen => 10 ) or die "\tCouldn't generate new server socket on port $port\n\n"; print "starting server\n\n"; $server->accept(); while ($server->recv($msg, 255)) { print "Got message $msg from $server->peername \n"; $server->send("Thanks"); } close($server); ------------ #!/bin/sh # comm-hack. Use this to send multiple commands to multiple hosts.
# Usage: ./scriptname [hostlist filename] [user] x=$1 #host list y=$2 #username #begin echo echo -n "Enter password: " read -s PASS echo echo -n "Enter commands: " read COMMANDS #for i in $(cat $x ); do echo -e "$PASS" | ssh '$y'@$i $COMMANDS; done for i in $(cat $x ); do echo "$PASS"| ssh "$y"@$i $COMMANDS; echo "Executed on host: $i @ `date '+%m/%d/%y%t'` " done fi
Monday, October 24, 2011
ARM TechCon
From: ARM TechCon <arm_techcon@eetevents.com>
Date: Mon, Oct 24, 2011 at 4:30 PM
Subject: Tuesday Digital Daily
To: john.sokol@gmail.com
|
Friday, October 21, 2011
Time up update your Rootkit's.
Some are as benign as adding links to your web pages, and inserting annoying advertizing, or just hosting Muslim extremist video's or pirated content. Other's want to turn your system in to a Zombie, which means turn it in to a node of a cloud for distributed nefarious activities. This could be like overloading an IRC channel to take it over, or as part of a DDOS attack. Other's are to SPAM, or crack captcha. Even worse to highjack your customers, steal data or passwords that may work on yet other systems.
Bottom line, There is money in your insecurity!
The current Model's of Operating systems are just broken. I mean it just wasn't designed for it and were overloading, trying to pile on more and more without redesigning the foundation.
The end result is many of one off efforts, reinventing the wheel over and over, and competing libraries when a more generalized solutions are now long overdue.
From Slashdot:
Most Sophisticated Rootkit Getting an Overhaul
"TDL4, a rootkit that helps build a powerful botnet, is pegged by security vendor ESET as one of the most sophisticated pieces of malware in the world. But its creators aren't resting on their laurels; they're rewriting some of the code from the ground up to make it difficult for antimalware to detect it, creating a hidden boot partition that guarantees malware code will be loaded even before the operating system is. It's part of a plan to turn TDL4 into a turnkey product that can be sold to other criminal operations."
VI, the TV Show?!?!
Vi is the main default Unix Editor.
read-eval-print loop (REPL)
I think it's a Lisp Terminology.
http://en.wikipedia.org/wiki/Read-eval-print_loop
This is basically a fancy way to say a Command Interpreter. Nothing more fancy then any Boot load such as U-boot does, or Bash, DOS, BC, Basic, Lisp, Smalltalk, Standard ML, Perl, Prolog, Scala, Python, Ruby's IRB, Haskell, APL, BASIC, J, Tcl or the many other interpreters available.
C and C++
Perl
Python
- PyPy a self-hosting interpreter
- Online Python Read-Eval-Print Loop (REPL)
If you have anything else to include please add a comment below.
Microsoft Roslyn: Reinventing the Compiler As We Know It
"Fatal Exception's Neil McAllister sees Microsoft's Project Roslyn potentially reinventing how we view compilers and compiled languages. 'Roslyn is a complete reengineering of Microsoft's .NET compiler toolchain in a new way, such that each phase of the code compilation process is exposed as a service that can be consumed by other applications,' McAllister writes. 'The most obvious advantage of this kind of "deconstructed" compiler is that it allows the entire compile-execute process to be invoked from within .NET applications. With the Roslyn technology, C# may still be a compiled language, but it effectively gains all the flexibility and expressiveness that dynamic languages such as Python and Ruby have to offer.'"
Wednesday, October 19, 2011
Saturday, October 15, 2011
What constitutes Unix?
"The main standard defining what constitutes a Unix OS is POSIX, an acronym for: Portable Operating System Interface for UniX" - http://www.dmoz.org/desc/Computers/Software/Operating_Systems/Unix
But this isn't completely agreed upon.
"Unix is a powerful, multi-user environment that has been implemented on a variety of platforms. Once the domain of servers and advanced users, it has become accessible to novices as well through the popularity of Linux and Mac OS X. With the notable exception of Microsoft Windows, all current major operating systems have some kind of Unix at their cores."
Read also:
http://kb.iu.edu/data/agat.html
But Windows may indeed be considered Unix because since NT it does support POSIX. And it called Interix.
To get it all you need to do is install Services for Unix (SFU).
So what this means is ALL major OS's today support POSIX and could be considered flavors of UNIX.
But I have some friends that fervently disagree. They don't even consider Linux to be UNIX because the code was not directly derived from the UNIX code, But most flavors of UNIX today can't directly trace there code back to the original Bell Labs code and yet there are no doubts when calling these OS's a UNIX variant.
Solaris, Mac OS for example.
Thursday, October 13, 2011
Dennis Ritchie Passed Away 8 Oct 2011
In addition to his work on UNIX and C, Dennis contributed to the Plan 9 operating system (1995), and the Inferno operating system, which was announced in April 1996.
Witout Unix and C, every major Operating systems today wouldn't be the same. The Internet may not even exist without Unix and C that is the bases for every web server, Router, and desktop OS today. Most smart phones are based on Linux which is a work alike version of Unix and on several occasions been accused of using Unix code it's so similar in functionality.
His book the C Programming Language has never been far from me since I have first picked it up back in the 80's. It's been my Bible. My whole life, my career, my businesses, most of my accomplishments were based on C, Unix and the Internet. When I first ran in to them as a Hacker, they came together as a natural package , a collection of Technologies along with many more. Technologies that were refined from years of college students screwing about trying to do "Cool things" . The result was cool. And more over Practical. So much code these days is just people getting caught up in testing every aspect of a language rather then the simplest cleanest way to get the job done.
The Unix Source code was brilliant. It was the first time I had ever seen well written code beside the little short basic programs floating around.
It was overwhelming at the time the amount of code. We spent month, no years, trying to download and store the 40 megabytes of source code. We started at 300 baud, and 360K floppies off a hacked phone company computer. Eventually I ended up working at Stanford and got it at 10 Mbps from a VAX server down the hall. First thing I did was try to get it to run on a PC. I can't tell you how precious that Source code was. How much risk and trouble I could get in to just for having it. How much I tried to learn all I could from it.
Mr. Ritchie, I owe you so much. I can't thank you enough. You are missed and will be remembered.
See my http://c-program.com/ that talks about the C programming Language Book.
Reference: http://en.wikipedia.org/wiki/Dennis_Ritchie
Writing A Compiler In Haskell that outputs to LLVM
Haskell Compiler Series
Haskell & LLVM Talk
Writing A Compiler In Haskell (Compiler Series Part I)
Playing with Alex (Compiler Series Part II)
Having Fun with Happy (Compiler Series Part III)
Wednesday, October 12, 2011
Fwd: New Meetup: Free: HTML5 training offered by Microsoft
From: "Silicon Valley HTML5 User Group" <info@meetup.com>
Date: Oct 12, 2011 12:42 PM
Subject: New Meetup: Free: HTML5 training offered by Microsoft
To: <john.sokol@gmail.com>
|
Free: HTML5 training offered by Microsoft Added by Siddharth Ram | |||||||||||
| |||||||||||
Add info@meetup.com to your address book to receive all Meetup emails To manage your email settings, click here Meetup, PO Box 4668 #37895 New York, New York 10163-4668 Meetup HQ in NYC is hiring! http://www.meetup.com/jobs/ |