Showing posts with label shell. Show all posts
Showing posts with label shell. Show all posts

2020-02-02

02/02/2020! The first palindrome date since 11/11/1111

It's been almost a decade since I've seen an "only happens every several hundred years" thing floating around that seemed too out-there to be real.

So let's debunk the "first palindrome date since 11/11/1111" shall we? Surely these happen more frequently than that. And for shame not using ISO-8601 format. Fortunately, today also works as an ISO-8601 palindrome: 2020-02-02.

I whipped up a quick bash script to scan for dates n-days before and after today's date looking for palindromes in ISO-8601 format as well as the two other formats that are commonly (ab)used here in the United States, DD-MM-YY and DD-MM-YYYY. It's inefficient, relying on a lot of calls to date(1) and rev(1) but it is what it is.


#!/bin/sh
export i=1
while true
do
# ISO-8601 or GTFO.
idtb=`date -v-${i}d +%Y%m%d`
idtf=`date -v+${i}d +%Y%m%d`

# MM-DD-YYYY format
ydtb=`date -v-${i}d +%m%d%Y`
ydtf=`date -v+${i}d +%m%d%Y`

# MM-DD-YY format
dtb=`date -v-${i}d +%m%d%y`
dtf=`date -v+${i}d +%m%d%y`

if [ "$idtb" -eq "`echo $idtb | rev`" ]
then
echo YYYYMMDD $idtb was a palindrome.
fi

if [ "$idtf" -eq "`echo $idtf | rev`" ]
then
echo YYYYMMDD $idtf will be a palindrome.
fi

if [ "$ydtb" -eq "`echo $ydtb | rev`" ]
then
echo MMDDYYYY $ydtb was a palindrome.
fi

if [ "$ydtf" -eq "`echo $ydtf | rev`" ]
then
echo MMDDYYYY $ydtf will be a palindrome.
fi

if [ "$dtb" -eq "`echo $dtb | rev`" ]
then
echo MMDDYY $dtb was a palindrome.
fi

if [ "$dtf" -eq "`echo $dtf | rev`" ]
then
echo MMDDYY $dtf will be a palindrome.
fi
export i=`expr $i + 1`
done

A quick run for a minute or so shows a lot of palindromes past and future.
MMDDYY 021120 will be a palindrome.
MMDDYY 022220 will be a palindrome.
YYYYMMDD 20211202 will be a palindrome.
MMDDYYYY 12022021 will be a palindrome.
MMDDYY 121121 will be a palindrome.
MMDDYY 122221 will be a palindrome.
MMDDYY 112211 was a palindrome.
MMDDYY 111111 was a palindrome.
YYYYMMDD 20111102 was a palindrome.
MMDDYYYY 11022011 was a palindrome.
MMDDYY 012210 was a palindrome.
MMDDYY 011110 was a palindrome.
YYYYMMDD 20300302 will be a palindrome.
MMDDYYYY 03022030 will be a palindrome.
YYYYMMDD 20100102 was a palindrome.
MMDDYYYY 01022010 was a palindrome.
MMDDYY 031130 will be a palindrome.
MMDDYY 032230 will be a palindrome.  

2015-04-01

Raspberry Pi random host generator

Say you have a really watchful network/systems administrator that keeps a close eye on new devices being joined to the network...

You know where this is going. It's April 1st.

Toss this into /home/pi, then make it executable.

#!/bin/sh
while true
do
  mac=`echo -n 00:03:BA; dd bs=1 count=3 if=/dev/urandom 2>/dev/null | hexdump -v -e '/1 ":%02X"'`
  newhost=`dd if=/dev/urandom bs=35 count=1 2>/dev/null | tr -dc "a-z"`
  echo $mac $newhost
  ifconfig eth0 down
  pkill dhclient
  hostname $newhost
  ifconfig eth0 hw ether $mac
  rm /etc/ssh/ssh_host_rsa_key /etc/ssh/ssh_host_dsa_key
  ssh-keygen -f /etc/ssh/ssh_host_rsa_key -N '' -t rsa -P ''
  ssh-keygen -f /etc/ssh/ssh_host_dsa_key -N '' -t dsa -P ''
  ifconfig eth0 up
  dhclient eth0
  ip addr show eth0 | grep "inet "
  echo "Sleeping..."
  sleep 60
done


You can add the below line before "exit 0" line at the end of /etc/rc.local on Raspbian to make it start up at boot.  You have a random host generator that spawns a new MAC Address, random host name and new SSH keys every minute or so.

nohup /home/pi/mac.sh >> /tmp/mac.out&

A few notes:

  • This will totally hose all of your SSH host keys on the pi.  Make backups of them if they're important to you.
  • I chose an OUI (00:30:BA) that I knew would not match anything else on the target network. You may wish to do some research and change the hard-coded OUI prefix in the code above.
  • The interface fluxing will also make remote management troublesome unless you have a wireless adapter that's on a more stable network, but this can betray you, as the host keys keep changing to match the wired interfaces. 
  • I took the additional step of leaving the Pi powered on for a few minutes before attaching the ethernet cable, so that it wouldn't ever show up on the network with a Raspberry Pi MAC address, since it had time to generate a new fake address before I hooked it in.
  • There are some very simple ways to defend against something like this.
  • It goes without saying, but pranks at work can lead to disciplinary action.
Also, thanks to the target of this April Fool's day prank for giving me a few extra ideas (included here, but not in the original implementation) after catching me in the act. 

2013-03-24

Decoding obfuscated JavaScript: Shell Script Edition

I've been playing with a bunch of malware lately. Most security researchers have run across obfuscated JavaScript, and we all have our favorite ways of defeating it. I'd written about one way to decode this sort of thing back in 2011.

Why bother decoding this stuff? Because encoded within this mess is another URL. Depending on the source of the obfuscated code it may be a link to a page full of exploit payloads or something similarly sinister. Unwrapping the layers of malware allows researchers to find out where the bad guys are actually hosting their stuff, and helps us identify providers who are willing to help with or at least turn a blind eye to cybercrime operations.


Lately, I've become more determined to handle javascript de-obfuscation outside the browser. Over the last year or so, I've been experimenting with a bunch of techniques to make sense of blocks of code that look like this:


I think it goes without saying that this looks like a monumental pain in the ass. The truth is, it's not as bad as it appears, but I didn't say it was going to be easy.

First, the basics. In JavaScript, FromCharCode() turns a decimal number between 0 and 255 into its ASCII character counterpart. You've all seen the ASCII table, right? Same thing.

I found a way to use printf in most shells to create a similar behavior, and I called this function "chr".  So let's start really simple. Here's a file containing a message encoded with CharCodes, and a quick way to decode it. It simply reads each number in (by replacing , with a space and using a for loop, then prints each character one at a time.

This is the basis for the rest of what we're about to do. Let's break down that obfuscated javascript:


The yellow highlighted area is a bunch of numbers separated by the letter w. This is stored in an array labeled "f". If you look after the yellow, you can see that the code uses the letter w to split it.

The blue highlighted area is the loop that handles decoding the numbers into characters. This is obfuscated code, so by definition they've made it a bit confusing, but the end result is that they keep appending each character to the end of the "s" variable.

The bulk of the conversion of number to charcode (ascii decimal number of the character to be rendered) is here: (w[j]*1+41)

This gets us to the essence of the article: You'll need to use some brain power. The first thing I do is turn the data block into a string of comma-separated numbers so they're easier to work with, and I put these in their own file, like so. You can do this inside a text editor with search/replace, or on the CLI if you like.


Then you need to figure out the math. In our example code, what is w? what is j? You'll probably run into a bunch of bizarre variable reassignment when trying to make sense of obfuscated code like this. Looking above, you can see between the yellow and blue blocks, w=f. So w is now a copy of that array full of numbers. Inside the blue block of code, you can see that j=i. So, w[j] points to the current number in the array.  But this number isn't the CharCode. There's still the "*1+41" part to deal with. Manually, we can see what's going on here. This is a very simple algorithm. The first 3 numbers are -32,-32,64.

-32*1+41 = -32+41= 9 - CharCode 9 is a tab.
64*1+41 = 64+41 = 105 - CharCode 105 is a lowercase "i"

Doing this manually would suck. The algorithm here is obvious. The CharCode is the number, plus 41. That's it.  Let's decode it with my script:

So what happened here? This is the source of my relatively simple script.

#!/bin/sh # Obfuscated JS Decoder # Ax0n - 2013-03-22 # ax0n@h-i-r.net # if [ -z "$1" ] then echo "$0 codefile algorithm" echo "c = placeholder for each number" echo "i = iterator" exit 1 fi chr() { # http://mywiki.wooledge.org/BashFAQ/071 # Turns a charcode into the ASCII byte printf \\$(printf '%03o' $1) } count=0 file=$1 shift algo="$*" for code in `cat $file | tr "," " " `; do chr `echo "$algo" | \ sed -e s/"i"/"$count"/g -e s/"c"/"$code"/g | \ bc | cut -f1 -d\.` | tr -d "\r" count=`expr $count + 1` done echo echo "----- done ----"
At the heart of the above script is a nifty function I found in the Bash FAQ. And then, I used a pair of sed expressions to replace the "i" and "c" placeholders within the loop that handles the output. We call on the "bc" command-line calculator to work all the math magic.  tr -d "\r" fixes some broken newlines found in some of the samples I decoded. Let's see it in action:




Now, let's look at the one in the video. It has a much more complicated algorithm than simply adding 41 to each digit before converting it to the ASCII byte! It's hard to read in the video, so here it is:


Fortunately, this one already separates the codes with commas, so we can pretty much just copy and paste the numbers into a text file for decoding. As you can see on the 3rd-to-last line, the algorithm it uses for each character is this:
((w[j]*1+e(x+3)+11))

Let's tear into it, shall we?
w[j] starts out just like the last example. We can see they set j=i and w is a copy of the array, so for each iteration, this is the digit. We can replace this with "c" in our algorithm expression.

That leaves us to figure out what e and x are. Also like the last example, e is set to "eval" at the end of the second line. We'll ignore it. Let's look for x= in the code.
There it is! j% - and remember, j = i, so it's the iterator. % is a mathematical modulus operator. -- that is, it divides two numbers and the output is the remainder. Example: 17 divided by 4 is 4 with a remainder of 1.
$ echo "17 % 4" | bc
1

(x+3) becomes essentially, (i % +3) and +3 just means "positive 3" in this context. We don't even need the +.

((w[j]*1+e(x+3)+11)) becomes ((c * 1 + ( i % 3 ) + 11)) or simply "c + ( i % 3 ) + 11"


You can see these two examples plus two more here (the password is "infected"):
http://stuff.h-i-r.net/bhlog.zip



2011-07-17

Sysadmin Sunday: parse strings with spaces using shell script

I run into this once in a while: I'm trying to perform some operation on a bunch of files or a big line of text, and a space in the filename or text file janks everything up. Take for example all these recordings from a podcast that got batch-named with spaces in them.


Chimera:Recordings axon$ ls
(110) - .mp3 (12) - .mp3 (18) - .mp3 (39) - .mp3 (79) - .mp3
(111) - .mp3 (15) - .mp3 (3) - .mp3 (70) - .mp3

I really don't want spaces in the names. No problem, just use ls -1 (the number one) to list the files on their own line, and use sed or something for renaming them and changing every space to a null character, right?

Chimera:Recordings axon$ for file in `ls -1`
> do mv "$file" `echo $file | sed s/" "//g`
> done
mv: rename (110) to (110): No such file or directory
mv: rename - to -: No such file or directory
mv: rename .mp3 to .mp3: No such file or directory
mv: rename (111) to (111): No such file or directory
mv: rename - to -: No such file or directory
mv: rename .mp3 to .mp3: No such file or directory
[truncated]

That did not go as planned...

There are a few interesting ways to solve this one. The actual reason for this problem is your shell's internal field separator. When iterating over some input (here, the results of "ls -1"), the shell interprets any kind of whitespace as a field separator, including spaces, tabs and newline characters.

Although there are some other clever ways to get around this limitation when dealing with filenames specifically, my favorite solution to this problem works on any whole line of input regardless its source, whether reading a text file and operating on it one line at a time or taking filenames as input from another command such as ls or find. You simply have to use something that can accept spaces and requires a newline character in order to set a variable. Of course, I'm talking about a rather unsavory (but totally viable) use of the read command, which most unixy shell-script writers are familiar with when they require user input. Check it:

Chimera:Recordings axon$ ls -1 | while read file
> do mv "$file" `echo $file | sed s/" "//g`
> done

Chimera:Recordings axon$ ls -1
(110)-.mp3
(111)-.mp3
(12)-.mp3
(15)-.mp3
(18)-.mp3
(3)-.mp3
(39)-.mp3
(70)-.mp3
(79)-.mp3

You can also remap the $IFS variable to contain a newline, but be sure to unset it afterwards (if using BASH, this will set it back to default), or your shell will act differently than you likely expect when you're done. Messing with the internal field separator can be useful for other things (such as parsing /etc/passwd or handling CSV files) but honestly I'd probably be more inclined to use awk for those. If we remap IFS to a newline, our original script that errored out above works just fine.

Chimera:Recordings axon$ IFS=`echo -en "\n\b"`
Chimera:Recordings axon$ for file in `ls -1`
> do mv "$file" `echo $file | sed s/" "//g`
> done
Chimera:Recordings axon$ ls -1
(110)-.mp3
(111)-.mp3
(12)-.mp3
(15)-.mp3
(18)-.mp3
(3)-.mp3
(39)-.mp3
(70)-.mp3
(79)-.mp3
Chimera:Recordings axon$ unset IFS


2010-10-11

Nessus XML parsing with awk

At the office, I use Nessus for automated network scanning and patch auditing. With credentials and proper tuning of the scan policy, Nessus is a very powerful tool for more than skript kiddie network scanning. This leaves me with a whole bunch of data to wade through on a weekly basis.

Usually, I only concern myself with the high-severity issues for weekly reports, then as I have time, I dig deeper into the more trivial problems. Still, this required me to manually open the scan files, filter them by severity, and export the data. I got tired of that and made a quick and really dirty XML parser (.nessus files are XML) with shell and grep. It was horrendously slow.

Andy, a fellow KC2600-er helped me wrap my brain around some of the finer points of awk to make it more efficient. This is slightly modified from the one I use at work, which is part of a bigger script that does other things. I figure it's useful for others who use Nessus regularly. The script is here.

Basically, it stores the HostName tag when it encounters it, then iterates through the lines, storing them temporarily until it runs into a line indicating a high-severity plugin has been triggered (severity level 3), then it spits out the host name and the plugin that was triggered. I probably could write the whole thing in awk, but I wrapped it in a little bit of plain old shell script.

Output looks something like this:


Windows
----------------------------------------------------
x.x.x.19:MS10-062: Vulnerability in MPEG-4 Codec Could Allow Remote Code Execution (975558)
x.x.x.19:Adobe Reader <= 9.3.4 / 8.2.4 CoolType.dll SING Font 'uniqueName' Field Parsing Overflow (APSA10-02)
x.x.x.20:MS10-066: Vulnerability in Remote Procedure Call Could Allow Remote Code Execution (982802)

Mac
----------------------------------------------------
x.x.x.8:Mac OS X AFP Shared Folders Unauthenticated Access (Security Update 2010-006) (uncredentialed check)

Linux
----------------------------------------------------
x.x.x.40:PHP 5.2 < 5.2.14 Multiple Vulnerabilities

2010-10-07

It only happens once every 823 years!

- OR -
Shell Scripting for Pedantry's Sake.


Today's "That can't be true!" moment hit me when I started seeing this making the rounds (in various different paraphrased versions) on Teh Intarwebs:

"This month has 5 Fridays, 5 Saturdays and 5 sundays-Only happens every 823 years!"

Truth be known, I don't really care about how many weekends are in a month except for the fact that I get three paychecks this month. That happens about twice per year, and that's always welcome! Once in a while, though, I just can't help it. I have to disprove something. I figured the easiest way to disprove this particular claim would be to write a shell script that used the "cal" tool, found in every unix variant known to mankind.

For there to be 5 Fridays, Saturdays and Sundays in a single month, there is a basic requirement for a 31-day month that begins on a Friday, and only then will the 31st fall on a Sunday to complete 5 "whole weekends" in one month.

Initially, I was thinking of ways to see what months started on a Friday. That would get me close. It would give me months such as February 2013, which have only 28 days. Then it hit me: Look for any month with a 31st day that falls on Sunday. Using "cal," I can simply roll through the calendar year looking for a line that begins with "31" and guarantee that the month will satisfy the requirements of having five Fridays, Saturdays and Sundays.

So here we go!

#!/bin/sh
ye=2010
mo=1
while true
do
until [ $mo -gt 12 ]
do
cal=`cal $mo $ye | grep ^31`
if [ -z "$cal" ]
then
echo -n ""
else
echo
cal $mo $ye
fi
mo=`expr $mo + 1`
done
mo=1
ye=`expr $ye + 1`
done


Output:

January 2010
Su Mo Tu We Th Fr Sa
1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31

October 2010
Su Mo Tu We Th Fr Sa
1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31

July 2011
Su Mo Tu We Th Fr Sa
1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31

March 2013
Su Mo Tu We Th Fr Sa
1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31

August 2014
Su Mo Tu We Th Fr Sa
1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31

May 2015
Su Mo Tu We Th Fr Sa
1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31


I don't know. It looks like these things happen more than every 823 years. You can rest easy knowing that it will actually happen a total of 825 times in the next 823 years. Yep, I counted them.

One of the derivatives mentioned October specifically, though. Perhaps this only happens once every 823 Octobers?

Slightly modified, we make the script check Octobers...

#!/bin/sh
ye=2010
mo=10
while true
do
cal=`cal $mo $ye | grep ^31`
if [ -z "$cal" ]
then
echo -n ""
else
echo
cal $mo $ye
fi
ye=`expr $ye + 1`
done

Output:

October 2010
Su Mo Tu We Th Fr Sa
1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31

October 2021
Su Mo Tu We Th Fr Sa
1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31

October 2027
Su Mo Tu We Th Fr Sa
1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31

October 2032
Su Mo Tu We Th Fr Sa
1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31



Nope. More than a decade at times, but not 823 years.

2010-04-27

Free OpenBSD shell accounts at devio.us

The nice guys at devio.us are offering free (or cheap premium) shell accounts on openbsd servers.

Free linux shells are everywhere, but I haven't seen OpenBSD shell accounts offered, at least for free, in recent memory. I manage an OpenBSD shell/tunneling/irc/whatever server for members of CCCKC, but it's not "free" so much as it's one of the many benefits and shared resources that come with being a paid member of our hackerspace.

Their policy is pretty straight-forward. If you pay $2 or $3 per month, you get a premium account that allows you to background a couple of processes, gives you a larger file quota, and the ability to create more databases. Regular accounts may not background processes or run a detached screen after logging off, only get one mysql database, and 100MB of storage space. Oh heck, just grok the services page.

In the FAQ, the devio.us team says they're OpenBSD advocates. That makes them cool in my book. Be sure to check out their manifesto, as well.

2009-01-16

Twitter Followers/Friends from the CLI

I started getting curious on twitter. I had two questions:
  1. Who am I following that's not following me back? (i.e. can Martin Roesch hear me? The answer is no, he can't)
  2. Who is following me that I'm not following back?
Already familiar enough with the Twitter API, I threw together some quick and ugly command-line foo, and @digitaljestin wanted to know how I did it. This is REALLY ugly, and could use a lot of refinement. That said:

I'll probably program a quick stand-alone newlisp or php tool for this over the weekend. Regardless, here's how I did it on the CLI.

First, Twitter will only hand you 100 friends and followers at once. If I were going to automate this, I would poll the followers_count and following_count attributes from http://twitter.com/users/show/username.xml to figure out how many "pages" I needed to fetch.

If you have 203 followers, you will have to do three requests for follower info. Same with friends (those whom you follow). I had over 200 (but less than 300) each. So I did 3 of each request.

I'm only interested in the screen_name attribute within the XML of each. Note that I'm doing a lot of cheap grep | awk crap here, so it just builds lists of screen names without any markup.

$ wget http://user:password@twitter.com/statuses/followers.xml \
| grep "<screen_name>" | awk -F"[\<\>]" '{print $3}' > followers.txt
$ wget http://user:password@twitter.com/statuses/followers.xml\?page=2 \
| grep "<screen_name>" | awk -F"[\<\>]" '{print $3}' >> followers.txt
$ wget http://user:password@twitter.com/statuses/followers.xml\?page=3 \
| grep "<screen_name>" | awk -F"[\<\>]" '{print $3}' >> followers.txt

$ wget http://user:password@twitter.com/statuses/friends.xml \
| grep "<screen_name>" | awk -F"[\<\>]" '{print $3}' > friends.txt
$ wget http://user:password@twitter.com/statuses/friends.xml\?page=2 \
| grep "<screen_name>" | awk -F"[\<\>]" '{print $3}' >> friends.txt
$ wget http://user:password@twitter.com/statuses/friends.xml\?page=3\
| grep "<screen_name>" | awk -F"[\<\>]" '{print $3}' >> friends.txt

Then, I just sorted them:
$ sort friends.txt > friends-sort.txt
$ sort followers.txt > followers-sort.txt

Using diff, it's easy to tell who is not following you, and who you aren't following.
The < shows lines that only appear only in the first file (ones you follow only). The > shows lines that only appear only in the second file (ones following you). Grepping for only lines that start with < and > avoids all the patch-file line offset stuff. Some diffs have varying syntax to do this, but letting grep filter it should work across more platforms.

$ diff friends.txt followers.txt | grep "[<>]" | sort
[excerpt]
< H_i_R
< Hak5
< KCWeather
< Scobleizer
< Veronica
< bacontwits
< beseKUre
< brightkite
< datalossdb
< hackadaydotcom
< ihacked
< ihackstuff
< kingpin_
< milw0rm
< mroesch
< obsessable
< om
< packetlife
< pauldotcom
< schneier
< textfiles
< wilw
< window
------------------ (split added by ax0n)
> BlackHatUSA
> Computersaurus
> HacClearwater
> HackersAlerts
> HackerspacesBot
> SOURCEBoston
> SecuritySatan
> quine
> reverz
> rsreese
> secureideas
> securitypro2009
> stopthemanga

2008-12-19

Asmodian's Workbench: The archive sorter

In the past when I have done backups for home computers, I used a cd-rom burner and dumped everything in a tarball. The problem is that the archive is full of stuff I don't need, want or remember anything about.

So to aid in figuring out whats what I turned to the wonderful unix command FILE.

The file command spits out what file format it thinks a given file is. Its does this through magic numbers.


#!/bin/bash
TEMP_DIR1=`mktemp -d -q /tmp/TMP1.XXXXXX`
TEMP_DIR2=`mktemp -d -q /tmp/TMP2.XXXXXX`
DATE=`date "+%m_%d_%y"`
ARC_FILE=$1
TF=`mktemp -q /tmp/TF.XXXXX`
cp $ARC_FILE $TF.tgz
tar -zx -C $TEMP_DIR1 -f $TF.tgz
NUM1=1
find=`find -X -P $TEMP_DIR1/. -type f`
for FILE in $find; do
TYPE=`file -b $FILE|tr [:space:][:cntrl:] \_`
FILTERED=`basename $FILE|tr [:space:][:cntrl:][:punct:] \.`
FILEFILTERED=`echo $NUM1$FILTERED`
mkdir -p $TEMP_DIR2/$TYPE
cp $FILE $TEMP_DIR2/$TYPE/$FILEFILTERED
echo $FILE >> $TEMP_DIR2/md5_file_list.txt
md5 $FILE >> $TEMP_DIR2/md5_file_list.txt
NUM1=`expr $NUM1 + 1`
done
mv $TEMP_DIR2 ~/recovered$DATE
rm -rf $TEMP_DIR1
rm $TF.tgz
rm $TF



This code has been tested on Mac OS X 10.5 . Linux handles the find and file commands differently. OS X either classifies it as a file (well gee now I know its a normal file... Thanks OS X, you were very helpful there...) or it responds with everything up to and including the picture size. Linux responds with some basics about the file or the whole mime-type, which comes in handy if you want to sub categorize. And I made use of the kick ass TR command (which Ax0n based a previous article about). The find command was useful too but once again there is a syntax difference between OSX and Linux.



Interesting Facts:
Wikipedia: Magic Numbers in files
Apple Man pages

2007-10-05

Shell Scripting: friendly command-line arguments

Ah, the joys of shell scripting! If you've spent any time on UNIX-like operating systems, you've probably encountered or written shell scripts. The theory is simple. For the most part, shell scripts simply execute shell commands in order. You find them everywhere. Simply booting a Linux or BSD host might execute scores of shell scripts. Scheduled processes like those launched with cron or at are usually shell scripts. As a system administrator or a hacker, well-programmed scripts can make your life and the lives of your users a lot easier. On the other hand, scripts that are arcane and cryptic can be more trouble than they're worth.

One of the major hang-ups of complex shell scripts is the strict syntax of the command-line arguments. When calling the command-line arguments from within the script, the first argument is referenced as $1, the next as $2 and so on. $0 is the name of the script itself as it was entered on the command line (including the path, if typed). Also, $# is a numeric variable that contains the number of command line arguments passed. Using the exit command is a way to make sure the script stops where it is without processing any further commands. Using exit 0 creates a "clean" exit, whereas exit 1 (or any other integer) is a way to symbolize an error. This doesn't matter much unless other scripts rely on the ones you're making. It's good practice to specify a proper exit status for your scripts, but it's not mandatory.

Most shell scripts that accept arguments require the end-user to know exactly what arguments to pass or they will simply fail. Take, for example, this script I wrote to get my wireless adapter online in OpenBSD.

-------------------------------------------------------------------------------


#!/bin/sh
sudo ifconfig $1 nwid $2 nwkey $3 up
sudo dhclient $1

-------------------------------------------------------------------------------

It requires me to select the device name of my wireless ethernet adapter, the SSID, and the WEP password. The command line could look like this:

wifi.sh ural0 mywlan 0x31337e1ee7

If I just executed wifi.sh without any arguments, the ifconfig would fail miserably on syntax alone, and dhclient would not know what ethernet adapter to use to get an IP address. The script would not work.

Some more advanced scripts will determine if you entered enough arguments. If you did not, it may give you some brief explanation as to what it wants for arguments. The "apachectl" script for controlling the Apache Web Server is a good example of this. If you run it alone, you are shown a list of arguments that it accepts:

usage: apachectl [ start | startssl | stop | restart | graceful |
status | fullstatus | configtest | help ]
<... output truncated ...>

This style of script is fairly straight-forward. You code it to accept one command line argument, referenced as $1 using the case command as shown in this simple example:

-------------------------------------------------------------------------------

#!/bin/sh
if [ -z "$1" ]
then
echo "usage: $0 [ start | stop ]"
exit 1
fi
case $1 in
start)
echo "You chose start!"
;;
stop)
echo "you chose stop!"
;;
*)
echo "I'm sorry, you didn't choose stop or start."
;;
esac

-------------------------------------------------------------------------------

You can fill in the echo commands with whatever you find useful. This is fairly mundane, and doesn't allow you to pass parameters or multiple flags to your script. For those unfamiliar with the "case" command, it's quite simple to use. If the contents of the variable referenced in the "case" line match the expression before the parenthesis, it executes the code on the following lines, and stops processing when it encounters two semicolons.

Let's face it, with just a single "case" structure and some error checking, you won't be writing any truly powerful shell scripts.

Enter "shift". Within a shell script, shift destroys $1 and shifts all the other arguments down by one, and decrements the value in $# by one as well in order to reflect the new (lower) number of command-line arguments left. The contents of $2 become $1, $3 becomes $2, etc. While you might not think that sounds too exciting, it will allow you to pull off some argument-processing trickery with a simple loop to read arguments. Check out this example:

-------------------------------------------------------------------------------

#!/bin/sh
until [ $# == 0 ]
do
case $1 in
foo)
echo "you have selected foo"
shift
;;
baz)
echo "you have selected baz"
shift
;;
bar)
echo "you have selected bar"
shift
;;
zot)
echo "you have selected zot"
shift
;;
*)
#ignore arguments we don't recognize, just shift them and move on
shift
;;
esac
done

-------------------------------------------------------------------------------

The until/do loop above simply keeps running the arguments through the case block until there are 0 arguments left, then exits. If you pass it an argument that is not in the case block, it simply does a shift and ignores it. It runs the arguments in the order we choose.

bash-3.1$ ./foo.sh foo bar
you have selected foo
you have selected bar

In the case of my made-up wireless configuration script, this isn't directly all that helpful. Another thing you can do, however, is run another shift within a case. This allows you to give your script many very flexible command flags, much like other UNIX commands. Using my wifi script as an example, I'll show you how it's done. For arguments that don't require a second parameter (such as -h and -d) just use one shift statement within the case. For arguments that do require a parameter (such as -d, -s, -k, or -p, use two shifts: one before you assign $1 to a variable, and again at the end of the case. Notice that the * catch-all case is simply there to shift un-recognized arguments. If we don't do this, our loop will hang forever because there will always be an argument that hasn't been processed.

This script puts it all together. Its arguments are just as flexible as most compiled programs. After the script has processed all of the arguments, I use a series of if statements to build the command line for ifconfig by appending to the $ifconfig_args variable, then run dhclient if desired.

There's a little extra scripting (more if statements) to make sure that a value follows the arguments that require another parameter. In the end, this is a pretty lengthy script, but it's almost bullet-proof and a lot friendlier than most shell scripts. People might not even know it's a script!

-------------------------------------------------------------------------------

#!/bin/sh
if [ -z $1 ]
then
echo "Try using '$0 -h' for help."
exit 1
fi
until [ $# == 0 ]
do
case $1 in
-h)
help=1
shift
;;
-d)
shift
if [ -z "$1" ]
then
echo "You must specify a device with -d"
exit 2
fi
device=$1
shift
;;
-s)
shift
if [ -z "$1" ]
then
echo "You must specify an SSID with -s"
exit 2
fi
ssid=$1
shift
;;
-p)
shift
if [ -z "$1" ]
then
echo "You must specify a password with -p"
exit 2
fi
password=$1
shift
;;
-k)
shift
if [ -z "$1" ]
then
echo "You must specify a key with -k"
exit 2
fi
wepkey=$1
shift
;;
-c)
dhcp=1
shift
;;
*)
shift
;;
esac
done

if [ "$help" ]
then
echo "Usage: $0 -d -s [-h] [-c] [-p | -k ]"
echo "-d Wireless Ethernet device (wi0, ural0, etc.)"
echo "-s The SSID of the network you wish to join"
echo "-h This help page"
echo "-c Start DHCP client"
echo "-p 5 or 13 character WEP password"
echo "-k 10 or 26 character hexadecimal WEP key"
exit 0
fi
if [ -z "$ssid" ]
then
echo "You must specify an SSID"
exit 1
fi

if [ -z "$device" ]
then
echo "You must specify a device"
exit 1
fi

ifconfig_args="$device nwid $ssid"
if [ "$password" ]
then
ifconfig_args="$ifconfig_args nwkey $password"
fi

if [ "$wepkey" ]
then
ifconfig_args="$ifconfig_args nwkey 0x$wepkey"
fi

ifconfig $ifconfig_args

if [ "$dhcp" ]
then
dhclient $device
fi

exit 0

-------------------------------------------------------------------------------

The ifconfig syntax I used in my examples is fairly platform specific to the BSD family, but you can change it to work on Linux, Solaris, or any other UNIX-like OS.

The UNIX userland contains hundreds of little utilities that can be strung together with scripts and pipes to create very powerful programs without having to spend a lot of time learning a new programming language. Hopefully you don't just learn how to make an ifconfig script out of this, but take what I've written as an example of how to improve your own scripts or inspire you to start creating your own scripts.