Zune doesn't like leap years

GenesisWave pointed this out on the Cowtown Computer Congress mailing list.

Q: Why did this occur at precisely 12:01 a.m. on December 31, 2008?

There is a bug in the internal clock driver causing the 30GB device to improperly handle the last day of a leap year.

What a riot. Well, your Zune will work tomorrow. In the meantime, check out i-Hacked.com's Zune Fix. They had the first working walk-through I saw.

Happy GNU Year! (ASCII art edition)


¤ø„¸¨°º¤ø„¸ ¸„ø¤º°¨¸„ø¤º°¨
¨°º¤ø„¸ HAPPY ¸„ø¤º°¨
¸„ø¤º°¨ GNU YEAR``°º¤ø„¸
¸„ø¤º ¸„ø¤º°¨ ¤ø„¸¨°º¤ø„

Props to Vikas for the ASCII art


Open Letter from Geeks to IT Recruiters and Hiring Managers

Preface: No, I'm not looking for a job, although I do get the occasional ping from headhunters. I've seen it before, and my friends (some currently unemployed) are seeing it still. My own boss is actually doing pretty good with the below tips.

For the love of all things good in the world, learn how to hire and employ a geek. You're doing it wrong.

Office Politics
Try to measure productivity in output, not in hours.

Geeks automate. Geeks script. Geeks compile. They summon computing power to get things done quickly on their behalf. If your geek seemingly spends all day on Twitter and Fark but somehow manages to still complete tasks ahead of schedule, your geek is multi-tasking. This is normal.

Assign tasks to the geeks who are most interested in them, not the ones with the most experience.

When geeks are interested, they are passionate. When they're passionate, they learn fast. You'll get more productivity out of an interested geek with no prior experience than you will with a bored drone who's been doing the same thing for the past five years. Sometimes, the one with the most experience is the one that's most interested. In those cases, you are a lucky manager!

Segregate the corporate, compensatory hierarchy from the leadership hierarchy.
With a team of geeks under you, one or more will eventually become to go-to guy (or girl) for certain things. You don't usually need to assign a "team lead" - Through meritocracy, the Alpha Geek will emerge. That Alpha Geek may lack seniority, but will have the most influence. It's best to let this occur naturally. It's awkward when the one who best fits the role has to answer to someone else just because they've been around longer. Furthermore, the members of your team will still go to the Alpha Geek because the wrong person has the "Team Lead" label. As Paul Glen puts it: Geeks don't hate hierarchy. They hate your hierarchy.

You'll know you've found the Alpha Geek when you see people from your team (and likely other teams) at said geek's desk getting advice or validation on a frequent basis.

Pre-hiring and interview

Have all screening and profile "paperwork" in one comprehensive online wizard or form.
Geeks do not like pens, pencils, or clip boards. We also despise giving you the same piece of information more than once on fifteen different sheets of paper. We'd rather not be sitting on an uncomfortable chair in a room that's far too brightly lit just so that we can give you the information that you want. It's easy to get the information to you electronically.

Only ask for information you need to make a hiring decision.
W2's, Direct deposit information, full fingerprints, home address and all that crap can be handled during orientation. The only personally identifiable information you need before hiring is a name.

Don't grill us on our resume and work history.
You don't hire a geek for what he or she did two years ago. You hire them for what they will be able to do for you now and in the future. Ask your geek to describe scenarios where problems arose that required them to pick up a new skill set to solve. All geeks worth their salt have stories like that and love telling them.

Instead of asking about skills that qualify them for the position, ask about their interest in the kind of work they think they'll be doing.
Remember: Interested geeks work harder. The above requirement will still let you H.R. types ask that oh-so-predictable question: "What is it that you think this company does?" while offering your candidate a chance to really show he or she will be a good match.

Recommended Reading

I saw Paul Glen speak at IT Security World 2008, and his book, Leading Geeks has a lot more sage advice for those who find themselves leading a technical team.


Bruce Schneier gave me a puzzle to solve...

I won this from a contest BT Counterpane hosted.

The "code" is simple:


I will, Bruce. I will.

Full HiR Reading Room review coming after I finish reading it.


Cracking Master Thumb-Wheel Padlocks

While on a bike ride today, I found this on the roadside:

It's a Master 175, as stamped into the bottom plate shown below. There are many locks in this series, including shrouded hasp models (177 series) and black coated ones (178 series). Internally, they're all identical. The only thing different is the color and the length of the hasp. Most people know that Master's cheap dial-combination locks are vulnerable to a variety of attacks. These are sold as "construction grade" locks. I figured I could put it to use if I could get the combination. I didn't feel like breaking it apart.

Tension probing
Many thumb-wheel combination locks like this (including the ones you see built-into attache briefcases) have a weakness that allows you to determine the combination by feel. You pull on the hasp or otherwise try to open the lock while spinning the wheels until you find one that is hard to turn. Turn it until it feels like it snaps into place. Move on to the next wheel that is binding, and work your way through them until the hasp opens.

The Master 17x locks don't suffer this vulnerability. Being "construction grade", I suppose that's a good thing. To open the lock, you must enter the right combination, then push the hasp in. If the combination is correct, it will pop all the way out and open. If it's not correct, it'll just retract to its locked position. This makes brute force attacks exponentially more difficult.

Another vulnerability with some thumb-wheel locks is a nifty bypass method. If you can trigger the hasp release without entering the combination at all, it'll open for you. This is usually done by wiggling some metal around the thumb wheels to probe the inner mechanics of the lock, then attempt to release the hasp that way. While that's good and well, I want to know the combination so that I can use it. Open or closed, this doesn't do me much good.

Cam probing
This is what I'm interested in. Each wheel has a cam attached to it. These cams have dimples, flat spots or notches which allow the gate to drop into place when the correct combination is entered. As the wheels' clearance between the metal bottom plate is pretty tight, I've opted to use a thin feeler gauge, made for rebuilding automobile engines. These tools are cheap, cheap. I think this one cost me $4 at AutoZone.

Each of these blades varies in thickness. The number on top is in thousandths of an inch, and the number below is in thousandths of millimeters. I found that the .0015" was simply too flimsy (it's about as thick as a sheet of aluminum foil, but made of steel and much stronger), so I went with the .002".

I had to do some tinkering. For starters, I didn't know which side of the wheel to probe, so I started with the left side. There's a ridge around the thumbwheel that keeps the feeler from going in more than 1/10" or so. I wiggled it around and eventually got it in between the metal plate and the thumbwheel, and inserted as far as it would go. Then, I started carefully turning the wheel while gently pressing the feeler inward. This promptly got me nowhere. I couldn't feel anything happening at all.

I moved to the right side of the thumbwheel, and I had it! I still had to weasel in around the ridge of the thumbwheel, though.

Note that the feeler is not sticking in very far. You can see the rounded edges of the feeler.

While turning the wheel, the feeler dropped noticably into place (in the middle of 2-3 on the 3rd wheel as shown), and I could feel it getting pushed back out if I tried to spin the wheel further.

Note the peculiar location of the flat spot on the cam. It's between two numbers on this model. That will help us in a moment.

Find all of the flat spots and jot them down, in my case, it was:
5/6, 7/8, 2/3 and 5/6

In this position, it shouldn't surprise you that my lock didn't open.

The gate is usually 90 degrees or 180 degrees from the wheel alignment mark. That means it's either directly opposite the alignment mark, directly above the wheels, or directly below them. Given that the flat spots were between two numbers, this rules out the gate being opposite the alignment mark. Turning the wheels 180 degrees would land them at: 0/1, 1/2, 7/8, and 0/1, which isn't really a combination Master had in mind.

I chose to rotate all of them 90 degrees upward, toward the lower numbers. Starting with the dials at the position you wrote down:

Nudge all four dials upward to the number that sat below the alignment mark

Then move all four dials upward two more spots (decreasing numbers unless you start at 0)

Press the hasp in, and it opens!

Similar methods work on a large number of locks. You just have to tinker. Now you know how almost all thumb-wheel padlocks work!

In Review
The instructions for cracking a lost/forgotten/unknown combination on a Master 17x series thumb-wheel padlock:

  • Hold the lock with the bottom plate facing you and the numbers right-side up.
  • Use a feeler gauge (I used .002") between the right side of the thumbwheel and plate.
  • Carefully turn the thumbwheel while applying gentle pressure on the feeler gauge.
  • Write down where the feeler sinks into the lock deeper. It will always be between two numbers on 17x locks.
  • Turn all four wheels to the locations you wrote down.
  • Turn all four wheels upward to the number on the bottom of the split.
  • Turn all four wheels upward two more whole numbers.
  • Press hasp in to open
Once you've done this a few times, it can be repeated in just a few minutes. The first time through took me about 15 minutes or so, because I didn't know for sure which side of the wheel to probe nor which direction to turn the wheels to activate the gate. I repeated the process for the photo shoot. Before I wrote this article, I tried it again after changing the combination on the lock, and it took me only 3 minutes.


You're Doing It Wrong: Whiteboard Security

Smart whiteboards can take what's drawn on them and print them, store them, e-mail them and a whole variety of other fun things. Panasonic is bringing password protection to these features.  [Via Engadget]

Darren Murph (whose snarky writing style I enjoy) says:
...The film and steel boards look pretty traditional at first glance, but underneath of that plain jane facade is a highly advanced security system. You see, each board can accept passwords, which will in turn restrict the ability to transfer information from the board to USB flash memory. For those cleared for access, the whiteboards can transfer on-screen information to a PC via USB, though we suspect you'll have to handle the encryption on your end. 'Course, neither of these will run you cheap, but you know your underground supervisor won't mind shelling out upwards of two large to make sure schematics to rule the world aren't intercepted by meddling rivals.
of course, you can practically hear read the sarcasm.  This is like putting a bank vault door on a 4-foot-tall chain link fence. There's little to keep someone from snapping photos of the whiteboard with a camera phone, or from sketching along with the presentation while stuff is being drawn.

Security. You're doing it wrong.



[Photo via Hack A Day]

I've been watching RepRap for a while.  I was reminded of it by a recent post on Hack A Day which focused on the new revision of the RepRap Motherboard. The project is self-described:
A universal constructor is a machine that can replicate itself and - in addition - make other industrial products. Such a machine would have a number of interesting characteristics, such as being subject to Darwinian evolution, increasing in number exponentially, and being extremely low-cost.

A rapid prototyper is a machine that can manufacture objects directly (usually, though not necessarily, in plastic) under the control of a computer.

The RepRap project is working towards creating a universal constructor by using rapid prototyping, and then giving the results away free under the GNU General Public Licence to allow other investigators to work on the same idea. We are trying to prove the hypothesis: Rapid prototyping and direct writing technologies are sufficiently versatile to allow them to be used to make a von Neumann Universal Constructor.

All good projects have a slogan, and the best have a slogan that reeks of hubris. RepRap is no exception. Our slogan is:

"Wealth without money..."
Impressive as the project and it's goals are, I personally find the miniature-advances in technology being made by the RepRap guys more inspiring than the RepRap itself. The Hack A Day post focused on the motherboard, which is based on the Sanguino platform. This new platform came out of RepRap's need for more functionality than the Arduino could deliver, and is a perfect example of hackers encountering a problem that would normally shut down or severely hinder a project. Then, not only did they find an intuitive way around it that was very useful for many other projects beyond the one that necessitated it, but they gave the solution back to the community as well.

That, my friends, is the spirit of the hacker.


Asmodian's Workbench: Suhosin Hardened PHP extension and patch.

Suhosin is a plug-in and patch for PHP. It places a white-list filter of actions which are allowed. It prevents a pile of PHP exploits from happening such as buffer overflows and certain kinds of injection attacks. You can find it at the Hardened PHP project web-page. It has a number of default items it blocks, one of which is the number of variables it allows to be posted and received.

You can configure it to either block potential attacks a and to log the results in unix syslog. you can also configure it to allow issues to occur and to only log events too.

You can control the Suhosin default values in your php.ini file.

Some php applications use an enormous amount of post variables so the default value (200) is probably too low. As I have explained to my co-worker, getting rid of the plug-in because your script uses too many post variables is probably not the best solution.

The solution in the that event is to modify the maximum request and post vars.

You can also tell suhosin to in the event of encountering a possible attack to run a different script or a http redirect instead. Like perhaps something like this:
(php.ini entry)


As you can see this has a number of interesting possibilities.

If you are interested in PHP and AMP (oamp,lamp ...etc) technologies See also:

The hardened PHP project:

Ax0n's OAMP (Apache, Mysql, PHP on OpenBSD) Article:

Asmodian X's Name based hosting mini-howto:

The PHP main website:

The Apache webserver website:


Sysadmin Sunday: AMP on OpenBSD 4.4

FYI - There's now a page that covers OAMP for all recent versions of OpenBSD.

At the beginning of 2008, I posted how to get Apache, PHP, and MySQL running on OpenBSD 4.2. This is commonly known as LAMP when running under Linux. I call this OAMP.

I decided to AMP my OpenBSD 4.4 rig, and followed those instructions. No surprise, things have changed a little bit.

Before installing packages
First and foremost, make sure you can easily install packages. In this case, I've added the FTP repository to the PKG_PATH environment variable, then I've made sure my user-level account has access to sudo.

OpenBSD's package system has finally become a little less flaky about remote packages and dependencies, so I finally added this to my user-level account's .profile:
export PKG_PATH
My user-level account is also in the wheel group, and the following line was uncommented from /etc/sudoers:
That means anyone in wheel can run anything with sudo as any other user (root by default)

Installing packages
Obviously, you've got OpenBSD 4.4 installed and patched. If not, you'd better get crack-a-lacking, homeslice! OpenBSD comes with Apache, so you're basically halfway there by installing OpenBSD.

The following command will install php5-mysql and mysql-server (obviously)
That will grab all the dependencies in one swoop, including the mysql-client package and php5-core. I'll snip the output to only display the relevant stuff, and address that afterwards
-bash-3.2$ sudo pkg_add php5-mysql mysql-server
--- php5-core-5.2.6 -------------------
To enable the php5 module please create a symbolic
link from /var/www/conf/modules.sample/php5.conf
to /var/www/conf/modules/php5.conf.

ln -s /var/www/conf/modules.sample/php5.conf \

The recommended php configuration has been installed
to /var/www/conf/php.ini.

Don't forget that the default OpenBSD httpd is chrooted
into /var/www by default, so you may need to create support
directories such as /var/www/tmp for PHP to work correctly.
--- php5-mysql-5.2.6 -------------------
You can enable this module by creating a symbolic
link from /var/www/conf/php5.sample/mysql.ini to

ln -fs /var/www/conf/php5.sample/mysql.ini \
Configuring Packages
As shown above, there's some work to do in order to make PHP work with Apache and make the PHP module load the MySQL client library.

I'm not a huge fan of symbolic links to sample files. I opted to copy them.
sudo cp /var/www/conf/modules.sample/php5.conf \

sudo cp /var/www/conf/php5.sample/mysql.ini \

Install the default database and start MySQL, then assign a password for the MySQL root user with the following commands (the output doesn't really matter)
sudo /usr/local/bin/mysql_install_db

sudo /usr/local/share/mysql/mysql.server start

sudo /usr/local/bin/mysqladmin \
-u root password 'your-password'

Now, we want to make sure that both Apache and MySQL start automatically.

Find the "httpd_flags" line in /etc/rc.conf and set it to literally two quotes: "" or, if you want to disable chroot (not recommended, but can make OAMP life much easier for you when you go to install webapps) set it to "-u":
sudo vi /etc/rc.conf
(find httpd_flags once editing the file)

# use -u to disable chroot, see httpd(8)

You can launch apache easily from the command line now by executing httpd, so you don't need to reboot. Use -u if you specified it in /etc/rc.conf.
sudo httpd
This is kind of a cheap and non-official way to do make MySQL start, but it works just fine and it's how I always do it. You're supposed to make an rc function and all that so you can enable/disable it in /etc/rc.conf. Bah. Add the following line to the end of /etc/rc.local.
sudo vi /etc/rc.local
(add the following line to the end)
/usr/local/share/mysql/mysql.server start
At this point, you have OpenBSD set up to start Apache and MySQL at boot, and both should be running. Additionally, the PHP module should be loaded. To test MySQL, use mysql_setpermission, a perl script that lets you manage and create databases and users.

mysql_setpermission -u root
Password for user root to connect to MySQL: your-password
## Welcome to the permission setter 1.4 for MySQL.
## made by Luuk de Boer
What would you like to do:
1. Set password for an existing user.
2. Create a database + user privilege for that database
and host combination (user can only do SELECT)
3. Create/append user privilege for an existing database
and host combination (user can only do SELECT)
4. Create/append broader user privileges for an existing
database and host combination
5. Create/append quite extended user privileges for an
existing database and host combination (user can do
6. Create/append full privileges for an existing database
and host combination (user has FULL privilege)
7. Remove all privileges for for an existing database and
host combination.
(user will have all permission fields set to N)
0. exit this program

Make your choice [1,2,3,4,5,6,7,0]:
If it shows the menu, then MySQL is running and the password you configured is working.

Test PHP and Apache by creating a phpinfo.php file in /var/www/htdocs/phpinfo.php:

Hit it with your web browser to make sure it loads. It should show you information about your PHP installation, including the MySQL module.

If you experience problems installing *AMP-based web apps, try:
  • Making a hard link to /var/run/mysql/mysql.sock within /var/www somewhere, and editing the php.ini file accordingly
  • Disabling apache chroot by setting httpd_flags="-u"
  • Checking the php, mysql, and apache error logs for more information about what went wrong


Firefox plugins for security and geeky fun

I don't run too many Firefox plugins, but I really love the ones I do use.  Here's a run-down. The title of each section will link directly to the plugin on the mozilla site.


Even if you don't use Firefox plugins at all, I recommend giving NoScript a try. From the NoScript website:

When you install NoScript, JavaScript, Java, Flash Silverlight and possibly other executable contents are blocked by default. You will be able to allow JavaScript/Java/... execution (scripts from now on) selectively, on the sites you trust. You can allow a site to run scripts temporarily, if you're just surfing randomly, or permanently, when you visit it often and you really trust it. This means that NoScript learns from your own browser habits and tends to disappear in the background after a while, but it promptly comes back to save your day if you stumble upon a malicious web page.
NoScript is updated frequently as malware blocking methods are improved. It was one of the first products to offer protection against clickjacking

FoxyProxy  allows you to set up multiple proxy configurations. This comes in handy when SSH Tunneling to your own proxy or just using public proxies for web filter evasion or privacy reasons.  FoxyProxy is a little unwieldy at first glance, but it's quite flexible; more so than other proxy management plugins. If that's a little over the top for you, a more minimalist plugin is SwitchProxy Tool
Security Reality Check by ax0n: Switching between multiple public proxies every 30 seconds might seem like a good idea for making yourself harder to track, but it also dramatically increases the number of places your traffic goes. You leave more footprints in more places, which could actually make it easier to track something back to you, even if it's harder to figure out everything you did.

Leet Key lets you transform text with a number of popular encoding algorithms, for example, when @lithium posts stuff like this.  Grr. 

Select the text, right-click, then hit the text transformer tool within Leet Key. In this case, it was not only Base64, but rot13 as well. The bad news is that you have to be able to guess what it's encoded with in order to use Leet Key. After having played with many different encoders, I can usually tell what it is that I'm looking at. 

Leet Key also lets you easily encode editable forms, so you can type something into a web mail client or forum posting form, then encode it on the fly before sending it.

User Agent Switcher is for testing how certain sites react to different user-agent strings, but I originally installed it so that I could trick Starbucks' WiFi into thinking I was using an iPhone (and thus, get free WiFi). I've found it useful for other things, though: Particularly when testing heavy JavaScript pages.


Ubiquity is a command line interface to Mozilla Firefox. This allows you to create small, re-usable custom functions and subscribe to third party functions. If, like me, you find yourself willing and ready to script-automate the repetitive things in life, You'll probably love Ubiquity. 

I'm paranoid by nature, and NoScript is the only plugin I leave enabled all the time. The rest of these I will only enable when I will need them. I'm leery about using a lot of Greasemonkey scripts, and don't really like loading my browser with dozens of add-ons. Do you have some must-have favorites that I'm really missing out on? 


Asmodian's Workbench: The archive sorter

In the past when I have done backups for home computers, I used a cd-rom burner and dumped everything in a tarball. The problem is that the archive is full of stuff I don't need, want or remember anything about.

So to aid in figuring out whats what I turned to the wonderful unix command FILE.

The file command spits out what file format it thinks a given file is. Its does this through magic numbers.

TEMP_DIR1=`mktemp -d -q /tmp/TMP1.XXXXXX`
TEMP_DIR2=`mktemp -d -q /tmp/TMP2.XXXXXX`
DATE=`date "+%m_%d_%y"`
TF=`mktemp -q /tmp/TF.XXXXX`
cp $ARC_FILE $TF.tgz
tar -zx -C $TEMP_DIR1 -f $TF.tgz
find=`find -X -P $TEMP_DIR1/. -type f`
for FILE in $find; do
TYPE=`file -b $FILE|tr [:space:][:cntrl:] \_`
FILTERED=`basename $FILE|tr [:space:][:cntrl:][:punct:] \.`
mkdir -p $TEMP_DIR2/$TYPE
echo $FILE >> $TEMP_DIR2/md5_file_list.txt
md5 $FILE >> $TEMP_DIR2/md5_file_list.txt
NUM1=`expr $NUM1 + 1`
mv $TEMP_DIR2 ~/recovered$DATE
rm -rf $TEMP_DIR1
rm $TF.tgz
rm $TF

This code has been tested on Mac OS X 10.5 . Linux handles the find and file commands differently. OS X either classifies it as a file (well gee now I know its a normal file... Thanks OS X, you were very helpful there...) or it responds with everything up to and including the picture size. Linux responds with some basics about the file or the whole mime-type, which comes in handy if you want to sub categorize. And I made use of the kick ass TR command (which Ax0n based a previous article about). The find command was useful too but once again there is a syntax difference between OSX and Linux.

Interesting Facts:
Wikipedia: Magic Numbers in files
Apple Man pages


ReactOS - Not quite there yet?

After doing my OS X updates, I installed ReactOS in Parallels. Install is reminiscent of Win2K, starts off with a blue text-only menu system, then goes through a GUI configuration after reboot. The splash screen is nifty, if nothing else.

The first thing I noticed was the lack of a web browser. I FTP'd down the Mozilla Firefox 3.0.4 Win32 installer, which ran fine. Trying to launch Firefox, however... BONK!

Maybe I need to try ReactOS on a real computer. It wouldn't be the first time something hasn't run quite right for me in a VM, but works perfectly well on a clunky old Pentium 3. More to come, I'm sure.

Most popular articles of 2008

2008 has been a good year for HiR Information Report. This marks eleven years of HiR, which started as a text-file e-zine in 1997. The core crew is still around and writing, although we spent quite a few years dormant.

2008's our first full year with the new blog format. It's time to highlight the most popular articles of 2008. This goes by hits acquired in 2008, so some of the posts may be older than 2008.

10) Shimming a cable lock
This article went somewhat viral in the lockpicking and bicycling community. It sees inconsistent waves of high traffic from forums and blog links, then goes weeks without a hit.

9) jLime Linux - Wifi Scanning
Jornada 6xx and 7xx-series handheld PCs seem to be getting cheaper and more popular. jlime is a viable alternative to the built-in WinCE OS.

8) Sysadmin Sunday: Pure-FTPd configuration
Pure-FTPd is a security-enhanced FTP server. It takes a little bit of elbow grease to get it working properly under Ubuntu, but Asmodian X outlined it clearly. Most people find the article while looking for configuration specifics such as quotas and virtual users.

7) Series: Web Filter Evasion
After the last post in the series was (finally!) completed, the series started getting links. The series was featured on The Edge of i-Hacked and recieved mention in the PaulDotCom Security Weekly Podcast #132

6) Sysadmin Sunday: OpenBSD/Apache/MySQL/PHP (OAMP)
The security and stability of OpenBSD meets the flexibility of an AMP server environment. It's a match made in heaven.

5) Sysadmin Sunday: Process Accounting
Are you seeing a trend? As a general rule, a lot of Sysadmin Sunday and UNIX-Specific content gets linked to. Process accounting is a relatively simple procedure, but it's a good one to know.

4) Testing an ATX Power Supply
This is one of those posts where I feel the HiR Community did a better job in the comments than I did in the article. There's a lot of useful info in there. They picked up where I left off and had some great suggestions for better testing methods.

3) Epoch posts - perl Epoch time and Epoch Fail
I have these two grouped together because they're tied on hits.
Epoch time is just a pain in the butt. It makes sense on a computer, but it doesn't help much in your log files. A quick perl one-liner was just what the world needed, and a lot of sysadmins search for it and find my epoch time post.

Epoch Fail took off for purely viral reasons, because very few people understood the xkcd cartoon. "What is an epoch fail?" and similar terms were the heavy-hitting keywords that linked to this post. Thanks for the traffic, Randall!

2) Unofficial Tethering Guide: LG Chocolate
Verizon is notoriously pesky when it comes to tinkering with phones they sell. Bluetooth OBEX, tethering, and things of that nature are made intentionally difficult. The relative affordability of the LG Chocolate vx8550 made it a popular one among geeks, and this article has remained very popular throughout 2008, with quite a few hits on a seemingly daily basis.

1) Series: Make your own lock picks
This was the most popular content on HiR this year. After my TSA/Lockpick post on i-Hacked got minor mainstream exposure, links to this series took off. Also, the final article on pick templates gets frequent hits from image search engines. I guess there are a lot of locksport enthusiasts out there, looking for templates!


I think I see the problem... [Soldering]

My NiMH Battery recharger has been acting up lately. One of the slots often (but not always) fails to charge. I finally took it apart today to check it out. The blue thermistor had somehow broken free from the negative terminal.

Thermistors are resistors that vary in value according to temperature. NiMH fast chargers almost always use some form of temperature sensor to determine the end of the charge cycle. That's what these thermistors are for.

I thought I'd introduce one of my favorite little gadgets. It's a Bernzomatic MicroFlame ST200.

A bona-fide soldering station will probably give you 150 watts or more of soldering power, fully adjustable. On full power, this little butane torch is probably in the 85-100W equivalent range, but is adjustable. It's considerably hotter than my 45W soldering iron, but not nearly as hot as my 150W soldering gun. On its lowest setting, I'd say it's about equivalent to 25 watts.

For little jobs and when I'm outside and away from electricity, this works quite well. It also heats up a lot faster than my 45W electric iron, which is why I chose it for this repair.

I also like the fact that it not only can be used as a torch for heating plumbing joints, loosening stubborn bolts, etc...

But it also features a hot air nozzle if you remove the solder tip. That's great for shrink tubing, de-icing locks, warming solder paste and removing surface-mount components.

Exploiting in-game logic flaws

Another one of my favorite games of days gone by is Maelstrom. I first discovered this my sophomore year of high school. Someone had installed it on quite a few Macs in the computer lab. It's a 3D-looking Asteroids rip-off. When I finally built my first Linux system for home use in the mid-90s, this was one of the first games I found that worked well. It had been open-sourced.

It's got all the features you'd expect of an Asteroids clone: Shields, UFOs that shoot at you from time to time, and... asteroids. There are also comets (shoot for more bonus points), surprise packages (catch them with your ship to get different weapons or other surprises, some good, some bad), metal asteroids you can't destroy, and the occasional black hole (or Maelstrom, if you will)

One thing I noticed early on, is that if you ever blow your ship up on the last asteroid, you don't actually lose a life, but any special weapons you have will go away.

But, if you look, your shields get replenished back to new when you start the next wave (level)

This happened because there are some logic problems at play. Primarily, the logic to reduce your lives by 1 (or to end the game if you're at 0 like I was) does not run as part of the block of code that initializes the ship's options to default when it's struck, and is somehow missed because of the end of a level. Was it intentional? I have no idea without checking the source code. It's always possible stuff like this is actually an easter egg.

This is the kind of bug that slips through the cracks and is only noticeable during code audits or caught with fuzzing or other black-box testing methods. Often, by invoking conditions that happen rarely, you can get unexpected results. If you can leverage those conditions to happen easily, you can reproduce the results as frequently as you wish. In Maelstrom, it's as simple as sacrificing your ship to knock out that last asteroid, knowing you'll get rewarded with shields for the next round. The same applies to operating systems, the firmware running on your mobile phone and cable box, and pretty much anything else that's been programmed by a human.

Have you found any little game bugs worth exploiting?


Operating System Junkie

I've talked about the computers in the Lab-O-Ratory before. Today, the laboratory is really, really noisy -- more so than usual. I'm always working on something and I'm coming to realize that I'm pretty much a hopeless operating system junkie.

The photo is from a few months ago. The SparcStation 20 on the top of that photo used to be my OpenBSD 4.1 DMZ box until the hard drive died. I shelved it for a while and replaced it with a fresh install of OpenBSD 4.4 on one of the noisy 1U rackmount cluster servers from the startup I worked at in early '06. That 1U now sits in the SS20's old spot on top of the Sun Ultra 5 (which happens to be one of my primary workstations, running Solaris 10)

A few weeks ago, a friend of mine gave me a pile of old SCA-80 hard drives -- 4GB, 9GB stuff that was going to get landfilled. I put some of them in the SS20 again, and decided to try NetBSD. I used this for the SSH Reverse Tunneling article. It's now on the top of the lower stack, below the FON and Jornada and on top of the beige low-profile server.

That's an IBM RS/6000 Model 250 -- 80 MHz (yes, really) with 256MB of RAM and a 36GB SCSI hard drive. Initially made to run IBM's AIX 3.x operating system, it still runs AIX 5.1 quite well as long as you keep the GUI from trying to start. After AIX 5.1, IBM dropped all support for Micro-Channel Architecture. Still, having AIX 5 to tinker with is beneficial to me. I have to deal with AIX (as well as Linux and Solaris) at my real job. Don't forget all the cool freeware that's been ported to AIX and Solaris!

The Dell PowerEdge server below it was going to be an AMP (Apache/MySQL/PHP) server for a client of mine who never ended up needing it. It gets re-purposed whenever I need a reasonably powerful machine or one with a lot of storage to test something on. I just installed FreeBSD 7 on it last night. This one's even more noisy than the 1U cluster server.

My wife bought a 4U rackmount case from Frogman back in the day, and it was originally built up as a telnet MUD server. Now, it's kind of the communal Windows 2000 test box. It's the HiR OpenArena server for the time being.

Then, there's the Dell Optiplex mini-tower out of frame that's still running BackTrack -- although after Mubix wrote about it, I plan on downloading Sumo very soon!

With so many of my own little projects going on and utilizing all these boxes, I'm currently running all of this in my apartment's second-bedroom-turned-laboratory:
  • OpenBSD 4.4 / x86
  • Solaris 10 / ultrasparc
  • jLime Linux (Mongo) / arm
  • Openwrt Linux 7.09 / mips
  • NetBSD 4.0.1 / sparc
  • AIX 5.1 / ppc
  • FreeBSD 7.0 / x86
  • Backtrack Linux 3 / x86
  • Windows 2000 SP4 / x86
That's not even considering that my wife's laptop is running Vista, and I'm running Mac OS Leopard on my own. I'm no stranger to lesser-known OS's either. I've written about OpenSolaris before. I used to run BeOS as my main platform at work. I've used Plan 9 before (in the late 90s) and I even helped a little bit with the now-defunct V2OS project in 2001.

What's next? I think I need to mix it up with ReactOS (Windows clone) or Haiku (BeOS clone) or perhaps try Plan 9 again...


More OpenArena Fun

First and foremost, this is just a concept, but L3DGEworld is an OpenArena-based visualization platform for network hosts. Currently, the main info-gathering daemon (Greymatter) only supports FreeBSD. I tried compiling it on OpenBSD and the code doesn't quite get along with OBSD's C Compiler. Fakematter, the demonstration daemon, however, does work. I just want to know if I can frag other sysadmins while I'm in L3DGEworld!

Also, The HiR Information Report OpenArena server is up and running using my wife's old rackmount server as the donor platform. It will probably get listed eventually on the Internet server list in-game. Otherwise, you can click "specify" and connect to openarena.h-i-r.net:

Maybe we'll have it set up for a weekly deathmatch or capture-the-flag brawl.


Open-Source gaming: OpenArena

I'm not a big-time gamer, but on occasion I get the urge to blow stuff up. I'm also not exactly rich. It would be silly for me to spend a lot of money on a console, or even $50 on an FPS that I probably won't play every day. It is for this reason that I really, really like OpenArena.

OpenArena is based on the ioquake3 engine. Ioquake3 is, in turn, based on GPL'd Quake3 code form id Software. While ioquake3 is little more than a framework for games, OpenArena takes it and packages the engine along with other GPL licensed content (level maps, weapons, character models and such) to give you a ready-to-play Quake3 clone, and for free!

Maybe in the near future, I'll set up an OpenArena server so we can duke it out. Otherwise, look for me (ax0n) playing on some of the Capture The Flag servers.


More BIOS update info.

I posted some time ago about upgrading a mobo without Windows or DOS. That process still needed one to use a floppy to perform the update. On more modern systems we can get by with nothing but the hard drive you already have installed. I recently picked up a Dell Optiplex GX280 from the usual source and of course it came without the hard drive and RAM, and it never had a floppy drive installed. I sourced a hard drive from my previous workstation, picked up some new RAM and booted the Ubuntu 8.10 install on the drive. The machine was still running the original A00 BIOS it shipped with. I then did a bit of digging and found out that Dell has been working to make updating the flash BIOS on their systems much easier under Linux -- so easy that it's literally just about cut-n-paste for the command line wary. A blog post gave the steps to get it done under Debian, and they worked just fine under Ubuntu. I already had the larger uni and multi repos enabled so all it took was to nab the correct .hdr BIOS image and get to work. It wouldn't take much work at all to get this into a point-and-click setup and have it be as dead simple to do as under Windows.

I'm surprised how easy this was to do, and this system is four years old. Linux is really coming into it's own.


The dangers of proliferation of shared FTP accounts

In a medium sized education organization which shall remain anonymous; FTP and windows file sharing is the file transport of choice for distributing small reports and data chunks. For a long time when a ftp account was needed the person would simply call up the first person who maintained a server that came to mind and had one created. The new ftp account was then fed into a script on the database server (mainframe or AS/400) which then once a day/month/year it spit out a report and uploaded it to the server where (someone/something/every one) picked it up and either loaded it into a different script, or dumped it into a spread sheet, did their business then deleted it and went to lunch.

This has gone on for decades with out some one pointing out that there is something wrong with this process.

Let me count the ways this is wrong:
1. It's not encrypted.
Packet sniffers are very easy to implement even on a switched LAN.
2. There is no way to prove that the remote host is what it says it is.
Server spoofing via DNS or Denial of Service.
3. Access control (in this case) isn't managed.
Static user names and passwords being passed in the clear.
4. Proliferation of potentially sensitive data
Just about every industry is required by law to protect certain kinds of data.
5. Use of old and un-maintainable server for warehousing information.
No warranty, use of old 3rd party software which is unmaintained, End-of-life OS.
It's potential for being 0wned is pretty high.
6. Total disregard of Intranet and Internet facing status of server.

Why? You ask has this issue been allowed to even occur?

Reason #1 Impending retirement. Why would some one who is retiring in 5 or so years would want to learn something new? Ftp and windows file sharing is well known. Ftp has been used on open systems since their inception so everybody supports it. (I mean the standard supports 7 bit file transfers, from the time when bits were expensive, really when is the last time you NEEDED to transfer something using 7 bits as opposed to 8bits?).

Reason #2 Bypassing the chain of command. Why follow protocol and make an official request when you can call the person maintaining the server and have them do it for you.

Reason #3 Maintainers versus dedicated IT staff. In most small and medium organizations, they cant afford dedicated IT staff so they give the position to some one who already does something else. The problem is that the person just puts out fires and performs maintenance. They don't keep up on industry issues and so long as the server limps along everything is fine.

Because the chain of command is bypassed the Network Administrator isn't aware of it. And the only way he or she will find out about it is either an audit, if it fails or if the server is totally Pwned and now is now selling generic Viagra. Should the latter be the case, a pile of finger pointing ensues and you can guess the rest.


The solution is finding a suitable replacement technology which is secure and possesses controls on access and availability yet is similar to an existing process so you take advantage of the users existing habits instead of putting them into an uncomfortable situation of learning some "NEW" computer process.

  1. Pre configuring the email client to use encryption. Email is one of those skills that every one knows or should know.
  2. Implement Ftp over SSL on a managed file server .(Windows, Linux, Novell ...etc) Most of them have some form of secure drive mapping or mounting which is done transparently to the user. This really is the best choice because most modern server platforms possess some form of auditing features which allow you to track access to resources and or files.
  3. Secure web application for reports and data. Automate the process and load it into a database then generate the reports on a web page or make it available as a download. A well designed web system can contain all of the controls to keep data safe. Surfing the web is a national pastime, provided you make a usable web interface.
Perhaps in conjunction with...
  1. Controlling movement of data. Prevention of use of external storage devices.
  2. Encrypting file contents using authentication. Smart cards, public/private keys, hardware keys...etc
Information security is not about being perfectly secure, it's about maintaining a good balance of security vs. usability.


Of eggs and baskets

Mainframe: All your eggs in one basket.
Virtualization: Quite a few eggs in one basket.
Servers: Every egg in its own basket.
Cloud computing: Scrambled eggs.


Web filter evasion part 5: SSH Tunneled proxies

View entire series: Web Filter Evasion

This was supposed to be posted quite a while ago to wrap up the Web Filter Evasion series, but I never got around to it.

Combining SSH Tunneling with web proxies is one of the more advanced ways to get past a web filter. You can increase your odds of being able to contact your SSH Server by running it on ports that are more likely to be allowed outbound access: 53 (DNS) and 443 (https) are good examples. If you're using a router or firewall at home, you might be able to do this with Port Address Translation or Port Forwarding.

If you can set up a system to SSH to (as described in the Reverse Tunneling and Quick & Dirty SSH Tunneling articles), you can forward the proxy port to any external proxy server (assuming it works) but the most reliable way is to run Squid Cache (which we've also discussed before) on your SSH server.

I also recently ran across a site that keeps a good up-to-date list of public proxies you could try. The problem with public proxies is that it's one more place your private information goes. Keep that in mind when using any old proxy. It might not be bad for checking 4chan or craigslist, but you might not want to pull up sensitive, personal information while using someone else's proxy.

Here is a bare-bones squid.conf file that works on the binary package for OpenBSD. It only listens on the localhost interface, so it's not likely to be abused by outsiders. It should work on any other Squid install, just make sure the access_log and cache directories exist with the right permissions (or change them in this config example to match what your squid install was using by default. Make sure to run "squid -z" to create the cache if you haven't already.

acl apache rep_header Server ^Apache
acl all src all
http_access allow all
http_port localhost:3128
access_log /var/squid/logs/access.log squid
broken_vary_encoding allow apache
coredump_dir /var/squid/cache
Once you do this, simply start a tunnel from your port 3128 to port 3128 on the proxy server. Configure your browser to use http://localhost:3128 as the proxy server (as shown in Part 4)

If you're running squid on your SSH server with the above configuration file, this will work. Notice I'm running my local proxy tunnel on port 3129, not 3128. Using "~#" over the SSH session, you can see the tunneled TCP connections.

$ ssh axon@labs.h-i-r.net -L3129:localhost:3128

-bash-3.2$ ~#
The following connections are open:
#4 client-session (t4 r0 i0/0 o0/0 fd 8/9 cfd -1)
#5 direct-tcpip: listening port 3129 for localhost port 3128, connect from ::1 port 55373 (t4 r1 i0/0 o0/0 fd 11/11 cfd -1)
#6 direct-tcpip: listening port 3129 for localhost port 3128, connect from ::1 port 55351 (t4 r2 i0/0 o0/0 fd 12/12 cfd -1)
#7 direct-tcpip: listening port 3129 for localhost port 3128, connect from ::1 port 55374 (t4 r3 i0/0 o0/0 fd 13/13 cfd -1)
#8 direct-tcpip: listening port 3129 for localhost port 3128, connect from ::1 port 55375 (t4 r4 i0/0 o0/0 fd 14/14 cfd -1)
From here on out, all the web traffic will go over the tunnel to the proxy.

Note: some browsers will try to resolve DNS OUTSIDE the proxy. This can cause things to not work properly, and might allow the owner of the network to see DNS requests for sites that are blocked. This could raise red flags. If you can, block outgoing DNS traffic from your workstation when tunneling your proxy traffic. This should force the browser to offload DNS resolution to the proxy server on the other end of the tunnel.

There are other ways to tunnel out as well, but I won't be covering them. OpenVPN, Free S/WAN and Hamachi are some other VPN/Tunneling solutions worth looking at.

View entire series: Web Filter Evasion


A generous gift - Into The Breach

Michael Santarcangelo, II has given us (the community) a generous gift. A while back, I wrote a review about his book, Into The Breach.

Today, he authorized HiR Information Report to give you access to the e-Book version for free.

Simply order the book online, then apply coupon code: hir31337

You can also purchase hard copies (and ask for an autographed one) as well. While you don't have to agree with Michael's take on things, It's a good read that might change the way you and your managers think about breach of data in the enterprise.

Thanks, Michael!

Reverse Tunnel with SSH

Inspired by a thread on the Hak5 Forums...

Sometimes, you might find yourself wishing you could poke arbitrary holes through a NAT or firewall. The potential uses (both good and evil) are nearly limitless. Perhaps you want to be able to log in to the computer in your dorm room while you're a thousand miles away on vacation with family.

You will need a few things before we start:

  1. Physical access to a computer on the "inside" of the network
  2. Access and permission to install software on that computer
  3. A system on the "outside" that can accept SSH connections from the Internet
It helps if the 2 computers involved are running Linux, Mac OS X, BSD or some UNIX variant, but you could probably use PuTTy on Windows for the "inside" computer in a pinch.

I'll be using NetBSD on my SparcStation 20 on the inside of the HiR Lab, and one of my web servers with a public IP address on the outside.

SSH Tunneling
is the process of encapsulating some other protocol within an SSH session. There are many advantages to this. Essentially, if you can get out of a network with SSH, you can get to almost any TCP service on the outside world, even if it's blocked.

Reverse SSH Tunneling is a similar process. If you can get out of the network with SSH, you can use that SSH tunnel to spawn a listening process on the outside of the network, from which you can get to anything internal.

Step One: Deploy SSH keys without a password
This has to be done from the "inside" computer. You can use your personal desktop system or a server where you have an account. It must be able to SSH out to the Internet.

I don't like to use public keys without passwords, but they do come in handy, particularly for tunneling. The un-protected public key should only be placed into the authorized keys list for UN-TRUSTED accounts that aren't in the wheel group or sudoers file.

I wrote about using passwordless SSH keys before. If you're using a UNIX variant, just follow the first few steps on generating and distributing the ssh keys. If you're using Windows, install PuTTy and follow the steps that Steve Friedl put together.

Step Two: Create and test your tunnel
I wrote a little about tunneling (forward and reverse) back in February. In this case, I'm going to tunnel SSH over a reverse SSH connection -- that is, I'm going to SSH from my inside box (NetBSD) to the web server (betaweb.h-i-r.net), and tell SSH to open a reverse tunnel port (2222) on the web server that connects to the SSH port on the NetBSD box (localhost:22). The -g option allows any remote host to use the tunnel. Otherwise, it will bind only to the localhost interface.
[axon@NetBSD]$ ssh axon@betaweb.h-i-r.net -g -R 2222:localhost:22
Last login: Sun Nov 30 14:13:08 2008 from netbsd.labs.h-i-r.net
Now, from anywhere in the world that can access port 2222 on betaweb.h-i-r.net, I can SSH through my home NAT to get to the internal NetBSD box:
Chimera:~ axon$ ssh -oPort=2222 axon@betaweb.h-i-r.net
Password: [my password]
Last login: Tue Oct 14 19:01:57 2008 from localhost
NetBSD 4.0.1 (GENERIC) #0: Wed Oct 8 01:06:02 PDT 2008
Welcome to NetBSD!

You don't need to reverse tunnel to localhost, and you could just as easily use this trick to reverse-tunnel your e-mail (POP/IMAP), VNC to a Windows/Mac desktop or even to an internal web proxy server.

Step 3: Automate!

If you just leave the SSH tunnel up and running, it won't likely last too long because of session timeouts. You'll need some process to start the tunnel. Here's where you can get creative. Obviously, a process involving the use of cron or at would be one way of going about it. A script running in the background (with nohup) or in a screen session could also take care of things nicely. One idea I had was to set up fetchmail to check a dummy e-mail account every 5 minutes or so. If there's new mail, it initiates the tunnel. Send mail. Wait. Tunnel. Another way would be to upload a file to a web or ftp site to trigger it. You could even make it read information from that file to create the tunnel for different protocols. I opted to create a configuration file on the web server that would be downloaded and used.

On the "inside" box, I created this script and called it tunnel.sh:
ssh betaweb.h-i-r.net rm .tunnel
args=`cat .tunnel`
rm .tunnel
ssh $args
Then, I added this to my crontab:
*/5 * * * * scp axon@betaweb.h-i-r.net:.tunnel ~/.tunnel && ~/scripts/tunnel.sh
Every five minutes, it will try to scp a file called ".tunnel" from the "outside" box. If it was successful, it runs the tunnel.sh script.

The tunnel.sh script removes the .tunnel file from the remote box (so that it doesn't try to spawn multiple tunnels every 5 minutes) and then uses the contents of .tunnel as the arguments for ssh.

I create a file on my laptop named ".tunnel" and add the following:
axon@betaweb.h-i-r.net -g -R 2222:localhost:22 sleep 600
Then, I upload it to the location and wait about 5 minutes. Then, I ssh to port 2222 of betaweb.h-i-r.net. The "sleep 600" is executed on the remote end, which means it will only stay connected for 10 minutes (600 seconds).

chimaera:~ axon$ scp .tunnel axon@betaweb.h-i-r.net:
axon@betaweb.h-i-r.net's password:
stdin: is not a tty
.tunnel 100% 58 0.1KB/s 00:00

[ wait 5 minutes ... cue Jeopardy theme song ]

chimaera:~ axon$ ssh -oPort=2222 axon@betaweb.h-i-r.net
Password: [my password]
Last login: Tue Oct 14 23:24:10 2008 from localhost
NetBSD 4.0.1 (GENERIC) #0: Wed Oct 8 01:06:02 PDT 2008
Welcome to NetBSD!


Considerations and Risks:
I've already stated that using public keys without a password is risky business.

The other shady business is directly executing anything using arguments pulled from some file on a remote server. All it would take is "; [insert evil command here]" added to the .tunnel file and havoc can potentially be wreaked on that internal box.

If you can get away with it, BY ALL MEANS use separate accounts that are not used for anything other than this tunneling process. Lock them down as best you can and be mindful of local privilege escalation attacks.

See? This is why admins put firewalls up in the first place. And don't go pointing your finger at me if you get in trouble for unauthorized tunneling.


We've got cards, yo! (and 2600 This Friday)

It's hard to tell, but there's green source code in the background of the front of the card. These are MiniCards by Moo.com. They've got a nice finish. I'll be handing them out to the HiR crew soon. We were all sick of being at conventions and events without being able to hand anyone some contact info. The e-mail address on the back will spam the whole team (so please don't abuse it?) Note: AsmodianX@, Frogman@, tmib@ and ax0n@ will get to individual writers at h-i-r.net.

Also, the KC 2600 meeting is coming up in one week. Same time and place. Oak Park Mall food court at 5:00PM.


Capacitor Plague

"Capacitor Plague" is the colloquial term for a vast range of electrolytic capacitor failures, most often used when several capacitors on a circuit board are bulging or have burst. The following photo shows the cooked acidic residue that sprayed out from several capacitors hiding under the mechanical part of one of my DVD players.

In my case, the residue had been cooked onto the backplane and corroded several surface mount components. This cheap DVD player is a write-off. I could probably fix it, but it would be more trouble than it's worth.

Capacitor plague is common on older motherboards and video cards, as well as in other chintzy consumer electronics. When it happens, there's usually a hiss or a pop from the device. If you catch it (and clean up the electrolyte using flux cleaner or high-grade isopropyl alcohol if any leaked) before it ruins something else, you can usually buy and solder in replacement capacitors of equal value. Remember, though, that many capacitors are polarity sensitive.

Electronics repair tips:

  • If polarity is marked on a component (by a colored band down the side of a capacitor or a band around the positive end of a diode), mark the positive terminal on the board before removing the component. I usually just put a black dot near the positive terminal on the solder side of the board using a permanent marker.
  • Avoid confusion of parts by replacing only one failed component at a time.
  • Use a grounded work mat and a wrist strap to avoid a static discharge that could damage the part you're working on.


Security Bloggers Network is back online

SBN is now powered by lijit networks. Here's the RSS Feed for SBN as well.

As of right now, Security Bloggers Network is the combined buzz of about 180 different blogs, all of which have at least partial focus on information security. Drop it into your RSS aggregator and start getting fed.

Update: Security4All has pointed out that the SBN site and feed aren't working right now. They were a bit ago. Keep your eyes on the links. It'll probably be back this weekend.

Okay, looks like it's online at www.securitybloggers.net now.


GMail In Terminal Mode - Really (with Links-SSL)

Sorry to make a second post about this, but I got all nostalgic about the Terminal theme for GMail, and it reminded me that you can actually access GMail with a text-only browser.

It's been a while since I've done it this way, but Gmail actually plays along quite nicely with the Links (not lynx) browser, as long as you have it compiled with SSL support. If you use Links in X11, you can even use the mouse to click on things.

GMail Themes? Terminal Mode!

GMail recently enabled a Theme feature. I couldn't resist using the new "Terminal" theme. I'm such a nerd.


Open-Source HDR photography with CHDK and qtpfsgui

Often, one photograph might have regions that are overexposed and others that are too dark. Details are lost in these regions. Explained simply, HDR is about taking over-exposed photos to get the detail from the dark areas and under-exposed photos to capture textures in the brighter areas, then tone-mapping them together into one high-contrast composite image. Here's what you need to make tone-mapped HDR images:

  • Tone-mapping software
  • A set of photographs. Not just any photographs:
  1. All taken from exactly the same perspective.
  2. Varied exposure. It's recommended you have at least 3: one underexposed, one "normal" exposed, and one overexposed.
  3. The more photos, the better (to an extent)
  4. Take the highest quality photos possible. That means cranking up the resolution, using the lowest ISO you can get away with, and shooting in RAW if your camera can handle it.
The tricky part, as you guessed, is getting those photos. Professional photographers with high-end rigs have autobracketing, burst mode and a host of other features that make it easier to obtain the kinds of photographs needed to make a tone-mapped HDR image. Where's that leave people (like me) with cheap point-and-shoot cameras?

I've written a little about The Canon Hacker's Development Kit, and even demonstrated CHDK at a 2600 meeting a while back ago.

CHDK is a firmware patch for Canon digital cameras. On inexpensive point-and-shoot Canons such as the PowerShot A530 my wife bought me for Christmas a while back ago, the firmware gets revamped, unleashing the ability to shoot in lossless RAW format, view live-updating histograms on-screen, run scripts, and even play games. You can obviously read more about it on the CHDK wiki, but Lifehacker had a decent write-up that you should check out.

Using CHDK to get a set of bracketed photos
While CHDK has a decent bracketing script on the wiki, you don't even need that. The default "Allbest" firmware has bracketing built-in. Read the CHDK installation instructions first. In fact, I recommend that if you haven't played with CHDK before, you bookmark this page, get CHDK installed, and play with it for a while first. Just to get used to it.

If you're local to Kansas City, I'd be more than happy to help you get up to speed. The script functionality is worth looking at. I'll cover it in a later article.

Pardon my "Screen shots" - I had to take them with a camera phone, so I hooked my Canon A530 up to the TV screen so that I could take photos of the menu without trying to focus on the tiny screen of my camera through the tiny screen on my phone.

Once CHDK is installed and started, fire up the Alt menu. Select:
Extra Photo Operations
--Bracketing in continuous mode
----TV bracketing value

Set that to at least 1/3 Ev as shown above (click for higher res). The bigger this number is, the fewer photos you'll need for a good range, but keep in mind that the more photos you get, the better. 2/3 or 1 Ev is a good compromise if you choose 5-7 photos. With 1 1/3 Ev or higher, you could try 3 or 5 photos.

To enable RAW (if supported)
RAW parameters
--Save RAW

Select that, but keep in mind that it will take a few extra seconds per photo to save the image, and the RAW image will take up a lot of space on the memory card. For this demo, I won't be using RAW mode. If you choose to use RAW, they can be processed with UFRaw, a freeware tool that turns RAW photos into high-quality JPEG images.

Bail out of the Alt menu and if it's not already in photo mode (instead of play/view mode), switch it over. In the Function menu, switch to Custom Timer drive mode. Hit the Menu button, and select the delay you want (a few seconds is a good idea, to avoid the jiggle as you hit the shutter) and the number of photos. I had 0 seconds delay in this photo, but you should probably go for 2 seconds or so. The camera should be on a tripod or stable surface.

Notice that in the last frame, CHDK has noted that bracketing is enabled for 2/3 Ev.

I then shot 7 bracketed photos of a Mt. Dew can. I used iPhoto to import these, but if you're an open-source zealot, I've had good luck using gtkam on FreeBSD and Linux with my Canon camera.

The hard part is done.

Tone-Mapping with qtpfsgui
A lot of the software for HDR is expensive, too. Photomatix is one of the more popular tools, and it's $99. qtpfsgui is an open-source tool based on QT. It runs on Linux, Windows and Mac OS X with little fanfare. While not as intuitive or easy to use as Photomatix, you still get more than you pay for with qtpfsgui. It's free, after all.

Load the images in, and select the option to automatically align them.

Click through the wizard and you'll have an HDR composite. Feel free to tinker with the values in the wizard to get the effects you want, but it's not needed if you just want to get started. In the resulting window, click "Tonemap the Hdr", then have fun exporting the images and playing with the values in the new window. Clicking "Apply" here will give you a composite with your settings. By default, it's a thumbnail size that renders quickly. You can see 4 different ones I did below with different settings.

Once you've got a thumbnail that you really like, bump the Result size up to something usable and apply it again. Then use File - Save As to export the shiny, HDR image. Wow and amaze your friends.

Notice how the brightest and darkest areas retain a lot of detail. Click here for full size.

Here's a more extreme example of HDR that I took this morning: