I'm a fan of ye-olde 8-bit hardware that couldn't dream of running a full-on Linux distribution. For the stuff that's just a little newer than that (think 200-600MHz), there's PuppyLinux.
Frogman introduced me to PuppyLinux when he brought a little thin client machine to the May 2600 Meeting, shown below:
In his case, this was an x86-compatible system with Windows CE. Puppy was booting off of a low-capacity SD Card, plugged into a USB reader visible plugged into the front of the machine. Puppy runs as a Live distro off of a CD, and easily installs to a USB drive, compact flash, IDE, SCSI or even CD-R media. When booted from a CD, you have the option to save the state to a file on removable media as well.
I'm just trying to get some old Pentiums and Celerons off the ground and into some state of usability. Puppy nicely fills the niche between bloated distros like Ubuntu and bare-bones disros like Gentoo and Arch. It's surprisingly friendly, surprisingly nimble, and once configured by a casual geek, would probably make a great desktop OS for the not-so-savvy.
Oddly enough, Getting it installed to the internal hard drive was a head-scratcher for me. The "Install" icon on the desktop is actually a package manager, not a system installer. A quick RTFM pointed out the procedure nicely (hint: it's buried in the taskbar menu). With GParted and the Universal Installer, pretty much any Windows power user can get Puppy installed with ease.
If you've got a kid coming of age and have been wondering what to do with that dusty old relic from the turn of the century, consider letting them cut their teeth on Linux. Better yet, take Puppy for a spin yourself. You just might like it. One can't have too many functional workstations, after all...