UNIX Tip: the tr command, newlines and more

tr is used to translate characters. It can translate all uppercase letters to lowercase. It can delete all occurances of a single character, and much more.

To turn all uppercase letters lowercase (a useful thing when parsing user input from a shell script), use this:

bash-3.2$ echo This Is CrAzY! | tr [A-Z] [a-z]
this is crazy!

Using tr -d, you can remove any characters from a string as well. Here, I remove several punctuation marks:

bash-3.2$ echo "You're right. That *is* crazy." | tr -d \'\*\!\?\.
Youre right That is crazy

One of the best ways to use tr, though, is for converting newline characters to something useful.

When running shell commands or writing shell scripts, most people can quickly adapt to using commands such as cut or awk to manipulate tabular data. See also: more awk recipes. Let's say we have a text file with the following values, and we want this data to be represented in CSV format, all on one line. This is where newlines make things tricky.

0.00
128.00
49.00
7031
4218

tr can tackle it easily, using "\n" to represent the newline character. Just translate newlines to commas, and you're good to go:

bash-3.2$ cat file | tr "\n" ","
0.00,128.00,49.00,7031,4218,

Similarly, you can take large chunks of text and replace all newlines with a space if you wish to un-do old-school text-editor formatting before importing into a word processor.

tr is a somewhat obscure command, but it can save a lot of headache, particularly when trying to format text output. The same result can be achieved (albeit with much more complex commands) with awk or sed, so this is a good one to know about, and it's found on almost every posix-compliant platform (and in cygwin)

2600 Meetings

It's that time again - The first Friday of the month! The Kansas City 2600 meeting will take place tonight at 5:00 PM (although I might be running a little late) at Oak Park Mall in the food court. Check the 2600 Meetings page to find one near you!

Crimping Twisted-Pair Ethernet (and Crossover)

In the Geek 100, I said that every geek should be able to Terminate category 5 cable with RJ-45 plugs and make a crossover cable. While home networking isn't nearly as daunting as it used to be with ubiquitous WiFi routers, I stand by my assertion that no geek has a valid excuse for being unable to crimp some cables. It's a useful skill to have and it's quite easy to learn.

All you need (aside from some Twisted-Pair ethernet cable that has bad ends or no ends) are some crimp-on RJ-45 plugs, a knife and a crimping tool. The plugs are less than $1 each at Radio Shack and cents each in bulk, knives are commodity and crimping tools like the one shown (I've had it for probably 15 years) are available for $20 or less. If your crimping tool doesn't come with a wire cutter, a pair of diagonal pliers might be useful as well. There are some much nicer crimping tools available that get really expensive, but the entry-level ones are probably just fine for the geek who only needs to run wires occasionally.


Take your knife and cut the sheath off about 3/4" of the end of the cable. Inside, there are four pairs: Blue, Orange, Green and Brown. Due to carryover from ye-olde telecom days, the pairs are divided into "Tip" (color striped with white) and "Ring" (solid color). Many cable manufacturers put a nylon strand, string, or tuft inside the sheath as well. This makes stripping a long length of sheath easy, but you can safely cut away whatever of it is exposed without harm.

Arrange the wires in either of the two configurations shown below. We'll get to putting the RJ-45 plugs on in a moment. For the time being, focus on the order of the wires.


These days, 568-A is kind of "the standard" but when I was learning, the industry was hell-bent on using 568-B. Honestly, as long as you use the same 568 scheme on both ends of the cable and you crimp the RJ-45 plug properly, it really doesn't make a bit of difference at all. The wires are ordered this way to reduce crosstalk and interference, so I advise you stick with one of the two diagrams above.

I have the wires ordered 568-B below. The tips (white-striped wires) are hard to determine in the photo, and sometimes they'll be tricky to identify in real life. You may wish to do this under an eye loop or a workbench magnifying lens if your eyesight is sub-par.


Use a wire cutter or a diagonal plier to cut the wires to the same length. It's important to keep the wires held flat and in the right order as shown above.


Slide the RJ-45 plug onto the cable end. Make sure the wires are still even and in the right order, and make sure that the sheath goes deep into the plug while still allowing the wires to "bottom out" in the plug. The sheath should be positioned so that the plastic tab that's crimped to hold the wires in place also holds the sheath. The wires should not be exposed out the bottom of the plug. (Thanks for the reminder, Ryan)


I stole this crimping "action shot" from the Loopback tester tutorial. Crimping the plug should be pretty straightforward. Make sure to give it a few good squeezes to ensure a good connection.


Make sure you wire both ends the same, and make sure the plugs are both installed facing the proper direction (with the metal pins pointed toward you and up). Here's another diagram I found on Wikipedia:


To make a crossover cable that can be used to connect two hubs/switches or two computers together (Without a hub or switch), simply wire one end as 568A and the other as 568B.

Xorg.conf for OpenBSD MacBook / Parallels

On a lot of hardware, X.org just works. OpenBSD is no exception. At certain resolutions and in particular on notebooks, X.org is not happy and getting it configured properly when that happens is usually a time-draining adventure.

This configuration file (xorg.conf) works great on OpenBSD 4.4 under Parallels Desktop on a MacBook and gives you full 1280x800 resolution at 24bpp color. It should also work for FreeBSD under Parallels as well, but I haven't taken it for a spin yet.

So far, I'm really digging OpenBSD 4.4. It might not be as easy to configure as Ubuntu or its Linux brethren, but it's rock-solid and I still prefer the BSD port and package systems to RPM or Debian-style package management.

Happy Morris Worm Day!

Twenty years ago to the day, Robert Morris, a Cornell student at the time, unleashed his worm (from the MIT campus!) on what little of the Internet existed in 1988.

To this day, no one really knows for sure how many computers were affected by the Morris Worm, which exploited any of several different vulnerabilities in order to replicate itself. Supposedly designed to be "harmless", it caused a large-scale denial-of-service attack, partly because of an error in the routine to check if it had already infected a given host.

Robert was the first person tried under the 1986 Computer Fraud & Abuse Act, and DARPA formed CERT Coordination Center in response to this incident. I found some interesting commentary about this via the Security Bloggers Network (of which HiR Information Report is a member). There's also some more information on the Wikipedia page: [Morris worm].

Creative Commons Photo Credit: Go Card USA on Flickr

The Geek 100: How'd you do? (Poll)

See the whole series: The Geek 100



This post contains a poll within an iFrame. If you don't see it in your RSS reader, visit HiR Information Report and take the poll, located on the right side of the site.

See the whole series: The Geek 100

The Geek 100 Pt. 5: Science and Electronics

See the whole series: The Geek 100

This is a list of 100 basic things and skills every geek should have. I've broken this series up into five parts. Let's face it: a list of 100 things would be tedious to wade through. Over the rest of the week, look for twenty more skills to show up daily. The skills assume you have done it in the past and can remember how to do it right now (or, like a good Geek, you've jotted it down in one of your notebooks). Having it in your personal notebook is okay. Scrambling to the Internet means you don't have the skill... yet.

Science. Every geek should be able to:

1. Build a dry-ice bomb
   
2. Build a gas-turbine engine from junkyard parts
   
3. Build a usable battery from household materials
   
4. Build an electric motor/generator from household materials
   
5. Build some form of a rocket motor
   
6. Comprehend and express orders of magnitude
   
7. Make a non-Newtonian "Oobleck" fluid
   
8. Make an explosion using only a few plastic containers, electricity and water
   
9. Know how to make hot-packs or cold-packs with simple chemical reactions
   
10. Use the Scientific Method
    

Electronics. Every geek should be able to:

1. Access debug mode on a mobile phone (preferably your own)
   
2. Build a simple FM transmitter
   
3. Build logic gates with discrete components
   
4. Burn an (E)EPROM
   
5. Calibrate and read an oscilloscope
   
6. Program a microcontroller
   
7. Properly use a digital multimeter to measure electric current
   
8. Solder surface-mount components
   
9. Use a logic probe
   
10. Use a schematic diagram to assemble a simple circuit from parts
    

See the whole series: The Geek 100