One last post from me on the inaugural CyberRAID event here in Kansas City.
Leadership
I'm not sure why I was chosen as team captain. Maybe it was because I knew more people on my team than everyone else, because I seemed more confident or because I was one of the first ones present. I wasn't nervous. I felt like I was as prepared as I was going to be. I had my plan, tools and more than half my life behind me that I've spent doing security work in some capacity or another. I thought I was ready to orchestrate a defense team, too, so I happily accepted the position of Captain at the start of the game when people told me "Go up to Dwight! Get our info!" It's not that I can't orchestrate a defense team, but I wasn't as ready as I had thought I'd be, and my plan fell apart pretty quickly.
My plan was to get people organized by what they're good at, and set them on a task. I kind of succeeded at that, but I did plenty of things wrong from the beginning. First, I was stressed but I haven't completely forgotten my manners. My requests probably sounded like a bossy demand followed by "please." Next, I didn't enforce these roles nor did I evaluate how it was going. Some roles changed without much communication. More than once, someone stepped on the toes of someone else who was already working on something. I'm thankful that there wasn't any apparent infighting on my team. Everyone remained rational.
I've lead teams on many projects in my career, but I have no formal management experience or training. I've been an IT worker in a crisis situation more times than I can count. I've never had to lead a crisis response, though. To manage people and work on technical things at the same time is a truly herculean task -- one that I feel I only barely stumbled through.
I learned a lot about myself and hardships faced by leaders in a crisis situation. I also gained a new perspective on IT workers. Among my team, I had some of the most brilliant and capable security minds I know of in the region. On the first day, we had grand ideas and the right mindset, but our implementation of them was slipshod at best. Our good communication skills were the only thing standing between what we had (which was still a good effort) and unabashed anarchy.
Things I learned about leadership and IT teams (and thus, what I need to work on myself):
#1: Technical leaders must lead first and foremost, and help second.
#2: Groups of geeks require a little bit of guidance to avoid replicating (or undoing) work.
#3: Good communication is vital, and its prerequisite is good rapport.
#4: Change management is an extension of good communication. Yeah, I went there.
On the technical side, there were so many things that I'd do different right from the start.
#5: Get visibility to the DMZ network and I'd immediately scan it from the outside while the firewall and system admins work to start locking down obvious things.
#6: Additional Virtual Machines on the DMZ. It would have been great to have a decent (and familiar) IDS on a Span Port. It might have also been fun to have a few honey pots sitting on the DMZ. Had I thought about this ahead of time, I could have totally made it happen. These are tools any one of us could have brought along for the ride in VMs.
#7: Egress filters from the start. There's no good reason not to. They make sense in the enterprise, and they make sense in the game.
Things I'm taking back to the office:
#7: Egress filters! I'm mentioning it twice. Blind SQL injection and RCE exploits are very popular, so crafty hackers and pen-testers often try to leverage these vulnerabilities to launch some process that can notify them that their exploit has worked. This might be popping an xp_cmdshell to launch ping with a special payload they can look for in return, or it may be something much more quotidian, such as a reverse_tcp or meterpreter call-back from metasploit. Again, egress filters make sense in the real world. To further this point, watch out for obscure tunneling through ICMP and DNS. Ideally the DNS server your DMZ uses should not allow recursion.
#8: We all know that despite our best efforts, a dedicated adversary will find a way in. For some reason, this exercise made it sink in a little better. It's been a while since I've worked somewhere that was breached on my watch. This further boosts my desire to learn more about modern post-breach activities both defensive (forensics, containment) and offensive (post-exploitation, pivoting). I have some serious reading and research to do!
Who else was at CyberRAID, CCDC, SANS ICE II or any other recent exercise like this? What did you get out of it?
Posts
