How to Prepare for a Capture the Flag Hacking Competition

Hacking isn't like the movies. You're not facing off against a worthy adversary in real-time, backed by your motley crew.

In fact, most security jobs are boring — and to be honest, you want them that way. Audit. Find vulnerabilities. Patch. Rinse and repeat. To keep sharp, lots of security professionals — both new and old — enter Capture the Flag (CTF) competitions, or use CTF challenges to learn.

For aspiring white hats, CTF challenges are a great way to learn hacking techniques, strengthen your problem-solving skills, and gain critical hands-on practice. CTF competitions deal the right level of pressure to keep things interesting while helping you to sharpen your skills.

For the pros, CTF competitions help you assess your skill level, challenge yourself among peers, and maybe even earn some bragging rights.

Here's how CTF competitions work, and a few tips on how to prepare for your first competition.

What are capture the flag competitions?

Capture the Flag hacking competitions are exactly like the first-person shooter game mode. One team of players attempt to locate and capture an opposing team's "flag" while also defending their flag. In CTF competitions, the flag is typically a snippet of code, a piece of hardware on a network, or perhaps a file. In other cases, the competition may progress through a series of questions, like a race.

They can either be single events or ongoing challenges — and typically fall into three main categories: Jeopardy, Attack-Defense, and mixed events.

Attack-Defense

This style of competition is much closer to the backyard capture the flag game than the Jeopardy style. In these types of events, teams defend a host PC while still trying to attack opposing teams' target PCs. Each team starts off with an allotted time for patching and securing the PC, trying to discover as many vulnerabilities as possible before the opponent attacking teams can strike. Teams receive points for staving off attacks from opposing teams and successfully infiltrating other teams. The team with the most points wins.

Jeopardy CTF

Jeopardy-style CTFs present competitors with a set of questions that reveal clues that guide them in solving complex tasks in a specific order. By revealing clues, contestants learn the right direction regarding techniques and methodologies that are needed going forward. Teams receive points for each solved task. The more difficult the task, the more points you can earn upon its successful completion.

Ongoing, online CTF competitions are most likely to be Jeopardy style. It's easier to play solo, and requires less coordination among players than an Attack and Defend competition.

Mixed Events

As the name suggests, mixed competitions are an amalgam of Jeopardy and Attack-Defend formats. Sometimes organizers will segment the competition into events. Other times organizers might split teams to compete in concurrent events of different styles.

What's the difference between these and hackathons?

Both CTF competitions and hackathons bring together teams to use their skills in a concerted fashion with a time limit. But that's about where the comparison ends.

CTF competitions encourage teams to subvert security systems and sidestep safeguards through known- or competitor-created exploits to earn points. In other words, it's a game.

Hackathons are more of a collaborative event that allows developers and programmers to showcase their creative talents by building a working application or program within an allotted time period while following specific criteria. They can be security-related, but hackathons are a generalized term.

The word 'hack' in hackathon refers to how an end product is 'hacked together,' which is a popular phrase in homebrew and DIY enthusiast circles, and not as in 'computer hacking.'

How to Prepare for Capture the Flag competitions

Unlike most technical certifications, CTF competitions are 100 percent practical. There's no multiple choice. To be successful, you've got to build up a strong knowledge base, and then draw from it. Though that sounds daunting, it's not that bad. Provided you've learned (or starting learning) white hat basics, you'll learn everything else you need from practice, practice, and more practice.

There are some great resources with challenging problem sets available for free.

PictoCTF

Plenty of aspiring white hats start with PictoCTF. It's actually intended for middle and high schoolers. For that reason, it covers the basics very well, provides many hints, and reveals challenges as an interesting storyline. If you're an adult, you can't compete for prizes, but the lessons are still excellent.

Smash the Stack

Among the most popular wargame sites, Smash the Stack hosts several wargames to attack operating systems, networks, and applications. Most wargames are always online, but they also have regular competitions. Due to its popularity, beginners can reference plenty of write-ups on GitHub, personal blogs, or even YouTube.

Over the Wire

Developed by a robust community of "good-looking hackers," OverTheWire has wargames for every skill level. The 34-level Bandit wargame is the perfect starting place for absolute beginners. Eventually, you can progress to the Manpage wargame. With each game on its own SSH port, even connecting to the individual games is a learning exercise.

Microcorruption

It's not a pretty website, but Microcorruption shows you how to exploit real-world software flaws with a debugger. Even better, you channel your inner Mission Impossible with a storyline that involves stealing a briefcase of bearer bonds. As they put it, "Should be a milk run. Good luck."

Google CTF

The Google CTF comprises 23 challenges, and one "Beginner's Quest." The challenges are available year-round, but the team competition aspect only runs for a weekend in the summer. Google pays out $100 for the best 21 write-ups and $500 for the 11 most creative solutions.

Note: These are merely a few popular examples of CTF challenge sites. They are by no means the only resources out there.

Types of Capture the Flag Questions

If even gamified learning sounds daunting, then find solace in the fact that CTF questions typically fall into five categories. You don't have to become an expert in every subject matter area, but you should have a working knowledge of each.

Question Type 1: Binary Exploitation

Binary exploitation comes down to making an application act differently than how it was intended to run. By making the application run differently, you're gaining valuable information that you'll use to alter or commandeer the target.

Common binary exploits use a technique known as memory corruption, which can enable an attacker to gain unauthorized privileges to the system that is running the application, or by hijacking the control flow of the application and injecting their commands directly into the system.

Question Type 2: Reverse Engineering

Sometimes the flag will be a string hidden inside the application code. Depending on the challenge type and level of difficulty the task, you might need to use reverse engineering.

Reverse engineering challenges require an intimate knowledge debugger and disassembler software. The goal: Take a compiled binary, rip it apart, and find out how it works.

You will want to be familiar with how the application uses control flow, loops, and conditionals so that you can figure out how to bend the program to your will, and then hopefully capture the flag.

Question Type 3: Web Exploitation

These question types cover a wide range of different methods to exploit web-based resources. While the methods are broad, there's are tools commonly associated with web exploitation, including Nmap, Wireshark, and Metasploit.

Some of the easier flags are even accessible through your web browser through "View Page Source" or the equivalent in your browser.

Question Type 4: Cryptography

Cryptography challenges are particularly fun. Even the definition for cryptography sounds fun. "Cryptography is the practice and study of techniques for secure communication in the presence of third parties." In practice, however, they can be difficult. Often enough, these questions are based on string conversions from one format to another. For instance, you might be given a file that starts like this:

8a10247f90d0a05538888ad6205882196f5f6d05c21ec8dca0cb0be02c3f8b09e382963f443aa514daa501257b09a36bf8c4c392d8ca1bf4395f0d5f2542148c7e5ff22237969874bf66cb85357ef99956accf13ba1af36ca7a91a50533c4d89b7353f908c5a166774293b0bf6247391df69c87dacc4125a99ec417221b58170e633381e3847c6b1c28dda2913c011e13fc4406f8fe73bbf78e803e1d995ce4dbd20aad820c9e387ea57408566e5844c1e470e9d6fbbdba3a6b4df1dd85bce2208f1944f1827d015df9c46c22803f41d1052acb721977f0ccc13db95c970252091ea5e36e423ee6a2f2d12ef909fcadd42529885d221af1225e32161b85e6dc03465cf398c937846b18bac05e88820a567caac113762753dffbe6ece09823bab5aee947a682bb3156f42df1d8dc320a897ee79981cf937390b4ae93eb8657f6ced9eccbe79394ca0575a81d1fa51443aa3e83e5e3cdb7a4c5897faa6b4ac123c1dde2dff4d7c5b77d2aa3c090cebce70340e7f0e0b754ca26b9c108ca0dbfdcd8aa230eb9420654b252ffc118e830179cc12b64b2819f81edcc2543d759c422c3850051d543c9bc1dcda7c2a6c9896e1161d61c3c13c80ee79c08379abf3e189f2ecf5f997db17db69467bf6dfd485522d084d6e00a329526848bb85414a7e6a

And scrolls forever. Your challenge: "In this file are a bunch of hex-encoded ciphertexts. One of them has been encrypted with ECB. Detect it." And that's an intro Cryptopals challenge.

In other cases, you'll have to encrypt or decrypt messages. You'll need to have a good handle on programming for cryptography. If you don't, it's a lucrative skill to attain.

Question Type 5: Forensics

This type of question in a CTF environment can cover a lot of ground, but it is quite common that you'll be asked to find files or information hidden within other file types. For instance, a simple jpg or png file could be manipulated to hold information such as text, or even an executable.

By digging into these files with scripts and tools, competitors can extract data (normally encrypted) and then run it against a series of other tools as they try to decode the coveted flag. There are many useful tutorials and write-ups online that can get you started.

Find your favorite type of CTF problem

As you learn your trade, you'll likely find that you're strong with, or particularly enjoy, one type of problem. Once you have found your favorite type of problems to solve, then specialize. It's perfectly acceptable to go deep into one subject matter area. In fact, it's recommended.

Write-ups are a key part of CTF competitions. Teams prepare documentation (called write-ups) about the vulnerabilities they found and the processes they used to exploit the vulnerability. Judges often use write-ups to evaluate teams, like in the Google CTF. Remember that white hats have a goal in mind, to develop a fix. Write-ups help track vulnerabilities and how to fix them. For that reason, they're great learning material for beginners.

When you're first learning, write-ups are a great resource to check your solution, or even provide a little help when you're stuck.

To reiterate, before you dive in and start looking at solutions, make sure that:

• You use write-ups only after you've solved the problem

• You don't use write-ups as a cheat sheet

To prepare for a CTF competition, you'll want to find and read as many write-ups as possible. They also make for good practice, even if they're way over your head. You can find past event results with questions, and you can try taking them on before you join a live tournament.

These past events are usually well documented, with solutions and problem-solving steps included on most of the write-up resources. Even if answer quality and completeness varies from author to author, you can look at multiple solutions to the same problems. You can then compare the best answers to your attempts, and then find out if you could learn any more about these types of problems by incorporating some of those techniques into your bag of tricks.

Get started hacking today

If you're brand new to hacking, then find a good course to teach you the skills. The best courses, like Keith Barker's new White Hat Hacking v10 course, start by setting up a Kali Linux practice lab, so you can get hands-on experience immediately.

When you're ready, work through the CTF challenges, review the write-ups, and maybe even enter a local competition. By competing in these competitions and following the challenges, you not only strengthen your knowledge and understanding of how the technologies work, but also how to select the appropriate responses to a particular challenge.

These skills very quickly add up and, over time, teach you valuable, real-world techniques that you could apply in your studies and working environment, allowing you to expand your Ethical Hacking toolkit.