What Should a Data Center Disaster Recovery Plan Include?

Have you ever wondered what Amazon’s disaster recovery plan looks like? AWS already has a lot of problems with US-East-1. Remember that huge outage across the internet right before Thanksgiving in 2020? 

While that outage wasn’t caused by an act of God, it gave us a glimpse of what could happen if Amazon’s data center in Virginia was wiped out by a nasty weather event. Let’s look at what a disaster recovery plan may look like in the data center. 

Ready to Learn the Skills You Need to Create Disaster Recovery Plans?

Even CCNPs have trouble building a data center disaster recovery plan. Data centers have a lot of interdependent, complicated systems. You need a good understanding of how data centers operate before you make a crash plan for it, and the CCNP exams test for those skills. 

If you’re ready to upgrade your IT skillset, consider taking our CCNP Service Provider certification course. We’ll get you prepped and ready for the exam, and help you learn everything needed to make a foolproof disaster recovery plan. 

What Does a Disaster Recovery Plan Look Like in a Data Center?

A disaster recovery plan is the Bible of the IT department during a disaster. It clearly defines what steps to take and whom to contact during an emergency. The last thing any business needs is for the IT department to be inventing new processes and troubleshooting random things while all its services are down. It’s easy to get confused in the chaos. 

A disaster recovery plan will look different for a data center compared to a typical business, though. Data centers are complex beasts that typically host systems and services for multiple customers. This doesn’t exclude data centers for large businesses either. Though large organizations may not be renting their compute cycles to other businesses like AWS, they have tons of satellite offices and distant locations that need access to its core information and services. 

The Sections of a Disaster Recovery Plan

With that said, what are the components of a strong disaster recovery plan for a data center? Here’s a list of each of the sections it needs to contain:

• Defining Goals

• Defining People and Stakeholders

• Explain the Tech Stack and Application Profiles

• List Inventory

• Explain Current Backup Procedures

• Define Immediate Recovery Procedures

• Explain Mobile Site Recovery

• Define Hot Sites and Their Responsibilities 

• Explain Data Restoration Processes

• Explain Rebuilding Plans

• Logs

• Catalog Simulations and Tests

• Track Changes

That seems like a lot of stuff, doesn’t it? The disaster recovery plan for a data center can easily be hundreds or thousands of pages long. Making matters more complicated, there is no such thing as a one-size-fits-all crash plan. A good disaster recovery plan needs to clearly define executable steps for each type of emergency. 

For example, a disaster recovery plan should have sections for fires, floods, earthquakes, etc. Each of these scenarios requires different procedures, and not creating procedures for a specific type of disaster could be costly.

If you want an entertaining example demonstrating why disaster recovery plans need to be so detailed, watch ‘Five Nights at Memorial.' Apple TV did a wonderful job dramatizing the events that occurred at Memorial Hospital in New Orleans during Hurricane Katrina. As it turns out, Memorial Hospital had a crash plan for every type of emergency except for flooding. The lack of foresight ruined a lot of lives and killed more than 40 people. Let’s learn from their mistakes.

While a data center isn’t triaging cancer patients, the loss of its systems and services to its customers could be equally as detrimental. Disaster recovery plans are not something to be taken lightly. 

Are Disaster Recovery Plans for a Data Center Difficult to Create?

A disaster recovery plan for a data center needs to be perfect. When an emergency happens, people can’t second guess what they need to do. They need to act with direction and confidence. 

This is the reason why the IT gurus that make these crash plans are paid so much. It’s also the reason why they hold certifications like the CCNP. 

If you’re not familiar with the CCNP Service Provider certification, it’s an IT cert offered by Cisco for network engineers to prove that they understand advanced networking concepts. The exams for the CCNP are very difficult to pass even for the most seasoned techs. So, people that have the CCNP are in high demand in large IT environments like data centers. 

It takes this level of knowledge to design, test, and implement a disaster recovery plan for a data center. Designing and explaining recovery steps is a massive thought exercise. Network engineers need to have the skills to explain in extreme detail how to recover from each type of emergency.

After disaster recovery plans are minted, they need to be tested, too.  A crash plan doesn’t work until you verify that it does. With that said, it’s not like you can drive a tank through your data center, break it, and then test your disaster recovery plan to find where it fails. This process needs to be simulated and simulating live services in a data center is not easy. Nonetheless, testing your disaster recovery plan is required. Full stop. 

Final Thoughts

Changes will need to be made to the disaster recovery plan as the data center naturally evolves. These changes need to be documented and tested. If disaster strikes, and one of the processes listed in the disaster recovery plan doesn’t ‘feel’ right, engineers need a way to quickly reference the historical context of procedures to make sure the current process is valid. 

Designing disaster recovery plans is one of the most complicated things anyone working in the IT industry will encounter. You will need experience, time, and knowledge to create an effective disaster recovery plan.