Disaster Recovery Plans

Preventive measures

Any procedures or systems in place that will proactively minimize the impact of a disaster.

What should be documented in detail when citing operational procedures?

Any steps or specific configuration details that are needed to restore 100% functionality to core systems and services.

Redundancies shouldn't be limited only to systems. What else should be included?

Anything critical to operations should be made redundant whenever possible.

What is considered preventative in terms of data loss?

Anything that's done before an actual disaster that's able to reduce the overall downtime of a disastrous event.

What does a risk assessment involve?

Brainstorming hypothetical scenarios and analyzing these events to understand how they'd impact your organization and operations.

How do you test early warning systems?

By simulating the conditions your monitoring systems are designed to catch.

Why is it critical that you have data recovery procedures clearly documented and kept up-to-date?

Data recovery following a disaster is hopefully something you will rarely do, so it's important to make sure that these procedures are updated since systems change and evolve throughout their lifetimes.

True or False: Disaster Recovery Documentation is more of a guideline and is not important to have on hand at all times.

False; It's important that critical documentation is accessible if an emergency scenario or disaster strikes.

True or False: The Disaster Recovery Documentation needs to contain every detail of the operations.

False; Links and references are sufficient.

True or False: You don't need to worry about operational procedure in terms of documentation.

False; Make sure that every important operational procedure is documented and accessible.

When designing a Disaster Recovery Plan, how can you make sure you have a sound backup and a recovery system, along with a good strategy in place?

Ideally, you should have regular but automated backups to backup systems located both on site and off site.

What is an example of detection measures to look at when designing a disaster recovery plan?

If up-time and availability is important for your organization, you'll likely have two internet connections; a primary and a secondary. You want to monitor the connection status of both of these links. Ideally, they should be configured to automatically fail over if one goes down. But you'll still want to be alerted when this happens, so you can investigate why the link went down and work on getting it back as soon as possible.

Why do you need to test your reactions and responses to alerts?

If your monitoring systems reliably trigger alerts but everyone just ignores them, then they won't be useful.

What is important for Disaster Recovery Documentation?

It's important to be prepared for a situation where typical documentation access methods are unavailable.

Detection measures

Meant to alert you and your team that a disaster has occurred that can impact operations.

Corrective or recovery measures

Measures enacted after a disaster has occurred.

What is another important preventative measure that should be evaluated and verified?

Operational documentation

Anything critical to operations should be made redundant whenever possible, what does this include?

Power delivery or supply, communications systems, data links, and hardware.

What does a disaster recovery plan cover?

Preventive measures and detection measures on top of the post disaster recovery approach.

What is included in operational documentation?

Setting up and configuring critical systems and infrastructure.

Why is timely notification of a disaster critical?

Since some steps of the disaster recovery plan might be time sensitive to ensure there is no data loss or equipment damage.

Along with temperature and humidity sensors, what else is critically important?

Smoke detectors and fire alarms

What do corrective or recovery measures involve?

Steps like restoring lost data from backups or rebuilding and reconfiguring systems that were damaged.

When you look into preventive measures, pay attention to what?

Systems that lack redundancy.

Corrective or recovery measures include actions that are taken to restore normal operations and to recover from an incident or outage. What does this include?

Tasks like restoring a corrupted database from a backup or rebuilding and reconfiguring a server.

What can alert you to failed cooling in the server room?

Temperature and humidity sensors

Preventive measure include what?

Things like regular backups and redundant systems.

When designing detection measures, you will want to monitor conditions of service and infrastructure equipment. What will give you insight into the performance of the system?

Things like temperatures, CPU load, and network load for a service monitoring for error rates and requests per second.

Why is it that when one system in a redundant pair suffers a failure, it's called a single point of failure?

This is because it only takes one failure now to completely take the system down.

What is the way to think about designing detection measures?

To evaluate what's most critical to the day-to-day functioning of the organization.

What is the goal of the disaster recovery plan?

To minimize disruption to business and IT operations, by keeping downtime of systems to a minimum and preventing significant data loss.

What is a good way to understand what to plan for when creating a disaster recovery plan?

To perform a risk assessment.

What is an effective way to keep operational documentation up-to-date?

To periodically verify that the steps documented actually work.

True or False: If a disaster takes out one system that's part of a redundant pair, this would prevent a complete service outage, but that would mean you aren't prepared for another disaster.

True

True or False: You want to conduct regular disaster tests to make sure systems are functioning and that your procedures to handle them are also up to the task.

True

True or False: Your disaster plan should include reference or links to documentation for corrective or recovery measure tasks.

True. It should include anything and everything that would be required to restore normal operations following some disaster.

What do corrective or recovery measures include?

Actions that are taken to restore normal operations and to recover from an incident or outage.

Risk assessment

Allows you to prioritize certain aspects of the organizations that are more at risk if there's an unforeseen event.

What should happen once the disaster has been detected and steps have been taken to either prevent an outage or at least minimize downtime?

Work should begin on restoring full operations of everything affected.

When evaluating what infrastructure services and access are absolutely vital for day-to-day function, what do you want to monitor for besides outages or complete failures?

You also want to monitor for conditions that indicate that a problem is likely to occur.

If there is a building evacuation, and you want to send everyone home for the day and have them work from home what should you do?

You should be prepared for the situation ahead of time and have a plan in place to ensure that everyone is able to work from home effectively.

What will you want to do when looking at detection measures while designing a disaster recovery plan?

You'll want to make sure you have a comprehensive system in place that can quickly detect and alert you to service outages or abnormal environmental conditions.

What else should you test besides the monitoring systems?

Your reactions and responses to these alerts.

Besides power loss events, what are other things that should be monitored to help head off any unexpected disasters?

1. Environmental conditions: inside several networking rooms. 2. Flood sensors: can alert you to water coming into the server room. 3. Temperature and Humidity sensors: can watch for dangerously high temps or some optimal moisture levels in the air. 4. Evacuation Procedures: If there's an evacuation of the building, you should be prepared to set up temporary accommodation so people can still work effectively.

What are the steps for designing a Disaster Recovery Plan?

1. Perform Risk Assessment 2. Determine Backup and Recovery Systems 3. Determine Detection & Alert Measures & Test Systems 4. Determine Recovery Measures

What are some examples of questions to ask when figuring out redundancies for a disaster recovery plan?

1. What would happen to your network if the building lost power? 2. Can your users continue to work at the fiber-optic data line if the building gets damaged by nearby construction work? 3. what would you do if your core router for the office is damaged?

Disaster recovery plan

A collection of documented procedures and plans on how to react and handle an emergency or disaster scenario, from the operational perspective.

Usually, a disaster will take out one system that's part of a redundant pair, or a replication scheme, which would prevent what?

A complete service outage.

Single point of failure

A component or entity in a system which, if it no longer functions, would adversely affect the entire system.

If a system critical to permit operations lacks redundancy, what should it have, just in case?

A redundant spare

What is an example of a preventive measure?

A standard for critical network infrastructure or service is to have redundant power supplies. They're often fed from different power sources like battery backup. This is designed to minimize the downtime that would be caused by one power supply failing or a power outage.