Principles of Incident Response and Disaster Recovery Chapter 7 Disaster Recovery: Preparation and Implementation
Objectives • Understand the ways to classify disasters, both by speed of onset and source • Know who should form the hip of the disaster recovery team • Understand the key functions of the disaster plan • Explain the key concepts included in the NIST approach to technical contingency planning • Describe the elements of a sample disaster recovery plan Principles of Incident Response and Disaster Recov
2
Objectives (continued) • Understand the need for simultaneous wide access to the planning documents as well as the need for securing the sensitive content of the DR plans
Principles of Incident Response and Disaster Recov
3
Introduction • Disaster recovery planning: preparation for and recovery from a disaster • Disaster may be an escalated incident or may be immediately classified as a disaster • In general, a disaster is an incident that cannot be contained or whose impact is not controllable • All business units of an organization need to be involved in disaster recovery planning, not just IT
Principles of Incident Response and Disaster Recov
4
Disaster Classifications • Disasters can be classified by cause: – Man-made: war, terrorism, cyberterrorism, etc. – Natural: fire, flood, earthquake, hurricane, lightning, tornado, etc.
• Disasters can be classified by speed of development: – Rapid onset: occur suddenly with little warning – Slow onset: occur over time and deteriorate the capacity of the organization to withstand
Principles of Incident Response and Disaster Recov
5
Disaster Classifications (continued)
Principles of Incident Response and Disaster Recov
6
Disaster Classifications (continued)
Principles of Incident Response and Disaster Recov
7
Forming the Disaster Recovery Team • Disaster recovery team is assembled by the MT • Should include from IT, InfoSec, and other departments • DR team is responsible for planning for DR and for leading the DR process when a disaster is declared • Must consider the organization of the DR team and the needs for documentation and equipment
Principles of Incident Response and Disaster Recov
8
Organization • DR team – Should include representatives from every major organizational unit – Should be separate from other contingency-related teams – May include senior management, corporate units, facilities, fire and safety, maintenance, IT, InfoSec
• May be advisable to divide the team up into subteams Principles of Incident Response and Disaster Recov
9
Organization (continued) • Subteams may include: – Disaster management team: command and control, responsible for planning and coordination – Communications: public relations and legal representatives to interface with senior management and general public – Computer recovery (hardware): recovers physical computing assets – Systems (OS) recovery: recovers operating systems – Network recovery: recovers network wiring and hardware Principles of Incident Response and Disaster Recov
10
Organization (continued) • Subteams (continued): – Storage recovery: recovers storage area networks and network attached storage – Applications recovery: recovers applications and reintegrates s back into the systems – Data management: recovers and restores data – Vendor : works with suppliers and vendors to replace damaged or destroyed materials, equipment, or services – Damage assessment and salvage: provides initial assessments of damage and recovers salvageable items
Principles of Incident Response and Disaster Recov
11
Organization (continued) • Subteams (continued): – Business interface: works with remainder of organization to assist in recovery of non-technology functions – Logistics: provides supplies, space, materials, food, services, or facilities needed at the primary site – Other teams needed to reestablish key business functions as needed
Principles of Incident Response and Disaster Recov
12
Special Documentation and Equipment • All team – Should have multiple copies of the DR and BC plans at home and office for immediate use when disaster occurs – Should have access to certain disaster recovery materials, including software, hardware, building blueprints, key phone numbers, emergency supplies, etc.
Principles of Incident Response and Disaster Recov
13
Disaster Planning Functions • Guidelines are found in NIST Contingency Planning Guide for Information Technology Systems • Planning process steps: – – – – – – –
Develop the DR planning policy statement Review the business impact analysis (BIA) Identify preventive controls Develop recovery strategies Develop the DR plan document Test, train, and rehearse Plan maintenance
Principles of Incident Response and Disaster Recov
14
Develop the DR Planning Policy Statement • DR policy should contain these key elements: – – – – – – – –
Purpose Scope Roles and responsibilities Resource requirements Training requirements Exercise and testing schedules Plan maintenance schedules Special considerations
Principles of Incident Response and Disaster Recov
15
Develop the DR Planning Policy Statement (continued) • Purpose: – Provide for the direction and guidance of any and all DR operations – Must include executive vision and commitment
• Business disaster recovery policy should apply to the entire organization • Scope: – Identifies the organizational units and groups of employees to which the policy applies
• Roles and responsibilities: – Identifies the key players and their responsibilities Principles of Incident Response and Disaster Recov
16
Develop the DR Planning Policy Statement (continued) • Resource requirements: – Identifies any specific resources to be dedicated to the development of the DR plan
• Training requirements: – Details training related to the DR plan
• Exercise and testing schedules: – Specifies the frequency of testing of the DR plan
• Plan maintenance schedules: – Details the schedule for review and update of the plan Principles of Incident Response and Disaster Recov
17
Develop the DR Planning Policy Statement (continued) • Special considerations: – May include issues such as information storage and retrieval plans, off-site and on-site backup schemes, or other issues
Principles of Incident Response and Disaster Recov
18
Review the Business Impact Analysis • Review the BIA within the DR context • Ensure that the BIA is compatible with the DR specific plans and operations • BIA is usually acceptable as it was prepared and released by the MT
Principles of Incident Response and Disaster Recov
19
Identify Preventive Controls • This function should have already been performed as part of ongoing information security posture • DP team should review and that data storage and recovery techniques are implemented, tested, and maintained
Principles of Incident Response and Disaster Recov
20
Develop Recovery Strategies • May be impossible to prepare for all diverse contingencies, but recovery strategies should be in place for the most likely disasters • DR strategies: – Go substantially beyond the recovery portion of database backup and recovery – Must include the steps to fully restore the operational status of the organization – Includes personnel, equipment, applications, data, communications, and services (power, water, etc.) Principles of Incident Response and Disaster Recov
21
Develop Recovery Strategies (continued) • DR strategies must include the enlistment and retention of qualified general contractors capable of assessing damage and rebuilding the facility • May want to include the general contractor in the DR training and rehearsals • If the primary site is a leased facility, include the leasing agency
Principles of Incident Response and Disaster Recov
22
Develop the DR Plan Document • DR planning document should contain specific and detailed guidelines and procedures for restoring lost or damaged capabilities • Steps: – DR team takes the IR plan and converts incidents to disasters – DR team adds additional disasters not in the IR document, and creates disaster scenarios – DR team develops 3 sets of activities for each scenario
• Activities during the disaster are placed first, then follow-up activities, and finally occasional activities Principles of Incident Response and Disaster Recov
23
Develop the DR Plan Document (continued) • Procedures during the disaster: – Procedures that must be performed during the disaster, if any – Grouped and assigned to individuals – May include evacuation plans, locations of shelters, fire suppression systems, other emergency reaction items – Must be readily available for use during a disaster
• Procedures after the disaster: – Procedures performed immediately after – May include crisis management procedures
Principles of Incident Response and Disaster Recov
24
Develop the DR Plan Document (continued) • Before the disaster: – Procedures to prepare for the disaster – May include data backup, disaster recovery preparation, training schedules, testing plans, copies of service agreements, business continuity plans, etc.
• DR addendums – One for each type of anticipated disaster – Includes the trigger, notification method, response time
Principles of Incident Response and Disaster Recov
25
Develop the DR Plan Document (continued)
Principles of Incident Response and Disaster Recov
26
Develop the DR Plan Document (continued) • Trigger: point at which a management decision to react is made • Planning for actions taken during the disaster: – Most important part is planning the actions before phase – Should create reaction scenarios
• Planning for events occurring after the disaster: – Includes recovery operations, identification of potential follow-on attacks, and forensics analysis – Must conduct an action-after review (AAR) Principles of Incident Response and Disaster Recov
27
Develop the DR Plan Document (continued) • Forensics analysis: process of systematically examining information assets for evidentiary material that can provide insight into the cause • After-action review (AAR): detailed examination of the events that occurred from detection to final recovery • Planning for actions taken before the disaster: – Includes preventive controls, risk management, team preparedness, stocking of critical consumables, execution of service and contracts
Principles of Incident Response and Disaster Recov
28
Plan Testing, Training, and Exercises • Training can be used to test the validity and effectiveness of the DR plan • Testing should be an ongoing activity, at least semiannually at the walk-through level • Final assembly of the DR plan can take place after testing and training
Principles of Incident Response and Disaster Recov
29
Plan Maintenance • Plan must be a dynamic document that is updated regularly • Revisit the DR plan at least annually to update plans, contracts, and agreements • Make necessary personnel and equipment modifications • Any change in the organization’s size, location, or business focus must be incorporated into the DR and plans, and the BIA should also be reviewed
Principles of Incident Response and Disaster Recov
30
Technical Contingency Planning Considerations • Technical contingency planning is based on the type of IT platforms: – – – – – – –
Desktop computers and portable systems Servers Web sites Local area networks Wide area networks Distributed systems Mainframe systems
Principles of Incident Response and Disaster Recov
31
Technical Contingency Planning Considerations (continued) • For each platform type, two perspectives are considered: – Technical requirements that should be considered, including preventive and recovery measures – Technology-based solutions that may be used
• Some contingency measures are common to all IT systems
Principles of Incident Response and Disaster Recov
32
Technical Contingency Planning Considerations (continued) • Common considerations include: – Frequency of backup and off-site storage of data, applications, and operating systems – Redundancy of critical system components – Documentation of system configurations and requirements
– Interoperability between system components and between primary and alternate site equipment to expedite system recovery – Appropriately sized and configured power management systems and environmental controls Principles of Incident Response and Disaster Recov
33
Desktop Computers and Portable Systems • Contingency considerations should emphasize data availability, confidentiality, and integrity • Should consider these practices: – – – – – – –
Store backups off-site Encourage individuals to back up data Provide guidance on saving data on PCs Standardize hardware, software, and peripherals Document system configuration and vendor information Coordinate with security policies and controls Use results from BIA
Principles of Incident Response and Disaster Recov
34
Desktop Computers and Portable Systems (continued) • Contingency strategies may include: – Document system configuration and vendor information – Standardize hardware, software, and peripherals – Provide guidelines on backing up data
– Ensure interoperability among components – – – –
Coordinate with security policies and controls Backup applications and store off-site Use alternate hard drives Image disks and standardize images
Principles of Incident Response and Disaster Recov
35
Desktop Computers and Portable Systems (continued) • Contingency strategies (continued): – Implement redundancy in critical system components – Use uninterruptible power supplies
Principles of Incident Response and Disaster Recov
36
Servers • Address server vulnerabilities by considering these practices: – Store backup media and software off site – Standardize hardware, software, and peripherals – Document system configuration and vendor information – Coordinate with security policies and controls – Use results from BIA
Principles of Incident Response and Disaster Recov
37
Servers (continued) • Contingency strategies may include: – Document system configuration and vendor information – Standardize hardware, software, and peripherals – Coordinate with security policies and controls
– Ensure interoperability among components – Backup data and store off-site – Use uninterruptible power supplies – Implement redundancy in critical system components
Principles of Incident Response and Disaster Recov
38
Servers (continued) • Contingency strategies (continued): – Implement fault tolerance in critical system components – Replicate data – Implement storage solutions
Principles of Incident Response and Disaster Recov
39
Web Sites • In addition to information about servers, these practices should be considered: – Document Web site – Web site programming should use documented change management – Web site coding should be relative, not absolute, allowing quick reconfiguration if needed – Coordinate contingency solutions with appropriate security policies and controls – Coordinate contingency solutions with incident response procedures – Use results from BIA
Principles of Incident Response and Disaster Recov
40
Web Sites (continued) • Contingency strategies may include: – – – – – –
Document Web site Code, program, and document Web site properly Coordinate with security policies and controls Consider contingencies of ing infrastructure Implement load balancing Coordinate with incident response procedures
Principles of Incident Response and Disaster Recov
41
Local Area Networks • Consider the following practices: – Physical and logical LAN should be well documented – System configuration and vendor information should be well documented – Coordinate with security policies and controls – Use results from BIA
• Identify single points of failure that affect critical systems or processes outlined in the BIA • Identify threats to the cabling system such as cable cuts, electromagnetic and radio frequency interference, and damage from fire, water, and other hazards Principles of Incident Response and Disaster Recov
42
Local Area Networks (continued) • Contingency strategies may include: – – – – – – –
Document the LAN Coordinate with vendors Coordinate with security policies and controls Identify single points of failure Implement redundancy in critical components Monitor the LAN Integrate remote access and wireless area network technology
Principles of Incident Response and Disaster Recov
43
Wide Area Networks • Consider the following practices: – Physical and logical LAN should be well documented – System configuration and vendor information should be well documented – Coordinate with security policies and controls – Use results from BIA
Principles of Incident Response and Disaster Recov
44
Wide Area Networks (continued) • Contingency strategies may include: – – – – – –
Document the WAN Coordinate with vendors Coordinate with security policies and controls Identify single points of failure Implement redundancy in critical components Institute service-level agreements
Principles of Incident Response and Disaster Recov
45
Distributed Systems • Consider the following practices: – Standardize hardware, software, and peripherals – Document system configuration and vendor information – Coordinate with security policies and controls – Use results from the BIA
Principles of Incident Response and Disaster Recov
46
Distributed Systems (continued) • Contingency strategies may include: – – – – – – –
Standardize components Document system Coordinate with vendors Coordinate with security policies and controls Consider server contingency solutions Consider LAN contingency solution Consider WAN contingency solution
Principles of Incident Response and Disaster Recov
47
Mainframe Systems • Consider the following practices: – Store backup media off site – Document system configurations and vendors – Coordinate with network security policies and system security controls – Use results from the BIA
Principles of Incident Response and Disaster Recov
48
Mainframe Systems (continued) • Contingency strategies may include: – – – – – – – – – –
Backup data and store off site Document system Coordinate with vendors Coordinate with security policies and controls Implement redundancy and fault tolerance in critical system components Consider hot site or reciprocal agreement Institute vendor service-level agreements (SLAs) Replicate data Implement storage solutions Use uninterruptible power supplies
Principles of Incident Response and Disaster Recov
49
Summary of Technical Contingency Planning Considerations
Principles of Incident Response and Disaster Recov
50
Summary of Technical Contingency Planning Considerations (continued)
Principles of Incident Response and Disaster Recov
51
Sample Disaster Recovery Plans
Principles of Incident Response and Disaster Recov
52
Sample Disaster Recovery Plans (continued)
Principles of Incident Response and Disaster Recov
53
Sample Disaster Recovery Plans (continued)
Principles of Incident Response and Disaster Recov
54
Sample Disaster Recovery Plans (continued)
Principles of Incident Response and Disaster Recov
55
Sample Disaster Recovery Plans (continued)
Principles of Incident Response and Disaster Recov
56
Sample Disaster Recovery Plans (continued)
Principles of Incident Response and Disaster Recov
57
The Combined DR Plan/BC Plan • Many organizations prepare DR and BC plans at the same time and combine them into a single plan • Must be able to reestablishment of operations at two different locations: – Immediately at an alternate site – Eventually back at the primary site
• Execution of a combined plan requires separate execution teams
Principles of Incident Response and Disaster Recov
58
Final Comments on the DR Plan • Planning process for the DR plan/BC plan should be tied to, but distinct from, the IR plan • These 3 processes should be tightly integrated to allow reaction teams to easily transition from incident response to disaster recovery and business continuity planning • Appendix B contains a sample NIST contingency plan • to keep the plan available but secure
Principles of Incident Response and Disaster Recov
59
Summary • DR planning is the preparation for and recovery from a disaster • Disasters can be classified by source (natural or man-made) or by speed of development (rapid onset or slow onset) • MT assembles the DR team, consisting of representatives from every major organizational unit • of the DR team do not serve on IR or BC team because of overlapping duties • DR team may consist of many subteams Principles of Incident Response and Disaster Recov
60
Summary (continued) • All of DR team should have multiple copies of the DR and BC plans available to them at home and office • DR policy is the first deliverable • Effective preventive controls implemented for security also facilitate recovery of information • DR plan should contain detailed procedures for restoring lost or damaged information, in 3 phases: – During the disaster – After the disaster – Before the disaster Principles of Incident Response and Disaster Recov
61
Summary (continued) • Training in the use of the DR plan can be used to test the validity and effectiveness of the plan • Testing of the plan is an ongoing activity, with each scenario tested at least semiannually at the walkthrough level
Principles of Incident Response and Disaster Recov
62