Contingency Planning at CMS 

Contingency planning at the Center for Medicare and Medicaid Services (CMS) is essential for protecting the organization from potential risks and ensuring the continuity of its operations. An Information System Contingency Plan (ISCP) is the cornerstone document of contingency planning for information systems, and every CMS FISMA system must have one in place. The ISCP provides a framework for responding to and mitigating the effects of unexpected events, such as natural disasters, data breaches, and public health crises. 

ISCPs outline risk management strategies, such as crisis management protocols, data backup and recovery procedures, business continuity plans, and roles and responsibilities. The plans generally include one or more of the following approaches to restore disrupted services:

• Restoring information systems using alternate equipment in case of an equipment failure
• Alternate data processing means 
• Alternate location(s) in case of a natural disaster 

Contingency planning also involves establishing clear communication channels between CMS and its stakeholders, such as healthcare providers, patients, and the general public. By being prepared for potential risks, CMS can ensure that its operations remain uninterrupted and that its stakeholders are kept informed of any changes. CMS utilizes guidance provided by the National Institute of Standards and Technology (NIST) SP 800-53 and the Federal Information Systems Management Act (FISMA) to inform its internal contingency planning process.

ISCP Testing, Training and Exercise (TT&E)

System/Business Owners are required to schedule and perform Testing, Training, and Exercise (TT&E) for their ISCPs annually. They must also oversee the development and completion of corrective action plans for vulnerabilities noted during the testing. Exercising an ISCP ensures that in the event of a system failure, the system team is prepared to take the steps necessary to protect security and privacy. 

To make sure that CMS FISMA systems can recover from outages or issues, it's important that everyone knows what they need to do, has been trained on how to fix problems, and that those solutions are tested to make sure they actually work. Therefore, every System/Business Owner and Information System Security Officer (ISSO) will implement a robust TT&E program for contingency planning. Your system’s impact level will determine the specific requirements of your TT&E program. As you develop and complete your TT&E, you will also need to update your ISCP as new information becomes available and changes to your system occur. 

A successful TT&E program should include several types of events to ensure the availability of a wide range of methods for validating various planning elements in the context of cyber incidents.

Tests

Tests are evaluation tools that use quantifiable metrics to ensure that a FISMA system or system component is functioning properly. A test is conducted in as close to an operational environment as possible; if feasible, an actual test of the components or systems used to conduct daily operations for the organization should be used. The scope of testing can range from individual system components or systems to comprehensive tests of all systems and components that support the ISCP. Tests often focus on recovery and backup operations; however, testing varies depending on the goal of the test and its relation to a specific plan.

Training

Training allows personnel to understand their roles and responsibilities within a system’s ISCP. Training opportunities teach staff skills such as decision making and offer information about best practices. It prepares the them for participation in exercises, tests, and actual emergency situations related to the ISCP. Training is typically split between a presentation on roles and responsibilities, and activities that allow personnel to demonstrate their understanding of the subject matter.

All training should be coordinated by and centrally documented with the ISSO. Training must include, but will not be limited to the following:

• Emergency response best practices
• Disaster declaration criteria and declaration authorities
• Functional recovery prioritizations and Recovery Time Objectives (RTOs)  of interdependent systems
• Validation of the approved recovery strategies and strategy implementation
• Verification of ISCP implementation procedures
• Validation of recovery personnel assignments, roles and responsibilities

ISCP Coordinators must develop a training program for all personnel assigned to recovery responsibilities within the ISCP. Training must be provided within 90 days of assignment to recovery responsibilities with refresher training conducted at least annually thereafter.

Exercises

An exercise is a simulation of an emergency designed to validate the viability of one or more aspects of an ISCP. In an exercise, personnel with roles and responsibilities within the ISCP meet to validate the content of the plan through discussion of their roles and their responses to emergency situations, execution of responses in a simulated operational environment, or other means of validating responses that does not involve using the actual operational environment. Exercises are scenario-driven, such as a power failure in one of the organization’s data centers or a fire causing certain systems to be damaged, with additional situations often being presented during the course of an exercise. 

The purpose of exercising an ISCP is to identify and fix deficiencies in the system itself and the overall planning process. ISCPs are not exercised to test the technical competence of personnel with recovery responsibilities. Exercises do serve as training for personnel who will be called upon to execute the ISCP in the event of a system outage. Exercises should include the following areas:

• Notification and escalation procedures
• System recovery on an alternate platform from backup media
• Internal and external connectivity
• Actual operational functional support from the recovered system
• System restoration
• Smooth resumption of normal operations

At CMS, there are two main types of exercises used to validate ISCPs: 

Tabletop Exercises

Tabletop exercises are discussion-based exercises where personnel meet in a classroom setting or in breakout groups to discuss their roles during an emergency and their responses to a particular emergency situation. A Facilitator presents a scenario and asks the exercise participants questions related to the scenario, which initiates a discussion among the participants of roles, responsibilities, coordination, and decision-making. A tabletop exercise is discussion-based only and does not involve deploying equipment or other resources. 

The primary goals of a successful Tabletop Exercise are: 

• Validation of Recovery Time Objectives (RTOs) and functional Maximum Tolerable Downtimes (MTDs)
• Validation of response and recovery procedures
• Guidelines and procedures for coordinated, timely, and effective response and recovery
• Call tree information verification
• Discovery of any weaknesses in the ISCP
• Verification of recovery procedures

Functional Exercises

Functional exercises allow personnel to validate their operational readiness for emergencies by performing their duties in a simulated operational environment. Functional exercises are designed to exercise the roles and responsibilities of specific team members, procedures, and assets involved in one or more functional aspects of a plan (e.g., communications, emergency notifications, IT equipment setup). Functional exercises vary in complexity and scope, from validating specific aspects of a plan to full-scale exercises that address all plan elements. Functional exercises allow staff to execute their roles and responsibilities as they would in an actual emergency situation, but in a simulated manner.

A successful Functional Exercise achieves the following goals:

• The ability to continue functional processing in backup mode
• Application/system interdependencies and data flow verification
• Compatibility of data backups with the primary and backup systems
• Data storage and recovery processes
• The ability to extend the system to users at alternate processing and telework sites

Selecting the correct exercise for your system

The type of exercise selected should reflect the FIPS 199 level of the system.

• Low-impact systems can be tested with a Tabletop Exercise
• Moderate-impact systems should undergo a Functional Exercise
• High-impact systems must utilize a full-scale Functional Exercise (also known as a Technical Exercise)  with system failover to the alternate site if required

Note: Actively exercising the system ISCP as part of a larger, coordinated technical exercise of the hosting system satisfies the annual requirement.

Developing your ISCP Exercise Plan

Developing a realistic and efficient ISCP Exercise is critical to the success of your system’s ISCP in the event of an outage. Because ISCP Exercises occur only once or twice a year, it’s important that a ISCP Exercise Plan is created and reviewed prior to each exercise. This ensures that all information is accurate and relevant, and that all roles on the team remain accurate. The ISCP Exercise Plan is approved by the System/Business Owner prior to the event. All exercise plans must include:

• An identified Exercise Facilitator for central management during the exercise
• Observers/Monitors for objective exercise evaluation
• Exercise participants
• Exercise objectives
• Exercise metrics to determine how well objectives were met
• Required materials
• Exercise timeline
• Any assumptions
• Exercise scenario to include scripts and injects

ISCP Exercise Plan preparation

Before drafting your ISCP Exercise Plan, it’s important that each member of the system team has done their part to ensure that the following items have been reviewed for accuracy and completeness. 

Before drafting your system’s ISCP Exercise, the System/Business Owner must have developed and approved:

• Maximum Tolerable Downtime (MTD) of the function(s) that is/are supported by the system
• Recovery Time Objective (RTO) of the system
• Recovery Point Objective (RPO) of the associated data
• Work Recovery Time (WRT) of the associated functional processes
• An up-to-date ISCP for the system
• The type of exercise (Tabletop or Functional) in accordance with guidance from CMS and NIST
• All relevant personnel with recovery responsibilities have been trained 

The system’s ISSO will work with the System/Business Owner to complete the tasks above. Additionally, all system team members must have completed the following tasks before any ISCP Exercise occurs: 

• Review the CFACTS CP control descriptions to ensure the plan as exercised is consistent with existing control requirements and implementation descriptions; if there have been changes to control requirements, you may need to update your approved recovery strategies
• Review the documented information system and business risks for any changes to the business process MTD, threats, vulnerabilities, or likelihood of occurrence for existing threats
• Determine and plan for the necessary logistics and supplies, such as booking conference rooms, setting up Zoom calls, sourcing a white board and markers, or providing note sheets for Data Gatherers

Drafting your ISCP Exercise Plan 

Set objectives 

Objectives are brief statements that have measurable outcomes. Measurable outcomes refer to specific and observable results that can be measured using data. They provide a way to track progress and determine the success of a particular activity or project.

Measurable outcomes are typically expressed in terms of specific goals, targets, or objectives. For example, a measurable outcome for your ISCP Exercise could be to have all system staff trained on new procedures within 90 days. This outcome can be easily measured by tracking successful completion of training within the set time period. Objectives should also track: 

Maximum Tolerable Downtime (MTD) - All ISCP Exercises must ensure all functional MTDs can be met and if not, either adjust the MTD(s) or upgrade the recovery procedures to reduce the amount of time permitted for the RTO.

Recovery Time Objective (RTO) - In order to ensure functional recovery, critical systems must be recovered quickly enough to allow for system operations, data loading and validation, and backlog processing. If the system cannot be recovered quickly enough to meet the functional MTD(s) then the recovery strategy must be upgraded to reduce the time required for the RTO.

Recovery Point Objective (RPO) – If data recovery and validation are insufficient to support the functional MTD(s) then the data backup strategy must be upgraded to support a more current (shorter) RPO.

Work Recovery Time (WRT) – In order to ensure the functional MTD(s) can be met, the time it takes to validate recovered data, update all data to current day and time and clear any transaction backlogs must be addressed If an exercise determines that the functional MTD cannot be met after the system is recovered within its RTO, and the data is recovered within its RPO, then all recovery strategies may need to be upgraded.

Validation of response and recovery procedures – WRT must be validated to ensure that the RTO and the processes necessary to achieve a normal state of functionality to include transactions are properly validated and do not exceed the MTD.

Verification of call tree information – Valid names and contact information are needed. Corrections to this list should also be made to the plan document.

Identification of inaccuracies or errors in the ISCP – Any errors must be identified and corrected.

Measurable outcomes are important because they help to focus efforts, set clear expectations, and evaluate progress. By defining specific, measurable outcomes, your team can determine whether they are on track to achieving the goals identified in the ISCP Exercise Plan and make adjustments as needed to ensure success. 

Determine time frame 

Each ISCP Exercise requires two time-frames: the actual time that is set aside for the exercise (normally 1 to 4 hours of active time spread across a number of days) and the elapsed time, which is the total number of days required to complete the CP Exercise in total. The elapsed time must be of sufficient length to encompass the system RTO, data RPO and the MTD of the function that relies on the system being tested.

Identify personnel & assign roles 

Based on the objectives and time frame, determine the personnel who are required to attend your ISCP Exercise. The System/Business Owner should also identify the following individuals with recovery roles in the ISCP:

Facilitator – The exercise Facilitator is the System/Business Owner or designee. The Facilitator is responsible for:

• Obtaining approval for the ISCP Exercise Plan
• Ensuring all personnel involved with the exercise are notified
• Providing pre-exercise and post-exercise briefings as required
• Conducting the exercise in accordance with the exercise plan
• Developing the AAR

Data Gatherers – The Data Gatherers should be the ISSO, CPC or their designee(s), and other functional experts as appropriate.  They are responsible for:

• Reviewing and being familiar with all information and procedures in the ISCP
• Reviewing and being familiar with the business processes that rely on the system to be exercised
• Reviewing and being able to determine, with the participants, when recovery procedures or other information in the ISCP do not meet the requirements of an effective ISCP

Participants – Participants are personnel who have recovery responsibilities that are relevant to the scope of the exercise as determined by the Facilitator and approved by the System/Business Owner.

Note: If the exercise is a Technical Exercise, the System/Business Owner, ISSO, and CPC will also coordinate with appropriate Information Technology (IT) infrastructure personnel for technical recovery expertise.

Determine assumptions and limitations

Assumptions refer to the beliefs or predictions that the ISCP is based on. For example, an ISCP for a data breach may assume that the organization's data encryption measures are effective or that the attacker's motive is to steal sensitive information. These assumptions help shape the response plan and determine the actions to be taken.

Limitations refer to the factors that may prevent the contingency plan from being fully effective. For example, a contingency plan for a power outage may be limited by the availability of backup generators or the capacity of the electrical grid. It is important to understand these limitations in order to develop a realistic and effective plan.

Develop injects 

Injects are hypothetical scenarios that are introduced into the ISCP Exercise in order to test the plan's effectiveness and identify any potential weaknesses. Injects offer different scenarios that could happen, and the ISCP Exercise participants are responsible for figuring out how to handle those scenarios. By introducing different injects, the team can see how well the plan works and make adjustments if necessary.

For example, let's say your team is exercising your ISCP for a breach event. You might introduce an inject scenario where the breach is more severe than initially expected, or where backup systems fail. By practicing how they would respond to these scenarios, the team can better prepare for a real emergency. NIST has created the Master Scenario Events List (MSEL), an outline of the simulated events and key event descriptions that participants will be asked to respond to during an exercise. Your team can reference the MSEL when drafting the ISCP Exercise Plan. 

Set a date

Establish a day-and-time to start your ISCP Exercise. Be sure that all team members with recovery responsibilities are available to participate for the entire duration of the ISCP Exercise. Obtain final approval from the System/Business Owner and the ISSO/CPC.

Conducting your ISCP Exercise 

Once your ISCP Exercise Plan has been completed and approved, your team is ready to conduct your ISCP Exercise. A successful ISCP Exercise will have active participation from all team members and identify areas for improvement and result in actions that are taken to improve the ISCP. 

1. Ensure all personnel who have been identified in the ISCP Exercise Plan are present. For any absentees, ensure a viable replacement is present.

2. Make sure that all personnel have the required information. The Facilitator should have their own copy of the ISCP, the developed ISCP Exercise scenario, prepared injects, and evaluation sheets. Participants should come to the exercise with their own copy of the ISCP. If they do not, this should be recorded as a deficiency/finding.

3. The Facilitator will kick off the Exercise by presenting the senior participant with the initial inject.

4. The team will follow the documented ISCP step by step.

5. As the participants respond to the first inject the Facilitator leads the discussion focusing on the recovery procedures in the ISCP. They will continue this process with each subsequent inject until normal operations are restored to the system within the Exercise and the ISCP Exercise is complete.

6. Upon conclusion, the Facilitator should have a quick discussion with the Data Gatherers to determine when their notes are due. The team should then immediately begin the process of compiling documentation of the exercise using the Tabletop Exercise Scenario Template and After Action Report (AAR) Template, as well as other documentation required to address any ISCP deficiencies. 

Post-Exercise activities 

There are a number of activities that must be completed immediately following your ISCP Exercise. The most important of these activities is the After Action Report (AAR). The AAR is a comprehensive review of your completed ISCP Exercise that identifies areas of strength, areas for improvement, and lessons learned. It provides a basis for ongoing refinement of the contingency plan. This helps to ensure that the plan is always up-to-date and effective.

Teams, led by the System/Business Owner, must complete the following steps after the ISCP Exercise is finished:

1. Conduct an initial out-brief with all persons identified in the scenario and record any lessons learned in the format provided in Tabletop Exercise Scenario Template and After Action Report (AAR) Template.

2. Collect all logs and exercise-related documentation from all personnel who participated.

3. Review all narrative comments.

4. In the event of a discrepancy between two participants (or data gatherers) giving different results for the same objective, discuss the results with them and, if possible, come to agreement.

5. When all results conflicts have been resolved, develop the AAR with significant results.

6. Include in the AAR any recommendations for improvements to any area of the system’s recovery plan or overall recovery capability. 

7. Attach the completed Exercise Scenario to the AAR.

8. Submit the AAR to the Business Owner for review and approval.

9. Update the ISCP with the exercise results, lessons learned, and any comments provided by the Business Owner.

10. Update ISCP training materials to reflect necessary changes to the ISCP as a result of the exercise and lessons learned.

11. The System/Business Owner and ISSO create a Plan of Action & Milestones (POA&M) for any weakness or deficiency in the ISCP that cannot be addressed in a timely manner, e.g. prior to the next ISCP testing date. This will identify the vulnerability and plan out the corrective actions necessary to reduce the weakness to an acceptable level.

ISCP Exercise roles and responsibilities

The following system team members are involved in the ISCP Exercise process: 

• System/Business Owner
• Information System Security Officer (ISSO)
• CMS Contingency Plan Coordinator (CPC)
• ISCP Exercise Facilitator

It is critical that each member of the system team understands their role in the execution of the ISCP, as well as their responsibilities related to ISCP Exercises. 

System/Business Owners 

All System/Business Owners are the leaders of the Contingency Planning process. As a result, they are responsible for the following when exercising an ISCP:

• Develop, distribute, and maintain ISCPs 
• Ensure each plan under their purview is exercised at least annually
• Ensure a technical test for each system is conducted at least every other year
• Review and correct plan deficiencies discovered during an exercise or outage in a timely manner
• Ensure the annual ISCP Exercise includes an analysis of the identified recovery strategies to ensure recovery strategies take full advantage of all possible cost savings and efficiencies
• Obtain appropriate resourcing to include funding and staffing, for recovery planning requirements
• Ensure all personnel with recovery responsibilities are trained to consider recovery preparedness part of their normal duties
• Determine and manage information system and data backup storage and alternate processing facility agreements
• Ensure a copy of the most current ISCP is maintained at the alternate processing location

Information System Security Officer (ISSO)

ISSOs serve as the partner to the System/Business Owner throughout the ISCP process. During the ISCP Exercise, the ISSO is responsible for: 

• Assist the System/Business Owner with training for staff related to the ISCP Exercise 
• Assist the System/Business Owner in correcting deficiencies and issues discovered during the ISCP Exercise process 
• Review all information and procedures in the ISCP
• Review the business processes that rely on the system to be exercised
• Review and determine, with the exercise participants, when recovery procedures or other information in the ISCP do not meet the requirements of an effective ISCP
• Submit updated ISCP documentation and information to CFACTS 

Contingency Plan Coordinator (CPC)

The CPC assists the System/Business Owner with their CP Exercise efforts. Sometimes the CPC and ISSO roles overlap during the CP Exercise process. Your individual team – led by your System/Business Owner – will determine the appropriate makeup for your team. During an exercise, the CPC will:

• Oversee and coordinate all CP Exercises
• Oversee and coordinate the recovery-related training and awareness program for all
• personnel
• Coordinate recovery team staffing with the System/Business Owner, CISO’s office, and Emergency Preparedness and Response Operations (EPRO) Office
• Assist ISSOs in event response until it is determined that contingency execution is not warranted

CP Exercise Facilitator

The CP Exercise Facilitator is a single individual identified in the CP Exercise Plan. The Facilitator is typically the System/Business Owner, but this is not always the case. Sometimes the System/Business Owner may designate another team member to serve as Facilitator. The Facilitator is responsible for the following:

• Obtaining approval for the CP Exercise Plan
• Ensuring all personnel involved with the exercise are notified of the exercise and that they are available to participate for however long the exercise is scheduled for
• Providing pre-exercise and post- exercise briefings as required
• Conducting the exercise in accordance with the exercise plan
• Developing the AAR

Tabletop Exercise Scenario Template 

The following template provides placeholder content for a Tabletop Exercise Scenario that you can copy and paste into a document. It is for planning your Tabletop Exercise and summarizing the outcomes. It is signed by the Data Gatherer. It is submitted to the Business Owner as part of the After Action Report (AAR).

Copy and paste the information below into a document to begin planning your Tabletop Exercise.

Exercise scenario format

System:

Date:

Type of exercise:

Person(s) planning the exercise:

Exercise Facilitator(s)

Facilitator name:

Facilitator name:

Exercise Data Gatherer(s)

Data Gatherer name:

Data Gatherer name:

Exercise participants

Participant name and role:

Participant name and role:

Participant name and role:

(add more as needed)

Timelines

Actual exercise time:

Exercise (simulated) time:

Exercise objectives

Objective 1:

Objective 2:

Objective 3:

(add more as needed)

Exercise scenario

Incident:

Impact to system(s):

Impact to operation(s):

Required supplies and documentation

List supplies and documentation that will be needed for the exercise.

• Item
• Item
• Item
• Add more as needed

Assumptions

Assumption 1:

Assumption 2:

Assumption 3:

(add more as needed)

Lessons Learned

(Use this space to summarize the lessons learned from conducting the Tabletop Exercise.)

Objective fulfillment

(Use this space to summarize whether objectives were met, and to provide details.)

• Objective 1 was / was not met. Specifically;
• Objective 2 was / was not met. Specifically;
• Objective 3 was / was not met. Specifically;
• (add more as needed)

Evaluation sheet

Objective 1: (re-state the objective here)

Comments:

Objective 2: (re-state the objective here)

Comments:

Objective 3: (re-state the objective here)

Comments:

Signature

_____________________________________

Data Gatherer’s name 
 

_____________________________________

Data Gatherer’s signature and date

After Action Report (AAR) Template

The following template provides placeholder content that you can copy and paste into a document to create your After Action Report (AAR). This is a comprehensive review of your completed CP Exercise that identifies areas of strength, areas for improvement, and lessons learned.

Copy and paste the information below into a document to begin your After Action Report. Then modify the details for your specific CP Exercise.

Introduction

A Tabletop Exercise was conducted for the <System Name (system acronym)> Information System Contingency Plan (CP) on <date>. 

Participants

The participants and their assigned roles are listed below.

Exercise Facilitator (Facilitates the CP Exercise and develops the AAR)

Name:

Organization:

Phone:

CP Coordinator (Ensures accurate damage assessment and system recovery)

Name:

Organization:

Phone:

Exercise Data Gatherer (Determines whether recovery procedures meet the requirements of an effective CP)

Name:

Organization:

Phone:

Recovery Management Team Member (Ensures accurate damage assessment and system recovery)

Name:

Organization:

Phone:

<System Name> Technical Lead (Ensures system is recovered to trusted state and verifies all processing and data integrity)

Name:

Organization:

Phone:

Scenario

The CP tabletop exercise was conducted in accordance with the <System Name> CP Exercise Plan, dated <date>. The exercise plan was developed around the following scenario:

<Synopsis of the scenario>

The exercise was developed to determine the following objectives:

• Determine weaknesses in the Contingency Plan
• Objective 2
• Objective 3
• <Add additional objectives as necessary>

The CP exercise evaluated the status of contingency planning for the system and provided a forum for identifying outdated contingency planning information and for providing updates as required.  The exercise plan and detailed results are contained in the Appendix to this report.

Summary of Exercise Results

Significant results from the exercise were:

• <Result one>
• <Result two>
• <Result three>
• <Add additional results as necessary>

Recommendations

The following recommendations are provided as a result of the exercise:

• <Recommendation one>
• <Recommendation two>
• <Recommendation three>
• <Add additional recommendations as necessary>

Signature

_____________________________________

Facilitator’s name and date

_____________________________________

Approved by and date

_____________________________________

System/Business Owner’s name and date

<System Acronym> System/Business Owner, <title>

Following this report, insert Appendix material as necessary (such as the exercise plan and any supporting documentation.)