Cyber Incident Exercises - Part 2: How - the Format and Level
This article is the second in a series on cyber incident exercises. This article explores two of the key questions that should be addressed early in the planning of any exercise.
Key points:
The design of a cyber incident exercise will be strongly determined by the exercise’s objectives.
There are two main formats for cyber incident exercises: table-top exercises and live-play exercises.
Exercises are generally conducted at one or more of three levels: the board level, the managerial level, and the operational level.
Introduction - exercise design is driven by requirements
The previous article in this series highlighted cyber security risk and introduced cyber incident exercises as a way of addressing this risk. We emphasised the importance of clarifying the objective of an exercise and establishing metrics to measure its effectiveness.
Having established the exercise’s objectives, the next step is to determine the format and level of the exercise. The exercise’s objectives will determine which parts of the organisation should participate and the manner in which they participate.
Exercise format
There are two main formats of exercise: table-top and live-play exercises. In the previous article in this series, we noted that a distinguishing feature of an exercise is that participants perform roles in the simulated environment of the exercise that they would expect to perform in real life. What differentiates a table-top and live-play exercise is the degree of fidelity with which that activity is simulated within the exercise.
Live-play exercises emphasise technical fidelity. The UK government notes that live exercises are “designed to test individuals and teams in real time, using techniques, drills and equipment in as close a replication of real events as possible.” The US National Institute of Standards and Technology (NIST) notes that in functional exercises (the term used for live-play exercises) participants can experience “performing their duties in a simulated operational environment.”
In contrast, table-top exercises emphasise abstraction. The UK government guidance notes that table-top exercises are “discussion based” and “typically used to examine, explore, and familiarise”. Participants are still exercising roles that they would be expected to perform in a real incident, but the format of the exercise is intended to achieve a different form of engagement. For this reason, table-top exercises are not generally conducted in real time, to allow for more considered discussion and engagement, and they do not require a high-fidelity environment simulating the ‘real thing’.
Exercise level
Exercises can be conducted at different levels within an organisation. The NCSC distinguishes exercises based on three levels:
Board level
Managerial level
Operational level
The choice of level shapes the types of activities that the exercise will involve. For example, a board-level exercise will involve assessing the strategic impact of the incident and engaging with external legal and regulatory bodies. In contrast, an operational-level exercise will be focused more on technical aspects of the detection, triaging, and response to the incident.
The appropriate level will thus be strongly determined by the objective of the exercise. If the objective is to test the operational effectiveness of the processes and technology used to detect and triage an incident then involving board-level participants would likely be unnecessary. Conversely, if the purpose of the exercise is to practise engaging with regulators, then the technical processes around incident detection may not need to be fully simulated within the exercise.
Exercises can be conducted that combine layers, for example combining operations with management, or combining all three levels. However, combining multiple levels adds complexity and introduces its own challenges, and so should not be assumed to be the default. If the focus is on one level, then it is usually easier to simulate the other levels than to incorporate them into the exercise. As always, the main thing is for the design to be shaped by the specific objectives of the exercise.
Further considerations
The format and level of an exercise will also partially determine the size of the exercise. Exercising across multiple levels will involve bringing together participants from across the organisation. The emphasis on fidelity in exercises means that participating teams need to provide a representative number of participants - the credibility of an exercise will suffer if one participant is expected to perform work that in a real incident would involve a large team.
In turn, the resource cost of the exercise will be closely related to the exercise’s format and size. As a rule, live-play exercises will be more resource intensive, because of the need to prepare a simulated but realistic environment in which participants can engage with and respond to the cyber security incident. At the operational level this will involve the preparation of technical infrastructure.
In general, a smaller table-top exercise will be less resource intensive. However, it is worth considering the opportunity cost of bringing together a group of senior board or managerial level participants for the duration of even a short exercise. A live-play exercise involving a large number of participants across multiple levels of an organisation will be an expensive activity, and hence will need to be justified as the best means of achieving the exercise’s objectives.
The next article in this series will examine the design of the scenario for the exercise itself, underlining the importance of understanding both the external threat environment and the organisation’s internal state.
Learning points:
An exercise’s format and level should be determined by the exercise’s objectives.
Live-play exercises simulate the working practices of participant functions with a high degree of fidelity.
Table-top exercises use abstraction to enable a more discursive and exploratory approach to exercising.
Exercises can be conducted at one or more of the following levels: board, managerial, and operational.
Combining levels introduces additional challenges and must be justified by the exercise objectives.
The format and level of an exercise will affect its size and its resource requirements.
Tyburn St Raphael is experienced at delivering cyber incident exercises. Whatever your goals, Tyburn St Raphael’s seasoned cyber incident exercisers will work with you to deliver the right exercise for your organisation. You can learn more about our cyber incident exercising offering here [executive TTX] or get in touch with us at info@tyburn-str.com.
