Cyber Incident Exercises - Part 6: The Psychology of Exercising
This bonus article expands on our recent series on cyber incident exercises. This article dives into academic research on psychology, pedagogy, and game development to explore how different exercise formats can deliver different learning experiences for participants.
Key points:
Different exercise formats will encourage different kinds of learning outcomes for participants.
The effectiveness of live-play exercises depends on creating a simulated environment that is close enough to the real thing that the participants’ experience in one will transfer to the other.
The effectiveness of table-top exercises depends on giving the participants the time and flexibility to engage with the subject matter at a critical and conceptual level.
Introduction - different formats, different outcomes
A key theme running through this series has been the importance of clearly specifying what an exercise is intended to achieve. Once this has been documented and metrics established, the exercise organisers can align the format, design, and scenario of the exercise to achieve these objectives. In the second article in this series we introduced the difference between table-top and live-play exercises, noting that these formats tend to align with different objectives.
In this article we will dive into research on the use of exercises for educational purposes. Doing so will reveal how the difference between table-top and live-play exercises aligns with two different forms of learning.
Exploring the outcomes possible through exercises
The different outcomes that might be achieved through exercises, simulations, and games are explored in the work of Peader Callaghan, a leading expert on the design of cyber security games and exercises. The graphic below is adapted from Callaghan’s master’s thesis.
In the graphic, the first question is whether the participants in the exercise have agency, understood as the ability to make choices that meaningfully affect the state of the exercise.
If the answer is no, then the exercise is what Callaghan terms a ‘message broadcaster’ - the purpose is a one-way transfer of the exercise designer’s understanding of the scenario being modelled. In such an exercise, participants would effectively be walked through what to do, or what not to do, in an incident.
Where exercises allow meaningful agency for the participants, Callaghan’s flowchart asks another question: “are the choices focused on speed of thought or deep processing/making connections?”
If the answer is the former - speed of thought - then the focus of the game is on practice. Such games work best when the participants already have some experience in the activity that the game is simulating. If the answer is the latter - deep processing or the making of new connections - then the game is aimed at conceptual transfer.
The first type of exercise asks the participants to respond in real time to events, drawing on skills that they are expected to have already. The other type is more considered, asking participants to discuss and engage in critical thinking. Drawing on work from psychology, Callaghan terms this ‘conceptual transfer’.
Applying this to cyber incident exercises
Looking back to Callaghan’s graphic, this would suggest that live-play exercises are likely to prioritise practice, while table-top exercises are likely to prioritise deep thinking. A live-play exercise happens in real time in a simulated environment that closely resembles the real thing. A simulated network environment can allow cyber security personnel to practise responding to incidents and threat hunting in a safe space that nonetheless closely parallels the ‘hands on keyboard’ experience of the real thing.
In contrast, a table-top exercise is more abstracted, allowing for discussion and reflection on the scenario as it develops. In such an exercise, the goal would be less to simulate the real systems and processes currently in place, and more to create the conditions to enable a group of participants to think deeply about how the world could be, challenging their own ideas, and drawing connections between seemingly unrelated ideas.
Another way of framing the distinction is through what Callaghan - following Perkins and Salomon - refers to as ‘low road’ and ‘high road’ transfer:
“Low road transfer requires that the task being as close to how it is going to be applied in real life. Examples of this kind of learning transfer are flight simulators and other VR training games. In contrast, conceptual transfer requires the participant to think about and engage with the material that has been presented.”
A key point about low road transfer is that it tends to be to some extent automatic or reflexive. The similarity between a flight simulator and the cockpit of a real aircraft means that a person familiar with one will rapidly apply the same skills to the operation of the other. The similarity to the live-play exercise format is clear.
Conversely, as Perkins and Salomon argue, high road transfer:
“depends on mindful abstractions [...] and a deliberate search for connections [...] Such transfer is not in general reflexive. It demands time for exploration and the investment of mental effort.”
Designing a table-top exercise to enable this kind of thinking is challenging and requires real buy-in from the organisation. Finding the right people, carving out the time to enable them to focus on the game, and creating the conditions to help them step outside of their day-to-day models and test them in a productive, inclusive environment is no mean feat. However, this effort can pay dividends if the objective of the exercise is to promote serious critical thinking about an organisation’s approach to responding to cyber incidents.
Learning points:
Exercises can achieve different forms of learning outcome; aligning format with the exercise objectives is therefore crucial.
Live-play exercises test the participants’ speed of thought in responding to a cyber incident.
These exercises are therefore better suited for practising key skills. Fidelity is therefore key to ensure ‘low-road’ transfer.
Table-top exercises instead encourage participants to think deeply about their response to a cyber incident.
Fidelity is less important in these exercises than creating the conditions for deep thought and engagement, promoting ‘high-road’ transfer.
Tyburn St Raphael is experienced at delivering cyber incident exercises. Whatever your goals, Tyburn St Raphael’s seasoned cyber incident exercisers will work with you to deliver the right exercise for your organisation. You can learn more about our cyber incident exercising offering here [executive TTX] or get in touch with us at info@tyburn-str.com.
