Case study: Millennium Challenge 2002 (Part Two)
Outline:
The MC ‘02 exercise provides a valuable case study of the challenges that arise in the design and conduct of exercises.
The representation of the adversary within the scenario is another instructive area to examine.
The ‘Red’ force was initially represented as an intelligent adversary that attempted to achieve its goals in a way that circumvented the Blue force’s advantages.
The capabilities available to the Red force initially reflected a credible assessment of the capabilities available to the real militaries that the exercise simulated.
Over the course of the exercise, this portrayal of the adversary was undermined by interventions that forced the Red team to act in unrealistic ways or denied them the use of existing capabilities.
This undermined the credibility of the exercise scenario, even when taking into account the exercise’s hypothesis of a more advanced ‘Blue’ force in the future.
In a separate article, we write about the importance of scenario design and the credibility of the adversary.
Introduction
The first part of this article introduced MC ‘02 and the controversy it provoked. We explored how this case underlines the importance of ensuring that exercise objectives are well-defined and clearly communicated across an organisation.
In this second part we will explore a second aspect of this case study: the depiction of the adversary. In part one we noted that actions such as refloating the Blue team fleet or assuming the success of the aerial landing made sense in the context of a resource intensive exercise intended to validate future concepts. However, in this section we will argue that the adversary depicted in MC ‘02 did not represent a credible and realistic threat - undermining the value of the exercise.
Portraying threats in an exercise
As a minimum, an exercise should portray a realistic adversary behaving intelligently. In this context, intelligence does not mean assuming a high level of critical thinking, education, or training. Rather it means that the adversary will actively pursue its own objectives, changing its behaviour if necessary to counter the defender’s actions.
Security is by its nature adversarial. A security incident is one in which there is ultimately a human adversary attempting to do your organisation harm, however many technological intermediaries there may be between them and you. That adversary is intelligent in the sense that they can plan and respond to stimuli, and hence they will not seek to make the defender’s life easy.
As well as recognising the adversary’s intent and ability to adapt, the exercise should also portray a realistic threat. In practice, organisations will face a range of threat actors from the highly capable to those that pose little practical threat. The choice of threat actor will be determined by the exercise objectives. Ideally it will reflect current threat intelligence about the actual actors targeting the organisation.
However, the format of exercising - with its challenges and resource requirements - also imposes certain guardrails around the choice of threat actor. There is little reason to conduct an exercise testing an organisation’s ability to defend against an inconsequential threat actor or against one that is effectively unstoppable. Exercises should therefore explore whether a credible adversary, operating using a credible set of tactics, techniques, and procedures, can achieve its goals despite the organisation’s security.
An unrealistic adversary undermines the goals of exercising
MC ‘02 began with an adversary with a realistic level of capability that acted intelligently. However, as the exercise progressed this was neutered, with the organisers forcing the adversary to act in an unintelligent manner and denying the Red team the use of capabilities that would be readily available to a real opponent.
Examples of this include forcing the Red control team to move military platforms out of concealed positions into open areas where they would be easier to target. Similarly, denying the Red team the ability to target the aircraft involved in the aerial landing enabled that part of the exercise to go ahead as required, but reflected an unrealistic assumption about the adversary’s actions.
Contrary to the claims of the exercise’s supporters, the scenario did not depict a technologically overmatched adversary facing the US military. Rather, it depicted a technologically overmatched adversary facing the US military and fighting in such a way as to maximise the effectiveness of that technological overmatch. In practice, a real opponent would seek to circumvent those advantages - a point that was made clear by decades of counterinsurgency operations in Iraq and Afghanistan, and in subsequent conflicts.
Conclusion
Organisations face two challenges in ensuring that exercises depict a credible adversary. The first is gaining accurate information on the intent and capabilities of real threat actors. At the very least, there should be a degree of external support and challenge in the creation of the threat scenario that informs the exercise. The proportion of organisations with the security and intelligence maturity to autonomously generate a credible cyber threat scenario is likely to be very small. This is one reason why most organisations work with external specialists to develop credible exercising scenarios.
The second challenge is less tangible but no less serious. Ensuring that the adversary is credible - and in particular that it acts intelligently - requires a commitment from the exercise participants and stakeholders to allow the scenario designer to present the adversary in this way. Doing so risks calling into question organisational strategy, processes and technology, spending decisions, and a host of other issues where participants and stakeholders will often be deeply invested in the status quo. It is easier and more comforting to develop an exercise scenario where the threat actor does what we hope they will and does not target our weaknesses. The benefit of exercising should come from preparing to respond to the most unexpected, damaging, disruptive lines of attack that an adversary could credibly pursue.
In a separate article, we write about the importance of scenario design and the credibility of the adversary.
Learning points:
A security incident by definition involves a human adversary that is seeking to do harm to the organisation.
Modelling that adversary is key to the credibility and effectiveness of any exercise.
The portrayal of the adversary should reflect a realistic assessment of the adversary’s capabilities.
The adversary should also behave intelligently, pursuing goals and changing behaviour as required to maximise its advantage.
Accurately reflecting adversary behaviour requires access to current intelligence and other specialist capabilities - most organisations will rely on external partners for this information.
Exercises are valuable when they depict a realistic adversary, behaving intelligently in a way that provides a strenuous test of an organisation’s security and response plans.
