 |
| Systems fail because engineers protect the wrong things, or protect the right things in wrong ways. - Ross Anderson, Security Engineering [Art by Shibara] |
Recently, yours truly ran into the use of analytical wargaming as a methodology to address policy problems at the intersectionality of AI risks, cyber security and international politics. While the critics of wargaming do espouse some valid points, it is an exceptionally useful tool when it comes to tackling a very specific kind of problems - where you need to not predict actions but rather anticipate the outcomes and consequences of actions. Because rationality doesn't work well with incomplete information, a lot of predictive research methods don't always work well for these kind of problems either. As forecasters say, quantification is useful but wisdom is more useful.
Anyways, system engineering principles suggest that it is best practice to build those components first which are most likely to fail or cause failure, so let us first look at the caveats generally associated with wargaming:
Reproducibility
A bit like the game of chess. While a bunch of independent games may proceed differently, they'd cause over time and iterations the general characteristics inherent in the game to emerge clearly. Wargaming is experimental, but not strictly an experiment. If we approach it as a simulation as it is, the lack of micro-level reproducibility turns out to be a feature and not a bug.
Rare Events
But a catastrophic risk is infact a very rare event. And the thing about such risk scenarios is that everything is chaotic and nobody knows the truth. Thus gamification based exercises in this context will only simplify reasoning about system behavior, so when the crisis actually happens, decision-makers can navigate the chaos without acting from a point of anxiety and fears.
Players Matter Too Much
Yes. This is indeed the biggest constraint which will shape how we design our game. Player expertise is paramount and it is best to design a game which explores questions relevant to those expertise. And gaming a crisis is not easy to do, especially without player commitment and appropriate preparations. So game designers and institutions need to atleast agree upon a baseline player capability.
 |
| A basic how-to-go-about gaming crises |
In a large sociotechnical environment, sometimes reliable systems can have unreliable components. Identifying those components requires us to develop conceptual models of interaction in that environment. This is exactly what wargaming does, it would reduce the strategic indeterminacy which burdens the AI risk governance efforts. I mean chaos engineering is nice and useful, but mission-critical systems involving intelligent robotics or autonomy and cyberspace need to be as deterministic as possible.
In coming days, edge robots would not only need to gather and store but also trend, time-series, and analyse their data. They would also have their own CI/CD peculiarities. As it is even today, old components in complex legacy products are not easy to replace as there often are third-party microcontrollers and actuators which are no longer in production. Gamification can help us find those weak links. It can also help decision-makers conceptualise differently, for example helping high-level politicians and planers better understand swarms as interoperating IoT instances and not just as lots of flying drones.
 |
| Slide from a presentation by Anja van der Hulst, research on forecasting crisis decision making behavior suggests role-play simulations outperforms other methods |
Mainly the main rationale for gamification is to discover structure of solutions - what elements are relevant for policy decisions, what unknown relations exist between these elements, how do player motivations arise and shift, how do actors (and agents) interact, what are the influences of time and exertion etcetera and so on. The Formula-1 teams for example make a really good use of simulations to harden safety. They simulate the vehicle and all its components in almost a digital-twin like fashion, running containerized virtual systems. These simulations reduce time-intensive testing and provide a good cognitive model for engineers and managers to confront unexpected failures. The AI risk community at present is anyways mostly concerned with the hypothetical impacts of statistical mistakes. Crisis-gaming is great for just that sort of thing.