Showing posts with label cyber. Show all posts
Showing posts with label cyber. Show all posts

Saturday, February 1, 2025

Infrastructuring AI in the Postcolony

The root of all evil is a premature policy optimization.

In the few decades of the history of computing in India there have been some spurts of proactivity, but by and large other than a large bodyshop in IT, we have been comfortable in a mode largely reactive to global developments, with an industry making negligible contribution to computing technologies. Just after liberalisation, C. R. Subramanian had undertaken a long-view analysis of the matter in his "India and The Computer" - with many of his suggestions ringing true to folks in AI today. Let me therefore reconstruct below the difficulties that were noted over 30 years ago for large scale "harmonious development" (not support and maintainance) of computing and software products in India:

1. The domestic markets in India are too small. To make a mark, we've to address the international high-technology demand, which means society being at the cutting-edge of technology and R&D. Considering quick internet based distribution, open-sourcing and market standardisation, any catch-up oriented strategy would have to fight against strong network effects as well. This will be very hard for domestic industry to do in presence of established foreign competition without some protections until they reach a certain stage of technological and business maturity. 

2. Subramanian clearly recommends 'standardisation first' to be one possible technology strategy, but he also says, "...there is a total lack of concern for this at the highest levels of the administration. Political support has not even been sought for this vital step." Based on my own interactions with the Bureau of Indian Standards and other IT sector actors, 30 years later Subramanian's characterisation of embedding political vision in India's technical standardisation strategy has not changed much. Stock policy formats of "one nation one..." lack compatibility with global standardization environment - one way to address this would be to dissociate IT standards from BIS (which comes under Consumer Affairs) and give it directly to MeitY and technology industry stakeholders bodies who have skin in the game.

3. He says, "the private sector in India does not traditionally invest in high-risk, new technology areas." One of the main reasons for this, perhaps owing to our socialist traditions, is that the industry actors (at least the smaller ones) are not seen with much trust and respect. Hans-Peter Brunner notes that 1984 was perhaps the first time since independence when domestic industry actors in India were given on paper a “respectful place”. The little industry guy is not really accorded the respect of a 'partner' in public-private interactions, not to mention of the babu expectations of profound subservience. Resultingly, the industry has stuck to relatively risk averse and by-the-book activities, having neither the financial incentives nor the resourcefulness for high-technology risks/innovation. 

4. Infrastructural ownership, he says is a "must from the point of view of national security and related developments in the space and defense areas. To allow the World Bank a say in the matter is inviting foreign interference in domestic technology aspirations related to self-reliance." Considering our situation in the semiconductor supply chains and an unwritten policy to export brains (the human infrastructure), and now also with data and platforms, who really has digital infrastructural ownership in India?

5. He notes that the "inconsistency in the various policies — fiscal, licensing, technology development and technology import - prevent harmonious development". The natural evangelist of technology in government anywhere is the military, which faces a policy conundrum in India between satisfying its immediate requirements for finished products vs sticking it out for the development of the "indigenous" industry - unless in view of Atmanirbharta it could take a more strategic role in domestic technology sector. I may remind the reader here that the true union of science with industry happened only in the WWII, which in the US resulted into a cultural osmosis where the military picked up management perspectives and businesses picked up military ones.            

But how soon can these difficulties be overcome? In a world where technology infrastructures are often intermingled with social and political imaginations, a mere exorcism by diplomacy and public relations cannot address India's currently vacuous strategy for technological leadership. Our national security as well as digital technology governance being run predominantly through a policing lense has not halped our cause either. Firstly, I must say that it is not the technical but the management standards in our organisations (including govt and startups) which need to catch up with the engineering talent.   

Thence, the AI hype can be appealing to broken institutions - and can lead quickly into a procurement mentality where run-of-the-mill, off-the-shelf analytics and interaction products (even LLMs wrapped into brands) are acquired in the name of promoting and enabling the AI research ecosystem. That is like getting a website made to enable the development of future internet. Shoppers have to shop, but without deeply trusting your own people, letting them not go, and persevering with them on novel ideas, the technological catch-up would not end very soon. Perhaps it is also time for us to deploy legally a broader definition of CSR finance to include certain kinds of R&D as well.

Consider also, that organisational innovations are needed to integrate technology as a doctrinal component at the highest levels. We do not need an "AI policy" as much as we need to formulate a multi-stakeholder industrial policy that integrates and addresses different sector-specific demands (and expectations) of deploying automation, data processing and cyber security. This is a must for national competitiveness, security and defense, and cannot simply be exorcised by witty diplomats.  

Mbembe writes on his experience in postcolonial Africa, that often the actual development of infrastructure has more to do with access to government contracts and rewarding patron-client networks, and not its technical function or nuances - and some note that "this is why roads disappear, factories are built but never operated and bridges go to nowhere"- as we might sometimes relate as well. Now imagine a super-intelligent AI being developed and deployed in the purview of a similar organisational paradigm. If that is the sovereign national discretion, so be it, but it might be useful to make available trustworthy infrastructural components - standardised and open building blocks, both logical and physical, that can be integrated (and disintegrated) at base/local levels as required by anyone - as the local is not very far from the global here - for pure technology competition, for all practical purposes, implies an overall escalation of force with non-lethal effects.  


Monday, September 11, 2023

Reconciling AI Governance and Cybersecurity

  

Recently, Sam Altman had been touring the world attempting (perhaps) a regulatory capture of global AI developments. No wonder that OpenAI does not like open-sourced AI, at all. Nevertheless, this post isn’t about AI development, but its security and standardization challenges. Generally, an advanced cyber threat environment, as well as the defenders’ cyber situation awareness and response capabilities (henceforth referred together as ‘security automation’ capabilities) are both overwhelmingly driven by automation and AI systems. To get an idea, take something as simple as checking and answering your Gmail today, and enumerate the layers of AI and automation that can constitute securing and orchestrating that simple activity. 

Thus, all organisations of a noticeable size and complexity have to rely on security automation systems to effect their cybersecurity policies. What is often overlooked is that there also exist some cybersecurity “metapolicies” that enable the implementation of these security automation systems. These may include the automated threat data exchange mechanisms, the underlying attribution conventions and knowledge production/management systems. All these enable a detection and response posture often referred to by marketers and lawyers as “active defense” or “proactive cybersecurity”. However, if you pick up any national cybersecurity policy, you’d be hard pressed to find anything on these metapolicies – because they are often implicit, brought into national implementations largely by influence and imitation (i.e. network effects) and not so much by formal or strategic deliberations.

These security automation metapolicies are important to AI governance and security because in the end, all these AI systems, whether completely digital or cyber-physical, exist within the broader cybersecurity and strategic matrix. And we need to be asking whether retrofitting the prevalent automation metapolicies would serve well for the future of AI or not. 

Avoiding Path Dependency 

Given a tendency towards path dependency in automated information systems, what has worked alright so far is getting further entrenched into the newer and adjunct areas of security automation, like the intelligent/connected vehicle ecosystem. Further, the developments in the security of software-on-wheels are being readily co-opted across a variety of complex automotive systems, from fully-digitised tanks that hold the promise of decreased crew-size and increased lethality to standards for automated fleet security management and drone transportation systems. Consequently, there is rise in vehicle SOCs (Security Operations Centers) that operate on the lines of cybersecurity SOCs and use similar data exchange mechanisms, borrowing the same implementations of security automation and information distribution. That would be perfectly fine if the existing means were good enough to blindly retrofit into the emerging threat environment. But they are far from it. 

For example, most of cybersecurity threat data exchanges make use of the Traffic Light Protocol (TLP), however TLP itself is only a classification of information – its execution, and any encryption regimes to restrict distribution as intended, is left to the designers of security automation systems. Thus there is need for not just more fine-grained and richer controls over data sharing with fully or partially automated systems but also ensuring compliance with it. Much of threat communication policies like the TLP are akin to the infamous Tallinn manual, in the way that these are almost an expression of opinions that cybersecurity vendors may consider implementing, or may not. It gets more problematic when threat data standards are expected to cover automated detection and response (as is the case with automotive and industrial automation) – and may or may not have integrated an appropriate data security and exchange policy for lack of any compliance requirements to do so.

Another example of inconsistent metapolicies, out of numerous others, can be found in the recent rise of language generation systems and conversational AI agents. The thing is that not all conversational agents are ChatGPT-esque large neural networks. Most of these have been in deployment for decades as a rules-based, task-specific language generation programs. Having a “common operating picture” by dialogue modeling and graph-based representation of context between such programs (as an organisation operating in multiple domains/theaters could require) was an ongoing challenge before the world stumbled upon “attention is all you need”. So now basically we have a mammoth of legacy IT infrastructure in human-machine interface and a multi-modal AI automation paradigm that challenges it. Organisations undergoing “digital transformation” not only have to avoid inheriting the legacy technical debts but also consider the resources and organisational requirements for efficiently operating an AI-centric delivery model. Understandably, some organisations (including governments) may not want a complete transformation right away. Lacking standardised data and context exchange between the emerging and the legacy automated systems, many users are likely to continue with a paradigm they are most familiar with, not the one which is most revolutionary. 

In fact much of cybersecurity today hinges on these timely data exchanges and automated orchestration, and thus these underlying information standards become absolutely critical to modern (post-industrial) societies and the governance of cyber-physical systems. Yet, instead of formulating or harmonising the knowledge production metapolicies needed to govern AI security in a hyper-connected and transnational threat environment – we seem to be falling into the doomer traps of existential deliverance and unending uncanny valleys. That said, one of the primary reasons for the lack of compliance and a chaotic standards development scenario in security data production is the lack of a primary governance agent. 

The Governance of (Cyber) Security Information

Present automation-centric cyber threat information sharing standards generally follow a multistakeholder governance model. That means these follow a fundamentally bottom-up life-cycle approach, i.e. a cybersecurity information standard is developed and is then pushed “upward” for cross-standardisation with ITU and ISO. This upward mobility of technical standards is not easy. The Structured Threat Information eXpression (STIX), which is perhaps the de facto industry standard now for transmitting machine-readable Cyber Threat Intelligence (CTI), is still awaiting approval from ITU.  Not that its really needed, because the way global governance in technology is structured, it is led by industry and not nations. The G7 have gone to the extent of formalising this, and some even blocking any diplomatic efforts towards a different set of norms. 

This works well for those nation-states which have the requisite structural and productive capacities within their public-private technology partnerships. Consequently, the global governance of cyber technology standards becomes a reflection of the global order. Sans the naming of cyber threat actors, this still had been relatively objective by nature so far. But it is no longer true with the integration of online disinformation into offensive cyber operations and national cybersecurity policies – not only the conventional information standards can run into semantic conflicts, newer value-driven standards over information environment are also popping up. Since the production and sharing of automation-driven social/political threat indicators can be shaped by and affect political preferences, as the threats of AI generated information and social botnets rises, the cybersecurity threat information standards also slide from a sufficiently objective to a more subjective posture. And states can do little to reconfigure this present system because the politics of cybersecurity standards has been deeply intertwined with their market-led multistakeholder development.

Cyber threat attributions are a good case in point. MITRE began as a DARPA contractor, and today serves as an industry wide de-facto knowledge base for computer network threats and vulnerabilities. Of the Advanced Persistent Threat groups listed in MITRE ATT&CK, close to 1/3 of cyber threats are Chinese, another ~1/3 come from Russia/Korea/Middle-East/India/South-America etc, and the remaining ~1/3 (which contain the most sophisticated TTPs, the largest share of zero-day exploitation, and a geopolitically aligned targeting) remain unattributed. We’ll not speculate here but an abductive reasoning about the unattributed threat cluster may leave readers with some ideas about the preferences and politics of global CTI production. 

A fact of life is that in cyberspace power-seeking states have been playing the role of governance actors and sophisticated offenders at the same time, so this market led multistakeholderism has worked out well for their operational logic – promulgating a global politics of interoperability. But it is bad for the production of cyber threat knowledge and security automation itself, which sometimes can get quite biased and politically motivated over the internet. Society has walked this path long enough to not even think of it as a problem when moving into a world surrounded by increasingly autonomous systems.  

A Way Forward 

With the social AI risks looming larger, states intending to implement a defensible cybersecurity automation posture today might have to navigate a high signal-to-noise ratio in cybersecurity threat information, multiple CTI vendors and metapolicies, as well as constant pressures from industry and international organisations about “AI ethics” and “cyber norms” (we’ll not venture into a discussion of “whose ethics?” here). This chaos, as we noted, is an outcome of the design of bottom-up approaches. However, top-down approaches can lack the flexibility and agility of bottom-up approaches. For this reason, it is necessary to integrate the best of multistakeholderism with the best of multilateralism. 

That would mean rationalising the present bottom-up setup of information standards under a multilateral vision and framework. Because while we do want to avoid partisan threat data production, we also want to make use of the disparate pool of industry expertise which requires coordination, resolution and steering. While some UN organs, like the ITU and UNDIR, play an important role in global cybersecurity metapolicies – they do not have the sort of top-down regulatory effect needed to govern malicious social AI over the internet, or implement any metapolicy controls over threat sharing for distributed autonomous platforms. Therefore, this integration of multistakeholderism with multilateralism needs to begin at the UNSC itself, or any other equivalent international security organisation. 

Not that this was unforeseen. When the first UN resolution was made assessing Information Technologies in 1998, particularly the internet, some countries had explicitly pointed out that these technologies will end up at odds with international security and stability, hinting at the required reforms at the highest levels of international security. Indeed, UNSC as an institution has not co-evolved well with digital technologies and the post-internet security reality. The unrestricted proliferation of state-affiliated APT operations is just but one example of its failure in regulating destabilising state activities. Moreover, while the council seems still stuck in a 1945 vision of strategic security, there is enough reason and evidence to relocate the idea of “state violence” in light of strategically deployed offensive cyber and AI capabilities. 

While overcoming the resilience of global order and their entrenched bureaucracies is not going to be easy, if reformed in its charter and composition, the council (or its replacement) could serve as a valuable institution to fill the void that emerges from the lack of a primary agent in guiding the security and governance standards driving security automation and AI applications in cyberspace. 

It Is The Process

At this point it is necessary that we call out certain misunderstandings. It seems that regulators have some ideas about governing “AI products”, at least the EU’s AI Act suggests the same. Here we must take a moment of silence and quiet to reflect on what is “AI” or “autonomous behaviour” – and it will soon dawn upon most of us that the present methods of certifying products may not be adequate for addressing adaptive systems rooted in continuous learning and exchanging data with the real world. What we’re trying to say is that the regulators perhaps need to seriously consider the pros-and-cons of a product centric vs a process centric approach to regulating AI. 

AI, at the end, is an outcome. It is the underlying processes and policies, from data engineering practices and model architectures to machine-to-machine information exchanges and optimisation mechanisms, where the focus of governance and standards needs to be, not at the outcome itself. Further, as software shifts from an object-oriented to an agent-oriented engineering paradigm, regulators need to start thinking about policy in terms of code and code in terms of policy – anything else will always leave a giant gap between intent and implementation. 

If the aforementioned chaos of today’s multistakeholder cybersecurity governance is anything to go by, for AI security and governance we need an evidence led (consider data that led to final CTI, and engaging with new types of technical evidence) threat data orchestration, runtime verification of AI-driven automation in cyber defense and security systems, clear non-partisan channels and standards for cyber threat information governance, and a multilateral consensus on the same. Focusing on the final AI product alone can leave much unaddressed and potentially partisan – as we see from the ecosystem of information metapolicies that drive security automation systems worldwide – hence we need to focus on better governing the underlying processes and policies that drive these systems and not the outcomes of those processes and policies.

Tuesday, March 7, 2023

On Unsavoury Memeplexes

 Tigress & Serpent, Thomas Landseer

When ChatGPT was released, soon arose several continuing Chinese whispers of the online variety that expressed discontent over the AI being too "lobotomised". These are also valid concerns after all, for information (the semantic significance of it, not the syntactical expression itself), or the deliberate absence of it, can often be political. And consequently, from a range of quarters, we've seen people clamming for their ideologically or theologically consistent AIs. Times are such, even computation can get cancelled. But this tendency to fortify, arm and train their memeplexes against adversarial memes under the pressure of structural changes is a phenomena that calls for deeper exploration, for much of what drives an abundant people (those who've sufficient access to food, mates, and circus), whether state identities or culture wars, are memeplexes undergoing constant unraveling.     

State identities are a rather interesting example here, because these also carry along a bureaucratic rationalization of their adopted memeplexes. Therefore, following the principle of co-evolution of politics and technology (i.e. institutions and techniques), states in peacetime will openly pursue only the degree of cultural politics that their established bureaucratic institutions may permit - such as the US evangelising worldwide its own cultural revolution, or the Chinese digging up old religious artifacts in western Tibet to expand their claims over Indian territories, or even some babus in India conveniently forgetting to discuss all pre-invasion indo-tibetan boundary arrangements with their Chinese counterparts in the hopes of a bff-ship (effectively secularising the wider political discourse over western Tibet). The role which bureaucracies play in the evolution of state identities and corresponding memeplexes is an excellent case of the chief evil of the (sometimes necessary) institutionalisation of anything - that the process take over the purpose. During existential turmoils, the phenomena may briefly slip into the-purpose driving the-process again, but soon the newly emerged processes again tether the purpose, establishing a cultural homeostasis until the inertia is disturbed. And thus, in the absence of a clearly defined, objective and actionable political purpose, pragmatism dictates that bureaucratic rationalizations orchestrate state identities.

In 1945, after taking over Japan, the local US administration had banned the images of Mount Fuji, samurais, and even performing arts such as sword-fighting movies and Kabuki plays, while at the same time promoting pacifist literature and debating fraternisation with Japanese women. In the eyes of west, Japan's collective mindset at the core was feudal, requiring considerable reeducation and civilizing for democratic functioning. And much of this grand effort came under the perview of an "Information Control Division". Over the decades the naming conventions have gotten a lot more politicaly correct, i.e. it'll be scandalous for a public department to be named Information Control Division these days, Ministries of War have become Ministries (and Departments) of Defense and so on. But bureaucratic institutions embody patterns of state behaviour, and changing their names does little to alter human affairs and organisational logics. The Twitter files might suggest that Information Control Division lives on, and Ukraine turning into West's collective proxy might indicate that the ministries of war too, live on. Survival requires old-timer memes to not only remain relevant but also interact competitively or even freeride with other memeplexes. The freeloading memeplex is a thing of beauty, a beast of prey in the realm of ideas. Like a parasite it benefits from susceptible memeplexes, and often us biological hosts have little recourse to deal with such epenthetic memes. Glorious ideals like The Law, Justice, and Freedom et cetera are all also great examples of such memes. 

Since we were speaking of state identities, justice too is an interesting meme to consider. Just a couple of centuries ago, if someone had harmed your loved ones, you might have sought to take revenge, maybe even banding together your community and friends into it. After all, revenge has its evolutionary logic - credible retaliation prevents future indiscretions i.e. an eye for an eye does not make the whole world blind, instead the arrangement ensures that people avoid poking into another's eye. But irrespective of the role which revenge plays in human affairs, and plays much greatly in state and international affairs, if someone harmed your loved ones today, it is expected that you'd not be going out seeking revenge on your own. Instead, you'd go to the police and the court (irrespective of their effectiveness) and hope that "The System" will take your revenge for you. Here is a meme that is treated differentially at different levels of society - for indignity and a lost sense of kindred honour, a man pursuing his revenge could be too barbaric and patriarchal in today's world, but the sovereign doing so is a "strategic imperative". 

This is infact the very essence of The Law among the sovereigns. Consider that for the unparalleled specialisation in enforcing his will, in the jungle the lion is the law. Over his subjects within his territory, the lion may exercise his jurisprudence, if he has any. That jurisprudence forms the law of the land, the constitution and other sacred documents of the people living in that land. But the meme of justice takes a U-turn when the matter to be resolved is not within a lion's territory, but among the lions that hold such territories. Our salaried lions then do not say that XYZ lion is breaking the law, instead, they say that XYZ lion is disturbing the order. Just with different communities of underlying hosts, the expected behavior around the memeplex shifts from being almost mathematically formalised to being merely customary. Moreover, modern memeplexes are also partly driven by and dependent upon constant connectivity and a (computer) networked society, and thus the bureaucratic rationalization of our world necessitates reigning in the unsavoury memeplexes via technical means of the computational variety as well. 

How unsavoury is your memeplex? 

Often such unsavoury memeplexes could originate from and target the lions themselves - an example is cyber threat information sharing about these memeplexes. A lot of cyber security discourse today (including the just released US' National Cybersecurity Policy) include online influence and disinformation threats. Some "do-gooders" at NATO's StratComCoE are also trying to extend the existing threat information sharing standards and languages to cover these online informational threats. However, such information sharing involves a fair share of subjectivity and semantic conflicts (some of which I go over here) which could make releasing such "memetic threat intelligence" a bit of political activity as well, as opposed to sharing conventional cyber threat information. It'll be a diplomatic soup if states or their CERT-type organisations do it, so there is an precipitating growth in dependence on private sector proxies - online platforms, OSINT organisations, and commercial information vendors - to do what in the old days would have been done by foreign services and information control divisions.    

Clausewitz, of all the philosophers, had rightly remarked that every period carries its own cultural grammar for war. And thus even decades old conflict today are being fought under a new cultural grammar in Ukraine. Often the biological hosts' vainglorious assumptions of modernity dictate that they sanitise and compartmentalise their memeplexes - change the content of books, redesign classics, listen to musicians not music, so on and so forth. Even after hundreds of thousands of years of human existence, information producers - whether it is Tulsidas or Roald Dahl - need to be retrofitted into "The 21st Century", and ironically even the 21st century AIs too. Which also triggers me to invoke Stanely Kubric's absolute masterpeice - 2001: A Space Odyssey - where the protagonist eventually wins the race of evolution by "lobotomising" his fellow AI. Interestingly, much of the popular AGI risk discourse today is effectively synthesised into the memeplex of that one Hollywood movie.     

Perhaps all of this also points at the default human tendency to let their beliefs be coloured by their desires. Buddha, as some Chinese communist babus may also be grappling with, postulated desire to be the root of suffering. But would the underlying biological substrates of living memeplexes ever be able to shun their desires and see things as they really are? I'd say never, for that is our prison, which Czeslaw Milosz captured beautifully when noting that men will continue to clutch at illusions when they have nothing else to hold to.   

Wednesday, July 21, 2021

On the utility of crisis-gaming AI risks

Systems fail because engineers protect the wrong things, or protect the right things in wrong ways.  - Ross Anderson, Security Engineering [Art by Shibara]

Recently, yours truly ran into the use of analytical wargaming as a methodology to address policy problems at the intersectionality of AI risks, cyber security and international politics. While the critics of wargaming do espouse some valid points, it is an exceptionally useful tool when it comes to tackling a very specific kind of problems - where you need to not predict actions but rather anticipate the outcomes and consequences of actions. Because rationality doesn't work well with incomplete information, a lot of predictive research methods don't always work well for these kind of problems either. As forecasters say, quantification is useful but wisdom is more useful.

Anyways, system engineering principles suggest that it is best practice to build those components first which are most likely to fail or cause failure, so let us first look at the caveats generally associated with wargaming: 

Reproducibility
A bit like the game of chess. While a bunch of independent games may proceed differently, they'd cause over time and iterations the general characteristics inherent in the game to emerge clearly. Wargaming is experimental, but not strictly an experiment. If we approach it as a simulation as it is, the lack of micro-level reproducibility turns out to be a feature and not a bug. 

Rare Events
But a catastrophic risk is infact a very rare event. And the thing about such risk scenarios is that everything is chaotic and nobody knows the truth. Thus gamification based exercises in this context will only simplify reasoning about system behavior, so when the crisis actually happens, decision-makers can navigate the chaos without acting from a point of anxiety and fears. 

Players Matter Too Much
Yes. This is indeed the biggest constraint which will shape how we design our game. Player expertise is paramount and it is best to design a game which explores questions relevant to those expertise. And gaming a crisis is not easy to do, especially without player commitment and appropriate preparations. So game designers and institutions need to atleast agree upon a baseline player capability.     

A basic how-to-go-about gaming crises

In a large sociotechnical environment, sometimes reliable systems can have unreliable components. Identifying those components requires us to develop conceptual models of interaction in that environment. This is exactly what wargaming does, it would reduce the strategic indeterminacy which burdens the AI risk governance efforts. I mean chaos engineering is nice and useful, but mission-critical systems involving intelligent robotics or autonomy and cyberspace need to be as deterministic as possible.

In coming days, edge robots would not only need to gather and store but also trend, time-series, and analyse their data. They would also have their own CI/CD peculiarities. As it is even today, old components in complex legacy products are not easy to replace as there often are third-party microcontrollers and actuators which are no longer in production. Gamification can help us find those weak links. It can also help decision-makers conceptualise differently, for example helping high-level politicians and planers better understand swarms as interoperating IoT instances and not just as lots of flying drones.

Slide from a presentation by Anja van der Hulst, research on forecasting crisis decision making behavior suggests role-play simulations outperforms other methods 

Mainly the main rationale for gamification is to discover structure of solutions - what elements are relevant for policy decisions, what unknown relations exist between these elements, how do player motivations arise and shift, how do actors (and agents) interact, what are the influences of time and exertion etcetera and so on. The Formula-1 teams for example make a really good use of simulations to harden safety. They simulate the vehicle and all its components in almost a digital-twin like fashion, running containerized virtual systems. These simulations reduce time-intensive testing and provide a good cognitive model for engineers and managers to confront unexpected failures. The AI risk community at present is anyways mostly concerned with the hypothetical impacts of statistical mistakes. Crisis-gaming is great for just that sort of thing.


Saturday, December 29, 2018

Misdirections & The Internet

"My sculptures are like no other known geometrical solid, but they look just like a cube, a pyramid or another known or absurd solid. This means that three observers at three different points would see three different solids." 
- Guido Moretti

The more intently we are occupied with something, the harder our system works to suppress distractions. Sometimes, we are skillfully distracted away from the real thing. Perception is constructive, following a predictive processing framework, which underlies why illusions occur. But our intent here is not to talk about magic, we'll do that some other time. Here we want to talk about the sometimes coercive influencing of perceptions, the skillful tunneling of our attention, the actors who may use that and the overarching cyber-enabled theater of these tricks.

Our actors may be groups/activists/hackers/guerillas/state/non-state/whatever with a political or reformist agenda. As per their situation, they may or may not use other and often more violent means of action. Our theater can be better conceptualized as a multi-agent system where each agent (actor) always maintains some local state carrying a representation of all the information which the actor has access to. Capabilities drive the actor intentions, but cultural ideologies, psychological and sometimes monetary benefits, as well as network externalities are primary drivers of behavior in this "theater".

There is emerging a lot of material on Russian and Chinese disinformation as well as hacking campaigns, it is arguable that this "material" itself is a countering mechanism of sorts. We don't know lots for sure so let us take another example, from the epic Mahabharat, day 15 of the Kurukshetra war. This because of two reasons, first that no one can object to the truth or falsehood of a mythology, and second, it makes a great example since we can see various elements of coercive influence operations in a single instance.

At the conceptual level, we can see multiple information classes and the leveraging of unique channel characteristics, also a great display of how an influential node can amplify emotions or curb resistance while building greater trust. There are different communication & influence paths, with a continuous notion of bombardment frequency and intensity. The core of disinformation was repeated thrice to Drona by different actors, while changing the nature of appeals with lots of fighting in between. This while the "celestials" were driving Drona's emotions and reasoning away from fighting, there is coinciding improvisation of inducement strategy throughout the operation. And most importantly from a cyber-driven perspective there is fake-forensics (dead elephant) to avoid inconsistency and leave a false trail backed by true evidences, which also allows for plausible deniability as well as perceptual ambiguity towards any bad self-knowledge among actors. Everything from indirect incitements to direct actions, interactions are directed towards the overall operational objective of attacking Drona's willingness to continue fighting by having him distracted and occupied with a falsehood.

The great strategist and the diplomatic finesse of neither fighting nor not-fighting. Image©watchalen
It is well known in general warfare, the general superiority of defense over attack, all things being equal. And even though military strategy is the domain of taking on the stronger opponent and taking on a weaker persistent opponent is what politicians were for, the advent of cyber-enabled operations has somewhat upended the traditional notions, including but not limited to the ideas of proportionality. David Deptula, when talking about armed drones once, noted that an actor can now project power without projecting vulnerability. This maximization of projected power while minimizing the projected vulnerability and the borderless quasi-intangible nature of cyber-enabled information operations, is markedly the defining shift in the conduct of war, arguably somewhat more threatening to the open-internet societies than the ones behind the firewalls.

Nevertheless, for big-brothers with many eyes, intelligence over internet traffic and telecommunications is only the baseline of information gathering. And with greater state capabilities, come innovations from the underground also. We are seeing ideological actors coordinating as leaderless and self-organizing trans-national networks, using less tech and even riding on international relations. Such systems (of networks) over time can become somewhat fault tolerant & resilient to a degree, if the environment is not shaped against them.

Sometimes actor actions can be attributable, but sometimes not attributable, and sometimes even misattributed. It is difficult to set expectations from future engagements if the prominent actions are a misdirection. With a leaderless system architecture comes the autonomy of peripheral attacks which puts the spotlight back over plausible deniability and perceptual ambiguity. We may see strategic shifts as stable actor behavior in such environments, even leveraging of crises and critical political events to widen any sense of distrust, doubt, disagreement and disarray among the adversary.

Come what may, our unconditional security proofs often rest over idealized assumptions. These illusions of security prevent us from making systemic changes. This may range from us not upgrading our vulnerable devices to our dear leaders overlooking contiguous societal necessities, like climate change for example. This is a pervasive human folly, we like the browsing experience so we don't turn off the JavaScript. And there is no best solution to this but akin to much of Sun Tzu tapestry is a Bruce Lee solution, be like water, LOL.