Tuesday, March 7, 2023

On Unsavoury Memeplexes

 Tigress & Serpent, Thomas Landseer

When ChatGPT was released, soon arose several continuing Chinese whispers of the online variety that expressed discontent over the AI being too "lobotomised". These are also valid concerns after all, for information (the semantic significance of it, not the syntactical expression itself), or the deliberate absence of it, can often be political. And consequently, from a range of quarters, we've seen people clamming for their ideologically or theologically consistent AIs. Times are such, even computation can get cancelled. But this tendency to fortify, arm and train their memeplexes against adversarial memes under the pressure of structural changes is a phenomena that calls for deeper exploration, for much of what drives an abundant people (those who've sufficient access to food, mates, and circus), whether state identities or culture wars, are memeplexes undergoing constant unraveling.     

State identities are a rather interesting example here, because these also carry along a bureaucratic rationalization of their adopted memeplexes. Therefore, following the principle of co-evolution of politics and technology (i.e. institutions and techniques), states in peacetime will openly pursue only the degree of cultural politics that their established bureaucratic institutions may permit - such as the US evangelising worldwide its own cultural revolution, or the Chinese digging up old religious artifacts in western Tibet to expand their claims over Indian territories, or even some babus in India conveniently forgetting to discuss all pre-invasion indo-tibetan boundary arrangements with their Chinese counterparts in the hopes of a bff-ship (effectively secularising the wider political discourse over western Tibet). The role which bureaucracies play in the evolution of state identities and corresponding memeplexes is an excellent case of the chief evil of the (sometimes necessary) institutionalisation of anything - that the process take over the purpose. During existential turmoils, the phenomena may briefly slip into the-purpose driving the-process again, but soon the newly emerged processes again tether the purpose, establishing a cultural homeostasis until the inertia is disturbed. And thus, in the absence of a clearly defined, objective and actionable political purpose, pragmatism dictates that bureaucratic rationalizations orchestrate state identities.

In 1945, after taking over Japan, the local US administration had banned the images of Mount Fuji, samurais, and even performing arts such as sword-fighting movies and Kabuki plays, while at the same time promoting pacifist literature and debating fraternisation with Japanese women. In the eyes of west, Japan's collective mindset at the core was feudal, requiring considerable reeducation and civilizing for democratic functioning. And much of this grand effort came under the perview of an "Information Control Division". Over the decades the naming conventions have gotten a lot more politicaly correct, i.e. it'll be scandalous for a public department to be named Information Control Division these days, Ministries of War have become Ministries (and Departments) of Defense and so on. But bureaucratic institutions embody patterns of state behaviour, and changing their names does little to alter human affairs and organisational logics. The Twitter files might suggest that Information Control Division lives on, and Ukraine turning into West's collective proxy might indicate that the ministries of war too, live on. Survival requires old-timer memes to not only remain relevant but also interact competitively or even freeride with other memeplexes. The freeloading memeplex is a thing of beauty, a beast of prey in the realm of ideas. Like a parasite it benefits from susceptible memeplexes, and often us biological hosts have little recourse to deal with such epenthetic memes. Glorious ideals like The Law, Justice, and Freedom et cetera are all also great examples of such memes. 

Since we were speaking of state identities, justice too is an interesting meme to consider. Just a couple of centuries ago, if someone had harmed your loved ones, you might have sought to take revenge, maybe even banding together your community and friends into it. After all, revenge has its evolutionary logic - credible retaliation prevents future indiscretions i.e. an eye for an eye does not make the whole world blind, instead the arrangement ensures that people avoid poking into another's eye. But irrespective of the role which revenge plays in human affairs, and plays much greatly in state and international affairs, if someone harmed your loved ones today, it is expected that you'd not be going out seeking revenge on your own. Instead, you'd go to the police and the court (irrespective of their effectiveness) and hope that "The System" will take your revenge for you. Here is a meme that is treated differentially at different levels of society - for indignity and a lost sense of kindred honour, a man pursuing his revenge could be too barbaric and patriarchal in today's world, but the sovereign doing so is a "strategic imperative". 

This is infact the very essence of The Law among the sovereigns. Consider that for the unparalleled specialisation in enforcing his will, in the jungle the lion is the law. Over his subjects within his territory, the lion may exercise his jurisprudence, if he has any. That jurisprudence forms the law of the land, the constitution and other sacred documents of the people living in that land. But the meme of justice takes a U-turn when the matter to be resolved is not within a lion's territory, but among the lions that hold such territories. Our salaried lions then do not say that XYZ lion is breaking the law, instead, they say that XYZ lion is disturbing the order. Just with different communities of underlying hosts, the expected behavior around the memeplex shifts from being almost mathematically formalised to being merely customary. Moreover, modern memeplexes are also partly driven by and dependent upon constant connectivity and a (computer) networked society, and thus the bureaucratic rationalization of our world necessitates reigning in the unsavoury memeplexes via technical means of the computational variety as well. 

How unsavoury is your memeplex? 

Often such unsavoury memeplexes could originate from and target the lions themselves - an example is cyber threat information sharing about these memeplexes. A lot of cyber security discourse today (including the just released US' National Cybersecurity Policy) include online influence and disinformation threats. Some "do-gooders" at NATO's StratComCoE are also trying to extend the existing threat information sharing standards and languages to cover these online informational threats. However, such information sharing involves a fair share of subjectivity and semantic conflicts (some of which I go over here) which could make releasing such "memetic threat intelligence" a bit of political activity as well, as opposed to sharing conventional cyber threat information. It'll be a diplomatic soup if states or their CERT-type organisations do it, so there is an precipitating growth in dependence on private sector proxies - online platforms, OSINT organisations, and commercial information vendors - to do what in the old days would have been done by foreign services and information control divisions.    

Clausewitz, of all the philosophers, had rightly remarked that every period carries its own cultural grammar for war. And thus even decades old conflict today are being fought under a new cultural grammar in Ukraine. Often the biological hosts' vainglorious assumptions of modernity dictate that they sanitise and compartmentalise their memeplexes - change the content of books, redesign classics, listen to musicians not music, so on and so forth. Even after hundreds of thousands of years of human existence, information producers - whether it is Tulsidas or Roald Dahl - need to be retrofitted into "The 21st Century", and ironically even the 21st century AIs too. Which also triggers me to invoke Stanely Kubric's absolute masterpeice - 2001: A Space Odyssey - where the protagonist eventually wins the race of evolution by "lobotomising" his fellow AI. Interestingly, much of the popular AGI risk discourse today is effectively synthesised into the memeplex of that one Hollywood movie.     

Perhaps all of this also points at the default human tendency to let their beliefs be coloured by their desires. Buddha, as some Chinese communist babus may also be grappling with, postulated desire to be the root of suffering. But would the underlying biological substrates of living memeplexes ever be able to shun their desires and see things as they really are? I'd say never, for that is our prison, which Czeslaw Milosz captured beautifully when noting that men will continue to clutch at illusions when they have nothing else to hold to.   

Saturday, June 11, 2022

A Networked Peace

"To live effectively is to live with adequate information."
 
Norbert Weiner had maintained that nature has an inherent tendency to degrade the organized. The organism, according to him, is opposed to death and disintegration "as message is to noise" - postulating a perpetual conflict between the self-organising and the organised. This is evident everywhere. Take, for example, the rules based world order. The thing about complex adaptive systems is that given their tendencies towards self-organisation, they generally find a way to work around any and all rules. The former then has to reassert itself to turn the self-organising into the organised, as long as it can. Thus the first secretary general of NATO had defined peace as keeping the Russians out, the Americans in, and the Germans down - and the secretary general of NATO today would continue to insist upon the same peace. But undoubtedly in an arena staged upon competing networks, one man's peace is bound to become another's war and vice versa.    
 
Notwithstanding, how do we ensure that in this struggle between the self-organising and the organised, a structure evolves and not destructs. Mr. Weiner had suggested that since all organisations are held together by communication, effective communication engineering is the key to steering society in a purposeful direction. Communication effects control. That is indeed the most fundamental insight of cybernetics, whose father understood society only through the study of the messages and communication facilities which belong to it. Thus the way to create a social structure is to restrict and regulate the information flows in that society, to destroy that structure is to destroy or divert those information flows, i.e. create new information flows. That is what the internet did too.
 
Enter digital networks and artificial agents. The internet hasn't lived upto the expectations of the founders of computing. JCR Licklider, for example, had prophesied that in the information age, unemployment would disappear from the face of the earth because of the opportunities offered by "an infinite crescendo of on-line interactive debugging". Seems unlikely to happen. A lot of the computing giants couldn't foresee the future that their dream machine, AI, would bring aboard while the humanity, globally and gently, sleepwalks into an era of perpetual warring and a painful transformation of the nation-state.
 
In 2011, a startup named Pacific Social Architecting Corporation, figured if they could change these information flows using online socialbots, they'd be able to alter the social architecture of a community. They tried and while their experiments met with some success, their startup did not. Almost two decades later, researchers writing for Nature Communications describe the use of similar software agents to produce "information gerrymandering" - forcing an asymmetric assortment of influence over an unsuspecting population and creating the majority illusion for specific nodes. Even in their highly-cited paper on social botnets, Yazan Boshmaf and others had suggested early on that each social bot in an online social network is going to be capable of two kinds of influence operations - those related to social interaction, and those related to the social structure. And we are only now witnessing the true meaning of that. To think about it, using networks of bots to create a very elaborate deception, even military deception, looks fairly doable with present developments in natural language and image processing. Unsurprisingly, some researchers did gone ahead and demonstrated the use of a guided social botnet to infiltrate and cultivate specific employees of targeted organisations(!).
 
Since communication effects control, the problem of Command and Control (C2) of operating such social machines becomes a problem of architecting a resilient and credible flow of information to effect specific organisational behaviours. The older methods of botnet command and control are no longer even suitable in this environment, as the use of social networks inevitable brings in a P2P architecture (i.e. each software agent can send encrypted commands to the rest of the network). We are thus seeing some very interesting C2 designs when it comes to bot deployments - from the use of smart contracts and blockchain to implement the C2 to embedding bot commands into Bitcoin transactions, we seem to be headed for interesting times in the cyberspace. To anyone still skeptical, do checkout this report on an IoT botnet that not just hacks your networked devices but also creates and operates socialbots using that infrastructure, even including a Machine Learning component that can be turned on or off depending upon the human users' responses on social media platforms.
 
Other than coordinating the bot network, identifying and prioritising the human users to reach using bots is also another challenge. These days, digital social networks enable a large number of unintended users to join an unfolding event i.e. the ongoing Russo-Ukrainian conflict. To those having to watch random networks form and dissolve from the existing scale free asymmetric networks of our modern human existence, following things must be highlighted about the human network to manipulate this "network-centric" environment:  
     -Which nodes have the decision rights?
     -What nodal decision rights can be accessed using software agents?
     -What is the pattern of interaction between these and other nodes?
     -What kind of inflow-outflow of information emerges from these networks? 

Coming back to the 1992 social architecting experiment, a key question that arises is whether such strategic use of software agent networks can overturn the importance of the existing key nodes in the human digital networks today? The key nodes are the information junctions, a kind of intelligence device within the network. Being at the junction, these nodes have power to regulate information flows, directly affecting a lot of other nodes, which also gives these a disproportionate amount of power over the rest of the network. It must be noted here how the information domain comes into a very intimate embrace with the cognitive and the political domains.

The original software robot was designed to provide an integrated and expressive interface to the internet as well as to change its behavior and choice of actions in response to transient system conditions. A coordinating network of such programs acting socially brings to table an important non-human agency in human affairs. Since computers today "record, relay, represent and inform our responses" to almost all political-military-economic conflicts as they unfold - today's digital networks potentially enable much coercive and destructive potential in the hands of robots. Our digital societies, as well as the their dominant military-industrial-media networks, both face the everyday dangers of interconnectivity i.e. networked terrorism, recurrent misinformation cascades, and computer/biological viruses to name a few. As Buddha used to say, everything is interconnected and life is suffering. In retrospect, it is a tribute to Turing that “bot or not” is turning into one of the defining questions of our hyper-connected age as the animal and the machine networks slowly renegotiate the structure of a new peace.

P.S. Credits to Scott Wolcott for the inspiration.
 

Wednesday, January 5, 2022

Metapolitics

Does your program have too many dependencies/relations with the outside environment?
 
Recently a question appeared on a prediction forum asking whether a nationalized AGI research group would arrive at AGI before the private sector. This is interesting because most AGI research is done privately at the fringes of AI community. Whether or not, and how well, dedicated nationalized organizations would be pursuing AGI is in some respects a little ill concerting as this has a metapolitical component - it agrees implicitly that AGI is great, it only differs on in whose custody it is indeed really great. 
 
Perhaps it is too early to speculate over how the computing and communication technologies have altered, or could alter, the size and quality of a state's political organization. Political organizations unfortunately are not software, you cannot simply download an upgrade to patch some issues and load even more additional features and functionalities. People will experience anxieties when structural changes lead to institutional changes, some of them will resort to violence, and then that violence will become a negotiation tactic for the politician. This. Happens. Everytime. In that sense the people are also a strategic dummy.  
 
Davidson & Rees-Mogg predicted that the computing and communication revolution would cause organized predatory violence to slip out from central control, and everyone except politicians will benefit from the death of politics. But they have their biases, I have mine. In August 1991, the hard-line coup plotters couldn't shut down Yeltsin's communications in Moscow because he had just acquired a cool new technology - the mobile phone. The history consistently suggests that computing and communication technologies have largely benefited the politicians. Carroll Quigley, writing in Weapons Systems and Political Stability, approached technologies in terms of their defensive/offensive nature - suggesting more that offensive power means larger, more intense political organization. Unfortunately, he died before his book could reach the computing age. Artificial Intelligence for example, could be described as a centralizing force, but a technology like that could hardly be assessed as inherently offensive or defensive. How will it then affect the political organization of modern nation-states if instead of the Deep Minds and the OpenAIs of the world, a nationalized organization happens to be the first one to develop AGI? 
 
Most of us live in a security bubble detached from the blood and gore of war, but it takes not long for things to slip back - Bosnia and plenty other places have proven time and again that the law is a consensus and no consensus lasts long enough to become The Law. That is why even if we hate them passionately, society needs politicians to negotiate that consensus on behalf of the crowd. Would an AGI make any difference to this, I can't really say. And about that AGI, well, its uncertainty is not the same as its low probability. The actual nature and requirements of AGI are yet to be discovered and its behaviors are yet to be added. Perhaps yet another job post seeking software architects with 108 years of ML experience needed for "extensible design" is in the works.

Tuesday, October 12, 2021

Rethinking GIS

Sambo sivam, jagame thandiram.
The deregulation of Indian geospatial data sector may have opened opportunities for some leeway in India’s strategic framework, though of course slight reorientations maybe required at comfortable places. The strategic security outlook over GIS still has some traces of the older days of Bhaskar and Rohini satellites where data handling rate used to be in few kilobytes. It maybe somewhat questionable to some, but we need to put the GIS’ software and hardware infrastructure at the very centre of a technology centric foreign policy and not at the peripheries of a shy and reticent overture towards being a global power. 
Some key recommendations:

  • Build regional cooperation over co-developing GIS assets and regional mapping capabilities, this would go a long way in redefining the nature of our neighbourhood.
  • Inter-governmental programs around GIS thus would also take own geography into account, meaning enhanced regional outreach of developmental programs using access, interoperability, and portability centric solutions.
  • We have to develop a horizontal and non-hierarchical work ethic in the organisations tasked with radical technical innovation, especially in the satellites-sensors-software triangle. It has worked wonders for many, it will be good for us too.
  • In the spirit of previous point, we need to create and globally promote open-ended and open-access GIS technology designs and standards. This cannot be done without seamless cooperation with academia and industry.
  • Again, in the spirit of previous point, the government needs to jointly develop a framework with the academia and GIS industry to enable them to independently pursue joint R&D goals with their counterparts in other countries.
  • We need to formulate a national data standardisation policy, which includes geo-tagging of all data objects in commercial as well as government sector. This way the existing information infrastructure can be easily transformed into a GIS based developmental and strategic platform.
  • Related to above point over standardisation, there should be a single source of “analysis ready” GIS data which can cut down painstaking data preparation, standardisation and fusion pipeline on the user side. Perhaps a cloud-native ML based geospatial processing service could help monitor and facilitate the whole thing.  
  • Whatever it takes, including stealthy reverse engineering or building edge AI capabilities into orbital platforms, we’ve to aggressively push Indian GIS capabilities to a level where it can be turned into a non-negotiable aspect of any joint space missions. 

Today the information acquisition costs are relatively democratised, what is wanting is the capacity building to augment the existing service based industries with earth observation. That would not only provide the impetus for the creation of a homegrown Indian earth observation industry, but also enable the strategic thinking to integrate the vagaries and vulnerabilities of our existence and its environment into how we run our geopolitics and strategic security.

Afterall the slowly evolving physical infrastructure of the internet, along with GIS assets on space and ground, and technologies like Artificial Intelligence – are forming the landscape of a new kind of territory where the pecking order has not yet been fully established. And therefore constructing India’s strategic interests as firmly embedded into the machine needs to go from perhaps an engineering problem to a political solution.

Wednesday, July 21, 2021

On the utility of crisis-gaming AI risks

Systems fail because engineers protect the wrong things, or protect the right things in wrong ways.  - Ross Anderson, Security Engineering [Art by Shibara]

Recently, yours truly ran into the use of analytical wargaming as a methodology to address policy problems at the intersectionality of AI risks, cyber security and international politics. While the critics of wargaming do espouse some valid points, it is an exceptionally useful tool when it comes to tackling a very specific kind of problems - where you need to not predict actions but rather anticipate the outcomes and consequences of actions. Because rationality doesn't work well with incomplete information, a lot of predictive research methods don't always work well for these kind of problems either. As forecasters say, quantification is useful but wisdom is more useful.

Anyways, system engineering principles suggest that it is best practice to build those components first which are most likely to fail or cause failure, so let us first look at the caveats generally associated with wargaming: 

Reproducibility
A bit like the game of chess. While a bunch of independent games may proceed differently, they'd cause over time and iterations the general characteristics inherent in the game to emerge clearly. Wargaming is experimental, but not strictly an experiment. If we approach it as a simulation as it is, the lack of micro-level reproducibility turns out to be a feature and not a bug. 

Rare Events
But a catastrophic risk is infact a very rare event. And the thing about such risk scenarios is that everything is chaotic and nobody knows the truth. Thus gamification based exercises in this context will only simplify reasoning about system behavior, so when the crisis actually happens, decision-makers can navigate the chaos without acting from a point of anxiety and fears. 

Players Matter Too Much
Yes. This is indeed the biggest constraint which will shape how we design our game. Player expertise is paramount and it is best to design a game which explores questions relevant to those expertise. And gaming a crisis is not easy to do, especially without player commitment and appropriate preparations. So game designers and institutions need to atleast agree upon a baseline player capability.     

A basic how-to-go-about gaming crises

In a large sociotechnical environment, sometimes reliable systems can have unreliable components. Identifying those components requires us to develop conceptual models of interaction in that environment. This is exactly what wargaming does, it would reduce the strategic indeterminacy which burdens the AI risk governance efforts. I mean chaos engineering is nice and useful, but mission-critical systems involving intelligent robotics or autonomy and cyberspace need to be as deterministic as possible.

In coming days, edge robots would not only need to gather and store but also trend, time-series, and analyse their data. They would also have their own CI/CD peculiarities. As it is even today, old components in complex legacy products are not easy to replace as there often are third-party microcontrollers and actuators which are no longer in production. Gamification can help us find those weak links. It can also help decision-makers conceptualise differently, for example helping high-level politicians and planers better understand swarms as interoperating IoT instances and not just as lots of flying drones.

Slide from a presentation by Anja van der Hulst, research on forecasting crisis decision making behavior suggests role-play simulations outperforms other methods 

Mainly the main rationale for gamification is to discover structure of solutions - what elements are relevant for policy decisions, what unknown relations exist between these elements, how do player motivations arise and shift, how do actors (and agents) interact, what are the influences of time and exertion etcetera and so on. The Formula-1 teams for example make a really good use of simulations to harden safety. They simulate the vehicle and all its components in almost a digital-twin like fashion, running containerized virtual systems. These simulations reduce time-intensive testing and provide a good cognitive model for engineers and managers to confront unexpected failures. The AI risk community at present is anyways mostly concerned with the hypothetical impacts of statistical mistakes. Crisis-gaming is great for just that sort of thing.


Sunday, March 28, 2021

The Narcissism of Technical Differences

A couple of years ago, the US quietly put a worldwide export control on convolutional neural network based geospatial imagery software, the reason being its potential use by foreign militaries. It is worth noting that these are commercial software products which if open-sourced, their hardware dependencies aside, would not be treated under international jurisprudence as products but instead as free-speech. Which brings us to the subject of export controls, dual-use emerging technologies, and legacy international institutions.
 

Technology Export & Information

 
Export-control in itself is an old and important tool of statecraft, serving economic as well as military functions. However the present dichotomous classification of technologies based on military and non-military usage is severely outdated and in fact only holds-up when dealing with conventional weapons. AI in itself isn't a weapon but an enabler and having AI-superiority proves hugely advantageous when extending general capabilities across all kinds of defense systems and platforms, giving the machine learning software-stack and hardware accelerators a strong military utility.  

To deal with the problematic usage of such dual-use technologies, the most prominent international regime is the Wassenaar Arrangement. The arrangement, with respect to its fundamental role of helping to prevent the malicious use of technology, is pretty ineffectual when it comes to Artificial Intelligence and its ICT based applications. The de-territorializing effects of cyberspace present a clear institutional, regulatory, and compliance gap, where arrangements like Wassenaar only end up imposing yesterday’s standards over tomorrow’s technology. For example the first time when Wassenaar was invoked, it targeted software during the 90s to stop international adoption of cryptographic techniques over things like e-mail communications. That turned out to be a lot more than just diplomatic failure.  

And Wassenaar isn't the only international regime facing difficulty with technological changes. The MTCR (Missile Technology Control Regime) for example has been updated to include long range UAVs and will need future changes sooner than later. It must be made clear that these controls do not work as intended. There is always great difficulty in enforcing contracts in emerging markets, there is an ever-present tradeoff between transparency and secrecy, and moreover in an environment of deception the compliance can not be truly verified even if states bring (they do) their own technical means of verification. This is a grand version of what game theorists call a POSG (Partial Observation Stochastic Game), really fascinating and quite intractable stuff.

A classic legend, of dual-use exploiting institutional ambiguities, sitting in the ruins of a medieval empire

Scholars have duly noted the inherent ambiguity in Wassenaar Arrangement, especially when dealing with intangible dual-use technologies. There is no consensus on whether the arrangement could adapt to technologies which are both socioeconomically foundational and militarily significant, without ironically turning itself into a dual-use multilateral instrument of coercive and economic statecraft. Such concerns are not unfounded in international politics, take for example when in 1993 the Americans got the Russians to cancel the transfer of cryogenic technologies to India in exchange for some US-Russia space cooperation. Software being fundamentally information, presents more compounded challenges for export controls given the largely unregulated flow of information across borders. Basically the trouble being that if you put sanctions over the export of fishes to someone, failing to prevent him from learning to fish is a foreign policy failure. Not surprisingly therefore, a recent US AI-policy report which declared advertising-technologies as "NatSecTech", contained that "Export controls should be utilized... to slow competitors’ efforts to develop indigenous industries in sensitive technologies with defense applications... (by) targeting discrete choke-points."

Proliferation & The Undefined

 
The Mahabharat is absolutely filled with instances of specialized and more devastating tools being limited-by-distribution to only select few, though those familiar with the text would concur that this non-proliferation regime did little to control privileged actor behavior and eventually an utterly morbid devastation followed up. Multilateral export control regimes today pursue a deterrence by denial framework towards global security. However, in case of ICT based goods when export controls suppress economic interests, proliferation shifts to a black market, even between member states, on shady darknet sites.

It takes no genius to see that legally binding restrictions on AI capabilities will be economically debilitating, so it is going to be really hard to enforce sovereignty over technology or even know which exact technologies to control. Let us consider a hypothetical scenario: Most of the power consumption (>90%) in AI operation happens due to what can be described as "data movement operations" — adjusting features, weights, biases, transferring intermediate results etc — so it is understandable that a better power management technique/toolkit could make the systems significantly more energy efficient. Now consider that in a not-too-distant future these computations are happening in autonomous edge devices fielded on both sides amid a lengthy military standoff in a remote inhospitable region such as the deserted upper reaches of the Himalayas. Would an innovative intangible power model here be treated the same way an advanced battery technology is treated? What if someone strategically open-sources it?
 
Neural Network Power Consumption

Given that even technologies used today in public transportation, agriculture and pollution monitoring, such as LIDAR and hyperspectral tracking, will be very easily directed to military usage; several researchers have suggested to use the phrase "omni-use" to describe AI and other related emerging ecosystem. It should be clear to whomsoever-it-may-concern that they cannot control the flow of omni-use intangible items and neither can it be ascertained decidedly what or from who an omni-use technology must be kept. In an area like this, traditional controls will only bring about shadow regulations and more of other unsavory economics.
 

Way Forward, Institutional Corrigibility

 
Most countries with strong R&D focus prefer their weapon-systems to be approached as black-boxes, not to be reversed engineered or subjected to exhaustive component analysis. On another side, in future insurgencies and urban conflicts, we could be seeing a lot of improvised weapon systems which utilize open-source and low cost OTS components, those which would just not be regulated with current approaches. What technologies are available to militaries greatly guides how societies evolve and technologies also have a tendency to become more efficient relatively quickly because of their near-constant evolutionary cycle of adaptation and reinvention. It is therefore fitting and important to have a better and continuous coupling with all stakeholders, and not get carried away by media headlines, trends and lobbying.

So building Institutional Corrigibility in our context can be thought of as designing mechanisms to facilitate objective stakeholders (from academia and industry) bring necessary outside correction in transgovernmental networks responsible for developing and implementing technology export controls. The multilateral international institutions must understand the difference between essential system complexity and accidental system complexity, and work every bit to minimize and first and obliterate the second.
_

Sunday, January 3, 2021

Cautious Quadrilateralities


There was the sound like a rumor without any echo of history, really beginning.

Following the recent Chinese overtures towards Arunachal, Ladakh, and Sikkim - notably sending troops, altering maps, taking land and calling for peace - there now hopefully would have enkindled some degree of sovereign urgency among Delhi's international security leaders towards a reassessment of some of our geostrategic priorities. So understandably (with some hindsight) while we might want to avoid symmetrical competition with an obviously bigger opponent and focus on emerging capabilities which could break an established stalemate and provide sustainable asymmetric advantages to constrain CCP actions in future. Delhi and even Dharamshala both understand this very well, after all. And other than the slowly simmering borders and periodic geo-economic swings, India also has a bit of a civilizational obligation to herself that Asia doesn't turn into a CCP fiefdom.

Under this kind of environment, there has lately been a resurgence and expansion of the old US-JP-AU-IN HADR (The Quad) alliance which in itself is a center of varied interests owing to its strong political proximity with maritime dominance and the welfares originating from a post-pandemic diversification of China based supply chains. There have even been some efforts to promulgate the idea that this arrangement is not against China. In all honesty, unless China itself joins the mix, any attempts to regulate the security of Indo-Pacific waters will have to take on a character which faces China as the opponent. This is a bit of a Tolstoyan case of the fate of nations being independent from the individual wills of their kings and ministers.

That said, there are valid reasons to reconsider the strategic utility of institutionalizing these emerging structures. Japan is a treaty-based US ally. Australia is a US ally which with RCEP is unlikely to break from an economic dependence on China. The US of course, is the incumbent politico-military arbiter of liberal world order. None of these shares a land boundary with or has massive trigger-happy territorial disputes with China. We do. But the quadrilateral isn't about India, it is about "Indo-Pacific". And that thick line needs more policy thickening, because given West's ideological and economic competition with China, it can be easy to surmise that the IOR or the Indo aspects of Indo-Pacific will recieve less attention than the Pacific aspects. Besides, a third of the worldwide maritime trade passes through the South China Sea that is being contested over, and the Chinese aren't exactly going to accept some fancy rules based order in strictly diplomatic manners.

It can also be expected that more and more states joining this arrangement will still try to maximize their relational benefits while minimizing the individual costs even if that isn't in the grander strategic interests of the arrangement itself. It is well known that specific goal oriented coalitions with clear objectives are better than a general-purpose bloc riding-the-waves of international waters. The military incentives here rest with the former, or in very localized regional arrangements between states trying to order their own backyards. Unless there are strong political and economic incentives, no one is going to make commitments of manpower and resources for expeditionary operations to defend foreign interests. So it can be fairly argued in this case that expansion for sake of expansion, while gaining more legitimacy in the eyes of public and possibly even some international institutions, would tend to dilute the coalitionary maritime defense potential unless carefully guarded against.

China too is aware of the possible maritime encircling and will focus on global economy to find its way out. Consequently it has revamped its foreign investment regime and launched a slew of techno-economic armamentarium including the central bank's digital currency which could in future help in renminbi internationalization, generating non-trade currency demands, and in getting the CCP a hold over China's burgeoning money networks and perhaps over the slippery oil shores as well. Considering 'all the things that need consideration' maybe the best way to think about the current stratagems is to affirmatively rephrase Derek Walcott that the sea is indeed the history.


Saturday, September 5, 2020

Fail Securely

As systems become more complex, they are also likely to fail in ways of which the exact mechanism might be harder to predict and understand beforehand. With machine-learning systems, there is a bigger problem than failure, which is exhibiting a potentially dangerous behavior as and when failure occurs, as part of the system/sub-system failure. This concern is understandably further magnified in the context of lethal autonomous systems. 

Exploitation surface of a generic ML pipeline

Given the increasingly diverse exploitation routes, the prevailing ideas suggest to have a well trained human operator who is periodically assessing whether an AI is misinterpreting its environment. But conventional human-in-the-loop mechanisms are ill-suited to handle spatiotemporal complexity. Consider for example a large number of potential targets in a tough or no geography and an extremely compressed timeline for making engagement. In this situation it is simply not feasible for the AI to refer back to the human operator every time an engagement has to be made.

If adapted suitably, the old concept of kill-boxes may produce a simple socio-technical solution to the problem of how to detach from conventional human-in-the-loop and embrace independent decision-making in military AIs while still keeping the operator to monitor incoming data. It is also in line with the existing cyber-security principles of network segmentation and granting access based on user's role/location/time etc.

By not wholly depending upon autonomous systems' ability to interpret context and limiting its "full-fledged use" within a human generated spatiotemporal compartment i.e. a kill-box, we would not only impart the AI operation a human-like non-zero probability of making high risk "alpha zero" moves, but also allow more secure failures that cannot exacerbate the larger conflict while providing all the benefits of deploying advanced autonomous technology. This is especially valid for the global-common type environments like the space and the ocean, and of course the internet too; where there is further need to research and manage AI security risks as most nations in such environments are virtually in a persistent struggle with their allies and adversaries simultaneously. 

A typical conventional kill-box. (image source: WikiLeaks)

Ideally, a military should develop its own autonomous systems instead of relying on commercial off-the-shelf products or even allies' systems for those may come with their own inductive biases and are sometimes less likely to fully support complex missions. There are obvious economic, organizational and foreign policy incentives for doing this. And most importantly it would let the machine behavior policy and AI failures be much more clearly defined and adversarially trained against in a manner that does away with insecure failures while also suiting the respective country's cultural sensibilities.

Speaking of latter, complex societies require a fair amount of organized coercion, socioeconomic incentives, cultural deterrence and mutation over a course of centuries to become eligible for the "civilization badge". So it should be natural to want your AIs to reflect those civilizational ethos. Understandably culture isn't an engineer's problem but the way technology (particularly ICT) accords vastly different modes of social interactions and restructures social and even political affairs, it sets the premise for engineering which can function as 'politics by other means'. Perhaps Kaczynski was right. And that's all the more reason to develop systems that embrace failures, and fail securely.

Thursday, March 5, 2020

Coalitions & AI

Following a previous article on multinational interoperability, some elementary inspection of complications was needed since the use of AI in the context of modern strategic systems exacerbates the pre-existing modalities governing the formation and functioning of coalitions.

Further to the differences in standards, cultures, and mutual trust, traditional international political-military alliances have the bulk of their operational interoperability challenges resulting from the diversity of doctrines, military technologies and soldiers’ skill-levels. Addition of Autonomous Weapons and Systems (AWS) greatly increases the already substantial operational complexity of these by introducing some major considerations into the alliance relations, chiefly:

THE DATA RIGHTS -
As the experiments with self-driving cars have shown, unlike the human soldiers, AWS will produce and collect huge amounts of multidimensional data, and there are obvious disputes about who’d have access to that data. The data will be of the external environment, also the AWS’ interaction data (with that environment and with other AWS including those of Allies and Non-Allies), the local learning and processing data, data on anomalies/lagging and security/cybersecurity incidents etc. These would present broader knowledge on the conflict and capabilities, and a picture which could be very useful to military planners and intelligence community. Because a lot of such data will be collected privately by autonomous agents, there are not just significant opportunities to achieve mutually beneficial sharing agreements between governments and organizations but also a whole new layer of interaction/information games between states. 

CONTESTED C3 - 
Related to who gets what data rights, is the matter of command arrangements. Decision-making, ethical guidelines, and operational procedures could differ significantly across allies when it comes to cyber-physical autonomous multi-agent systems. Therefore AI systems present a very different set of problems in alliances’ command & communication design. This also presents an opportunity for those nations which have better C3 infra and multilateral options of integration to take the upper-hand in setting the terms of the security cooperation.

BYZANTINE AI -               
The autonomous agents which display arbitrary, faulty, or malicious behavior are termed as Byzantine AI. These could be the result of enemy exploitation also. The management of such resources in an alliance requires special attention and creation of formal procedures, quick consensus mechanisms, and channels of risk communication to interact and intervene, including on behalf of allies, while avoiding any claims of mutual interference. For an example of bad interference, recently Tesla remotely turned-off the autopilot of a customer’s vehicle because some payment wasn’t made, such "security features" may have drastic consequences in armed conflicts, and so far we’ve been lacking a valid governance framework for managing byzantine assets. 

RESOURCE PROTECTION -
Assuming that most wars wouldn’t be a decisive all-out armageddon but follow some sort of political nash-equilibria and that today’s friends can turn into tomorrow’s foes, there emerge some concerns regarding maintaining certain elements of competitive advantage over your allies. There need to be modalities of ML process and intrasystem communication protection, development of mutually-agreed upon safety testing tools, and architectural modularity in the systems deployed, among probably many other things. Further work in modularity and resource protection mechanisms in an alliance would not just help solve the immediate interoperability challenges of a man-machine force, but also take us a step in the direction of being able to address the problem of technology/equipment life-cycle.

Clearly, one player signals too much.
There is a growing body on research in AI alignment which addresses the issues related to cooperation failures, AI’s lack of evolutionarily established signalling mechanisms, approaches for joint optimization, and open-source interactions between AI systems of asymmetric capabilities. All of which should be the top AI safety research questions for states looking to go deploy AI for security related tasks with their allies. And give the fact that most of these tasks are going to be actuated via edge devices, further expansion of security anti-patterns is needed in context of multi-agent learning systems.

Since AI integration is an inevitable and irreversible process, hopefully international alliances would churn out (or adopt) a framework which finds wider applicability across the spectrum of cooperation-competition activity. Also what promptly comes to mind is Napoleon’s somewhat ironic remark that if he must make war, he’d prefer to do it against a coalition. Which is still very relevant, even for security engineers and foreign policy wonks.  
__

Monday, July 22, 2019

Sadbuddhi in Sets

"We tend to forget that throughout history many decision-makers were delighted to accept “double or nothing” tactics if the odds looked sufficiently favorable." Kahn

Too constrained to maximize?
Alexander Kotov, the Chess grandmaster with eyebrows worth a queen sacrifice and probably also a KGB fellow, was quite on the mark while describing a disposition to make judgmental errors and go with a misplaced gut while being constrained with overanalysed choices. The agency of 'constrained maximizers', not necessarily always as rational as in the western conception, has its students in the arena of conflicts beyond the chess board, rooted in its potential for both serendipity as well as misfortune. A Machiavellian adage therefore rightly suggests that true prudence consists in knowing how to distinguish the character of troubles and in the choice to take the lesser evil. In Yogic philosophy there is a word for that, Sadbuddhi, referring to developing the much touted virtue of timely knowing the right thing to do - arguably resolving the groupthink/polythink syndromes that beset most decision making units. This writing is inclined to touch upon political culture and the decision making therein, so we'll follow the lens of intra-species aggression. And although coalationary intra-species aggression spans our non-relatives too, from lions and wolves to even numerous ant species among others, we'll stick to apes likes us.

The term "political culture" itself is recent, though the practices have existed throughout. Like humans, chimpanzees too have a coalationary psychology — that is an inclination to form closely structured groups/ecosystems with strategic hostility toward outgroups, and sometimes even outliers. Other than cooperation, reciprocity and role division - the division of meat and other resources among chimps follows a process of sharing in which power, sex, and violence tend to interact in rather human ways. So different groups of chimps in their territories end up having very different behaviours and social customs, developing different kind of tools and manifesting different communication mechanisms thusly. Reductionism it is, but a chimp order rises with its own crude geopolitics.

The human society too, going by Dunbar's logic, functions in sets of humans. The successful sets have and attract more members (Roman allies rebelled in 80s BC not for secession but for overt Roman citizenship), while losing few to others. This is compounded by the networked nature of an information society. Yet these sets and their members, like those of chimps, never function outside the laws of direct or indirect reciprocity towards each other i.e. I'll help you but when we meet again then you help me. This basic evolutionary dynamics makes Proportionality perhaps the only natural law of war, and other conflicts between sets.

Basis 'Proportionality', the nuclear MAD doctrines would appear as optimized bluffing, for all WMD may be CBRN weapons but not all CBRN weapons are WMD, but that discussion is for another day. As it is, all such interpretations, by nature of self-interest, are biased and susceptible to 'syndromes of the self' i.e. self-deception and self-centeredness. Coming back, it is safe to say that the set-membership nature of society causes, especially in the conditions of present degree of connectedness, an evolving instance of political boundary setting, from physical boundary to the ideological ones shaping member identities.

Basic Structural Coupling
It is well known, that for a system to stay "alive" it must preserve its identity. Though since the gap between perception and reality is also a socio-psychological threat surface, the system must keep maneuvering the environment around it and be fluid enough to evolve along. In the age when goal setting of AIs is termed not an engineering but a political problem, the notions of 'structural coupling' between systems and environment therefore apply equally to political cultures as well. An advice from John Boyd "the maneuverist" suggests decision makers to approach people, ideas, and machines — in that order. However the how-to of such abstractions remains subject to systemic pressures and some degrees of operational and organisational elusiveness, often threatening a conceptually devolving set.

Pursuing a Systemic Sadbuddhi thus becomes a matter of acting on and perceiving reality without letting our beliefs be colored by our desires (and fears), and as Robert Jervis has recommended, making our 'Assumptions, Beliefs, and Predictions' explicit and inquisitory, for it underlines that the predictions are always resting on some assumptions. Taking human society as a whole, the present chimp world order reflects the inertia of the predominant last great war positions. As the economic center-of-gravity shifts from Atlantic to the Pacific and technology gives chimps men equalizing effects over adversaries, how the decision makers of resource constrained competing sets battle it out remains to be seen. One thing is evident though that usually he who would pay the piper, historically speaking, will call the tune.
__