Ian S. Lustick
“Social radar” is a metaphor: in other words, a model. A model projects something known about a “source” onto something less well known—a target. We see the moon (the target) differently when we hear the metaphor: “The moon is a ghostly galleon.” In effect, the poet asks the reader to use “ghostly galleon” as a model for the moon. How we visualize the moon when we use that model results from our individual expectations, emotions, and images associated with the idea of a “ghostly galleon.” Under the influence of the metaphor, i.e., when employing “ghostly galleon” as a model for the moon, we see the moon not as an astronomical object but as a spooky vision, floating silently, mysteriously, perhaps ominously, through the clouded darkness. Taking “social radar” seriously as a metaphor, and therefore as a model, can illuminate many aspects of this vision for forecasting.
Download Here
Matthew Reichert, Miguel Garces, Ian S. Lustick
This poster was presented at the American Political Science Association Annual Meeting on August 29th, 2014 in Washington, D.C.
We attempt to build a multi-level agent-based model that combines micro-level interactions within an agent landscape and macro-level rationalistic decision-making processes. Adapted from James Fearon’s bargaining model of war, two agent-based landscapes attempt to bargain in order to avoid costly war. Without perfect information, war sometimes occurs and we investigate the potential landscape-level causes.
Download Here
If you are interested in replicating this experiment, please contact the authors for replication resources.
Miguel Garces, Matthew Reichert, Ian S. Lustick
In this paper, we begin with a thought exercise to generate a definition of rare events that captures those events that pose the ‘rare events problem,’ and introduce the concept of a ‘state space’ as a useful way to conceptualize it. Second, we review three approaches that seek to increase the N of rare event data: one from King, Keohane, and Verba (1994) and two from King and Zeng (2001), identifying the strengths and limitations of each. Next, we propose agent-based modeling as an alternative approach to the rare-event problem that moves past existing limitations. Finally, we provide a proof-of-concept for the agent-based modeling approach, using a model of Virtual Egypt with alternating inductive and process-tracing type techniques to study the rare event of a military takeover in Egypt in July 2013.
Download Here
If you are interested in replicating this experiment, please contact the authors for replication resources.
Brandon Alcorn, Miguel Garces, Ian S. Lustick
In this paper we will report on an attempt at Validation & Verification of our agent-based country models using empirical “medium hostility” data from the W-ICEWS program. We will investigate how well our model compares to week-to-week changes in hostility in Bangladesh, Egypt, Pakistan, the Philippines, Thailand, and Venezuela.
Download Here
If you are interested in replicating this experiment, please contact the authors for replication resources.
Ian S. Lustick, Matthew R. Tubin
The original DOD objective which the process of Verification, Validation, and Accreditation exists to serve was to evaluate the credibility of models and simulations. The red-line between verification (following the blueprint correctly) and validity (accuracy for a particular domain) was convenient, but suppressed consideration of flaws in design concepts. Protocols for evaluating models in domains where theory or data is not unchallenged, e.g. social science, requires clarifying the meaning of verification as “construct validity” and developing a model of verification and validation which situates these operations across multiple levels of an epistemological hierarchy. Pathologies of either inference or generalizability can present themselves at any level and be inherited by others.
Download Here
Brandon Alcorn, Miguel Garces, Ian S. Lustick
In this paper we report and discuss results of an effort to adapt country-level modeling
techniques developed in DARPA’s Integrated Crisis Early Warning System
(ICEWS) program to produce forecasts and analyses at a much more granular level,
focusing on districts in Kandahar and Kunduz, Afghanistan. The challenge of this
work in Afghanistan is not only to bridge theoretical models to operational needs at
an appropriate level of granularity, but also to overcome modeling obstacles that are
unique to Afghanistan, at least in comparison with the national models that were the
focus of the ICEWS project. We briefly describe our modeling methodology,
including specific steps taken to account for the challenges listed above, before
moving to a analysis of the results of our experiments. Findings and forecasts are reported from baseline models of Kandahar and Kunduz provinces built with the
most recent open-source data available as well as several “what-if” and “in light of”
analyses that explore counterfactual futures of Kandahar and Kunduz provinces
associated with an ISAF drawdown, militia disarmament, and a string of political
assassinations.
If you are interested in replicating this experiment, please contact the authors for replication resources.
Ian S. Lustick
Verification and Validation are legally required evaluations that the Defense Department
must make of any system, model, or simulation it deploys. With extensive experience and
detailed protocols for verifying and validating natural science based systems, DoD now faces the problem of how to conduct such evaluations for the social science based systems, models, and simulations it is increasingly interested in acquiring. Employing an “epistemological ladder” or hierarchy linking ontology to data, this paper will analyze the verification and validation process as entailing analogous questions about warrantability of inferences and accuracy or generalizability of findings appropriate at any step in the ladder. Analysis of the confusing impact of the “two cultures” idea and its importance within influential philosophy of science texts, along with the particular history of validation and verification concepts within the American military, will be used as the basis for presenting a unified approach to the problem. The usefulness of the approach is tested in a mapping exercise of key debates between quantitatively and qualitatively oriented political science methodologists and of the conventional wisdom reflected in recent DoD documents.
Ian S. Lustick; Edited volume by Kanchan Chandra
The purpose of this volume is to demonstrate that a rigorous conceptual framework can enable constructivist insights to be deployed for the solution of a variety of theoretical and empirical problems. In this chapter I offer a use case in which the framework set forth by the editor, if not the exact details of its entire vocabulary, is employed to solve a difficult empirical and policy-relevant problem. The general problem involves is to evaluate a future for Pakistan involving the secession of its Punjabi core – a future whose probability experts have had difficulty assessing. Since secession is itself a rare event, and secession of the center an even rarer event, data relevant to addressing this problem must be generated by a computer simulation model designed and implemented in conformance with available social theories, including constructivist theory, along with information and Pakistani society relevant to the categories of those theories. The thrust of this chapter is to demonstrate that by integrating constructivist approaches to political contestation, via the framework offered in this volume, which specific knowledge of a complex and important case – the future of Punjabi-dominated Pakistan – an agent-based modeling approach can be used to analyze the conditions under which secession of the center can take place and to estimate its likelihood.
If you are interested in replicating this experiment, please contact the authors for replication resources.
Miguel Garces, Brandon Alcorn
Traditionally agent-based modeling has been a field that focuses largely on abstract models that offer insights into real world phenomena, but rarely with enough complexity to play a major role within academic or policy discussions about particular problems in particular places. In this paper we outline an approach to building country virtualization models that combines traditional micro-level interactions in an agent-based model with macro-level theory to simulate complex political systems in a range of countries in Southeast Asia. We discuss the advantages, and potential downfalls, to adding complexity into agent-based models and present results that highlight some of the analytic advantages to having multiple levels of analysis present in a model and a distribution of counterfactual model results by which we can analyze and forecast political outcomes.
Download Here
If you are interested in replicating this experiment, please contact the authors for replication resources.
Lustick Consulting was awarded a contract through the Office of Naval Research to develop V-SAFT.
We will provide a completed Virtual Strategic Analysis and Forcasting Tool (V-SAFT), an integrated, semi-automatic tool for producing and exploiting virtualized models of complex sociocultural arenas.
The V-SAFT ultimate objective is to provide commanders with the capacity to monitor the velocity, scope, and magnitude of change in politically fragile societies. Model-building instruments within V-SAFT will double as situation awareness instruments for operators, featuring wiki-style windows into the human terrain of complex societies. V-SAFT will enhance forecasting capabilities by extending the range of questions commanders can ask and by greatly improving the reliability of answers. In particular, V-SAFT will be designed to assess the likelihood of rare but high-impact events and to afford trace-back opportunities to support mitigation strategies and contingency planning.