Archive of ‘Paper’ category
Matthew Reichert, Miguel Garces, Ian S. Lustick
This poster was presented at the American Political Science Association Annual Meeting on August 29th, 2014 in Washington, D.C.
We attempt to build a multi-level agent-based model that combines micro-level interactions within an agent landscape and macro-level rationalistic decision-making processes. Adapted from James Fearon’s bargaining model of war, two agent-based landscapes attempt to bargain in order to avoid costly war. Without perfect information, war sometimes occurs and we investigate the potential landscape-level causes.
Download Here
If you are interested in replicating this experiment, please contact the authors for replication resources.
Miguel Garces, Matthew Reichert, Ian S. Lustick
In this paper, we begin with a thought exercise to generate a definition of rare events that captures those events that pose the ‘rare events problem,’ and introduce the concept of a ‘state space’ as a useful way to conceptualize it. Second, we review three approaches that seek to increase the N of rare event data: one from King, Keohane, and Verba (1994) and two from King and Zeng (2001), identifying the strengths and limitations of each. Next, we propose agent-based modeling as an alternative approach to the rare-event problem that moves past existing limitations. Finally, we provide a proof-of-concept for the agent-based modeling approach, using a model of Virtual Egypt with alternating inductive and process-tracing type techniques to study the rare event of a military takeover in Egypt in July 2013.
Download Here
If you are interested in replicating this experiment, please contact the authors for replication resources.
Brandon Alcorn, Miguel Garces, Ian S. Lustick
In this paper we will report on an attempt at Validation & Verification of our agent-based country models using empirical “medium hostility” data from the W-ICEWS program. We will investigate how well our model compares to week-to-week changes in hostility in Bangladesh, Egypt, Pakistan, the Philippines, Thailand, and Venezuela.
Download Here
If you are interested in replicating this experiment, please contact the authors for replication resources.
Ian S. Lustick, Matthew R. Tubin
The original DOD objective which the process of Verification, Validation, and Accreditation exists to serve was to evaluate the credibility of models and simulations. The red-line between verification (following the blueprint correctly) and validity (accuracy for a particular domain) was convenient, but suppressed consideration of flaws in design concepts. Protocols for evaluating models in domains where theory or data is not unchallenged, e.g. social science, requires clarifying the meaning of verification as “construct validity” and developing a model of verification and validation which situates these operations across multiple levels of an epistemological hierarchy. Pathologies of either inference or generalizability can present themselves at any level and be inherited by others.
Download Here
Brandon Alcorn, Miguel Garces, Ian S. Lustick
In this paper we report and discuss results of an effort to adapt country-level modeling
techniques developed in DARPA’s Integrated Crisis Early Warning System
(ICEWS) program to produce forecasts and analyses at a much more granular level,
focusing on districts in Kandahar and Kunduz, Afghanistan. The challenge of this
work in Afghanistan is not only to bridge theoretical models to operational needs at
an appropriate level of granularity, but also to overcome modeling obstacles that are
unique to Afghanistan, at least in comparison with the national models that were the
focus of the ICEWS project. We briefly describe our modeling methodology,
including specific steps taken to account for the challenges listed above, before
moving to a analysis of the results of our experiments. Findings and forecasts are reported from baseline models of Kandahar and Kunduz provinces built with the
most recent open-source data available as well as several “what-if” and “in light of”
analyses that explore counterfactual futures of Kandahar and Kunduz provinces
associated with an ISAF drawdown, militia disarmament, and a string of political
assassinations.
If you are interested in replicating this experiment, please contact the authors for replication resources.
Ian S. Lustick
Verification and Validation are legally required evaluations that the Defense Department
must make of any system, model, or simulation it deploys. With extensive experience and
detailed protocols for verifying and validating natural science based systems, DoD now faces the problem of how to conduct such evaluations for the social science based systems, models, and simulations it is increasingly interested in acquiring. Employing an “epistemological ladder” or hierarchy linking ontology to data, this paper will analyze the verification and validation process as entailing analogous questions about warrantability of inferences and accuracy or generalizability of findings appropriate at any step in the ladder. Analysis of the confusing impact of the “two cultures” idea and its importance within influential philosophy of science texts, along with the particular history of validation and verification concepts within the American military, will be used as the basis for presenting a unified approach to the problem. The usefulness of the approach is tested in a mapping exercise of key debates between quantitatively and qualitatively oriented political science methodologists and of the conventional wisdom reflected in recent DoD documents.
Ian S. Lustick; Edited volume by Kanchan Chandra
The purpose of this volume is to demonstrate that a rigorous conceptual framework can enable constructivist insights to be deployed for the solution of a variety of theoretical and empirical problems. In this chapter I offer a use case in which the framework set forth by the editor, if not the exact details of its entire vocabulary, is employed to solve a difficult empirical and policy-relevant problem. The general problem involves is to evaluate a future for Pakistan involving the secession of its Punjabi core – a future whose probability experts have had difficulty assessing. Since secession is itself a rare event, and secession of the center an even rarer event, data relevant to addressing this problem must be generated by a computer simulation model designed and implemented in conformance with available social theories, including constructivist theory, along with information and Pakistani society relevant to the categories of those theories. The thrust of this chapter is to demonstrate that by integrating constructivist approaches to political contestation, via the framework offered in this volume, which specific knowledge of a complex and important case – the future of Punjabi-dominated Pakistan – an agent-based modeling approach can be used to analyze the conditions under which secession of the center can take place and to estimate its likelihood.
If you are interested in replicating this experiment, please contact the authors for replication resources.
Miguel Garces, Brandon Alcorn
Traditionally agent-based modeling has been a field that focuses largely on abstract models that offer insights into real world phenomena, but rarely with enough complexity to play a major role within academic or policy discussions about particular problems in particular places. In this paper we outline an approach to building country virtualization models that combines traditional micro-level interactions in an agent-based model with macro-level theory to simulate complex political systems in a range of countries in Southeast Asia. We discuss the advantages, and potential downfalls, to adding complexity into agent-based models and present results that highlight some of the analytic advantages to having multiple levels of analysis present in a model and a distribution of counterfactual model results by which we can analyze and forecast political outcomes.
Download Here
If you are interested in replicating this experiment, please contact the authors for replication resources.
Brandon Alcorn, Miguel Garces, Allen Hicken
In this paper we report on the deployment of “VirThai,” a virtualization type agent-based model of contemporary Thailand, to produce predictions for events of political and policy interest over the course of 2011. Predictions are inferred from distributions of outcomes across large batches of counterfactual futures. We will assess the performance of the model as a forecasting tool, as a technique for understanding the mechanisms that drive political outcomes, and as a means for stimulating new insights or lines of reasoning among country experts.
Download Here
UPDATE: This paper has since been published in the 12th volume of the Stanford Journal of East Asian Affairs. Download Here
<![CDATA[
Ian Lustick, Brandon Alcorn, Miguel Garces, Alicia Ruvinsky
This paper suggests that computer-assisted agent-based modeling has the ability to move beyond abstract representations of political problems to theoretically sound virtualizations of real-world polities capable of producing probabilistic forecasts from distributions of stochastically perturbed model trajectories. In contrast to statistical approaches, this technique encompasses both prediction and explanation, with every distinctive trajectory traceable backward from the occurrence or non-occurrence of an event of interest through the branching points and mechanisms that led to it. In this paper we illustrate our technique for building a country-scale model from corroborated theories, focusing on the “Dynamic Political Hierarchy” module that integrates theories of cross-cutting cleavages, nested institutions, and dynamic loyalties. We present our forecasts for significant political events in Thailand for the year August 2010-July 2011. Drawing on this case we demonstrate how the challenges of internal validity can be met in complex formal models and conclude by emphasizing the importance of advances in visualization techniques for parsing large amounts of interrelated time-series data.
Download Here
UPDATE: This paper has since been published in Volume 24 Number 3 of the Journal of Experimental & Theoretical Artificial Intelligence. Download Here