Ian S. Lustick
Verification and Validation are legally required evaluations that the Defense Department
must make of any system, model, or simulation it deploys. With extensive experience and
detailed protocols for verifying and validating natural science based systems, DoD now faces the problem of how to conduct such evaluations for the social science based systems, models, and simulations it is increasingly interested in acquiring. Employing an “epistemological ladder” or hierarchy linking ontology to data, this paper will analyze the verification and validation process as entailing analogous questions about warrantability of inferences and accuracy or generalizability of findings appropriate at any step in the ladder. Analysis of the confusing impact of the “two cultures” idea and its importance within influential philosophy of science texts, along with the particular history of validation and verification concepts within the American military, will be used as the basis for presenting a unified approach to the problem. The usefulness of the approach is tested in a mapping exercise of key debates between quantitatively and qualitatively oriented political science methodologists and of the conventional wisdom reflected in recent DoD documents.