This is my review of Kevin Clarke and David Primo’s new book, A Model Discipline: Political Science and the Logic of Representations, which tackles the role of theoretical and statistical modeling in political science. Phil Arena has a great review covering the book’s main points. My goal is to avoid duplicating his efforts – so I suggest reading his review.
Since I try to keep a foot in both evolutionary ecology and political science, I have been exposed to the methods of both. I am, to my knowledge, the only non-political-science grad student to attend an EITM summer school, so may have more insight into the context of their argument. But since most of my methods training has been in on the ecology side, this will be somewhat of an outsider’s view.
First, I really liked this book and agreed with most of the authors’ philosophy of modeling. One of their main points is that the goal of theoretical models is rarely prediction. Most often they are tools for reasoning about the world. They might help evaluate the logical consistency of ideas, define terms and clarify arguments, or identify important new areas of research – but these many uses are not always clear to the non-modeler. For example, in my inaugural blog post, I wrote about how Hamilton’s rule is the result of a simple model intended to demonstrate the mechanisms of kin selection and that the big misunderstanding of E.O. Wilson’s crusade against kin selection is mistaking it for a predictive model.
The standard story I keep hearing in political science is that the field has taken the “theoretical models as predictive models” to the extreme. I have been warned, repeatedly, that the field’s top journal, APSR, will not publish purely theoretical models without some nod at an empirical “test” of the model. Because of my experience at EITM, I agree with Phil that, the relationship between theory and statistics are hardly monolithic.
Primo and Clarke’s view of theoretical models would be very uncontroversial in ecology, population biology, population genetics, and related fields. These, like political science, are fields of complex dynamics systems with important interactions at different levels and a heavy reliance on observational data for the large scale processes. (International relations and ecosystem ecology are both hindered by a sample size of one planet and the difficulty of logistically and ethically conducting large-scale experiments.)
Clarke and Primo argue that theoretical models are like maps. Maps are designed for a specific purpose and what determines a good map is whether it is useful for the purpose for which it is designed. In the same way that it is weird to argue about whether a map is “true” or “false” (since all maps are false in most respects), it is weird to argue about whether a model is true or false.*
I was surprised by their discussion of empirical models since Clarke and Primo did not take their argument to its seemingly logical conclusion by arguing for a model selection approach to empirical analysis (as opposed to the standard null hypothesis testing – NHT). The NHT view of the world is that models are true and false and the job of science is to reject the false ones. In a null hypothesis test, the first step is to pick a “null” model (which, in practice, is almost always a very terrible model) and assume it is true.
The model selection view of the world assumes that all models are false – or incomplete views of the world – but that some models are better than others for specific purposes. They try to distinguish between models based on some criteria of usefulness. One criteria might be out-of-sample prediction. Failing that, they can use criteria based on information theory. Another (important) criteria could be theoretical relevance based on a priori reasoning.
I found this omission surprising, not because I’ve seen a lot of model selection statistics in political science** (I haven’t), but because, in ecology, as soon as you start reading about statistical models not being “true” or “false,” this seems to be next argument. The standard reference for this approach is Burnham and Anderson’s Model Selection and Multi-Model Inference: A Practical Information-Theoretic Approach. (Here is a short review.)
Overall, I recommend this book anyone interested in modeling in political science. I especially found their taxonomy of model types helpful (see Phil’s review for details). As someone who aspires to both a theoretical and empirical modeling program in political science, I hope that the authors manage to shift the field’s view of modeling more towards their own.
* – They are not the first to make this analogy. I couldn’t remember whether I first saw it in McElreath and Boyd or Miller and Page – so I looked it up and it was in both.
** – Phil Arena pointed out to me that Kevin Clarke has written about model selection elsewhere. But that makes it even more surprising to me that it was not in the book.