I have read Andrew Gelman’s blog for about five years, and gradually, I’ve decided that among his many blog posts and hundreds of academic articles, he is advancing a philosophy not just of statistics but of quantitative social science in general. Not a statistician myself, here is how I would articulate the Gelman View:
- The purpose of social statistics is to describe and understand variation in the world. The world is a complicated place, and we shouldn’t expect things to be simple.
- The purpose of scientific publication is to allow for communication, dialogue, and critique, not to “certify” a specific finding as absolute truth.
- The incentive structure of science needs to reward attempts to independently investigate, reproduce, and refute existing claims and observed patterns, not just to advance new hypotheses or support a particular research agenda.
- Because the world is complicated, the most valuable statistical models for the world will generally be complicated. The result of statistical investigations will only rarely be to give a stamp of truth on a specific effect or causal claim, but will generally show variation in effects and outcomes.
- Whenever possible, the data, analytic approach, and methods should be made as transparent and replicable as possible, and should be fair game for anyone to examine, critique, or amend.
- Social scientists should look to build upon a broad shared body of knowledge, not to “own” a particular intervention, theoretic framework, or technique. Such ownership creates incentive problems when the intervention, framework, or technique fail and the scientist is left trying to support a flawed structure.
- Measurement. How and what we measure is the first question, well before we decide on what the effects are or what is making that measurement change.
- Sampling. Who we talk to or collect information from always matters, because we should always expect effects to depend on context.
- Inference. While models should usually be complex, our inferential framework should be simple enough for anyone to follow along. And no p values.
He might disagree with all of this, or how it reflects his understanding of his own work. But I think it is a valuable guide to empirical work.