Empirical Etiquette

Empirical etiquette is a set of fundamental principles for conducting and presenting empirical research. It provides a sufficient level of transparency to other scholars about the data generating process, variable construction, and analysis so that future researchers can reproduce the project and attempt faithful replications in other contexts. 

I aspire to good empirical etiquette in all of my research. And I often provide this guidance to authors in my role as Associate Editor for Administrative Science Quarterly 

1. Aspire to reproducibility – at least reproducible by you and ideally by others.

2. Describe the ideal experiment in an unconstrained world. What treatment would you administer? How? Who would receive it? What would you measure?

3. Discuss how closely your actual design approximates that ideal experiment. 

4. Describe the data generating process. What behaviors produced the data you analyze?

5. If conducting an experiment – lab or field – then pre-register your research plan.

6. Elaborate on the primary threat to causal inference. Clearly articulate your identification strategy and its fundamental assumptions.

7. Reproduce established results with a simple approach. Enable readers to compare your results to prior work. 

8. Add complexity, as necessary, to deliver “differences-in-inferences” (i.e., What distinct insight is produced by your theoretically-motivated analysis?).

9. Discuss the economic significance of the results. Interpret the estimates. Discuss how well the analysis accounts for variance in the phenomenon.

10. Before concluding, evaluate the key results when subjected to more sophisticated or more restrictive approaches. Specify boundary conditions on the results.