I thank the PS editors for the opportunity to respond to the reply by Daniel Maliniak, Ryan Powers, and Barbara Walter.
-
1. The evidence of fraud and non-replicated results mentioned in my article’s introduction relate to reporting flexibility because each reduces the trust that social science deserves.
-
2. My article concerned the potential for politically biased estimates due to selective reporting in a politically imbalanced field. I suggested preregistration and preacceptance to address this concern because these mechanisms can be adopted immediately to directly reduce or eliminate selective reporting for results presented as confirmatory in applicable studies. I did not call for mechanisms to increase ideological diversity because, among other reasons, I do not think that increased ideological diversity would be an efficient or particularly effective mechanism for reducing political bias in research reporting. For an analogy, sentencing matrices that eliminate sentencing discretion would be a more efficient and effective mechanism than increased racial diversity among judges for reducing racial bias in criminal sentencing, in the aggregate and in individual cases.
-
3. Table 4 in Maliniak et al. (Reference Maliniak, Powers and Walter2013) reported results for 10 models estimating the gender gap in citations to international relations articles, with each successive model including additional controls. Maliniak et al. (Reference Maliniak, Powers and Walter2013) noted that gender in their kitchen sink model “remains…statistically significant” (p. 905), but my reanalysis indicated that the p-value for that model is 0.223 when including articles from 2007. This omission of 2007 articles from the kitchen sink model is the referent regarding Maliniak et al. (Reference Maliniak, Powers and Walter2013) in the passage: “[p]reviously-discussed examples are consistent with a preference for statistically-significant results” (Zigerell, this issue). Maliniak et al. (Reference Maliniak, Powers and Walter2013) indicating that the gender gap coefficient is not statistically significant in certain subsets of years does not make the omission of 2007 articles from their key model inconsistent with a preference for statistically-significant results.
-
4. Maliniak et al. found it odd that I did not report results for their other table 4 models when including articles from 2007. But none of these other models control for the journal that an article was published in. The journal provides information about article quality and visibility, which are theoretically related to the number of citations an article receives. I did not think it necessary to discuss models that omit such a theoretically-important control, either for illustrating reporting flexibility (this issue) or for a substantive discussion of the gender citation gap (Zigerell Reference Zigerell2015). My reanalysis volunteered a model that added theoretically-relevant controls (Zigerell Reference Zigerell2015, Figure 1), with a gender gap estimate larger and a p-value smaller (p=0.027) than in the Maliniak et al. (Reference Maliniak, Powers and Walter2013) kitchen sink model, which provides better information about the gender citation gap than a model omitting available data for relevant controls would.
