Are Referees and Editors in Economics Gender Neutral?

Author(s): David Card, Stefano DellaVigna, Patricia Funk and Nagore Iriberri

Women are under-represented in Economics, and more so the higher the rank. While numerous explanations have been offered for this gap, an abiding concern is that stereotype biases or other forms of discrimination lead decision makers to undervalue the contributions of women. Producing high quality research and publishing in high impact journals is the most important task for an academic to be promoted, summarized as the publish or perish paradigm. We address the question: Are referees and editors in economics gender neutral? 

We present a comprehensive study of the role of gender in the editorial process (Card et al., 2020, QJE) based on submissions to four leading journals in economics between 2003 and 2013: the Quarterly Journal of Economics, the Review of Economic Studies, the Journal of the European Economic Association and the Review of Economics and Statistics. Our main findings can be summarized as follows: 

Concerning the referee process, we first document evidence for assortative matching: female authored-papers (compared to male-authored papers) are more likely to be assigned to female referees (results for mixed-gender papers are omitted for the sake of brevity). Does this matter for the referee recommendations? We find this not to be the case: referee gender does not affect the evaluation of neither, all male or all female-authored papers. Therefore, authors should not care whether their papers are sent to male or female referees!

Although female and male referees evaluate (the same paper) in a similar way, this does not necessarily imply that referees set the same bar for male and female authored papers (for simplicity, we omit mixed-gender groups in this discussion). It is possible that referees of both genders are biased for or against female-authored papers. To make further progress we need to make comparisons across papers by different gender groups, taking into account differences in quality, which we measure by accumulated google cites from submission to mid 2015. We then investigate whether conditional on referee recommendations, characteristics such as gender have any predictive power. Intuitively, if referees correctly evaluate papers according to their quality, author-characteristics such as gender should not predict citations, once referee recommendations are controlled for. On the other hand, if female authored papers - conditional on referee recommendations - accumulate more/fewer cites than all-male papers, then we could argue that female-authored papers face a higher/lower bar in the publishing process. 

What do the data tell? We find that all female-authored papers receive about 25% more citations than observably similar all male-authored papers. This is consistent with female researchers facing a higher bar in the publishing process. We calculate that the R&R rate for all-female papers would rise from 12.3% to 18.6%—a 50% increase, if editors were to maximize citations. Different characteristics of papers written by males and females may play some role in the observed citation premium, but is unlikely to explain the whole citation premium in our main sample.

Concerning the editorial decision, we find that editors largely follow the referee’s advice and therefore do not “undo” the bias coming from the referees. Editors are gender neutral in valuing advice from male and female referees. We find that the quality of referee-reports written by male and female referees is similar, and rightfully, editors do not put different value on them either.