INTEGRATED SCIENTIST MAGAZINE

Knowledge
Reviewers of Reviews

—SUMMARY NOTE—

Scientific publications rely heavily on reviewers to ensure they are of the highest quality and rigor. AGU's anonymous peer review mechanism keeps their contributions secret from authors. We use gender, age and race/ethnicity information from our member database to identify reviewer demographics. AGU is actively working on expanding co-reviewing chances for early career scientists. Earth and space sciences have a higher percentage of women in early career cohorts than older cohorts.
Last updated on 13 September, 2021

Scientific publications rely heavily on reviewers to guarantee that they are of the highest quality and rigor possible. It’s crucial that we know who they are and where they are because of our anonymous peer review mechanism, which in most cases keeps their contributions secret from authors. If you’re interested in learning more about AGU’s reviewer demographics, this is the time to do so. Over the last few years, we’ve shared our previous research on reviewer demographics [Hanson and Lerback, 2017; Lerback et al., 2020] and worked to expand the reviewer pool in order to engage a wider range of perspectives and ultimately publish more inclusive science that benefits a larger sector of the society. Increased scientific production and submissions need a growing requirement for peer reviews.

It is possible for AGU to identify reviewer demographics in a variety of ways. When writers and reviewers are matched in our submission system with their member profiles, we use gender, age and race/ethnicity information from our member database. When a member’s gender is not specified in their profile, we use the gender-name database Gender API, which uses initial name and country to guess, and we keep guesses with a +90% confidence score when possible. ‘ If you’re a reviewer outside of the United States, you won’t be able to use our current race/ethnicity data because it’s based on US census categories. But we are currently improving our system so that the categories appropriately describe the global population. In order to determine a person’s country of residence, the submission system looks at their profile. We use the suffix of their email address for those who don’t have it (when it identifies a country).

Female reviewers’ acceptance rates are lower than those of their male counterparts, although the percentage of women who accept review requests has risen nominally each year since 2016 (see the figure below for more information on the percentage of women who accept review requests).

Percentage of women who were invited to review a product and how many accepted.

A particular target for invitations to women is difficult to establish because reviewing receives less professional credit than authoring publications, leading research groups, chairing departments and other activities.. Concerns have been expressed about the impact of too many women scientists having to evaluate papers on their ability to focus on other aspects of their careers.

As a result, younger scientists will gain more from evaluating publications than those who are more advanced in their careers. This is because the Earth and space sciences have a higher percentage of women among early career scientists than among older cohorts. Expanding co-reviewing chances will allow senior scientists and those in their early career to work together on reviews, which AGU is actively working on. Peer review training for early-career scientists will be facilitated by this initiative, which aims to increase the number of chances for future reviewers.

To see how our corresponding authors’ geographic locations compare to the invited reviewer’s, check out the following chart. Despite the fact that we’ve observed an increase in the proportion of review invitations sent to regions where we’ve had the largest growth in authors submitting papers, such as China, the increase in invitations is not as large as the increase in accepted articles from China. In addition, the proportions of invitations extended to reviewers from the United States and Europe have decreased, but their representation in our pool of authors who get their papers published is still higher than their proportion in our pool of reviewers.

It’s critical to break down gender statistics by age range in order to identify specific areas for improvement. These graphs indicate the demographics of reviewers, writers, and members of the American Geophysical Union.

Men in their 30s and 40s are the majority of those invited to evaluate articles (40s-60s). Women in their 30s make up the majority of the review panel.

A typical comparison between invited reviewers and members is presented below. According to the writers and members, editors feel more at ease inviting persons in their mid- and later-career stages of development. This could be owing to the fact that these reviewers are part of the editor’s professional network, or because they are experts in their field.

Increasing the number of possibilities for younger scientists to participate in peer review could alleviate some of the load on mid-career scientists, who are likely to have more teaching, administrative, and/or family-related responsibilities.

Our reviewer pool’s diversity can be assessed by calculating the number of invitations received per person in various demographic groupings. How often are we inviting the same men and women to the same events, for example? By dividing the total number of invites sent out by the number of unique email addresses in that demographic, we can see how many invitations each person will receive, as shown in the graph below.

According to the results, we invite far fewer females than males, as well as far fewer people of “unknown gender.” Since the beginning of time, we’ve noticed that we’ve been inviting the same men and women less frequently.

There is also a decline in the number of invited reviewers per country or region. More than any other country-region, the United States is being invited by editors more frequently than any other. As a result of this, our reviewer pool has grown by 2020, as seen by the fact that we are inviting fewer people from a certain region.

There was a lot of worry about how the epidemic would influence people’s ability to submit papers and to review them, and this was a major concern for the scientific community. In May 2020, we’ll look at how the epidemic affected our submissions; we’ve collected enough data from the past year to do so.

No statistically significant difference was found between pre-pandemic and post-pandemic periods when it came to submissions overall and across all demographic groups (based only on the corresponding author). Compared to prior years, editors and associate editors invited more reviewers in their 20s and 30s and fewer in their 40s and 50s. According to the data below, women’s rates dipped a few points in 2020, while men’s rates rose a few points.

Stay-at-home orders may have contributed to an increase in paper submissions and reviewer activities (more submissions, a greater number of peer reviews) (see survey conducted by the American Geosciences Institute on work habits in 2020). When we looked at AGU submission and review rates, we found no evidence that the pandemic had any influence on productivity or job satisfaction among women (and men) with children.

Pin It on Pinterest

Share This