Introduction

Being human and living in the world means that we inherently bring our past experiences, learning, and socialization to everything we do, including engaging in the peer review processes, whether as an author, referee, editor, etc. We are biased and thus carry those biases in all that we do and experience. Merriam-Webster provides a basic definition of bias: “a tendency to believe that some people, ideas, etc., are better than others that usually results in treating some people unfairly.” In addition to this basic definition, we need to understand bias as socially constructed by in-groups and out-groups. Out-group bias occurs when we think our in-group is better than the out-group.

Social power structures also come into play in peer-review work. In their chapter, “Conceptualizing Structures of Power,” the authors of the open textbook Introduction to Women, Gender, Sexuality Studies unpack this concept:

“By power we mean two things: 1) access to and through the various social institutions mentioned above, and 2) processes of privileging, normalizing, and valuing certain identities over others. This definition of power highlights the structural, institutional nature of power, while also highlighting the ways in which culture works in the creation and privileging of certain categories of people.”

As you move through this module it is important to remind yourself that bias and power structures are inherent in society. This module will help you in beginning to uncover what those systemic or personal biases and power structures may be. Some of this work may incite feelings. That is okay; you are human. It may be uncomfortable work, but it is in discomfort that we begin to learn. If you are feeling discomfort, take a moment to pause and reflect why that is. What has brought you to this state of discomfort? Your prior experiences? Your beliefs or attitudes?

You will revisit the ideas and discoveries you make in this module throughout the rest of this course, especially in Module 4: Critically Examining Established Peer-Review Practices.

At the end of this module you should be able to:

• Examine personal bias(es)
• Explore how bias can manifest in peer review
• Identify policies intended to mitigate bias
• Identify and analyze bias and power structures in peer-review systems

Activities and Exercises

Do: Harvard Implicit Bias Test (3:1)

Take one or two of the Harvard Implicit Bias tests. (From this linked page read the content, and then click on “I wish to proceed.”) If you have taken some of these before, that’s okay. Spend 2-5 minutes after the test free writing, or recording yourself thinking aloud based on these prompts: What did you learn? How did it make you feel?

Listen and/or Read: Citation Politics and Editorial Alteration of Reviewer Reports (3:2)

Listen to the 14-minute Canadian Broadcasting piece, The Politics of Citation: Is the Peer Review Process Biased Against Indigenous Academics? (The audio is linked under the headline picture.)

Either read the news article by Cathleen O’Grady from Science Magazine, “Delete Offensive Language? Change Recommendations? Some Editors say It’s OK to Alter Peer Reviews”  or listen to the 45-minute podcast episode from The Black Goat, You Took the Words Right Out of My Mouth. Both discuss the same instance of an editor altering a peer reviewer’s report. I highly recommend reading both the article and listening to the podcast episode if you have the time.

Review and/or Watch: Peer Review Bias (3:3)

Review this image on peer review bias from Samir Haffar and co-authors’ article Peer Review Bias: A Critical View and/or watch the supplemental video (it’s about 7 minutes long).

Do: Revisit Your Flow Chart and Brainstorm (3:4)

Find the peer-review process flow chart you created in Module 1: What Is Peer Review? (1:4). For each step on the flow chart reflect on the following questions:

• Which actor has power?
• Where may bias come into play?
• Who has the ability to hold others accountable for their bias?

After you have examined each step, brainstorm a list of ways to lessen the harmful consequences of power and bias in the peer-review process.

Do: Review and Compare Journal Policies (3:5)

This activity asks you to explore the policies sections of two different journal websites. Then, you will compare them. For each journal listed, review the policies and written guidelines regarding peer review. Make a mental (or written) list of the biases and power structures that these policies intend to mitigate. Also note if you feel there is power or bias that is unaddressed by the documentation. After reviewing both journals, which do you feel does a better job explaining the process and expectations? Which does a better job of eliminating bias and harmful power structures? If it is helpful for you, make a table comparing the contents

• Atmospheric Chemistry and Physics – Review the policies, which are linked under the “policies” drop-down menu on the home page, as well as the topics under the “peer review” drop-down.
• American Speech and Hearing Association journals – Review the page What to Expect in Peer Review, as well as the contents of their Peer Review Excellence Program (in three sections: The Peer Review Process, Peer Review Policies, and Peer Review Procedures).

References

CBC Radio. (2018, February 23). The politics of citation: Is the peer review process biased against Indigenous academics? CBC. https://www.cbc.ca/radio/unreserved/decolonizing-the-classroom-is-there-space-for-indigenous-knowledge-in-academia-1.4544984/the-politics-of-citation-is-the-peer-review-process-biased-against-indigenous-academics-1.4547468

Delete offensive language? Change recommendations? Some editors say it’s OK to alter peer reviews. (n.d.). Science. Retrieved February 10, 2022, from https://www.science.org/content/article/delete-offensive-language-change-recommendations-some-editors-say-its-ok-alter-peer

Haffar, S., Bazerbachi, F., & Murad, M. H. (2019). Peer review bias: A critical review. Mayo Clinic Proceedings, 94(4), 670–676. https://doi.org/10.1016/j.mayocp.2018.09.004

Kang, M., Lessard, D., Heston, L, & Nordmarken, S. (2017). Introduction to women, gender, sexuality studies. University of Massachusetts Amherst Libraries. https://press.rebus.community/introwgss

Merriam-Webster. (n.d.). Bias. In Merriam-Webster.com dictionary. Retrieved March 1, 2022, from https://www.merriam-webster.com/dictionary/bias

PREP: The ASHA Journals Peer Review Excellence Program. (n.d.). ASHA Journals Academy. Retrieved March 1, 2022, from https://academy.pubs.asha.org/prep-the-asha-journals-peer-review-excellence-program/

Project Implicit. (n.d.). Harvard Implicit Bias Test. Retrieved March 1, 2022, from https://implicit.harvard.edu/implicit/takeatest.html

You took the words right out of my mouth. (2020). Retrieved February 20, 2021, from https://blackgoat.podbean.com/e/you-took-the-words-right-out-of-my-mouth

Wikipedia (2021). In-group and Out-Group. Retrieved April 7, 2022 from https://en.wikipedia.org/wiki/In-group_and_out-group