Introduction

In this module we will expand on the opportunities and challenges presented in Module 2: Opportunities and Challenges in Peer Review, as well as the work you did in Module 3: Bias and Power Structures in Peer Review. This module offers you more time to think critically about established peer-review norms and practices, as well as reflect on your own personal biases and how you might thoughtfully work with your awareness of them. When we think critically, we think about the players in peer review processes, why things are the way they are, and if they have to be that way.

A Note on Critical Examination

This module asks you to critically examine established peer-review practices. The module approaches critical examination as it is rooted in Critical Theory, a historical and philosophical approach that studies social and political contexts: “It must explain what is wrong with current social reality, identify the actors to change it, and provide both clear norms for criticism and achievable practical goals for social transformation” (para. 3).

By the end of this module you should be able to:

• Identify concepts and practices that reinforce bias and power structures in peer review
• Examine peer-review systems for bias and imbalances of power
• Identify established anti-bias practices
• Develop strategies for eliminating bias in peer-review systems

Activities and Exercises

Read and Reflect: Editorial and Ethical Policies (4:1)

Review the editorial and ethical policies for peer review at the American Journal of Public Health. (Scroll about halfway down the page to get to the peer review section.)

Read the Ethical Guidelines for Peer Reviewers from the Committee on Publication Ethics (COPE).

Take some time to consider and reflect on these questions:

• Does what you read from the American Journal of Public Health policies match the Ethical Guidelines outlined by COPE? How do they, and/or how do they not?
• What are the strengths and weaknesses of the COPE guidelines? Would you change anything about them?

Do and Reflect: Defining Rigor (4:2)

Use this prompt to free-write for five minutes: I know something is rigorously researched and of quality when…

Next, spend some time (~10 minutes) looking for a concrete definition of rigor or quality in academic research and publication in your field. You could look on the open web or via handbooks or encyclopedias at your library. Then, when you’ve found a definition that you like, write it down. Reflect on the following questions in a journal or aloud on a recorder:

• Did my personal definition match the one I found ?
• What concrete measures are used to evaluate rigor?
• Who defined it for my field?
• In their 1996 article, Ways of knowing, culture, communication and the pedagogies of the future, Paul Wildman and Sohail Inayatullah assert, “The search for rigour is also often the call for the elimination of difference” (p. 733-734). Based on what you have found, do you agree or disagree with this statement? Why? Why not?

Read: Anti-Racism in Scholarly Publishing (4:3)

Read the Introduction to Antiracism Toolkit for Organizations from the Coalition for Diversity & Inclusion in Scholarly Communication and one more toolkit section of your choice. Also review the Library Publishing Coalition’s Roadmap for Anti-Racist Practice and Anti-Racist Scholarly Reviewing Practices: A Heuristic for Editors, Reviewers, and Authors.

Do: Develop an Anti-Bias Plan (4:4)

Based on the readings from this module, as well as any knowledge or questions you have gained throughout this course, develop an anti-bias plan for a journal of your choosing. This is really a free-form exercise. You could decide to make a detailed document much like you read from the Library Publishing Coalition, or it could be a document that outlines a mission statement and values for a publication of your choosing. Whatever it is, consider what you have learned, and outline concrete things that you would consider the most important parts of an anti-bias plan for peer-reviewing work at a journal publication.

References

Anti-racist scholarly reviewing practices: A heuristic for editors, reviewers, and authors. (2021). Retrieved from https://tinyurl.com/reviewheuristic

Bohman, J., (2021, Spring). Critical theory. In E. N. Zalta (Ed.) The Stanford Encyclopedia of Philosophy. Stanford University. https://plato.stanford.edu/archives/spr2021/entries/critical-theory

COPE Council. (2017). Ethical Guidelines for Peer Reviewers. https://publicationethics.org/resources/guidelines/cope-ethical-guidelines-peer-reviewers

Haffar, S., Bazerbachi, F., & Murad, M. H. (2019). Peer review bias: A critical review. Mayo Clinic Proceedings, 94(4), 670–676. https://doi.org/10.1016/j.mayocp.2018.09.004

The LPC roadmap for anti-racist practice (n.d.). Library Publishing Coalition. Retrieved March 1, 2022, from https://docs.google.com/document/d/1obgebVSlcGjIf4c6PA2zaPqCmyVSWJmL66BRcpq1waU/edit#

Toolkits for equity in scholarly publishing project volunteers (2021). Introduction to the antiracism toolkit for organizations. Coalition for Diversity and Inclusion in Scholarly Communications, Toolkits for Equity. https://doi.org/10.21428/77410d6b.5bc04707

Wildman, P., & Inayatullah, S. (1996). Ways of knowing, culture, communication and the pedagogies of the future. Futures, 28(8), 723–740. https://doi.org/10.1016/0016-3287(96)00031-6