Introduction

In this module we dive a little deeper into the nuances of peer review. The peer-review system has been the subject of much debate. In this module we will take a cursory look at some of the opportunities that peer review affords and the challenges it presents.

Although the activities and exercises in this module will offer you exposure to some of the opportunities and challenges, it is important that I present them here.

Opportunities:

• Authors/researchers get meaningful feedback on their work
• Process helps to ensure the quality of scholarly work/knowledge production

Challenges:

• Long publication timelines
• Acknowledging referees’ hidden labor
• Ethical dilemmas

These are certainly not exhaustive lists, and subsequent modules will offer you space and time to reflect on these challenges, as well as the power structures and biases within peer-review systems and processes.

At the end of this module you should be able to:

• Identify some positive aspects of and problems with peer review
• Consider ethical dimensions of the peer-review process
• Interpret peer-review challenges and compose responses to them

This module begins the unveiling of what I hope will be an opportunity for you to discover the multitude of layers of complexity in current peer-review systems and processes, but it is not the end of this discovery. In fact, all subsequent modules will build on the understandings you have gained from this module and future exercises will ask you to dive more deeply into challenges and opportunities.

Activities and Exercises

Do: Brainstorm (2:1)

Set a timer for five minutes and write down (or record yourself thinking aloud) what you think are the opportunities and challenges presented by peer review. Save this list; you will use it later.

Read and Do: Why Peer Review? (2:2)

Read Peer Review in Scholarly Publishing Part A: Why Do It? by Kiron Koshy and co-authors. Make a list of the arguments presented in this short piece. Save this list; you will use it later. Identify one of the arguments presented and compose a social media post (e.g. Twitter, TikTok, Facebook, etc.) substantiating your agreement or your disagreement with the argument. If you feel comfortable sharing what you have composed on social media, please use the course hashtag: #PeerReviewPrimer.

Listen and Reflect: Scientists Aim To Pull Peer Review Out of the 17th Century (2:3)

Listen to this four-minute segment from NPR’s Weekend Edition program. While you’re listening, or after, reflect on the problems presented by the individuals interviewed in the piece. What lingering questions do you have about these problems?

Listen and Read: Peer Review Labor (2:4)

Listen to two minutes of the audiobook What is Peer Review? (A Short Guide) by Jo VanEvery, minutes 11:30-13:30, discussing the labor of peer review.

Read Recognition in Peer Review by Haseeb Irfanullah posted on the Scholarly Kitchen and this editorial from Nature Genetics: “What’s Taking So Long?”

Read Owning the Peer Review Process by Charlotte Roh.

Read and Reflect: Sharing by a Reviewer on Social Media (2:5)

Read this case report from the Committee on Publication Ethics: Sharing by a Reviewer on Social Media. Reflect on this case:

• If you were the reviewer would you have published the tweet?
• What do you think of the advice that COPE gave?
• Taking into account your own experiences, what you already know about peer review, and what you have read and understood from this course so far, what advice would you have given?

Do: Responding to Comments (2:6)

Find 3-5 egregious reviewer comments from Shit My Reviewers Say on Tumblr. Now make creative reactions to each one you selected. Your reaction could be finding an appropriate gif, a poem, a song, or a dance video that would capture your feelings and reactions if you had received these comments. If you feel comfortable, please share your reactions on social media using the course hashtag: #PeerReviewPrimer.

Do: Edit your Lists (2:7)

Now that you have learned a bit more about the opportunities and challenges offered by peer review, modify the lists you made in the first activity in this module (2:1) and the list of arguments made by Koshy and co-authors in Peer Review in Scholarly Publishing Part A: Why Do It? (2:2). Thinking back to what you either agreed with or disagreed with before, have you changed your mind? Add reasons why one should peer review, and add more challenges that you have identified or thought of.

References

VanEvery, Jo. (n.d.). excerpt from: What is peer review? (a short guide).  Retrieved December 20, 2021, from https://soundcloud.com/jovanevery/an-excerpt-from-what-is-peer-review-a-short-guide

Harris, R. (2018, February 24). Scientists aim to pull peer review out of the 17th century. NPR. https://www.npr.org/sections/health-shots/2018/02/24/586184355/scientists-aim-to-pull-peer-review-out-of-the-17th-century

Irfanullah, H. (2021, October 12). Recognition in Peer Review. The Scholarly Kitchen. Retrieved December 20, 2021 from https://scholarlykitchen.sspnet.org/2021/10/12/recognition-in-peer-review

Koshy, K., Fowler, A. J., Gundogan, B., & Agha, R. A. (2018). Peer review in scholarly publishing part A: Why do it? International Journal of Surgery: Oncology, 3(2), e56. http://doi.org/10.1097/IJ9.0000000000000056

Roh, C. (2022). Owning the peer review process: If we have to do this work, we should own it. College & Research Libraries News, 83(3). https://doi.org/10.5860/crln.83.3.100

Sharing by a reviewer on social media. (n.d.). COPE: Committee on Publication Ethics. Retrieved December 20, 2021, from https://publicationethics.org/case/sharing-reviewer-social-media

Shit My Reviewers Say. (n.d.). Retrieved December 20, 2021, from https://shitmyreviewerssay.tumblr.com/

What’s taking so long? (2003). Nature Genetics, 33(1), 1–2. https://doi.org/10.1038/ng0103-1