Rethinking Peer Review: Usability Testing and Evidence

All, Guides

I recently participated in a Twitter conversation about peer review that made me realize that I haven’t blogged about my peer review process. This is still a work in progress because I want to add some additional foundational work inspired by Caitlin Tucker’s Blended Learning and Starr Sackstein’s Assessment Hacks. Shelby Scofield writes about using station rotation for workshop on Edutopia that touches on this work. Stay tuned for my experiments with this work in the Spring Semester! This is how peer review works in my classroom:

Community First

Before we begin even thinking about peer review we create class community. We get to know each other so we can build trust and understanding. We do this by talking and sharing and recognizing the various ways individuals impress us (see badges). If you want your students to fully engage and contribute to a peer review workshop then you need to get them invested in each other first.

Explore

Before we can peer review we need to spend time experimenting with and experiencing the form of text we will write. This process usually involves a lot of preliminary writing, but also reading, viewing, listening, and talking about the form and the topic it will address. Along the way students develop their own ideas about the topic and generate a lot of text as well as identify some mentor texts to guide their work. If you want your students to offer meaningful and constructive advice to their peers then they need to understand the deliverable they are reviewing and see what successful writing looks like in this specific rhetorical context.

Make the Rules

After we explore and experiment, I then introduce the formal assignment we are working toward. Students are usually comfortable with this news especially once I remind them of how much work they have already accomplished (words written, claim drafted, sources located, etc.). I share the student learner outcomes and goals for the assignment and then we get to work and create a scoring guide together. This rubric is a single-trait list of the qualities that successful examples of this writing deliverable should possess. For example, my students recently crafted this for our American Dream argument paper: Introduction draws reader in and offers a strong, well-supported claim about the American Dream. We usually develop lists of 5-10 criteria depending on the complexity of the rhetorical context. If you want an useful and usable rubric for your writing assignment then why not build it with your students?

Usability Test

After students have prepared a somewhat polished draft of their deliverable, we are ready to run our usability tests. Each student is required to read five deliverables and respond to the criteria we have identified then provide evidence if they believe the deliverable has successfully met the criteria or suggest a path for the writer to explore if it has not (often by asking a question – this is what I model the most for them in class). Although this is not a true usability test and not even truly a discount usability test, this model helps students better understand the reader-writer contract and to see their writing as a product separate from themselves. This process gives the reviewers a specific process to follow and familiarizes them with the scoring guide as well as more mentor texts to guide their final revisions. My students report that this is often the first time they found peer review useful for their writing and that they enjoy the process.

I am fairly satisfied with all the steps in this process, but as I read my students final deliverables for the semester I know that there is still room for improvement and that is why I want to add a step between exploring and making the rules where we will dive deeper into specific elements of craft to help students develop stronger drafts and I know that peer review will also play a key role that work as well. But my goal is to design that process so that students are leading that work with the goal to reduce my own grading burden.

What does your peer review process look like? Have you used usability testing to support student learning?

Image: Evidence by Nick Youngson CC BY-SA 3.0 ImageCreator

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.