Peer Grading in Online Classes: Does it Work?

Is student grading good enough to use? A loaded question – and though it is context-dependent, the answer is yes. Yesterday I peer-graded six student midterm exams in the Introduction to Sociology course I’m taking through Coursera, along with 30,000 other enrolled students. In this learning environment, also known as a MOOC, peer grading is the only option, though it did motivate me to research its effectiveness and applicability to online courses for credit (the context for this post). Intrigued, I could see the potential in light of the online program at my workplace, not only from a time-saving standpoint for the instructor, but also for the potential in enhancing the student learning experience.

Enhanced Learning
I see the value of peer grading for what the student gets out of it, more than for the time it saves the instructor. I experienced this value first hand as mentioned above. The exam in question consisted of two short answer questions each requiring 250 words, and an essay response requiring 750 words. Though tedious [90 minutes of my undivided attention], I was far more familiar with the course concepts after the exercise, it was well worth the time.

Several of my classmates reported the same phenomenon via the discussion board – the deeper learning experienced while grading their peers. Below are snippets from select posts on the discussion board from the Coursera Introduction to Sociology course:

“…I actually enjoyed the peer review part mostly because of looking at different answers, giving me more perspective from an group of people all from different cultures. Not only that, but I felt I was able to be unbiased while grading my own work ….” Greg

I’ve learned a lot through the peer assessment, and even though maybe the scores will not be perfect, everybody who goes through that process will have now a better and more complete understanding of the first half of the course.” Horacio

For what it’s worth, one of the essays I corrected didn’t seem too good on first reading but when I checked it against the rubric, lo and behold, most of what was required was there.” Ron

“In my experience as an educator for over 25 years, it’s often the case that we think our grades should be higher than they are. ….the rubric was great, very thorough and complete…” Kendall

Accuracy
How accurate is peer grading? Another interesting question, and it depends upon the perspective. The viewpoint that most educators are familiar with, is where the instructor’s grade is the benchmark – his or her grading is the standard for measuring accuracy.

This is the method used in a research study by two biology professors at the University of Washington which determined that on a per-question basis, students were more generous in grading, actually 25% more, “0.27 points—roughly a quarter point on each 2-point question”. However, despite the differing grades, authors support peer grading and suggest further research be done to examine its value and the role peer grading can play in enhancing student learning. (Freeman & Parks, 2010).

Mechanics
How can student grading be effective within an online environment? Effectiveness depends upon the thoroughness and specificity of the rubric. A grading tool that guides the student through assessment of short answer and essay questions is critical. Below is an example from the research study mentioned earlier [a biology course was used for the  study]. There may be five or six of these questions for each point within a given question.

Sample answer: If the two species mate on different fruits,
then no gene flow occurs and they are reproductively isolated.

Rubric

  • Full credit (2 pts.): Clear articulation of logic that mating on different fruits reduces or eliminates gene flow—a Prerequisite for speciation to occur.
  •  Partial credit (1 pt.): Missing or muddy logic with respect to connection between location of mating and gene flow, or no explanation of why reductions in gene flow are Important.
  • No credit (0 pts.): both components required for full credit missing; no answer; or answer is unintelligible.  (Freeman & Parks, 2012)

Recommendations
To begin peer grading in an online course consider the following:

  • Create your own rubric that provides standards for each point the question is worth. For instance if a given questions is worth 6 points, 6 statements will need to be developed, similar to the one above.
  • Create detailed instructions for students that clearly outline how peer grading will work.
  • Set-up the process so that students grade a minimum of 3 peers’ assignments/exams and self-grade his or her own.
  • Average the peer scores and include the student’s self-graded assignment.

Though creating a peer review exercise is time consuming at the outset, rewards are tremendous.  First, for the potential time saved by the course instructor in grading, and second, [perhaps the most important] is the value that peer grading provides for the student.

References:
Freeman, S. & Parks, C. How accurate is peer grading (2010). CBE—Life Sciences Education. Vol. 9, 482–488.

Resources:
Peer Review, Peer Grading, JaZahn
UCLA’s Calibrated Peer Review, Eric Mazur.  This is a software program/platform that can handle and support peer grading for large classes and/or institutions that plan to implement peer grading in several classes.

15 thoughts on “Peer Grading in Online Classes: Does it Work?

  1. Pingback: Piaget & Vygotsky: Outdated or adaptable? | Stephen Engel-Hart

  2. Pingback: #IOLchat Report: Grading Challenges and Online Teaching - Online College.org

  3. Pingback: Once upon a course in Princeton… Review of Coursera.org experience – David I. Agogo

  4. Pingback: Review of the Coursera Human Computer Interaction Course | Reflections

  5. Thank you for posting about this topic. I am a high school teacher who uses peer grading in my classroom. The one I use most is on Collaborative Work Skills for when my students work in group settings. It is a 32-point rubric with each section being worth 4 points. I tell them it is the same as getting evaluated at work. I ask them two questions: Do you want someone to get a raise, promotion, or bonus when you did most of the work? How truthful would you be?

    I am currently enrolled at Walden University learning about to be an instructional designer for online learning. I was glad to see that peer grading is carried out in a successful way, because I wondered if it was possible.

    For my current Walden course, I am required to keep a blog on instructional design issues. I used your blog as a reference for my Week 2 Blog.

  6. You correctly point out that in the right context, peer grading can be a useful tool to enhance learning. I think the key to the success of this model lies with a learner. Is the individual self-motivated and interested in the subject? Does the individual have a personal stake in learning the material?

    Peer grading works in the MOOC model because there is nothing at stake for the learner except to expand knowledge of the subject matter. Participants want to learn more about a particular subject matter and the learning is conducted for its own sake.

    Can this model (or should this model) be applied to higher education? I believe the answer again depends upon the context. I think this model would see greater success among graduate level students or for higher level undergraduate programs as long as a clear and appropriate rubric is established. These courses tend to be smaller, more focused and the students taking these classes are more interested in the subject. I would also like to see what impact peer grading has on an individual’s own research. Does the competition for time between research and peer grading negate the benefit?

    Regarding the accuracy of peer grading, can it be any worse than what is already occurring in higher education? Several years ago, there was an article in the New York Times about the efforts Princeton University was taking to reign in grade inflation (Foderaro, 2010). Of course, the problem of grade inflation is not limited to Princeton. Stuart Rojstaczer provides a detailed analysis on this issue at gradeinflation.com. Could peer grading be used to help balance grade inflation that occurs in higher education? Possibly – although with such fierce competition between schools for students, accreditation, and a top spot in U.S. News & World Reports “Best Colleges” list, I do not know if peer grading will change the way institutions do business. If you look at peer grading as a way to enhance learning then it is worth looking into.

    Foderaro, Lisa W. (2010, January 29). Type-A-Plus Students Chafe at Grade Deflation. The New York Times. Retrieved from http://www.nytimes.com/2010/01/31/education/31princeton.html?_r=1

    • Dear Russell
      You bring up an excellent point that peer grading works successfully when students are motivated and interested in learning for personal enrichment which contrasts with grading in higher education environment at the undergraduate level. The article you included is excellent as it sheds light on the challenges of implementing peer grading at the undergraduate level. One of the key challenges appears to be the competitive nature of grades as perceived by students, and thus the pressure on faculty to grant the ‘A’s when it appears to be the ‘standard’ rather than the exception. In the sociology course I’m taking through Coursera, the professor is a Princeton Professor, and he mentioned the realigning of standards for granting grades in one of his lectures. He did not emphasize the problems it caused, but I would be interested to learn more from his perspective.

      I appreciate your perspective — the implementation of peer grading is certainly far more complex in courses that ‘count’ towards a degree, even though it has some merits for enhancing learning. Further exploration and discussion into peer grading will be a worthwhile endeavor.

      Thanks for your thoughtful response, and the link to the article. Debbie

  7. Pingback: Hacking Assessment: Redesigning the Numbers Game « Remixing College English

  8. Pingback: Peer Grading in Online Classes: Does it Work? | online learning … « Performance By Design

  9. I’ve been taking the Coursera HCI course, and have enjoyed the peer grading component. As the assignments have been largely creative, it’s been very useful to see other people’s work.

    The assessments follow a clear rubric. There’s been a trial grading section each week, where you learn to assess appropriately. You mark a few assignments and see if you’re close to the grade that the Professor gave. Once you’re good enough you graduate on to ‘real’ marking. (I’m not sure if it’s possible to fail this process, and if there’s a concomitant penalty to one’s grade. I haven’t dared to try!)

    It certainly helps to have graded other people before you go back and grade your own work. It does encourage you to view things more humbly and objectively.

    The pedagogy section of the Coursera website is pretty confident on the literature underlying student peer reviews and crowd-sourcing in this context: https://www.coursera.org/about/pedagogy

    In terms of how we deliver online peer assessment as part of a course, it’s worth thinking about how the timing works.
    In any given week in the HCI course, learners have to carry out peer assessment and work on their own assignment (in addition to viewing the lecture and doing the quick quiz). This means that assessment and assignment can compete with each other for attention, and there’s not much of a formal incentive to lavish attention on the peer assessment.

    Whilst I think the peer evaluation mechanism has been very good, perhaps the overall HCI course would be improved if there were alternating weeks of creation and assessment. This could encourage deeper reflection – and could be used to drive reflective peer discussion.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s