Home » Reflections » Rubrics Matter: Our Upcoming Peer Assessment

Rubrics Matter: Our Upcoming Peer Assessment

Posted by on Tuesday, August 13, 2013 in Reflections.

by Derek Bruff

At yesterday’s study group meeting, we briefly discussed the upcoming peer assessment activity in #MapMOOC. I’m excited to try and tell a story through a map, but I’m a bit worried about the quality of feedback I’ll receive from other MOOC students. Given the research on peer assessment conducted by the CFT’s graduate assistant Katie McEwen last year, I’m convinced that the effectiveness of a peer assessment activity depends largely on the quality of the rubric. And the rubric in #MapMOOC is unfortunately fairly simple:

  • Does this map tell a complete story?
  • Is the story that this map tells presented in a compelling way?
  • Is the map designed in a way that reflects the use of best practices in cartographic design and geospatial analysis?
  • Does this map have an aesthetic look and feel that reinforces the objectives of the story it tells?

We’re asked to give our peers a 0, 1, 2, or 3 on each of these questions. What do each of those scores mean in the context of these four questions? Who knows. I might give a 1 to something you feel deserves a 3. Without clearer criteria, there’s not likely to be any consistency in how the maps are assessed across the course.

This is, in fact, I point I made at a workshop last week for our TA Orientation staff. After giving the workshop participants some student work on a math problem, I asked them to assign a grade to that work between 0 and 5. One of the student responses received a 0 from one participant and a 5 from another! Without a clear, somewhat objective rubric, there’s no consistency in grading nor any real way to interpret a score one receives.

The #MapMOOC peer assessment also includes a free-response question meant to encourage students to give each other specific feedback on their maps. Given the widely varying backgrounds and motivation levels of students in this course, I’m not expecting to get useful feedback from all of my assessors–maybe one if I’m lucky.

With that somewhat depressing context, I’ll reiterate that I’m looking forward to building a map that tells a story. I’m also looking forward to seeing the maps you all build! We might not receive very useful feedback on our work from our fellow MOOC participants, but I’m glad to know that I’ll have a local audience interested in what I make!

Image: “The Calm After the Show,” Thomas Hawk, Flickr (CC)


One Comment on “Rubrics Matter: Our Upcoming Peer Assessment”

Hi Derek,

It looks as though you are not able to see the rating-level explanation in the Coursera interface until after the official review period begins, and I can see how until then all you have to go by is what I listed on the project description page, which is not the complete rubric. I have in fact provided rating-level explanations for each numerical rating, since otherwise I’d agree that the rubric would be mostly useless.

I’m making changes right now to the final project description page to fill in the rest of those details so that this is completely described before the review period begins.

-A.

Anthony Robinson on August 13th, 2013 at 2:32 pm

Leave a Reply