Michael is a Professional Practice Fellow at the Higher Education Development Centre. This article will appear in an up-coming issue of ‘Akoranga’, HEDC’s quarterly publication.
In December 2008, the University of Otago upgraded Blackboard to version 8. This version has a number of new features, including a facility for self and peer assessment.
What is Self and Peer Assessment?
With self assessment a student submits their assignment in Blackboard, and then, after the deadline, criteria are provided and the student evaluates their submission. Evaluation can involve assigning marks (which can be set up to allow for partial credit, or just right/wrong), and/or textual feedback.
With peer assessment the process is the same, except that instead of evaluating their own assessment, students evaluate a given number (e.g. 3) of other students’ submissions.
How do I create a peer/self assessment?
Go to the control panel, click on the name of a content area (like “Course Documents”), then select “Self and Peer Assessment” from the drop-down menu on the top right and click on go.
You then specify information for the assessment, including the deadline for submitting assessment, and the period during which peer and/or self assessment is to take place (which must start after the submission deadline).
You then need to add questions and criteria. Click on Modify (in the row of the newly created assessment), and then on Assessment Canvas. Then click on the “+Question” (top left) to create a question. One a question has been created, you can modify the marking criteria for that question (“Criteria”, then “+Criteria” top left).
For more information, see ITS’ training courses (http://www.otago.ac.nz/its/training/).
How might I use it?
There are various approaches and reasons for using self/peer assessment.
One reason is to reduce staff workload by shifting marking to students. The literature on peer-assessment has a range of studies that compare peer-assigned and staff-assigned marks. The conclusions are variable: some studies show very high levels of agreement, others do not. Given some of the issues and additional work required to make peer-assessment work effectively (see below), it appears that using peer-assessment to reduce staff workload is not sensible.
Where self and peer assessment make sense is in relation to life long learning. Peer assessment is an opportunity to develop critical thinking. Self assessment is an opportunity to develop the ability to reflect on one’s progress, knowledge and skills. The combination can be particularly valuable: peer assessment can provide feedback that allows students to gradually “callibrate” their self-assessments.
In deciding how to use self and/or peer assessment the following are some questions that should be considered:
- Will assessment be self, peer, or both? If peer, then how many peers will assess each submission?
- Is the assessment used to aid learning (formative) or to assess whether learning occurred (summative)?
- What is being assessed? An essay? Short questions? Is there a single assessment, or is peer/self assessment used repeatedly to provide regular feedback?
- What marks are allocated? Are there marks for the assessment, for the peer reviews, for the self assessment, or for some combination? Are marks given based on the quality, or on participation?
- What marking is done by staff? If there are significant marks and the assessment is summative, then staff marking becomes more important. On the other hand, if assessment has relatively limited marks and is formative, then staff marking may not be necessary (although it might be useful initially to give peer-assessors feedback).
What pedagogical pitfalls do I need to watch out for?
Assessing is a skill. Students will need help to develop this skill. It is not reasonable (or effective) to just provide them with submissions and say “mark these”. One possibility is to discuss sample submissions in lectures/tutorials to illustrate the assessment process and thinking: what is a good assessor looking for? why do they assign marks? What makes a good submission?
Clearly defined marking criteria are essential. One possibility (for small classes) is to develop these collaboratively with students. However, it is more likely that staff will develop criteria.
Finally, it is a good idea to be explicit about aims and expectations: tell the students what you expect of them, explain why you are using peer/self-assessment, discuss reflection and critical thinking, and their relationship to life long learning.
What technical pitfalls do I need to watch out for?
As is often the case with new features, there are some niggling issues. It is worth being aware of these, since these issues can be mitigated by being careful with how self and peer assessment is used. These issues are somewhat exacerbated by the nature of peer assessment: it’s not possible to thoroughly test things without having multiple accounts.
A few issues concern the allocation of assessment to peers.
One issue is that allocation is done when the deadline is reached. Since there is no facility to modify the allocation manually, it is not possible to handle late submissions.
Another issue is that the allocation assigns missing submissions. Suppose we have a course with three students, Alex, Cameron and Jamie. Alex and Cameron both submitted assessments, but Jamie didn’t: perhaps Jamie has stopped attending the paper and the paperwork has not yet been processed. Alex and Cameron will each be asked to evaluate Jamie’s (empty) submission. Worse, Jamie will have been allocated Alex and Cameron’s submissions to evaluate, and these will not be evaluated.
Mitigating strategies include using peer assessment once membership of the paper is stable; not assigning significant marks to it; and being prepared to manually work around the system, for example, manually providing copies of Alex and Cameron’s submissions to another student, or to assess them yourself.
There are also a few slight annoyances with the way the user interface is structured.
The Results page allows the lecturer to see, for an individual student, the peer feedback and marks. One minor issue is that the page is sorted by evaluator. So, if Alex and Cameron evaluated Jamie’s submission, the order in the page will be something like:
- Alex’s feedback on question 1
- Alex’s feedback on question 2
- Cameron’s feedback on question 1
- Cameron’s feedback on question 2
- etc.
This makes it a little awkward to consider a given question and see all of the feedback on it at once. The mitigation strategy here is simply to avoid setting too many questions. Another issue is that the Results page doesn’t show the answer given, just the feedback on it.
Overall, although the self and peer assessment feature has some issues, it can still be a useful tool. If you are interested in using it we’d love to hear from you!
Dr. Michael Winikoff
Acknowledgements: I would like to thank Annemaree Senior for her assistance.
Further reading:
Dochy, F., Segers, M. and Sluijsmans, “The use of self-, peer and co-assessment in higher education: A review”,Studies in Higher Education,24:3, 331—350, 1999. DOI: 10.1080/03075079912331379935
Keith Topping, “Peer Assessment Between Students in Colleges and Universities”, Review of Educational Research, 68:3, 249-276, 1998.
DOI: 10.3102/00346543068003249
(especially pages 265-267)
