Your Librarian
Peer Review and Evaluating OER
Crowdsourced Quality Control
In academia, we have been using crowdsourced quality control since time immemorial. We call it Peer Review. Sometimes OERs are vetted through a formal peer review process that is very similar to what is used on formally published textbooks and scholarly journals; other times the process is self-selecting and informal. Some OERs are not vetted at all, and the quality control is entirely up to the faculty who wants to adopt or adapt the OER.
Here is a short video that explains how, in aggregate, crowds can be very intelligent and astute. NOVA scienceNOW | Wisdom of the Crowds | PBS (4:49).
OER repositories and lists generally have features like:
- Hit counts (a popularity vote)
- Comments (more qualitative information)
- Ratings
An increasing number of reputable OER repositories and lists have an actual peer review system as well. The peer reviewers are given training and assigned content to review based on their areas of expertise.
The Educator Who Is Using the Content Is Responsible for Evaluating the Content
As always, the faculty member who is selecting the content has to take responsibility for making sure that it is appropriate to the audience and learning objectives, correct and accurate, accessible and usable, etc. The educator as content expert has always been expected to be vigilant with educational resources, even ones that come from traditionally authoritative sources.
Evaluate for:
- Subject matter, for accuracy, objectivity, completeness.
- Style and presentation, to make sure that it is appropriate for your learners' level, learning style and modality, etc.
- Pedagogy, to make sure it fits well with the kinds of assignments and assessments you want to create.
- Technical aspects, to make sure that you and your learners will be able to use it without undue inconvenience. This includes accessibility for disabled learners and fellow faculty and staff!
Open Educational Resources can fall short of formally published educational materials in terms of their level of polish, but you don't have to just deal with it. You can tweak and polish them until they fit perfectly. That is not something that is possible with traditionally published educational materials, which are copyrighted and often have DRM (digital rights management).
Rubric for Evaluating OER
Here is a rubric to help you evaluate OER: Rubric: Evaluating OER.
Let it guide you but don't have it dictate your choices. Each of the categories in it is just something to think about. Scoring "poor" or "unknown" in one or two categories doesn't have to be a deal breaker, especially if you are willing to make changes to the OER before using it.
