MOOCs Desperately Seeking Quality

cropped-mooc-banner1MOOCs, despite what the critics say have transformed higher education. They have spawned new vehicles for online learning, reaching new groups of learners who want and need an alternate form of traditional education. Thanks to the MOOC format learners now have numerous pathways to furthering their education. Granted, not all MOOC programs meet the definition of open, but as programs expand as we’ve seen with certificate-granting MOOCs, MOOCs for college credit, professional development MOOCs, and others, there is a pressing need for benchmarks of quality.

This need is more apparent after the recent publication of two reports: Babson’s thirteenth annual “Online Report Card” and “In search of quality: Using Quality Matters to analyze the quality of Massive, Open, Online Courses (MOOCs)”, (Allen, Seaman, Poulin & Straut, 2016; Lowenthal & Hodges, 2015).

The Babson report devotes (only) two of its sixty-plus pages to MOOCs, yet what’s most telling is the fact that the report is in its final year of publication. Though there are a variety of contributing factors, a compelling one as described in the report’s introduction, “distance education is clearly becoming mainstream” (pg. 3). In other words online education is growing up. ‘Online learning’ is simply becoming ‘learning’. The report outlines the number of organizations dedicated to online education, who report on and address issues specific to online learning. Many do address quality standards as it relates to developing and delivering online programs, as is the case with Quality Matters rubric, Online Learning Consortium’s (OLC) Five Pillars, and California State University Chico’s rubric for online instruction, yet all fall short in specifying standards for MOOCs.

This void is a concern given the number of students engaged in learning using the MOOC format which is significant. Estimates are in the range of millions—one source states there were 35 million enrollment in 2015 (“MOOC Enrolment”, 2016). Given these numbers the question is—how will MOOC learning be advanced and improved if MOOC quality isn’t addressed by organizations involved in online education? Researchers Lowenthal and Hodges bring some of these issues forward in their paper, “In search of Quality”.  They apply the Quality Matters™ rubric to six MOOCs, offered by three providers, Coursera, edX and Udacity:

The six identified MOOCs were analyzed using the Quality Matters Rubric Standards with Assigned Point Values, which involves a type of content analysis by three different reviewers using a standard coding scheme. [Quality Matters] QM has a rubric for Continuing and Professional Development that would be appropriate to use on MOOCs (Adair et al., 2014). However, we intentionally chose to use QM’s higher education rubric rather than the continuing and professional development focused rubric because of the increased initiatives about offering college credit for MOOC completion. In other words, a MOOC should score as well as a traditional online course if it is going to be worth college credit.  (Lowenthal & Hodges, 2015)

Not surprisingly, after the QM peer-review assessment all six MOOCs failed to meet QM’s passing grade of 85%. The QM rubric consists of a set of standards grouped into eight dimensions (below); in the study, most MOOCs failed in two dimensions, #5 and #7.

  1. Course overview and introduction
  2. Learning objectives
  3. Assessment and measurement
  4. Instructional materials
  5. Learner interaction and engagement
  6. Course technology
  7. Learner support
  8. Accessibility (Quality Matters, 2014)

The apparent failure of the MOOCs in this study may give fodder to MOOC critics, yet I suggest that failure stems not from the MOOCs, but from:  1) applying a tool (QM rubric) to a MOOC, which inherently serves a variety of learning purposes and needs, e.g. not just for credit, but for professional development, personal interest, etc. and 2) assessing a MOOC on dimensions such as ‘learner interaction and engagement‘ and ‘learner support‘ doesn’t make sense in context of a MOOC, specifically at the level the QM standards articulate. Considering the massive component of MOOCs, it’s almost a given that facilitating structured, mandatory engagement and active learning is next to impossible. Furthermore since MOOC students are able to choose the level of engagement based upon their learning needs, including this as a standard doesn’t fit with the intent of the course.

The study acknowledges many of these points, and serves as a vehicle for discussion about applying quality standards to courses that align with the MOOC format. The authors also highlight a critical point, if the MOOC format is used as a vehicle for granting college credit, as it appears to be, quality benchmarks are essential.

Final Thoughts
A unique approach to quality assessment (and course design) is needed; one that heeds the needs of learners, the constraints and advantages of the delivery platform, and ensures a quality learning experience. Going further, I also suggest that before establishing quality standards, institutions would do well to first identify the primary purpose and intent of the MOOC. Categorizing a MOOC based on its purpose, then establishing quality standards is a good place to start.

Allen, I. E., Seaman, J., Poulin, R., & Straut, T. T. (2016). Online report card: Tracking online education in the United States (Rep.). Babson Survey Research Group. Retrieved from

Lowenthal, P. R., & C. B. Hodges (2015). In search of quality: Using Quality Matters to analyze the quality of massive, open, online, courses (MOOCs). The International Review of Research in Open and Distributed Learning, 15(5). Retrieved from

MOOC enrolment surpassed 35 million in 2015. (2016, January 05). Retrieved from