A MOOC Quality Scorecard applied to Coursera Course

In this post I review a recently completed Coursera course using a quality scorecard approach to measure and quantify five key dimensions of the course.

I’m in the final week of a Coursera MOOC, Sports & Society that for the most part has been lackluster and disappointing. I expected a university level course that would provide  learning and perspectives into the social and cultural dimensions that affect sports participation and perceptions across different cultures. It missed the mark. Granted, the majority of Coursera courses are ‘lite’ versions of the real thing—few mimic the workload or rigor of the face-to-face counterparts, which is fine [even preferred when taken for personal development] given the courses are not promoted as such.

Though a lite version does not mean that meaningful and deep learning cannot occur. I’ve completed two other Coursera courses, Introduction to Sociology, and E-learning and Digital Cultures. Both courses provided rich learning with scholarly materials, challenging assignments and opportunities to gain knowledge beyond the course site. Though the Sports & Society course wasn’t completely inadequate, in fact some things were done well, but for all the effort and resources that went into the course, it missed the mark quite significantly in terms of providing conditions for meaningful learning to occur. I see an opportunity to share here with readers what contributed to a mediocre MOOC learning experience. To provide an illustrative framework for this review, I’ve created a MOOC quality scorecard review, that is [loosely] based on a quality scorecard approach and my   course design experience.

Course Overview
I have a keen interest in the topic of Sports & Society, specifically youth and college sports and its effects on youth development – the emotional, physical, and educational dimensions. I won’t get into details here, but I write about this topic on another one of my blogs, school and sports, and research the topic frequently. Coursera’s Introduction to Sociology expanded my interest in the topic, which has led me to several excellent resources on Sports Sociology.

To put this course review into context, following is the course description for Sports & Society. I want to emphasize that they were no objectives, goals or purposes outlined for the course, which made defining the scope of the course a challenge, as well as determining my learning goals.

Course Description: “Sports play a giant role in contemporary society worldwide.  But few of us pause to think about the larger questions of money, politics, race, sex, culture, and commercialization that surround sports everywhere. This course draws on the tools of anthropology, sociology, history, and other disciplines to give you new perspectives on the games we watch and play. We will focus on both popular sports like soccer (or “football,” as anyone outside America calls it), basketball, and baseball, and lesser-known ones like mountain-climbing and fishing.”   

MOOC Quality Scorecard Review: Coursera, Sports & Society

Criterion Scoring
0 points = Not Observed
1 point = Insufficiently Observed
2 points = Moderate Use
3 points = Meets Criterion Completely

1)  Quality of Instructional Methods: Sports & Society:  Score: 1/3

Criteria for scoring
♦ Course includes objectives that provide direction for course ♦ instructor demonstrates expertise in subject area, presents topics that relate to course objectives in cohesive manner ♦ course environment encourages student to make connections/construct new knowledge and/or demonstrate knowledge ♦ instructor presents alternative viewpoints ♦ students encouraged to draw upon personal experience and encouraged to apply/reflect on course concepts.

My Comments
The course felt disjointed, topics were unrelated and did not appear to move towards an objective or goal. It was a passive learning experience, with little encouragement or opportunities to apply content, reflect or share content/resources. The course was instructor-focused, content was primarily delivered via professor lecturing. Instructor did not go into deep analysis of principles, apply theoretical constructs or provide different perspective on topics – for example, neglected to mention other  institution’s in society, and the role/influence each play in sports, i.e. governments, International Olympic Committee, NCAA (in the US for instance) etc. Another, the instructor presented a narrow point-of-view on the topic – Business of Sports, suggesting that big corporations are solely responsible for the commercialization of professional and college sports.

sports & society, Coursera

2) Quality, Depth & Breadth of Course Materials: Sports & Society: Score: 2/3

Criteria for scoring
♦ Variety of content sources with breadth of perspectives ♦ content goes beyond professor video lecture ♦ numerous links provided to open source content with variety/breadth ♦ students encouraged to contribute to course content and/or have access to venue within course site to share/discuss findings ♦ readings include scholarly sources with peer-reviewed papers via downloadable PDF format and/or open access resources ♦ current content included to provide relevant context/perspective for topic.

My Comments
Video lectures were informative, though primarily featured professor lecturing. There were some references made to the readings. Weekly readings varied, with average number of pages assigned between 15 and 30. It might have been helpful to have  optional readings each week, for students wanting to delve deeper into a topic (there were two or three).  Two of the weekly featured selected chapters from a non-fiction book—one from a book the professor authored. A link was provided for those interested in purchasing the book at a discounted price for Coursera students, which comes across as a tactic to sell the book, more so when it is the only reading for the given week. Only two or three readings were from scholarly sources/peer-reviewed journals (here is an example of readings from week 6, here, (a chapter from a non-fiction book)and  an optional reading here). Few materials encouraged deep analysis or described theoretical principles of topic.

3) Interaction: Student & Social Engagement: Sports & Society: Score: 2/3

Criteria for scoring
♦ Students are provided a venue within course site to interact with other students ♦ forums are monitored to ensure discussion is respectful, non-threatening and safe ♦ Twitter hashtag created for course ♦ students encouraged [not mandated] to engage with other students via platforms outside of course site ♦ instructor/ TA engages within forum discussions that are specific to course topics to promote higher order thinking ♦ live events [Google hangout/webinar] for students to watch/engage via real-time comments, i.e. professor with guest, groups of students discussing reading etc.

My Comments
Four Google Hangout events scheduled, though two were cancelled due to technical difficulties. The two that did work featured guest speakers, and select class members for discussion, and allowed for students to post live comments while watching. These were very good.  Forums had few students commenting and contributing, but this allowed for easy to follow discussions threads. Instructor did get involved in some discussion forums. A Twitter hashtag was not assigned to course to promote interaction or sharing. This is a missed opportunity, as Twitter is an excellent platform to engage students throughout the course, to share content, promote blog posts, encourage chats & connections, etc.

4) Activities & Assessments: Sports & Society: Score: 1/3

Criteria for scoring
♦ Learning activities prompt students to share their learning ♦ activities leverage international perspectives of student body ♦ instructions for assignment include descriptions of the purpose and rationale i.e. why learners are doing the assignment/activity ♦ assessments provide further opportunity to learn ♦ assessments encourage students to find information ♦ assessment(s) align with course outcomes [as per certificate].

My Comments
Little encouragement or opportunity to apply content, to reflect and share.  Assessments were five-question quizzes, two per week—one on assigned reading and the other lecture content. Neither assessments nor activities promoted higher order thinking skills, analysis, evaluation or critical thinking.

5) Interface of Course Site / Instructional Design: Sports & Society: Score: 3/3

Criteria for scoring
♦ ‘Start-here’ section [orientation] included for introduction to course and site itself ♦ course instructions and requirements for each week/module are detailed ♦ clear expectations for assignment completion, peer review process and/or quizzes/tests outlined and accessible ♦ clear guidelines for certificate requirements as applicable ♦ instructional materials are accessible and easy to use ♦ links to technical support.

Comments
Very good interface on Coursera site, easy to navigate. Includes a ‘start-here’ orientation page. Instructions and expectations concise and thorough. Schedule and due dates page was helpful to clarify dates.  Technical support available.

MOOC Scorecard for Sports & Society: Total  9/15

I’ll close with this: I appreciate that Cousera provided this course for free. I also appreciate the time that the professor put into this course; his work and effort, which no doubt was considerable. The score above is based upon my personal viewpoint. The purpose of this post is to provide insight into what could improve the course, and for other educators to learn from one student’s perspective.

Resources:

• A Quality Scorecard for the Administration of Online Programs, The Sloan Consortium
• MOOCs, Robots, and the Secrets of Life, A comprehensive piece about two MOOC experiences, of which the Coursera experience missed the mark.
• Coursera Condensation, the author presents a different argument, but one that is worth considering.