A MOOC Quality Scorecard applied to Coursera Course

In this post I review a recently completed Coursera course using a quality scorecard approach to measure and quantify five key dimensions of the course.

checkmarkI’m in the final week of a Coursera MOOC, Sports & Society that for the most part has been lackluster and disappointing. I expected a university level course that would provide  learning and perspectives into the social and cultural dimensions that affect sports participation and perceptions across different cultures. It missed the mark. Granted, the majority of Coursera courses are ‘lite’ versions of the real thing—few mimic the workload or rigor of the face-to-face counterparts, which is fine [even preferred when taken for personal development] given the courses are not promoted as such.

Though a lite version does not mean that meaningful and deep learning cannot occur. I’ve completed two other Coursera courses, Introduction to Sociology, and E-learning and Digital Cultures. Both courses provided rich learning with scholarly materials, challenging assignments and opportunities to gain knowledge beyond the course site. Though the Sports & Society course wasn’t completely inadequate, in fact some things were done well, but for all the effort and resources that went into the course, it missed the mark quite significantly in terms of providing conditions for meaningful learning to occur. I see an opportunity to share here with readers what contributed to a mediocre MOOC learning experience. To provide an illustrative framework for this review, I’ve created a MOOC quality scorecard review, that is [loosely] based on a quality scorecard approach and my   course design experience.

Course Overview
I have a keen interest in the topic of Sports & Society, specifically youth and college sports and its effects on youth development – the emotional, physical, and educational dimensions. I won’t get into details here, but I write about this topic on another one of my blogs, school and sports, and research the topic frequently. Coursera’s Introduction to Sociology expanded my interest in the topic, which has led me to several excellent resources on Sports Sociology.

To put this course review into context, following is the course description for Sports & Society. I want to emphasize that they were no objectives, goals or purposes outlined for the course, which made defining the scope of the course a challenge, as well as determining my learning goals.

Course Description: “Sports play a giant role in contemporary society worldwide.  But few of us pause to think about the larger questions of money, politics, race, sex, culture, and commercialization that surround sports everywhere. This course draws on the tools of anthropology, sociology, history, and other disciplines to give you new perspectives on the games we watch and play. We will focus on both popular sports like soccer (or “football,” as anyone outside America calls it), basketball, and baseball, and lesser-known ones like mountain-climbing and fishing.”   

MOOC Quality Scorecard Review: Coursera, Sports & Society

Criterion Scoring
0 points = Not Observed
1 point = Insufficiently Observed
2 points = Moderate Use
3 points = Meets Criterion Completely

1)  Quality of Instructional Methods: Sports & Society:  Score: 1/3

Criteria for scoring
♦ Course includes objectives that provide direction for course ♦ instructor demonstrates expertise in subject area, presents topics that relate to course objectives in cohesive manner ♦ course environment encourages student to make connections/construct new knowledge and/or demonstrate knowledge ♦ instructor presents alternative viewpoints ♦ students encouraged to draw upon personal experience and encouraged to apply/reflect on course concepts.

My Comments
The course felt disjointed, topics were unrelated and did not appear to move towards an objective or goal. It was a passive learning experience, with little encouragement or opportunities to apply content, reflect or share content/resources. The course was instructor-focused, content was primarily delivered via professor lecturing. Instructor did not go into deep analysis of principles, apply theoretical constructs or provide different perspective on topics – for example, neglected to mention other  institution’s in society, and the role/influence each play in sports, i.e. governments, International Olympic Committee, NCAA (in the US for instance) etc. Another, the instructor presented a narrow point-of-view on the topic – Business of Sports, suggesting that big corporations are solely responsible for the commercialization of professional and college sports.

starn_logo

sports & society, Coursera

2) Quality, Depth & Breadth of Course Materials: Sports & Society: Score: 2/3

Criteria for scoring
♦ Variety of content sources with breadth of perspectives ♦ content goes beyond professor video lecture ♦ numerous links provided to open source content with variety/breadth ♦ students encouraged to contribute to course content and/or have access to venue within course site to share/discuss findings ♦ readings include scholarly sources with peer-reviewed papers via downloadable PDF format and/or open access resources ♦ current content included to provide relevant context/perspective for topic.

My Comments
Video lectures were informative, though primarily featured professor lecturing. There were some references made to the readings. Weekly readings varied, with average number of pages assigned between 15 and 30. It might have been helpful to have  optional readings each week, for students wanting to delve deeper into a topic (there were two or three).  Two of the weekly featured selected chapters from a non-fiction book—one from a book the professor authored. A link was provided for those interested in purchasing the book at a discounted price for Coursera students, which comes across as a tactic to sell the book, more so when it is the only reading for the given week. Only two or three readings were from scholarly sources/peer-reviewed journals (here is an example of readings from week 6, here, (a chapter from a non-fiction book)and  an optional reading here). Few materials encouraged deep analysis or described theoretical principles of topic.

3) Interaction: Student & Social Engagement: Sports & Society: Score: 2/3

Criteria for scoring
♦ Students are provided a venue within course site to interact with other students ♦ forums are monitored to ensure discussion is respectful, non-threatening and safe ♦ Twitter hashtag created for course ♦ students encouraged [not mandated] to engage with other students via platforms outside of course site ♦ instructor/ TA engages within forum discussions that are specific to course topics to promote higher order thinking ♦ live events [Google hangout/webinar] for students to watch/engage via real-time comments, i.e. professor with guest, groups of students discussing reading etc.

My Comments
Four Google Hangout events scheduled, though two were cancelled due to technical difficulties. The two that did work featured guest speakers, and select class members for discussion, and allowed for students to post live comments while watching. These were very good.  Forums had few students commenting and contributing, but this allowed for easy to follow discussions threads. Instructor did get involved in some discussion forums. A Twitter hashtag was not assigned to course to promote interaction or sharing. This is a missed opportunity, as Twitter is an excellent platform to engage students throughout the course, to share content, promote blog posts, encourage chats & connections, etc.

4) Activities & Assessments: Sports & Society: Score: 1/3

Criteria for scoring
♦ Learning activities prompt students to share their learning ♦ activities leverage international perspectives of student body ♦ instructions for assignment include descriptions of the purpose and rationale i.e. why learners are doing the assignment/activity ♦ assessments provide further opportunity to learn ♦ assessments encourage students to find information ♦ assessment(s) align with course outcomes [as per certificate].

My Comments
Little encouragement or opportunity to apply content, to reflect and share.  Assessments were five-question quizzes, two per week—one on assigned reading and the other lecture content. Neither assessments nor activities promoted higher order thinking skills, analysis, evaluation or critical thinking.

5) Interface of Course Site / Instructional Design: Sports & Society: Score: 3/3

Criteria for scoring
♦ ‘Start-here’ section [orientation] included for introduction to course and site itself ♦ course instructions and requirements for each week/module are detailed ♦ clear expectations for assignment completion, peer review process and/or quizzes/tests outlined and accessible ♦ clear guidelines for certificate requirements as applicable ♦ instructional materials are accessible and easy to use ♦ links to technical support.

Comments
Very good interface on Coursera site, easy to navigate. Includes a ‘start-here’ orientation page. Instructions and expectations concise and thorough. Schedule and due dates page was helpful to clarify dates.  Technical support available.

MOOC Scorecard for Sports & Society: Total  9/15

I’ll close with this: I appreciate that Cousera provided this course for free. I also appreciate the time that the professor put into this course; his work and effort, which no doubt was considerable. The score above is based upon my personal viewpoint. The purpose of this post is to provide insight into what could improve the course, and for other educators to learn from one student’s perspective.

Resources:

10 thoughts on “A MOOC Quality Scorecard applied to Coursera Course

  1. Pingback: A MOOC Quality Scorecard applied to Coursera Course _ online learning insights | 磨課師知識庫

  2. Pingback: Learning designs for MOOCs | Online Learning Designs

  3. The MOOC scorecard is a great idea, and I hope you continue to use it for assessing future courses you’re involved in.

    As far as rigor is concerned, have you tried any more technical courses? I know I got my butt kicked in Principles of Economics for Scientists and Probablistic Graphical Models, both on Coursera.

    If you don’t mind, I may incorporate your scorecard as well when reviewing my MOOC experiences.

  4. Thank you for this critical assessment – as someone who is interested in both online learning and sociology of sports, I found this fascinating. I have been trying to develop some ideas for mooc online learning designs that provide that kind of application of the insights to experience and sharing of different perspectives that you missed on this mooc. Is that the challenge for moocs in this kind of discipline area? Was it done better on your other courses – what about on your intro to sociology mooc?

  5. Debbie,

    Thanks very much for the thoughtful post. Using the Sloan scorecard is a great tool. I just completed a coursera course – Nutrition, Health, and Lifestyle: Issues and Insights – that I likely would have graded much in the same way as your sports course. Very lecture intensive, with lots of good info, got a bit redundant, but a ton of links and further direction. On the other hand, I am two weeks into a coursera course – Introduction to Art: Concepts & Techniques – that is absolutely fascinating to me. However, the latter course is much less intensive in terms of lecture, does not really provide much direction for further study. But the instructor did a fantastic job of contextualizing Dada and Surrealist movements in time and space, coupled with applying their techniques in participant art projects.

    I have been reflecting more and more on these online experiences. I am still very keenly interested in MOOCs. In the past year, I have enrolled in perhaps 8 coursera courses, completed 3, have 1 in process, and dropped out of 4. Two of the ones I dropped were because it was over my head or I simply ran out of time. Two I dropped for lack of interest.

    I have been thinking more about how I use MOOCs. Those that I might use immediately in my work/career I am quite a bit more judgmental about. If the career interest offerings are all intro surficial material, I feel I have wasted my time. I am a lot less critical of the above two noted here that were more of personal fulfillment. I can accept more of a hit or miss approach on the latter. My total MOOC experience seems to tie in well with general democratization of education perspectives put forward by the coursera founders.

    Here are some thoughts:

    When I was a kid in the 1950s we had three TV channels. Today with cable there are hundreds and I become increasingly bored with TV, but there are a few programs that really resonate with me and I am loyal too. Are MOOC opportunities becoming much the same?

    The novelty of MOOCs is past. Now is the time to get down to the real business of what these things are going to become. I think that is part of where your post goes. The coursera will need to do more than just put up content. I suspect that in the same way the “Rate my Professor” sites are seemingly popular in higher ed, similar tools might develop for MOOC content – along the lines of the Sloan model you present. (Hmm . . . sounds like a good entreprenurial venture!)

    In this way, I suspect that MOOCs are going to be much like the bricks and mortar classroom, continuing education offerings, study groups, book groups, etc. etc. – some are as interesting as watching paint dry and others are fantastic. As someone who ultimately got my graduate degrees in anthropology with a focus in archaeology, if I think back to my formal coursework, and some of those classes were absolutely awful where I struggled just to keep my eyes open! It’s not about online vs. bricks and mortar.

    My experience with MOOCs to date is that for the one’s I have completed, I would be happy to pay 20.00 just for the content. As a university professor myself, I am not opposed to paying for the production cost.

    I remain excited to see where these processes are going.

    • HI Robert,
      Thanks for your thoughts which are most insightful. You make a good point, that the novelty has worn off as far as MOOCs are concerned, and now the real work will begin – how can we leverage the MOOC delivery model to meet education needs of various groups? It is not a one-size-fits all scenario as your comment highlights.

      MOOCs have great potential, but determining why and how the model will be used is key – one cannot evaluate its effectiveness unless we measure it against something – which you allude to – for professional development? As classroom supplement? etc.

      I too look forward to see where higher education is going! Thanks for commenting Robert. As always your thoughts and comments are most appreciated.

  6. Pingback: A MOOC Quality Scorecard applied to Coursera Course « Analyzing Educational Technology

  7. I enrolled in this course – my first experience with a MOOC – to broaden my experiences with elearning to help extend my current vocational teaching. I started with much enthusiasm -as you note. the interface, design and instructions were very clear. However I only lasted 3 weeks, and not only due to external demands.
    I enjoyed the lectures and readings for their broad reach (in terms of sports covered), and some of the online discussion threads, particularly instructor involvement.
    But the quiz-based assessment was a big disappointment. Useful for checking grasp of key facts and terminology from the lectures (much of which I knew already – as would many sports fans enrolling in this topic), but without any need to analyse, reflect, question, or connect to other learning.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s