A Guide for MOOC Course Developers and Facilitators: “The MOOC Case Book”

Following is a review of “The MOOC Case Book: Case Studies in MOOC Design, Development and Implementation” the second-place winner of the Association for Educational Communications and Technology (AECT)’s Division of Distance Learning (DDL) Book Award, 2016.

Image of Book Cover: The MOOC Case Book: Case Studies in MOOC Design, Development and Implementation
“The MOOC Case Book”, Linus Learning (2015)

The “MOOC Case Book” is a collection of case studies written by (mostly) educators sharing their experiences developing, delivering, and supporting a Massive Open Online Course (MOOC) from the perspective of one of eight dimensions of Khan’s e-learning framework. Khan’s Framework provides structure to the daunting task of developing a MOOC. Not only from a course design perspective but by providing strategies for the host of factors that impact the success of a MOOC. Factors associated with the Technological dimension for instance that involve the MOOC’s platform features, and the Institutional dimension that encompasses student services such as academic advising or institution policies. There’s also the dimension of Resource Support which addresses support for faculty in course design and technical online support for students. Details of Khan’s Framework are covered in chapter two; the image below provides a snapshot.

The book is geared to readers involved in developing or facilitating a MOOC. It provides guidance and knowledge; readers can learn from the experiences of chapter-authors who have ‘been-there-and-done-that’—who have invested the time and energy needed to pull a MOOC together.  Included in the studies are two written from the student perspective, providing further depth to the stories shared. Readers interested in learning how institutions and educators are using digital platforms to deliver online learning—the methods, challenges and barriers faced, will find the studies within instructive, even entertaining.

Following is an overview of Khan’s Framework and highlights of a handful of the case studies. One disclaimer, I wrote one of the case studies—chapter #3 “Pedagogy and MOOCs: Practical Applications of Khan’s E-Learning Framework”. The chapter, as the title suggests, focuses on the Pedagogical dimension of the Framework.

About Khan’s Framework
As an instructional designer I’ve often heard from faculty and design teams the drawbacks of using a model (instructional design models such as ADDIE or the Dick, Carey & Carey model) for the instructional design process. Drawbacks mentioned include words such as, ‘cumbersome’, ‘too linear’, rigid’, and ‘inhibits creativity’. Khan’s Framework is different from traditional design models; it’s holistic. Not only does it address the design phase, but goes beyond by including elements critical to a MOOC’s success and sustainability. It expands to delivering and assessing the MOOC and supporting MOOC stakeholders. These elements, delivery and sustainability in particular are critical. More so when MOOCS are developed and implemented by higher education institutions; there are a host of issues  administrators need to know and make decisions on.

e-learning_framework_model
Khan’s E-Learning Framework

How the Book is Structured
There are eleven sections—one dedicated to each of the eight dimensions of the Framework. There are twenty-three case studies, with a great diversity in the institutions represented—Ivy league and public institutions, as well as  small private schools. The MOOCs cover topics ranging from remedial college-prep courses, English composition, poetry, statistics, global health and more. The courses are delivered on platforms readers will recognize, Coursera and EdX, and some specific to an institution; authors share the challenges, barriers and lessons learned when working with the features inherent to each.

Application
Badrul Khan’s Framework preceded the MOOC phenomenon yet its applicability to the MOOC scenarios is impressive and telling. Chapter-authors describe the factors they dealt with in detail in light of one of the eight dimensions (chapters 3 – 25). Chapter 1 gives the reader a solid overview of the factors associated with each dimension, giving context to each case study. Table #3, “Issues for Addressing MOOC Learning Environments” is especially helpful with its list of questions specific to each dimension; course development teams will find these helpful during the design phase and after the course is launched. Below are select questions specific to the Pedagogical dimension (p. 11):

  • How well does the MOOC course plan align with the course goals and outcomes?
  • Does the course provide a clear description of what learners should be able to do at every stage of the course?
  • How well is the instructional strategy being used to target each objective?
  • How good is the content? How well do learners interact with it?
  • How well does the course design contribute to an interactive and flexible learning environment?

Case Study Highlights
In respect of time I’ll highlight just a few case studies to provide a glimpse into what the book provides. Chapter 5 focuses on the Technological dimension. The author shares how digital tools, e.g. Social media and online surveys, as well as MOOC platform features were used to deliver differentiated learning for students (pp. 63 – 79). In chapter 11 the reader is given a behind-the-scenes glimpse into the development of a MOOC, “Statistics One”, offered on the Coursera platform by Princeton University. The lead author also instructor of the MOOC, analyzes the course through the lens of the Evaluation dimension. He shares the challenges of assessing student learning, specifically creating and implementing assignments, primarily a result of the features inherent to the MOOC platform (pp. 136 – 162).

The key factors associated with the Management dimension of Khan’s framework is described in chapter 13 where the chapter-author from Penn State University highlights the school’s experience delivering the MOOC “Epidemic: infectious Disease and Dynamics”, deemed by Penn State as a great success. Success defined by Penn State as adhering to a detailed project management timetable which delivered the MOOC on time and in the highest quality possible. Success also includes the fact that Penn’s MOOC achieved a higher than average completion rate (compared to average completion rates of MOOCs on Coursera) at 14%, and a rating of number one science course as ranked by Coursetalk (p. 183).

Closing
There are many applications for The MOOC Case Book; it can be a useful tool for MOOC course design teams, students of instructional design and educational technology, and for higher education institution leaders involved in MOOCs. As Curtis Bonk writes in the book’s Forward, though MOOCs are not and can never be a solution to the challenges facing education, they can expand our thinking and perspectives on the future of education that lends hope to better educational world (p. xvii).

  • “BADRULKHAN.COM.” BADRULKHAN.COM. n.d. Web. 25 Jan. 2017.
  • Corbeil, Joseph Rene, Maria Elena Corbeil, and Badrul H. Khan. The MOOC Case Book: Case Studies in MOOC Design, Development and Implementation. Ronkonkoma, NY: Linus Learning, 2015. Print.

 

MOOC Quality Comes Down To This: Effective Course Design

“Design brings forth what would not come naturally”
                                —Klaus Krippendorff, Professor of Cybernetics, Language,and Culture

designThere’s little data to go on to determine the quality of learning outcomes in Massive Open Online Courses (MOOCs). Traditional education measures the quality of learning with a variety of assessment methods against a set of established criteria or objectives. But MOOCs don’t fit into the traditional education mold and given it’s usually unclear what the intended outcomes are for MOOCs, assessment is challenging (Gaebel, 2013). If the objective is to deliver quality learning there’s little to go on except for low completion rates and even smaller percentages of student rankings of their perceptions of learning (via end-of-MOOC surveys). A recent study attempts to address MOOC quality by assessing instructional quality of 76 MOOCs—50 xMOOCs offered on dedicated MOOC platforms and 26 cMOOCs or connectivist MOOCs (Margaryan, Bianco & Littlejohn, 2015). My aim in this post is to share results of the study and outline the framework used to evaluate the MOOCs in an effort to highlight how a course design framework is critical to developing quality learning experiences within MOOCs.

Factors Affecting Quality Course Design
Course design is a critical to delivering quality learning through online courses and MOOCs, yet it’s rarely mentioned in literature and articles discussing MOOC and online course outcomes. This study fills a gap. It determined that while most MOOCs were well-packaged, design quality was low. Out of a possible 72 points that each MOOC could score, not one MOOC scored above 28 points (p. 82).  Reasons vary, but I see it as absence of one or a combination of: skill set of course designer(s), time, and/or a structured process that includes a course design framework.

My view is that an online course takes on a persona of the instructor where the course guides and promotes part of the learning process as an instructor would. This thinking requires a different design approach—a different mindset than one used for traditional courses. A well-designed course also provides a learning path that students can follow and influence. A path that includes: quality, varied and curated resources, methods that encourage active learning whether individually or within self-selected groups, places for students to engage and share where they also act as contributors to the course. The latter is key—students should be able to shape the course through application of course concepts using their existing knowledge and experience.

Overview of the Study & “First Principles of Instruction”
The study analyzed quality through the lens of the Merrill+ model, a framework based on the “First Principles of Instruction” framework of David Merill (2002). Merill’s model is remarkably thorough, detailed and thoughtful in its inclusion and application of learning theories and approaches incorporating components of R. Gagné and H. Gardner’s theories as well as models of instructional design. First Principles also aligns closely with Malcolm Knowles’ adult learning theory andragogy, which I’ve discussed in previous posts. There’s significant research in support of Merrill’s theory suggesting it’s a credible, perspective tool to evaluate curriculum design of traditional, online courses and MOOCs (Frick et al. 2007; Margaryan et al., 2015).

Screen Shot 2015-12-11 at 11.15.33 AM
Merrill’s’ model is grounded in the practical as shown by figure 1 which describes how his five principles focus on problem-focused learning (Merrill, 2002).

Ten Principles of the Merrill+ Model
Margaryan and Collins added five additional principles to Merill’s First Principles to create Merrill+ Model which builds on Merrill’s philosophy and synthesizes contemporary instructional theories and practices (2014). The 10 principles of Merrill+ Model:

Learning is promoted…

  1. Problem Centered Learning: …when learners are engaged in solving real-world problems
  2. Activation: …when learners activate existing knowledge, experience or a skill set as a foundation for creating new knowledge and/or skills.
  3. Demonstration: …when learners observe a demonstration [that includes learning of new knowledge via a primary resource] of the skill [knowledge] to be learned.
  4. Application: …when learners apply their new knowledge or skill through discussion, written work, or creation of an artifact to solve a problem.
  5. Integration: …when new knowledge is integrated and into the learner’s context
  6. Collective knowledge: …when learners contribute to the collective knowledge of a subject or topic
  7. Collaboration: …when learners collaborate with others to expand knowledge of individuals and a community of practice
  8. Differentiation: …when learners are provided with different avenues of learning, according to their need, e.g. scaffolding
  9. Authentic resources: …when quality learning resources are curated from and applicable to real world problems
  10. Feedback: …when learners are given expert feedback on their performance

Closing Thoughts
There are other course design frameworks that can be used as alternatives to the Merrill+ Model, Khan’s e-Learning Framework (I’ll be writing about Khan’s Framework next month) and the Dick and Carey model for instance. Some institutions develop their own course design model as Purdue University did with its IMPACT model. The key to MOOC quality is selecting, then following a framework grounded in learning theory that supports an effective course design process that delivers quality learning experiences.

References

  • Frick, T., Chadha, R., Watson, C., Wang, Y., & Green, P. (2007). Theory-based course evaluation: Nine Scales for measuring teaching and learning quality. Retrieved from  http://www.indiana.edu/~tedfrick/TALQ.pdf
  • Gaeleb, D. (2013). MOOCs: Massive open online courses. (Tech.). European University Association. Retrieved from http://www.eua.be/Libraries/publication/EUA_Occasional_papers_MOOCs
  • Margaryan, A., Bianco, M., & Littlejohn, A. (2014). Instructional quality of massive open online courses (MOOCs). Computers & Education. 80. 77–83. doi:10.1016/j.compedu.2014.08.005
  • Merrill, D. M. (2002). First principles of instruction. Educational Technology Research and Development, 50(3), 43-59. doi:10.1057/9781137394644.0012

How Five Web Design Principles Boost Student Learning in an Online Course

“Our team realized quickly that we needed to do a better job cross referencing material on our course site. For example if we mention syllabus, we must link to it. Some students we have learned want a great deal of guidance” MOOC instructor, Karen Head (2013)

great-designIn the quote above, without realizing it, the instructor was referring to the concept of ‘user experience’. And it’s not guidance students wanted so much as an intuitive learning experience. Creating a user-friendly course site begins with incorporating web design principles. Even the most basic of principles customized to online course design reduces barriers associated with virtual learning by minimizing distractions, highlighting concepts and making resources readily accessible. Embedding a link into the phrase ‘assignment guidelines’ for instance, when the assignment is referred to within a course page, is an example of making resources readily available (if the assignment guidelines are within the syllabus, refer students to the page number). This reduces the amount of time students spend searching and frees up time for learning.

The challenge of designing online courses is not only pedagogical, but also technical, which is the category that ‘usability’ falls under. We are at the point with online learning where pedagogy and technology are interdependent; where a well-designed, user-friendly course with a clear learning path needs to adhere to technical principles as well as pedagogical ones. Technology is a new form of pedagogy. The course site design, how content is presented, is an aspect of online pedagogy. In this post I cover five principles of web design that are essential to online course design.

Retail sites frequently adhere to best practices for web design given customers (users) are more likely to spend time and money on an attractive, intuitive website. I suggest educators use similar web design principles to support their students.

Before we examine the principles, defining user experience (UX) is in order. There are numerous definitions of user experience but the one below specific to web design, incorporates key elements of the entire experience:

“User experience (UX) is about how a person feels about using a system. User experience highlights the experiential, affective, meaningful and valuable aspects of human-computer interaction (HCI) …. It also covers a person’s perceptions of the practical aspects such as utility, ease of use and efficiency of the system. User experience is subjective in nature, because it is about an individual’s performance, feelings and thoughts about the system. User experience is dynamic, because it changes over time as the circumstances change…” All About UX

Five Principles of Web Design Applicable to Online Course Design

1. Design for the user
This seems obvious—design a course from the student’s perspective, yet it’s an atypical approach. When designing a web page for a course site, always ask ‘how will this look to the student’? Anyone involved in online course design needs to take an online course as a student. Completing at least one week of course work in a MOOC for example, gives one an entirely different perspective on course design—guaranteed. Design the course from the student’s perspective—always.

2. Consistency
For online course, consistency is probably the most under-utilized principle. Specifically in terms of how resources are titled, labeled and/or placed within the course site. I’ve taken many courses where the same resource, an article for instance, is referred to by two different names—in the syllabus it’s titled one way, and in the course site another. Confusing. Same goes for assignments, calling an assignment by slightly different names, even by one word suggests there are two assignments, not one. Another, posting the same document in two different locations within the site suggests there are two different documents. The time students spend searching, checking, comparing etc. is valuable learning time that is spent on logistics. Consistency is key.

3. White Space
Effective use of white space emphasizes key concepts, improves comprehension (up to 20%) and reduces cognitive overload (Lin, 2004). White space is the part of a web page that is left blank or unmarked. It’s the (white) space between columns, text, images, and margins on the page. This space provides visual relief to the reader and improves readability. Avoid using big blocks of text. Break it up with a graphic, or block of white space or increased line space. See examples below.

white-space-usability-web-design-img2
Example of text with little white space.
white-space-usability-web-design-img3
Same text as above but with increased line spacing. Effective use of white space improves readability.

4. Simplicity
In keeping with the idea of white space is simplicity. A cluttered page with three or more colors of font, sizes of font and images placed sporadically throughout that are of different type and size creates a chaotic-looking virtual classroom. It’s far easier to study and focus on learning in a physical classroom that is organized with minimal distractions. The same goes for an online classroom. Keep it simple, two colors of font, same size and style throughout, organized and consistent pages creates a Zen-like classroom where students can focus on course content and application of concepts. Learning is enhanced greatly.

“The way information is organized and presented to students affects not only the usability of information, but the usability of the course itself” (Young, 2014)

Screen Shot 2015-04-06 at 9.25.36 AM
Tabs grouped by category (Coursera)

5. Use Tabs Effectively
Imagine opening a file drawer that is full of file folders with inaccurate or missing labels. The same principles of file labels apply to web sites except rather than alphabetized listing, it’s an order that makes sense to the student. For example the ‘start here’ tab should be at the top of the menu not third or fourth down the list (which happens more often than you would think). Tabs should be two words (max 3 words), and with descriptive language, ‘Start Here’, ‘Week One, or ‘Student Support’ for example.  Use sub-tabs if possible, and if not group tabs into categories (screenshot right). Also avoid CAP LETTERS for titles of tabs. CAPITAL LETTERS can appear loud and abrasive on a website (there are exceptions as the screenshot above right demonstrates).

Conclusion
Developing an online course is a multidimensional process. Usability is one dimension often neglected; understandably so given that most educators approach online course design with little expertise in web design. Yet a little goes a long way—by implementing just the basic web design principles, educators can create an intuitive learning path that gives students the boost they need to invest more time in learning, not searching.

References/Resources

How to Create a Video Strategy for MOOCs: Costs and Considerations

“Harvard has built what amounts to be an in-house production company to create massive open online courses, or MOOCs…[it] has two video studios, more than 30 employees, and many freelancers — an astonishing constellation of producers, editors, videographers, composers, animators, typographers, and even a performance coach to help professors get comfortable in front of a camera.The Boston Globe

Harvardx Studio Filming for MOOC
In the HarvardX video production studio, Harvard historian & museum curator being filmed for a MOOC.  Image credit: Katherine Taylor for The Boston Globe

I saw scores of dollar signs when I read about Harvard’s production studios created solely for the purpose of producing videos for MOOCs to put on the edX platform. The article featured in The Boston Globe about Harvard’s new studio, describes videos that share traits similar to documentaries rather than the typical lecture videos featuring a professor speaking to the camera, typical of xMOOCs. Here’s the catch though, the money spent on these production costs for MOOC videos, which is dear, may not always be worth the investment according to recent studies (Guo, Kim & Rubin, 2014), (Hollands & Tirthali, 2014).

For readers considering, or are in the process of developing a MOOC I’ve outlined guidelines that will help in the development of an instructional strategy for the delivery of the course content for xMOOCs (MOOC featured on a platform such as Coursera or Open2Study).  I’ve drawn from recent research on video production and student engagement specific to xMOOCs—one study out of MIT using data from edX, and the other a noteworthy report released this May “MOOCs: Expectations and Realities”.

Following are considerations and questions to guide the development and choice of the content delivery methods, including videos for MOOCs. Don’t be misled by the flashy [and expensive] studios that Harvard established, thinking that this is a requirement for putting on effective MOOCs. This high bar set by Harvard, may be unjustified, more so when analyzing why institutions choose to offer MOOCs, and how they fit into the vision, and long-term strategy. For the most part institutions’ reasons for offering MOOCs are vague, and few establish metrics to measure the effectiveness of MOOCs, including return on investment, as discussed in Hollands & Tirthali’s report:

“…Most institutions are not yet making any rigorous attempt to assess whether MOOCs are more or less effective than other strategies to achieve these goals.

The Numbers $$$
The two primary cost drivers of MOOC production are the hours invested by faculty members, administrators, instructional designers, technical support and the costs associated with the quality and type of delivery method for the course content. Videos, the typical mode for xMOOCs, can range between high and low production values. The estimated costs for high quality video production is $4,300 per hour of finished video (Hollands & Tirthali, p 11). High quality video production typically involves a team of at least five video experts each involved in one aspect of the process, including  filming, sound, lighting, editing and project management.

  • Development costs of MOOC vary significantly: as low as $38,980, Teacher’s College, Columbia University, to a range between $203,770 – $325,330, Large Midwestern University (Hollands & Tirthali, p 12)
  • Harvard’s costs as per the Boston article, $75,000 and $150,000, though depending upon the method for calculating, it’s difficult to compare to the study quoted above.
philip-guo-edx-first-blog-figure
Chart showing student viewing time in minutes.

Guiding Questions and Considerations for Creating a Video Strategy

  • Consider goals for each module/week within the MOOC when planning for content delivery. What delivery method will communicate content to students effectively? Is there an opportunity for student-developed or student-curated [and shared] content sources?
  • Consider a variety of content delivery methods: video, open source content (video and other), interactive online resources, etc.
  • If using video, consider between formats, tutorial-style (illustrating a procedure, step-by-step, i.e. Khan Academy-style) and lecture.  Within each format there are variations influenced by filming technique [screen cast, podcast, filming on laptop, studio, etc], media choices, etc. Research shows students engage differently with each (Guo, Kim & Rubin).
  • Consider: the average engagement time of any MOOC lecture video maxes out at 6 minutes, some as few as 4. However, students engage with tutorials quite differently, often pausing, re-watching, fast-forwarding, etc. (Guo, Kim & Rubin).
  • Are there existing open and accessible content sources on the web that can illustrate a course concept [rather than filming from scratch]?
  • Call students to action to use and apply content from video, i.e. via a  discussion forum or upcoming assignment.
  • Finally, plan the strategy upfront where: 1) each content delivery mode (video, etc) is planned by module/week (content is outlined and scripted when necessary), 2) content within delivery mode links directly to goals of given week, and 3) there is a requirement for students to apply and use the content, for example in discussion forum, assignment or quiz.

References:

Further Reading:

MOOC Development Advice from Instructors that Have ‘Been-There-Done-That’

Recomended stamp
“Recommended” istock

I’ve collected several YouTube video clips of faculty and instructors sharing their experiences and giving advice for those developing a MOOC. Readers adapting a face-to-face course to a xMOOC format may find this post helpful. Who better to get advice from than faculty or instructors that have developed at least one MOOC and have-been-there-done-that?

The video clips below are not longer than four minutes a piece; I’ve summarized each and categorized the advice into four categories:  1) Video Production, 2) Pedagogy, 3) Course Design and 4) Instructional Strategy. This should help readers determine which clips are worth watching. The video clips feature instructors that developed one or more MOOCs for one of three platforms, iVersity, Coursera, and Open2Study.  One clip is in German, from a professor teaching a course on the iVersity platform. It focuses on video production techniques specific to a math focused class, and is very worthwhile to watch given the techniques demonstrated.

1. “Reflecting on the MOOC Experience
Categories: Pedagogy, Course Design, Video Production
Platform: Open2Study
Course: “Understanding Common Diseases
Summary: Course instructors surprised to discover some aspects of the MOOC format applicable to their face-to-face courses
Takeaways

  • Required a completely different skill set to teach a MOOC—most challenging: i) teaching in short blocks [condensing the course content], ii) filming lectures—talking to a camera not a live audience
  • Discovered a new teaching method for teaching in their traditional program, by incorporating MOOC format into face-to-face program
  • Instructors spent 5 days filming approximately 40 videos, 6 to 8 minutes each

 

2. “What Makes a Good MOOC with Dr. Charles Severance”
Categories: Pedagogy, Instructional Strategies
Platform: Coursera
Course: Internet History,Technology and Security
Summary: Provides excellent tips for adapting a face-to-face course to the MOOC format
Takeaways:

  • Implement a course design and instruction strategy for the MOOC that is different from one’s face-to-face course, though make it personal—inject humour and personality into course

3. “10 Steps to Developing on Online Course [MOOC]”
Categories: Course Design, Video Production
Platform: Coursera
Course: Think Again: How to Reason and Argue”, Coursera
Summary: Practical steps for creating a low-tech MOOC [with one camera, and small development team]
Takeaways:

  • Break course content down into smaller segments (chunks), plan the course outline (syllabus)
  • Write the scripts for the videos
  • Practice and more practice before recording the lectures

4.  “Making-of MOOC “Algorithmen und Datenstrukturen” von Prof. Dr. Oliver Vornberger”
Category: Video Production
Platform: iVersity
Course: Algorithmen und Datenstrukturen
Summary: Demonstrates various video production techniques for filming mathematical calculations and physical demonstrations
Takeaways: Variety of lecture formats holds interest for students. A video production team can support execution of diverse formats: green screen, whiteboard and demonstrations with white background

5.  “The Making of an Online Course: Ronen Plesser”
Category: Video Production
Platform: Coursera
Course: “Introduction to Astronomy”
Summary: Professor shares how experiments/demonstrations traditionally done in a face-to-face classroom, translate into the MOOC format
Takeaways:

  • A highly skilled video production team helped produce high quality videos
  • Certain demonstrations turned out better in the recording studio than in classroom, and vice versa

Further Reading: