Need-to-Know-News: BBC gets into MOOCs, Global MOOC Report for $2,500 & Competency Education gets Boost

MP900405500This ‘Need-to-Know’ blog post series features noteworthy stories that speak of need-to-know developments within higher education and K-12 that have the potential to influence, challenge and/or transform traditional education as we know it.

1) BBC gets involved with MOOCs
Four universities in the United Kingdom are partnering with the British Broadcasting Corporation (BBC) to include never seen before film footage from World War One via its archives. Times Higher Education reports that FutureLearn will host the four MOOCs. The venture with BBC promoted as a creative approach, will be different from the traditional lecture format typical of the majority of xMOOCs:

“FutureLearn chief executive, Simon Nelson, said that many Moocs had been “rightly criticised as just being a repackaging and redistribution of the traditional lecture format, and that some universities were using the internet to “pump out videos”, rather than using their courses to tell a story. He said he hoped that working with the BBC would help institutions to be more creative.” 

Insight: This venture is a great opportunity for institutions to demonstrate they can reach non-traditional xMOOC students (traditional MOOC students: holding an undergraduate degree or higher), and engage learners not familiar with online learning or self-directed education. Not to mention introducing a novel method for delivering content. The four MOOCs launch soon—in October so we won’t have to wait long to see the response.

Screen Shot 2014-07-25 at 11.14.36 AM

Image from “BBC helps produce First World World Moocs” by Chris Parr

2) Report on MOOCs for $2,500
It’s not only Udacity and Coursera that are starting to make money from xMOOCs. It seems that there’s even demand for research reports about MOOCs—$2,500 a pop, or $10,000 for enterprise wide access. The report, published by Research and Markets, The World’s Largest Market Research Store described as follows:

“The report, the Global Massive Open Online Courses Market 2014-2018, has been prepared based on an in-depth market analysis with inputs from industry experts. The report covers the Americas and the EMEA and APAC regions as well as the key leading countries in this market. It also provides in-depth overview of the revenue generation models adopted by the vendors in the market and discusses the key vendors operating in this market It also includes discussions of  “…the market segmentation based on student demographics and course preferences, as well as the market landscape and its growth prospects in the coming years.” 

Insight: xMOOCs are big business, though not necessarily for higher education institutions.

 3)  Competency Education Gets a Bigger Boost
There’s bi-partisan support for competency-based education in the United States, a bill passed this week by the senate will allow student aid to go towards thirty academic programs that are experimenting with a range of innovative higher education academic programs that lead to degrees, many including degree-tracks grounded in competency-based education. According to sources of Inside Higher Ed,  there are nearly 350 institutions that do offer, or plan to offer a competency-based degree track (Fain, 2014).

Representative John Kline of Minnesota, the Republican who chairs the House education committee, called the legislation a “good first step” to figuring out what works and doesn’t work for competency-based education.Inside Higher Ed

The programs fall under the Experimental Sites Initiative (ESI), funded by the Department of Education. The program is still accepting applications.

Insight: The idea of experimentation with students pursuing higher education makes me uneasy. If such programs enroll students at risk for not completing a two or four-year degree, I would hope that the programs do not further jeopardize the students’ chances of success.  I think back to the Udacity and San Jose University experience with MOOCs, where one of the pilot programs was a MOOC format for remedial math.

“Another factor in the disappointing outcomes may have been the students themselves. The courses included at-risk students, high school students and San Jose State students who had already failed a remedial math course.” Inside Higher Ed

Failure rates were higher in the MOOC experiment than in the face-to-face class.  What about those students?  Granted new programs have to be piloted in some way, I would hope however, that there is a plan in place to address any negative outcomes students may experience as a result of the experimental programs.

You can keep up to date with developments in education and related sectors by following me on Twitter, @OnlineLearningI 

Three (BIG) Barriers to Student Participation in xMOOCs

This post outlines three barriers that can deter, discourage and/or intimidate students from participating in xMOOCs (MOOCs offered on platforms associated with higher education institutions, i.e. Coursera, iVersity, edX, etc).

construction_barriers

“Construction Barriers”  Photo by Lyn Topinka

The xMOOC model that emerged in 2012 has not changed much in 2014, with completion rates and participation rates just as low as they were when concrete data on completion rates appeared in 2013 (Parr, 2013). Though there are a variety of factors that contribute to low completion rates, I suggest that three barriers, 1) technology, 2) poor usability & course design, and 3) anonymity contribute significantly to low student participation levels and completion—barriers that deter, discourage and in some cases intimidate students. Also, in some instances, barriers one and two are potential barriers in closed, online classes (as those offered as for-credit courses at public and private institutions).

To illustrate points one, technology and two, poor usability and course design, below is a selection of screenshots featuring actual student comments and questions (names obscured) taken from several MOOCs offered on Coursera. Comments below are representative of typical experiences and frustrations of students participating in MOOCs. In some instances, the examples included are similar to frustrations students experience in closed online courses, which I’ve encountered when working with faculty in online course design, and as a lead curriculum developer for online programs at a private university. I close by discussing the third barrier, anonymity in online learning, specifically in MOOCs.

1) Technology 
Examples below feature student challenges with accessing course content and engaging in events due to bandwidth and internet access limitations.

Screen Shot 2014-07-11 at 8.22.26 AM

File size is a common problem in MOOCs and small courses

Screen Shot 2014-07-11 at 3.41.19 PM

Connectivity issues are common due to bandwidth, and even limitations of the devices used

Screen Shot 2014-07-11 at 8.22.50 AM

Restricted access to certain sites in some countries

Examples below feature students’ frustrations with applications (discussion forums, etc) within the MOOC platform itself which put up barriers to student participation and engagement, for example, i) discussion forums (volume of student posts and organization), ii) synchronous events offered via Google Hangout or other platform which often fail due to technical glitches, or because of students’ lack of technical ability, etc.

Screen Shot 2014-07-12 at 3.54.13 PM

Discussion forums often become unwieldy; though more common in MOOCs it also happens in closed, small online courses

Screen Shot 2014-07-12 at 3.53.21 PM

Discussion forums in most MOOC platforms have options for ‘subscribing’, where participants receive alerts of new posts within that particular forum, though not all students are familiar with this settings and don’t know how to turn the notification emails off (or on). It’s helpful to provide participants with the instructions of how-to do so (among other features) in an orientation or introduction to the course

Screen Shot 2014-07-12 at 3.37.18 PM

The forums with large numbers of participants can be overwhelming to the point that there is little opportunity for reflection or deep discussions. In closed online classes it helps to have focused discussion questions per thread, and if more than 20 participants to break the class into smaller groups

Screen Shot 2014-07-12 at 4.20.15 PM

Google hangouts and other platforms used for synchronous events, are not immune to technological glitches. A practice run prior to the event helps (granted even still, problems occur),  and having a back-up plan is recommended

2) Poor Usability and Course Design
Usability refers to how effectively students can navigate, interact and engage with the course interface, find the content they need, determine what they need to do to engage, etc. How user-friendly the course is (or is not) is a function of how the course content and pages are organized, what is featured on the course home page for example, or where the course announcements show up, even how the course tabs appear in the navigation menu. Usability falls under the umbrella of course design; it is a component with its own principles and guidelines that impacts the students overall course experience and learning outcomes in online spaces. Usability adds another layer of complexity to designing learning experiences mostly due to the newness of online platforms as delivery mediums for education.

Course design is a broad and deep topic, which I can’t address at all adequately in this post, but below are some examples that are representative of issues that frustrate students, and can deter learning outcomes that have to do with how an assignment’s instructions are worded, presented to the student, or even designed in the first place.

Screen Shot 2014-07-12 at 3.56.30 PM

Instructions for student assignments or activities need to be written with exceptional clarity. This means expanding on details is necessary, including examples, and reinforcing instructions and expectations via course announcements or live sessions when the course is in session. Another issue is the use of  consistent terminology throughout the course.  In this above example ‘thread’ and ‘post’ were used interchangeably, when in fact they mean different things, thus confusing the students.

Screen Shot 2014-07-12 at 4.50.56 PM

Another example of student confusion when there is inconsistency or conflicting information in the course

Screen Shot 2014-07-12 at 6.52.58 PM

Frequently, it is student assignments that generate the most confusion among students in virtual formats, often due to unclear instructions, or those that require students to use technical applications (e.g. to create a digital artifact) that they are not familiar with

3) Anonymity
A view on participants posting anonymously within a MOOC from iVersity:

“MOOCs offer an environment that may engage introverts. Online anonymity can make students comfortable expressing themselves in forums…participating in course conversations online may give students confidence to contribute in traditional classrooms and work environments.” iVersity blog post on Anonymity

I disagree with iVersity’s position, and with Coursera and edX, which both allow anonymous posting within discussion forums. Anonymity does not contribute to effective online learning communities such as MOOCs; it’s counterintuitive to the premise of a learning within a community, where the idea is that learners actively engage, and learn with, and from each together. Several papers have identified the benefits of learning communities in distributed (online) learning environments (Dede, 2004), with some emphasizing the value of communities in MOOCs especially (Kop, Fournier & Mak, 2006). What is consistent in the research is the idea of trust and a set of common values or goals among learners.

The type of support structure that would engage learners in critical learning on an open network should be based on the creation of a place or community where people feel comfortable, trusted and valued, and where people can access and interact with resources and each other. (Kop, Fournier & Mak, 2006, p. 88)

Learning in a virtual community, where students go outside of their comfort zone, are challenged to consider alternative perspectives and build a personal learning network for example, requires a level of rapport, familiarity and trust between classmates and instructors. This sense of community can and does happen in small, closed learning environments, and in cMOOC learning communities, but experiencing a sense of community in xMOOCs is far more difficult to accomplish with many of variables making it so, anonymity is just one. With this learning approach (and others) assumed by MOOC platform providers, I see xMOOCs destined to be static resources posted on the web—open courseware such as MIT OCW.

Closing Thoughts
Learning in xMOOCs is far more complex than what the MOOC platforms seem to be able to address. Low completion rates are just one metric of how students’ views of MOOCs are at odds with what the expectations of the MOOC providers. The three barriers discussed here, technology, usability and anonymity are just one piece of a bigger problem that MOOC platform providers will need to address if they are interested in creating a communities of learning where students actively engage, contribute and learn.

Further reading:

How to Create a Video Strategy for MOOCs: Costs and Considerations

“Harvard has built what amounts to be an in-house production company to create massive open online courses, or MOOCs…[it] has two video studios, more than 30 employees, and many freelancers — an astonishing constellation of producers, editors, videographers, composers, animators, typographers, and even a performance coach to help professors get comfortable in front of a camera.The Boston Globe

Harvardx Studio Filming for MOOC

In the HarvardX video production studio, Harvard historian & museum curator being filmed for a MOOC.  Image credit: Katherine Taylor for The Boston Globe

I saw scores of dollar signs when I read about Harvard’s production studios created solely for the purpose of producing videos for MOOCs to put on the edX platform. The article featured in The Boston Globe about Harvard’s new studio, describes videos that share traits similar to documentaries rather than the typical lecture videos featuring a professor speaking to the camera, typical of xMOOCs. Here’s the catch though, the money spent on these production costs for MOOC videos, which is dear, may not always be worth the investment according to recent studies (Guo, Kim & Rubin, 2014), (Hollands & Tirthali, 2014).

For readers considering, or are in the process of developing a MOOC I’ve outlined guidelines that will help in the development of an instructional strategy for the delivery of the course content for xMOOCs (MOOC featured on a platform such as Coursera or Open2Study).  I’ve drawn from recent research on video production and student engagement specific to xMOOCs—one study out of MIT using data from edX, and the other a noteworthy report released this May “MOOCs: Expectations and Realities”.

Following are considerations and questions to guide the development and choice of the content delivery methods, including videos for MOOCs. Don’t be misled by the flashy [and expensive] studios that Harvard established, thinking that this is a requirement for putting on effective MOOCs. This high bar set by Harvard, may be unjustified, more so when analyzing why institutions choose to offer MOOCs, and how they fit into the vision, and long-term strategy. For the most part institutions’ reasons for offering MOOCs are vague, and few establish metrics to measure the effectiveness of MOOCs, including return on investment, as discussed in Hollands & Tirthali’s report:

“…Most institutions are not yet making any rigorous attempt to assess whether MOOCs are more or less effective than other strategies to achieve these goals.

The Numbers $$$
The two primary cost drivers of MOOC production are the hours invested by faculty members, administrators, instructional designers, technical support and the costs associated with the quality and type of delivery method for the course content. Videos, the typical mode for xMOOCs, can range between high and low production values. The estimated costs for high quality video production is $4,300 per hour of finished video (Hollands & Tirthali, p 11). High quality video production typically involves a team of at least five video experts each involved in one aspect of the process, including  filming, sound, lighting, editing and project management.

  • Development costs of MOOC vary significantly: as low as $38,980, Teacher’s College, Columbia University, to a range between $203,770 – $325,330, Large Midwestern University (Hollands & Tirthali, p 12)
  • Harvard’s costs as per the Boston article, $75,000 and $150,000, though depending upon the method for calculating, it’s difficult to compare to the study quoted above.
philip-guo-edx-first-blog-figure

Chart showing student viewing time in minutes.

Guiding Questions and Considerations for Creating a Video Strategy

  • Consider goals for each module/week within the MOOC when planning for content delivery. What delivery method will communicate content to students effectively? Is there an opportunity for student-developed or student-curated [and shared] content sources?
  • Consider a variety of content delivery methods: video, open source content (video and other), interactive online resources, etc.
  • If using video, consider between formats, tutorial-style (illustrating a procedure, step-by-step, i.e. Khan Academy-style) and lecture.  Within each format there are variations influenced by filming technique [screen cast, podcast, filming on laptop, studio, etc], media choices, etc. Research shows students engage differently with each (Guo, Kim & Rubin).
  • Consider: the average engagement time of any MOOC lecture video maxes out at 6 minutes, some as few as 4. However, students engage with tutorials quite differently, often pausing, re-watching, fast-forwarding, etc. (Guo, Kim & Rubin).
  • Are there existing open and accessible content sources on the web that can illustrate a course concept [rather than filming from scratch]?
  • Call students to action to use and apply content from video, i.e. via a  discussion forum or upcoming assignment.
  • Finally, plan the strategy upfront where: 1) each content delivery mode (video, etc) is planned by module/week (content is outlined and scripted when necessary), 2) content within delivery mode links directly to goals of given week, and 3) there is a requirement for students to apply and use the content, for example in discussion forum, assignment or quiz.

References:

Further Reading:

How and Why Institutions are Engaging with MOOCs…Answers in Report “MOOCs: Expectations and Reality”

How do institutions use MOOCs; and to what end?  •  Why do institutions pay thousands of dollars to develop and offer a MOOC on an external platform?  •  How do institutions determine the effectiveness of their MOOC efforts?  •  What are the costs associated with producing and delivering a MOOC?

All good questions; questions that policymakers, administrators and other stakeholders within higher education institutions that are considering MOOCs or already engaged with, want [or should want] answers to. The 200+ page report “MOOCs: Expectations and Reality” by Hollands and Tirthali of Columbia University attempt to answer these questions by surveying 83 faculty members, administrators, researchers and other actors within 69 education institutions. The report delivers on the promise of its title—how and why institutions engage, and provides the reader with even more insights.

The report is meaty, worthy of review for anyone with a vested interest in MOOCs of any type. In this post I provide a brief overview of the report, but focus specifically on one aspect of the ‘how‘. I highlight the resources required to develop a MOOC—how many people it requires, the job titles, the [estimated] costs associated with development. This may be useful for readers considering developing a MOOC for a platform such as Coursera or another, or for a cMOOC using a collective course design approach. This report brings into focus just how resource-hungry MOOCs are, and after reading the report, readers considering developing or contributing to the development of a MOOC might feel enlightened, encouraged, or perhaps even discouraged; at the very least, will have a better understanding of MOOCs and their place in higher education institutions.

 Overview

Who sponsored the report?  The Center for Benefit-Cost Studies of Education (CBCSE), a research center at the Teachers College at Columbia University. The mission of the center is “to improve the efficiency with which public and private resources are employed in education“.  Note: the report is open and available for download.

Purpose of the Study: Given the work of the CBCSE, and its pursuit of improvement of cost efficiency in education, the report is an extension of its mission. The purpose as outlined in the report, “the study serves as an exploration of  the goals of institutions creating or adopting MOOCs and how these institutions define effectiveness of  their MOOC initiatives“.

Screen Shot 2014-05-19 at 9.50.39 PM

Figure 1 ‘MOOCs: Expectations and Reality’ (p 22)

Report Snapshot:  The report sample includes 83 administrators, faculty members and researchers, all of which were interviewed, at 62 institutions. The institutions: public and private universities, community colleges, platform providers, research organizations, for-profit education companies and a selection of institutions deemed ‘other’ including one museum (p 180). Of the 62 institutions in the sample, 29 at the time of the study were offering or using MOOCs in some way; the remaining were either not participating or taking a wait-and-see position.

Why a MOOC? One of the reasons this report is instructive for the education community is the inclusion of the data about why institutions offer MOOCs. Many have asked why some institutions (several public higher education institutions) have spent thousands of dollars, invested considerable resources into this method of education delivery to the masses that has yet to be evaluated and tested for effectiveness. The chart below summarizes the six reasons identified.

Screen Shot 2014-05-20 at 8.25.43 AM

‘MOOCs: Expectations and Reality’ (Hollands & Tirthali, p 8)

The above table is merely a snapshot.  Each goal is described in further detail within the report. A case study featuring an institution accompanies each which gives a contextual example of the reasons.

A snapshot of How? MOOCs are resource intensive efforts, and the report validates this. Development of a MOOC, and the facilitation of the course once its live (accessible to students) requires significant amounts of time and energy from individuals across several departments within the institution. The faculty member (or members) acting as the subject matter expert for the MOOC requires a team, each with different areas of expertise to support him or her in bringing the content to life and creating an environment of learning for hundreds, if not thousands of course participants.

“Number of faculty members, administrators, and instructional support personnel involved MOOC production teams seldom included fewer than five professionals and, in at least one instance described to us, over 30 people were involved. Faculty members typically reported spending several hundred hours in the production and delivery of a single MOOC” (p 11)

Example of Human Resources Requirements: Case Study 11

Case study 11 provides an excellent example of the commitment of resources needed for developing a course for a MOOC platform which in this example is Coursera. The institution in the case is an unnamed MidWestern University (p 144). The school invited faculty with prior media experience to develop a five to eight week MOOC. This study is representative of the human resources required for development of a MOOC.

Human resources requirements by job title for course development of a MOOC:

2 x Faculty Members: (Subject Matter Experts)

1 x Project Manager: Leads the project, coordinates all elements of development. Liaise with departments as needed within the institution. Manages the project timetable; keeps project on time and on budget

4 x Curriculum Design Team Instructional Designer (works with faculty to present course content and create a learning environment with it on the course home page). • Instructional Technologist (works with instructional designer) • Video Production Liaison (works with faculty member in production of videos, and liaise with video production team)

5 x Video Production Team:  Production Manager •  Camera operators/equipment technicians • Audio-technician

In this case study, videos were produced at a high quality, using a full video design team. The final costs were calculated using records from the institutions, though the report authors made some estimates due to lack of detail on some aspects of human resource inputs.

Screen Shot 2014-05-20 at 8.59.34 AM

‘MOOCs: Realities and Expectations’ (p 144). One of the two data tables accompanying case study #11. Table 7 gives the range of hours spent on MOOC design (p 144)

Lecture Videos: Costs and Student Engagement
One of the primary drivers of costs in MOOC development (for platforms such as Coursera, FutureLearn, etc.) is video production. The more complex the video, for instance addition of graphics, multiple cameras used for shooting, post-filming editing, the higher the costs. Low-tech efforts,  where there might be one camera person, or even the faculty member self-recording on his or her laptop requires far fewer resources.  Some institutions seek a higher quality finished product, which in turn demands a high level of production using a team of video professionals. Accordingly, the costs vary dramatically. ‘MOOCs: Expectation and Realities’ estimates high quality video production at $4,300 per hour of finished video (p 11).

One may be tempted to think that the higher the video quality, the better the learning outcomes. However a report published recently by EDUCAUSE, What Makes an Online Instructional Video Compelling? suggests that students engagement with videos relies upon several factors, including whether or not the video links to an assignment within the course. Furthermore, the average viewing time of videos is less than five minutes (Hibbert, 2014). What this suggests is that videos presenting content must be carefully and strategically planned for during the course development phase, and tied closely to the instructional strategy. Higher production costs does not necessarily mean higher student engagement or learning outcomes.

Closing Thoughts

The report discussed here, ‘MOOCs: Expectation and Realities’ is an important contribution to the MOOC discussion in higher education. In my opinion one of the greatest benefits of the report is the spotlight it puts on the resources required for developing a MOOC, in contrast to the reasons why institutions engage with MOOCs. When one examines closely the reasons, it appears that the amount of resources invested, in some cases is extreme. I agree with the authors in the point they make in the executive summary,

” [we]…conclude that most institutions are not yet making any rigorous attempt to assess whether MOOCs are more or less effective that other strategies to achieve these goals” (p 11).

I’ll add one more point to this, and that’s the need for a complete and comprehensive approach to course design, (applicable to any course) that involves from the beginning, a thorough needs analysis that determines the goals of the organization and how the [potential] course fits into it. It’s only after this analysis that the course design process can proceed.

References:

Hibbert, M. (2014). What Makes an Online Instructional Video Compelling?. EDUCAUSE Review Online. Retrieved from: http://www.educause.edu/ero/article/what-makes-online-instructional-video-compelling

Hollands, F. M., & Tirthali, D. (2014). MOOCs: expectations and reality. Full report. Center for Benefit- Cost Studies of Education, Teachers College, Columbia University, NY.  Retrieved from: http://cbcse.org/wordpress/wp- content/uploads/2014/05/MOOCs_Expectations_and_Reality.pdf

MOOC Design Tips: Maximizing the Value of Video Lectures

“Which kinds of videos lead to the best student learning outcomes in a MOOC?”
How Video Production Affects Student Engagement: An Empirical Study of MOOC Videos (Guo, Kim & Rubin, 2014)

Reel of FilmAn excellent question that design teams and instructors of MOOCs want to know—which kinds of videos lead to the best student learning outcomes in a MOOC?  According to a recent study conducted by researchers for the edX MOOC platform, this was the most pressing question posed by the course design teams working with its partner institutions. Given that most MOOCs offered through higher education institutions platforms such as edX, iVersity, or Coursera use video lectures as the primary content delivery source, it is a critical question that preoccupies many if not most MOOC instructional design teams. Adding to this need-to-know element is the fact that video production is most often the highest cost associated with MOOC production. MOOC video production can range from a few hundred dollars and run up to the thousands. This post suggests how institution can use resources effectively in the video production process with the primary goal of supporting students’ learning outcomes.

The report released by edX last week gives design teams some concrete data to examine. I’ve emphasized below the recommendations and practical application points from the paper for readers who might be part of a design team for MOOC, online course, or for those with an interest in video production for instructional videos.  There are limitations to the study outlined in the paper, though the depth of the analysis does provide data worthy of consideration. 

The report, the first of its kind according to the authors Guo, Kim & Rubin, analyzes students’ engagement* with lecture videos gathered from data extracted from over 6.9 million video watching sessions across four edX courses.  *Student engagement is defined in the study by:

  1. Engagement time: the length of time that a student spends of a video. This is the same metric used by YouTube. Though researchers acknowledged the limitation of engagement assessed from this one-dimensional perspective.
  2. Question/Problem Attempt:  Almost one-third of the videos across the four courses featured an assessment problem directly following the video, usually a multiple-choice question designed to check a student’s understanding of the video’s contents. “We record whether a student attempted the follow-up problem within 30 minutes after watching a video.”

Videos Types for MOOCs
Lectures are divided into two primary types for the study, [which mirrors most MOOCs]: 1) lecture videos for content delivery—presented via an instructor/professor (‘talking head’ is the term used in the paper), and 2) a tutorial/demonstration, a step-by-step problem solving walk-through, common in computer science courses, courses featuring mathematical concepts or science courses featuring lab demonstrations.

Video Production Format
For analysis purposes, researchers coded the videos examined in study using six primary video production formats, which I’ve summarized below, along with production styles not mentioned in the study.

1) Lecture-Style Video Formats:

  • Instructor(s) with/without Presentation Slides: Features instructor(s) lecturing, with or without PowerPoint slide presentation slides inserted throughout with instructor ‘voice over’ while slide is displayed
  • Office Setting: close-up shots of the instructor filmed at his or her office, typically instructor speaks directly to camera
  • Classroom Setting: video captured from a live classroom lecture
  • Production Studio Setting: instructor recorded in a studio with no audience, typically speaking to the camera

2) Tutorial/Demonstration Video Formats:

  • Video Screencast: of the instructor demonstrating a concept, i.e. writing code in a text editor, or command-line prompt (in the case of computer science courses), using spreadsheet or document
  • Instructor Drawing Freehand on a Digital Tablet, using a software program, which is a style popularized by Khan Academy videos (click here to view an example)

Other Formats not mentioned in the study:

  • Instructor interviewing another expert or guest speaker
  • Instructor delivering lecture in another setting related to the course (though not always), for example an ecologist giving lecture at the beach, an art historian in a museum, etc.
  • Panel Discussion of experts on specific course-related topic

Which format to use? The primary factor that determines which format to use are the objectives of the MOOC or course, and the course content. The course design team typically selects the video formats during the course design phase when the instructional strategy is created, for example: the formats of the video are chosen, the content chosen for each, related student activities or assessments selected, etc.

The second factor determining which format to employ is the amount of resources (dollars) available for video production. This determines right off the bat which tool, program or hardware will be used for the video production. Important to note, the amount of resources invested in video production does not scale to how much students’ learn or to MOOC completion rates. For example, I completed a course on Canvas Network, Statistics in Education for Mere Mortals (my course review here). The course featured video lectures and tutorials, all created by the instructor using low-budget technology. Lectures appeared to be filmed on the instructor’s laptop using a web cam, (power point slides were added, so there was some editing). Each module featured a tutorial, a screen cast where the instructor demonstrated application of various formulas to a data set. I found the professor, Lloyd Rieber, encouraging and personable; he also delivered the content concisely in lecture videos and tutorials. Interestingly, the course completion rate was over 10%, higher than typical MOOC completion rates that are usually lower than 7%.

Key Findings of Study

  • Shorter videos are more engaging. Student engagement levels drop sharply after 6 minutes
  • Engagement patterns differ between the two video formats; engagement higher with the lecture style videos (‘talking head’) which researchers suggest is due to more “intimate and personal feel”
  • Several MOOC instructors interviewed for study felt more comfortable with the classroom lecture format, however this format did not translate well online, even with much editing in production studio
  • For tutorial/demonstrations videos, the Khan-style format where instructor draws on tablet and narrates, was found to engage students more effectively than screen casts. A contributing factor—instructors ability to situate themselves “on the same level” as student
  • Video producers and edX design teams determined that pre-production planning had the largest impact on the engagement effect of the videos. Researchers used a data set within the study to test this idea

Practical Recommendations for Course Design Teams

  1. Identify type and format for each video lecture using course objectives and module breakdown as a guide, and budget. Plan each lecture for the MOOC format and its potential students. Consider copyright terms for images used in videos and slides. Plan ahead by selecting appropriate images, free from copyright during the planning phase
  2. Invest in pre-production planning phase. Segment course content into chunks, using six-minutes per video as a guideline. Identify purpose for each video lecture, and key content points to deliver within each.  Write script for each [lecture video format] and have instructor practice before filming—reduces filming and editing time

  3. For tutorial/demonstration videos introduce motion and continuous visual flow into tutorials, along with extemporaneous speaking so that students can follow along with the instructor’s thought process. Complete basic outline of video beforehand, not full script to be read word-for-word
  4. Provide more personal feel to videos. Try filming in an informal setting (such as the instructor’s  office) where he or she can make good eye contactit often costs less and might be more effective than a professional studio. Coach instructors to use humour, personal stories and convey enthusiasm where possible

Closing Thoughts
MOOCs are here to stay, which makes studies like this one valuable for helping educators be more effective through course design. This study brings us closer to finding the answer to the question which kinds of videos lead to the best student learning outcomes in a MOOC?  Yet it’s a start, there is still much more to be done in understanding how students learn in massive courses, and how institutions can be more effective with investment of its resources for increasing student learning outcomes.

Further Reading:

MOOC ‘Jam': Highlights from a Jam on Digital Pedagogy

This post includes takeaways from a ‘MOOC Jam’, a synchronous discussion online I participated in with a group of educators about digital pedagogy. 

MOOC_Jam_Image

‘Jam’ by John Wardell (Flickr)

“What is a Jam?  A Jam is an asynchronous, typed, online discussion designed to work around your schedule. The goal of a Jam is to gain perspective and solicit ideas that inform the community. After the Jam is over, you can read the exchange and the posted resources, which will remain available for several weeks.” MOOC Jam II, Digital Pedagogy   (The three threads of asynchronous discussions in this jam are: 1) Competencies for teaching online, 2) Developing Faculty Competencies, and 3) Learner Analytics for Faculty)

Screen Shot 2014-04-01 at 11.16.09 AM

Screen Shot of participants online during the Jam participating in the threaded discussions (only partial shot of participants)

This past Tuesday, I participated in a  MOOC Pedagogy Jam via the website Momentum, a platform created for stakeholders to discuss critical issues related to education, sponsored by the Bill and Melinda Gates Foundation. The purpose of the platform is to provide a space to host online events about topics related to online education, with the ultimate goal of the Jams ‘to gain perspective and solicit ideas that inform the community’.  I participated in the first MOOC Jam this past November. The topic, “Peer Review of a Framework for MOOCs” hosted by George Siemens, focused on the design of the MOOC Framework. Siemens, creator of the Framework, sought input from the community of participants.

The topic of this Jam, [which turned out to be more of a synchronous discussion] was digital pedagogy, divided into the three threaded discussions as mentioned above. Each discussion featured a moderator, responsible for responding to participants and furthering the discussion, and another moderator summarizing key themes of the discussion each hour. I chose to participate in ‘Competencies for teaching online: describing effective pedagogy’ given its description— “An exchange on how information is delivered to students, how they are engaged as active learners and community is built and how learning is assessed”.

Screen Shot 2014-04-03 at 3.55.00 PM

Screen shot of one the three threaded discussions of the Jam held on the Momentum platform

Digital Pedagogy: Themes and Highlights
Following are my insights from the discussion on digital pedagogy and I’ve included comments from other participants. (Jam II, Momentum, Digital Pedagogy).

The discussion was rich with ideas, insights and provided a glimpse into the issues and challenges with online instruction. Though the title of the Jam featured ‘MOOCs’, much input from contributors pertained to closed, online courses which created an interesting discussion by highlighting one of the primary challenges in online education—the application of appropriate pedagogical methods, which will vary depending upon the learners, the delivery method and goals of the course.

Themes:

1)  Part of the discussion was devoted to the contrast and challenges between learner-directed and instructor-directed learning. The fact that much discussion focused on this issue highlights one of the challenges with MOOCs; a MOOC, due to its scale and format lends itself to be learner-directed. It’s not surprising then that MOOCs attract learners that already know how to learn, are motivated and educated. Several Jam participants discussed methods to get learners involved in learning, how to encourage students to engage and participate [typically in the context of closed online classes].

“I’ve done something similar to engage students in action research with me. I was teaching Web Development and was not happy with the development framework we were using. So as a class we researched the pros and cons of various frameworks and decided as a class (with my approval) which one to use. This worked well – they had “buy in” as we used to say.  Beyond that, I set basic specifications as to what they were to include in their work product, but allowed them to choose the subject matter (content). I also had to give approval before they began coding.”

I see the above challenge highlighting two opportunities: 1) to provide support to students to learn how to be self-directed, and 2) to provide skill development for educators and course designers in how to be flexible and adapt instructional strategies by assessing learners, the learning context and creating appropriate learning experiences, implementing pedagogical methods that match the learning needs.

One Jam participant shared an initiative that his institution recently started for its students; a program designed to address much of what was discussed here.

“California State University, Monterey Bay, is creating an online training module for training in baseline skills in web technologies for collaboration and other soft skills, such as team working relationships. Selecting appropriate pedagogy again depends upon an analysis of the learners — goes back to careful and thrustful planning”

2)  Considerable discussion focused on how to get students to interact, collaborate and engage with peers in online classes, and what the instructors role is in facilitating group formation, participation and learner engagement. Though this theme is similar to the theme mentioned above, interesting thoughts on group formation and collaboration emerged—should it be encouraged, facilitated or left for students to form spontaneously? And if so, how? This relates to the motivation of the learner, which is quite different when students are in for-credit classes versus ‘free’ and open classes  [MOOCs] that are driven by interest and desire to learn—essentially self-directed.

But a key trade-off when you have non-static groups, as Michaelsen, Fink et al have looked at is that you lose the crucial accountability factor and or the time to form constructive group norms/roles etc. — this then leads to the ‘freeloaders’ issue that gives groupwork such a bad rep.  There is the challenge — in a MOOC context, can you establish stable, productive learning groups with accountability, positive norms, roles etc to really activate the engagement, peer learning and other benefits of group learning?”   

The comment above is interesting—is it really possible or desirable in a MOOC environment that the responsibility for group accountability and productivity rests with the instructor?

3)  The session wrapped up with discussion that focused on supporting learners, helping learners to learn in a MOOC format.  The question appears to be—how can this be accomplished, is it through course design, or while the course is live, accomplished via course facilitators?  Or do we need to teach students how to learn in a MOOC?

I think one of the goals for a MOOC is enabling learners to make connections, share, collaborate and learn from one another. Rather than thinking about self-directed or facilitator directed maybe we need to think about how we can create ways that encourage learners to support one another?”

I am very interested in how learners can and do support one another’s learning in MOOCs. Do you have some thoughts in mind about the answer to this question? What can we build into the design that supports and encourages peer-peer learning?”

Closing Thoughts
Discussions, similar to those within this Jam, create excellent opportunities to get the issues and challenges facing education, specifically online education, out in the open.  It also helps stakeholders identify what needs to be discussed and explored within their own institutions. There are commonalities across all institutions when it comes to online education, and ironically the very barriers affecting these issues, exist within institutions at all levels. Fortunately there is progress—many institutions are experimenting, collaborating and striving to adapt to cultural shifts, increase access, yet still provide high quality, relevant education. Are there similar discussions happening within your institution?

An Unique Approach to a xMOOC — Learner-Centric Course Design

course6671

Applied Sustainability” Fanshawe College

Not all MOOCs are created equal. I know of one, Applied Sustainability, delivered by a community college in Ontario, Canada that defies what many have come to expect from a Massive Open Online Course offered through a university-sponsored platform such as Coursera, Open2Study or edX. What makes Applied Sustainability offered by Fanshawe College unique?  It’s the learner-centric course design that makes it different, where the focus is on students and their learning. This approach contrasts with an instructor-centric design—an approach which emphasizes content delivered by faculty with a learning strategy that follows accordingly. Though both formats are applicable in given learning situations, few student-centric course designs in xMOOCs exist, which is what motivated me to share a learner-centric design example with readers. It’s a worthy endeavor for educators to explore and consider different approaches to course design for MOOCs, to improve upon what already exists, and bring xMOOCs to the next level.

How Fanshawe’s MOOC is Unique:  Fanshawe’s MOOC takes the form of an educative journey that engages students in the learning process with active, and practical learning assignments that make learning meaningful, relevant and specific to each student.

The interview approach supported the practical focus of the MOOC – as the interviewer was learning along with the student, as opposed to having an expert deliver a presentation.”  MOOCs at Fanshawe

This approach contrasts significantly with the course design model that the majority of MOOCs delivered on university platforms feature.  The pedagogical methods used in xMOOCs  are remarkably similar to the traditional classroom format—course content delivered via the subject matter expert. xMOOCs typically are associated with top-tier higher education institutions, and are led by faculty members, sometimes referred to as ‘super professors‘, or ‘talking heads’, labeled (not so kindly) by MOOC skeptics. The instructors deliver content that mirrors the classroom method quite closely, but without the accessibility and feedback capability that instructors offer. Fanshawe’s format is closer to the original version of MOOCs as its co-creators [Downes and Siemens] introduced in 2008, where the focus was on learning within a network where knowledge is co-created, not from one expert i.e. the super-professor, but from many sources including the learners within the course.

Screen Shot 2014-03-26 at 7.25.07 AM

Image from Presentation Slide featuring student project, posted during Webinar with Contact North, with Wendy Wilson describing the MOOC’s development strategy

“For the students, the most significant benefits they reported on the first offering of this MOOC were the changes they made in their thinking, behaviour, and habits concerning sustainability – for some it was truly life changing.The wide range of students from around the world appreciated the truly applied nature of the course.”

Course Description: Applied Sustainability: The course is designed to emphasize the practical and the personal elements of sustainability, with each of the six modules featuring on-site video interviews in locations and with experts involved with different aspects of sustainability. The themes of the modules focus on: water, waste and wastefulness; homes; streets and neighbourhoods; cities and regions; policies and certifications; and a final section on our community, your community. Applied Sustainability, Desire2Learn

Below is a brief outline the characteristics of Fanshawe’s MOOC that make it unique. For readers interested in learning more, the Pockets of Innovation series on Contact North’s website provides further background and description—Designing and Offering a Massive Open Online Course (MOOC) at Fanshawe College.

Characteristics of the Learner-Centric Design

1) Diffusion of subject matter expertise: Rather than one subject-matter expert leading the course, there is an interviewer who travels to visit numerous subject matter experts in the field. The interviewer acts as the ‘host’ of the course, going on exploratory journey along with the students to learn about applied sustainability by conducting interviews with experts in the field.

“Three short videos (8 to 10 minutes) highlight three specific themes through visits to facilities directly related to waste and water such as the Greenway Wastewater Plant and the Upper Thames River Conservation Authority (pictured below). Staff interviews, facilities tours, and demonstrations of their ongoing work are presented.  The interview approach supported the practical focus of the MOOC – as the interviewer was learning along with the student, as opposed to having an expert deliver a presentation.  Topic-related links are provided to articles, report summaries, videos, TED talks, and other resources.”

Screen Shot 2014-03-26 at 7.22.07 AM

Presentation Slide posted during Webinar with Contact North, featuring Wendy Wilson describing the MOOC’s development strategy

2) Reliance on open resources, with a collection of curated resources. Students accessed content on the course topic (sustainability) primarily from the experts in the field, on the field trips with interviewer, and by exploring list of curated resources provided on the course site. The resources are from a variety of sources, with very few coming from scholarly or peer-reviewed sources.

3) Customization of learning experience. The Fanshawe MOOC provided options for students to customize their learning by offering four options for course participation. The four levels of achievement—Green, Silver, Gold and Platinum.

“… At the Silver level, students take part in weekly discussion on such topics as the use of rain barrels and antibiotics in agriculture.  The Gold level involves completion of a weekly task, such as undertaking a three-day waste audit at home, posting a photo of the accumulated waste, and reflecting on personal consumption and disposal habits. The Platinum level of achievement requires the completion of one project over the length of the course, assisted by a project manager from Fanshawe. The project-options include creating a Green Gaming Journal by using a blog, Tumblr, YouTube, or a podcast to comment on the green elements of a video game with ecological themes, such as SimCity. Another choice involves using QGIS, an open source geographic mapping system, to map a neighbourhood and analyze issues such as park vs residential space usage.”

Conclusion
We can credit MOOCs for generating discussions among educators and stakeholders about technological advancements and education, specifically how technology can be used to improve access, quality and cost. Yet we need to move discussions about online education forward, and one dimension which needs attention is how to create learning experiences for students that are relevant and meaningful. The current format of most xMOOCs are not much different from the traditional instructor-focused model, yet student-centered course design is a viable option very worthy of our time and energy.

Update: After this post went live, two other institutions reached out to share MOOC, learner-center models.  Open2Study, customizes courses for the MOOC format; you can read more from the comment posted. Another, Penn State’s Center for Online Innovation in Learning is developing a ‘flex-MOOC’ framework, which appears to have much potential.

Further Reading:

Is Learning Scientific or Organic?

Lafyette_organic_photo

Lafayette College, Organic Garden

Is learning scientific or organic? A scientific perspective suggests that learning is explained by how the brain works, is rationalized by scientific study. Organic learning on the other hand occurs through the culture in which one lives—where learning is derived from, or characteristic of culture and society.

The presentation, subtitled Learning Theory & MOOCs delivered by Boise State University professor Norm Friesen at a conference in Shanghai, presents the alternate viewpoint that learning is cultural—that learning comes about through experience and immersion in culture rather than by a process explained by scientific study of behaviors. Not a new idea, though it’s worth examining for two reasons: first, in light of the transformative influence technology is having on our behavior as a society and culture as we speak—and second, that current teaching practices are based on learning theories derived from scientific studies done decades ago. Considering these two factors, is there a disconnect in the practices and methods implemented at education institutions today? The presentation sheds light on this question—it’s thought-provoking, so much so I want to share it with readers.

Presentation Overview 
The slideshare with audio is about an hour-long; three-quarters of it focuses on learning theory which is the focus of the highlights outlined below. MOOCs were discussed briefly at the end of the presentation.

Learning as a Science
The idea of learning as a science began in the 1900’s with the American psychologist Edward Thorndike. Interested in learning, he laid the foundation for learning as science through his research on testing and exam procedures in industrial settings, and behavior studies that used animals as test subjects. Subsequent theories, behaviorism, cognitivsm and constructivism followed, all built on the similar premise that learning is biological, or mechanistic; the brain acts as a center for processing, storing, recalling and directing responses to stimuli. Over the last hundred years education institutions built practices, methods and policies around the principles of the theories. Teaching is a reflection of this scientific perspective; methods of instruction, assessment and testing embrace the theoretical principles.

In recent years, it’s gone further where the human brain is compared to a computer. Common terms used to describe learning and the brain include storing, processing, retrieving, short-term memory, etc. To that end, knowledge is taught in schools with the goal of maximum efficiency, technology often used as a method to increase efficiency, ie. automating teaching functions grading tests, essays and even feedback from robots in group work.

Learning as a Science: Examples
The scientific approach to learning, where the brain is viewed as the ‘processor’ of learning, drives our education practices, yet still the evidence of exactly how learning occurs within the brain is inconclusive. Friesen, in his presentation describes the science of learning, as “learning = x”.   There is no shortage of examples that reinforce the point that education builds on this scientific approach:

The American Psychological association devotes an entire section on its website to teachers: “Research in Brain Functioning and Learning: The importance of matching instruction to your child’s maturity level”.

Another organization, Learning & the Brain®, seeks to bring “neuroscientists and educators together to explore new research on the brain and learning and its implications for education“.

Learning is Organic
Friesen challenges the scientific viewpoint with a slide introducing culture as the driver of learning:

What if we were to say...
“We depend for survival on the inheritance of acquired characteristics from the culture pool rather than from a gene pool.

“Culture [would] then become the chief instrument for guaranteeing survival, with techniques of transmission being of the highest order of importance.” 

Friesen goes further and suggests that:

  • Learning is not cause and effect
  • Learning is cultural not scientific (brain as the machine)
  • Learning is contingent upon culture
  • Learning changes over time

There are two theorists that Friesen uses to support the perspective of considering a cultural approach to educating students—one is Jerome Bruner, a psychologist readers are likely familiar with, who is considered a cognitivist (thus supporting the science dimension). However, Friesen emphasizes that Bruner states that learning changes in fundamental ways based upon culture, example of technology.

Screen Shot 2014-02-06 at 8.13.23 AM

Slide from Dr. Norm Friesen’s Slideshare, Learning Theory & MOOCs (2013)

Dr. Daniel Trohler is a scholar that studies the role of culture in education; Friesen shares Trohler’s perspective put forth in one of his book’s on the topic, The knowledge of science and the knowledge of the classroom Using the Heidelberg Catechism. In this example Trohler examines school textbooks over the ages and makes the point that pedagogy derived from science presents a constructed picture to students, that is a part of a whole, becomes an object of teaching for maximum efficiency.

Closing
Professor Friesen’s presentation is thought-provoking–my takeaway is how it challenges the traditional approach education. Professor Friesen doesn’t appear to suggest we abandon the learning science altogether, but he does present MOOCs as cultural technique of transmission. Though I suggest that the current-day xMOOCs  are an extension of classroom pedagogy, it is only the delivery format that is different. The same science is behind the methods used, ie. lecture, testing, etc.  It is the cMOOC format, developed by Downes and Siemens that reflects a different approach to learning, a cultural and organic approach accomplished  through a connected experience, where knowledge is constructed by the individual from existing knowledge within a network. Learning is pulled by the learner, and not pushed. This sounds like organic learning, a reflection of our connected and networked culture, doesn’t it?

Resources:

Photo Credit: ‘Lafayette Organic Garden’, Flickr: Lafayette College’s Photostream