Need-to-Know-News: edX goes Corporate, Wired Magazine/USC Partner to Create Degree & More on Competency Education

This ‘Need-to-Know’ blog post series features noteworthy stories that speak of need-to-know developments within higher education and K-12 that have the potential to influence, challenge and/or transform traditional education as we know it.

1) edX Goes Corporate
Udacity the for-profit MOOC provider did an about-face a few months ago, shifting its focus from the higher education market to vocational education, partnering with big tech companies. Coursera too is reaching out to companies looking for ways to generate a revenue stream. Now edX is going the corporate route. Most disappointing given its not-for-profit premise, which differed significantly from the others—”(edX is) committed to research that will allow us to understand how students learn, how technology can transform learning, and the ways teachers teach on campus and beyond“.  This past Wednesday, October 1, edX announced the launch of professional education classes on topics including energy, entrepreneurship and cybersecurity, priced at up to $1,249 a person, with volume discounts available for some employers (Korn).

Why? According to CEO of edX, Anant Agarwal, “This goes to our sustainability story. Though edX is a nonprofit enterprise, it still needs cash to develop the free courses taken by nearly three million participants world-wide”. 

When considering the statement above in conjunction with one that Agarwal made in another interview, one with Wired magazine last month, “…effective uses of the MOOC model are only beginning to take shape. Enrollment in edX courses has doubled over last year, and he (Agarwal) believes we’re on the verge of an era he calls MOOC 2.0. “We’ve been growing as others are throwing in the towel” (Lapowsky), one wonders if he meant MOOC 2.0 as the corporate-MOOC—the not-for-free version of MOOCs.

Insight: MOOC providers do not (and never did) have a sustainable financial model to offer free courses indefinitely. It sounds noble—offering free education to learners worldwide. But somebody has to pay eventually. Development costs run into the thousands (paid for by the university-partners), operating costs considerable. MOOCs are not ‘free’. We all pay for free education in different ways; now it’s running dry and the only way to go it appears is to go corporate.

post_wired_logo_150x602)  Wired Magazine and USC Team-Up to offer “Real World Degree”
Another twist this week on an education partnership—University of Southern California (USC) announced its partnership with Condé Nast and Wired Magazine (Condé Nast is the parent company) to offer a degree program. And, as a journalist at Wired puts it “it’s a real credential, not just a certificate with the WIRED logo stamped” (Wohlsen). This is perhaps the most odd combination for an education partnership I’ve read about to date. There’s other businesses involved too, Qubed Education, which is joint venture between higher-ed investment firm University Ventures and Condé Nast, and an online degree consultancy company Synergis Education.

Taking the best from USC and WIRED, we can teach discipline and disruption, business fundamentals and the very latest innovation models from Silicon Valley. This is going to be thrilling

Insight: Businesses and now education institutions are capitalizing on an underserved market in the education sector, which is the adult learner that works full-time with some or little higher education. Yet the implications for traditional higher education are many— higher education institutions (and students) become a testing ground for business experiments and models, it draws funds away from higher education institutions, and the practice could be viewed by some, as undermining the integrity of higher education.

3) (Another) Course Management Platform geared to Competency-Based Education 
A couple of weeks ago I shared a story about a new course management provider, Helix Education. The system is different from your traditional LMS, it’s created to deliver a single platform to serve competency-based education programs (CBE), on-campus, online, or continuing education formats (Helix).  This week, another LMS launch by Motvis Learning. It’s also a  platform focused on CBE, though it’s referred to as a ‘relationship management system‘ rather than a LMS.

For students, the system looks more like a social network than a learning management system. When they log in, students are greeted by an activity feed, showing them a tabbed view of their current projects, goals and feedback. A column on the right side of the screen lists connections and to-dos, and a bar along the top tracks progress toward mastering competencies. (Straumsheim)

Insight: Competency based education has more potential for disruption to the higher education model than MOOCs ever will.

4) Multi-Language MOOC on Ed-Tech starts October

The 27th of October we will launch the third edition of the Learning Design Studio for ICT-based Learning Activities MOOC. The course will last 5 weeks and a group of facilitators will support you in the task of designing your own learning activities and lessons. The course will be offered in six languages: English, Spanish, Catalan, Greek, Slovenian and French.”

For more information: http://handsonict.eu/join-the-mooc/

 

Need-to-Know-News: BBC gets into MOOCs, Global MOOC Report for $2,500 & Competency Education gets Boost

MP900405500This ‘Need-to-Know’ blog post series features noteworthy stories that speak of need-to-know developments within higher education and K-12 that have the potential to influence, challenge and/or transform traditional education as we know it.

1) BBC gets involved with MOOCs
Four universities in the United Kingdom are partnering with the British Broadcasting Corporation (BBC) to include never seen before film footage from World War One via its archives. Times Higher Education reports that FutureLearn will host the four MOOCs. The venture with BBC promoted as a creative approach, will be different from the traditional lecture format typical of the majority of xMOOCs:

“FutureLearn chief executive, Simon Nelson, said that many Moocs had been “rightly criticised as just being a repackaging and redistribution of the traditional lecture format, and that some universities were using the internet to “pump out videos”, rather than using their courses to tell a story. He said he hoped that working with the BBC would help institutions to be more creative.” 

Insight: This venture is a great opportunity for institutions to demonstrate they can reach non-traditional xMOOC students (traditional MOOC students: holding an undergraduate degree or higher), and engage learners not familiar with online learning or self-directed education. Not to mention introducing a novel method for delivering content. The four MOOCs launch soon—in October so we won’t have to wait long to see the response.

Screen Shot 2014-07-25 at 11.14.36 AM

Image from “BBC helps produce First World World Moocs” by Chris Parr

2) Report on MOOCs for $2,500
It’s not only Udacity and Coursera that are starting to make money from xMOOCs. It seems that there’s even demand for research reports about MOOCs—$2,500 a pop, or $10,000 for enterprise wide access. The report, published by Research and Markets, The World’s Largest Market Research Store described as follows:

“The report, the Global Massive Open Online Courses Market 2014-2018, has been prepared based on an in-depth market analysis with inputs from industry experts. The report covers the Americas and the EMEA and APAC regions as well as the key leading countries in this market. It also provides in-depth overview of the revenue generation models adopted by the vendors in the market and discusses the key vendors operating in this market It also includes discussions of  “…the market segmentation based on student demographics and course preferences, as well as the market landscape and its growth prospects in the coming years.” 

Insight: xMOOCs are big business, though not necessarily for higher education institutions.

 3)  Competency Education Gets a Bigger Boost
There’s bi-partisan support for competency-based education in the United States, a bill passed this week by the senate will allow student aid to go towards thirty academic programs that are experimenting with a range of innovative higher education academic programs that lead to degrees, many including degree-tracks grounded in competency-based education. According to sources of Inside Higher Ed,  there are nearly 350 institutions that do offer, or plan to offer a competency-based degree track (Fain, 2014).

Representative John Kline of Minnesota, the Republican who chairs the House education committee, called the legislation a “good first step” to figuring out what works and doesn’t work for competency-based education.Inside Higher Ed

The programs fall under the Experimental Sites Initiative (ESI), funded by the Department of Education. The program is still accepting applications.

Insight: The idea of experimentation with students pursuing higher education makes me uneasy. If such programs enroll students at risk for not completing a two or four-year degree, I would hope that the programs do not further jeopardize the students’ chances of success.  I think back to the Udacity and San Jose University experience with MOOCs, where one of the pilot programs was a MOOC format for remedial math.

“Another factor in the disappointing outcomes may have been the students themselves. The courses included at-risk students, high school students and San Jose State students who had already failed a remedial math course.” Inside Higher Ed

Failure rates were higher in the MOOC experiment than in the face-to-face class.  What about those students?  Granted new programs have to be piloted in some way, I would hope however, that there is a plan in place to address any negative outcomes students may experience as a result of the experimental programs.

You can keep up to date with developments in education and related sectors by following me on Twitter, @OnlineLearningI 

Three (BIG) Barriers to Student Participation in xMOOCs

This post outlines three barriers that can deter, discourage and/or intimidate students from participating in xMOOCs (MOOCs offered on platforms associated with higher education institutions, i.e. Coursera, iVersity, edX, etc).

construction_barriers

“Construction Barriers”  Photo by Lyn Topinka

The xMOOC model that emerged in 2012 has not changed much in 2014, with completion rates and participation rates just as low as they were when concrete data on completion rates appeared in 2013 (Parr, 2013). Though there are a variety of factors that contribute to low completion rates, I suggest that three barriers, 1) technology, 2) poor usability & course design, and 3) anonymity contribute significantly to low student participation levels and completion—barriers that deter, discourage and in some cases intimidate students. Also, in some instances, barriers one and two are potential barriers in closed, online classes (as those offered as for-credit courses at public and private institutions).

To illustrate points one, technology and two, poor usability and course design, below is a selection of screenshots featuring actual student comments and questions (names obscured) taken from several MOOCs offered on Coursera. Comments below are representative of typical experiences and frustrations of students participating in MOOCs. In some instances, the examples included are similar to frustrations students experience in closed online courses, which I’ve encountered when working with faculty in online course design, and as a lead curriculum developer for online programs at a private university. I close by discussing the third barrier, anonymity in online learning, specifically in MOOCs.

1) Technology 
Examples below feature student challenges with accessing course content and engaging in events due to bandwidth and internet access limitations.

Screen Shot 2014-07-11 at 8.22.26 AM

File size is a common problem in MOOCs and small courses

Screen Shot 2014-07-11 at 3.41.19 PM

Connectivity issues are common due to bandwidth, and even limitations of the devices used

Screen Shot 2014-07-11 at 8.22.50 AM

Restricted access to certain sites in some countries

Examples below feature students’ frustrations with applications (discussion forums, etc) within the MOOC platform itself which put up barriers to student participation and engagement, for example, i) discussion forums (volume of student posts and organization), ii) synchronous events offered via Google Hangout or other platform which often fail due to technical glitches, or because of students’ lack of technical ability, etc.

Screen Shot 2014-07-12 at 3.54.13 PM

Discussion forums often become unwieldy; though more common in MOOCs it also happens in closed, small online courses

Screen Shot 2014-07-12 at 3.53.21 PM

Discussion forums in most MOOC platforms have options for ‘subscribing’, where participants receive alerts of new posts within that particular forum, though not all students are familiar with this settings and don’t know how to turn the notification emails off (or on). It’s helpful to provide participants with the instructions of how-to do so (among other features) in an orientation or introduction to the course

Screen Shot 2014-07-12 at 3.37.18 PM

The forums with large numbers of participants can be overwhelming to the point that there is little opportunity for reflection or deep discussions. In closed online classes it helps to have focused discussion questions per thread, and if more than 20 participants to break the class into smaller groups

Screen Shot 2014-07-12 at 4.20.15 PM

Google hangouts and other platforms used for synchronous events, are not immune to technological glitches. A practice run prior to the event helps (granted even still, problems occur),  and having a back-up plan is recommended

2) Poor Usability and Course Design
Usability refers to how effectively students can navigate, interact and engage with the course interface, find the content they need, determine what they need to do to engage, etc. How user-friendly the course is (or is not) is a function of how the course content and pages are organized, what is featured on the course home page for example, or where the course announcements show up, even how the course tabs appear in the navigation menu. Usability falls under the umbrella of course design; it is a component with its own principles and guidelines that impacts the students overall course experience and learning outcomes in online spaces. Usability adds another layer of complexity to designing learning experiences mostly due to the newness of online platforms as delivery mediums for education.

Course design is a broad and deep topic, which I can’t address at all adequately in this post, but below are some examples that are representative of issues that frustrate students, and can deter learning outcomes that have to do with how an assignment’s instructions are worded, presented to the student, or even designed in the first place.

Screen Shot 2014-07-12 at 3.56.30 PM

Instructions for student assignments or activities need to be written with exceptional clarity. This means expanding on details is necessary, including examples, and reinforcing instructions and expectations via course announcements or live sessions when the course is in session. Another issue is the use of  consistent terminology throughout the course.  In this above example ‘thread’ and ‘post’ were used interchangeably, when in fact they mean different things, thus confusing the students.

Screen Shot 2014-07-12 at 4.50.56 PM

Another example of student confusion when there is inconsistency or conflicting information in the course

Screen Shot 2014-07-12 at 6.52.58 PM

Frequently, it is student assignments that generate the most confusion among students in virtual formats, often due to unclear instructions, or those that require students to use technical applications (e.g. to create a digital artifact) that they are not familiar with

3) Anonymity
A view on participants posting anonymously within a MOOC from iVersity:

“MOOCs offer an environment that may engage introverts. Online anonymity can make students comfortable expressing themselves in forums…participating in course conversations online may give students confidence to contribute in traditional classrooms and work environments.” iVersity blog post on Anonymity

I disagree with iVersity’s position, and with Coursera and edX, which both allow anonymous posting within discussion forums. Anonymity does not contribute to effective online learning communities such as MOOCs; it’s counterintuitive to the premise of a learning within a community, where the idea is that learners actively engage, and learn with, and from each together. Several papers have identified the benefits of learning communities in distributed (online) learning environments (Dede, 2004), with some emphasizing the value of communities in MOOCs especially (Kop, Fournier & Mak, 2006). What is consistent in the research is the idea of trust and a set of common values or goals among learners.

The type of support structure that would engage learners in critical learning on an open network should be based on the creation of a place or community where people feel comfortable, trusted and valued, and where people can access and interact with resources and each other. (Kop, Fournier & Mak, 2006, p. 88)

Learning in a virtual community, where students go outside of their comfort zone, are challenged to consider alternative perspectives and build a personal learning network for example, requires a level of rapport, familiarity and trust between classmates and instructors. This sense of community can and does happen in small, closed learning environments, and in cMOOC learning communities, but experiencing a sense of community in xMOOCs is far more difficult to accomplish with many of variables making it so, anonymity is just one. With this learning approach (and others) assumed by MOOC platform providers, I see xMOOCs destined to be static resources posted on the web—open courseware such as MIT OCW.

Closing Thoughts
Learning in xMOOCs is far more complex than what the MOOC platforms seem to be able to address. Low completion rates are just one metric of how students’ views of MOOCs are at odds with what the expectations of the MOOC providers. The three barriers discussed here, technology, usability and anonymity are just one piece of a bigger problem that MOOC platform providers will need to address if they are interested in creating a communities of learning where students actively engage, contribute and learn.

Further reading:

How to Create a Video Strategy for MOOCs: Costs and Considerations

“Harvard has built what amounts to be an in-house production company to create massive open online courses, or MOOCs…[it] has two video studios, more than 30 employees, and many freelancers — an astonishing constellation of producers, editors, videographers, composers, animators, typographers, and even a performance coach to help professors get comfortable in front of a camera.The Boston Globe

Harvardx Studio Filming for MOOC

In the HarvardX video production studio, Harvard historian & museum curator being filmed for a MOOC.  Image credit: Katherine Taylor for The Boston Globe

I saw scores of dollar signs when I read about Harvard’s production studios created solely for the purpose of producing videos for MOOCs to put on the edX platform. The article featured in The Boston Globe about Harvard’s new studio, describes videos that share traits similar to documentaries rather than the typical lecture videos featuring a professor speaking to the camera, typical of xMOOCs. Here’s the catch though, the money spent on these production costs for MOOC videos, which is dear, may not always be worth the investment according to recent studies (Guo, Kim & Rubin, 2014), (Hollands & Tirthali, 2014).

For readers considering, or are in the process of developing a MOOC I’ve outlined guidelines that will help in the development of an instructional strategy for the delivery of the course content for xMOOCs (MOOC featured on a platform such as Coursera or Open2Study).  I’ve drawn from recent research on video production and student engagement specific to xMOOCs—one study out of MIT using data from edX, and the other a noteworthy report released this May “MOOCs: Expectations and Realities”.

Following are considerations and questions to guide the development and choice of the content delivery methods, including videos for MOOCs. Don’t be misled by the flashy [and expensive] studios that Harvard established, thinking that this is a requirement for putting on effective MOOCs. This high bar set by Harvard, may be unjustified, more so when analyzing why institutions choose to offer MOOCs, and how they fit into the vision, and long-term strategy. For the most part institutions’ reasons for offering MOOCs are vague, and few establish metrics to measure the effectiveness of MOOCs, including return on investment, as discussed in Hollands & Tirthali’s report:

“…Most institutions are not yet making any rigorous attempt to assess whether MOOCs are more or less effective than other strategies to achieve these goals.

The Numbers $$$
The two primary cost drivers of MOOC production are the hours invested by faculty members, administrators, instructional designers, technical support and the costs associated with the quality and type of delivery method for the course content. Videos, the typical mode for xMOOCs, can range between high and low production values. The estimated costs for high quality video production is $4,300 per hour of finished video (Hollands & Tirthali, p 11). High quality video production typically involves a team of at least five video experts each involved in one aspect of the process, including  filming, sound, lighting, editing and project management.

  • Development costs of MOOC vary significantly: as low as $38,980, Teacher’s College, Columbia University, to a range between $203,770 – $325,330, Large Midwestern University (Hollands & Tirthali, p 12)
  • Harvard’s costs as per the Boston article, $75,000 and $150,000, though depending upon the method for calculating, it’s difficult to compare to the study quoted above.
philip-guo-edx-first-blog-figure

Chart showing student viewing time in minutes.

Guiding Questions and Considerations for Creating a Video Strategy

  • Consider goals for each module/week within the MOOC when planning for content delivery. What delivery method will communicate content to students effectively? Is there an opportunity for student-developed or student-curated [and shared] content sources?
  • Consider a variety of content delivery methods: video, open source content (video and other), interactive online resources, etc.
  • If using video, consider between formats, tutorial-style (illustrating a procedure, step-by-step, i.e. Khan Academy-style) and lecture.  Within each format there are variations influenced by filming technique [screen cast, podcast, filming on laptop, studio, etc], media choices, etc. Research shows students engage differently with each (Guo, Kim & Rubin).
  • Consider: the average engagement time of any MOOC lecture video maxes out at 6 minutes, some as few as 4. However, students engage with tutorials quite differently, often pausing, re-watching, fast-forwarding, etc. (Guo, Kim & Rubin).
  • Are there existing open and accessible content sources on the web that can illustrate a course concept [rather than filming from scratch]?
  • Call students to action to use and apply content from video, i.e. via a  discussion forum or upcoming assignment.
  • Finally, plan the strategy upfront where: 1) each content delivery mode (video, etc) is planned by module/week (content is outlined and scripted when necessary), 2) content within delivery mode links directly to goals of given week, and 3) there is a requirement for students to apply and use the content, for example in discussion forum, assignment or quiz.

References:

Further Reading:

How and Why Institutions are Engaging with MOOCs…Answers in Report “MOOCs: Expectations and Reality”

How do institutions use MOOCs; and to what end?  •  Why do institutions pay thousands of dollars to develop and offer a MOOC on an external platform?  •  How do institutions determine the effectiveness of their MOOC efforts?  •  What are the costs associated with producing and delivering a MOOC?

All good questions; questions that policymakers, administrators and other stakeholders within higher education institutions that are considering MOOCs or already engaged with, want [or should want] answers to. The 200+ page report “MOOCs: Expectations and Reality” by Hollands and Tirthali of Columbia University attempt to answer these questions by surveying 83 faculty members, administrators, researchers and other actors within 69 education institutions. The report delivers on the promise of its title—how and why institutions engage, and provides the reader with even more insights.

The report is meaty, worthy of review for anyone with a vested interest in MOOCs of any type. In this post I provide a brief overview of the report, but focus specifically on one aspect of the ‘how‘. I highlight the resources required to develop a MOOC—how many people it requires, the job titles, the [estimated] costs associated with development. This may be useful for readers considering developing a MOOC for a platform such as Coursera or another, or for a cMOOC using a collective course design approach. This report brings into focus just how resource-hungry MOOCs are, and after reading the report, readers considering developing or contributing to the development of a MOOC might feel enlightened, encouraged, or perhaps even discouraged; at the very least, will have a better understanding of MOOCs and their place in higher education institutions.

 Overview

Who sponsored the report?  The Center for Benefit-Cost Studies of Education (CBCSE), a research center at the Teachers College at Columbia University. The mission of the center is “to improve the efficiency with which public and private resources are employed in education“.  Note: the report is open and available for download.

Purpose of the Study: Given the work of the CBCSE, and its pursuit of improvement of cost efficiency in education, the report is an extension of its mission. The purpose as outlined in the report, “the study serves as an exploration of  the goals of institutions creating or adopting MOOCs and how these institutions define effectiveness of  their MOOC initiatives“.

Screen Shot 2014-05-19 at 9.50.39 PM

Figure 1 ‘MOOCs: Expectations and Reality’ (p 22)

Report Snapshot:  The report sample includes 83 administrators, faculty members and researchers, all of which were interviewed, at 62 institutions. The institutions: public and private universities, community colleges, platform providers, research organizations, for-profit education companies and a selection of institutions deemed ‘other’ including one museum (p 180). Of the 62 institutions in the sample, 29 at the time of the study were offering or using MOOCs in some way; the remaining were either not participating or taking a wait-and-see position.

Why a MOOC? One of the reasons this report is instructive for the education community is the inclusion of the data about why institutions offer MOOCs. Many have asked why some institutions (several public higher education institutions) have spent thousands of dollars, invested considerable resources into this method of education delivery to the masses that has yet to be evaluated and tested for effectiveness. The chart below summarizes the six reasons identified.

Screen Shot 2014-05-20 at 8.25.43 AM

‘MOOCs: Expectations and Reality’ (Hollands & Tirthali, p 8)

The above table is merely a snapshot.  Each goal is described in further detail within the report. A case study featuring an institution accompanies each which gives a contextual example of the reasons.

A snapshot of How? MOOCs are resource intensive efforts, and the report validates this. Development of a MOOC, and the facilitation of the course once its live (accessible to students) requires significant amounts of time and energy from individuals across several departments within the institution. The faculty member (or members) acting as the subject matter expert for the MOOC requires a team, each with different areas of expertise to support him or her in bringing the content to life and creating an environment of learning for hundreds, if not thousands of course participants.

“Number of faculty members, administrators, and instructional support personnel involved MOOC production teams seldom included fewer than five professionals and, in at least one instance described to us, over 30 people were involved. Faculty members typically reported spending several hundred hours in the production and delivery of a single MOOC” (p 11)

Example of Human Resources Requirements: Case Study 11

Case study 11 provides an excellent example of the commitment of resources needed for developing a course for a MOOC platform which in this example is Coursera. The institution in the case is an unnamed MidWestern University (p 144). The school invited faculty with prior media experience to develop a five to eight week MOOC. This study is representative of the human resources required for development of a MOOC.

Human resources requirements by job title for course development of a MOOC:

2 x Faculty Members: (Subject Matter Experts)

1 x Project Manager: Leads the project, coordinates all elements of development. Liaise with departments as needed within the institution. Manages the project timetable; keeps project on time and on budget

4 x Curriculum Design Team Instructional Designer (works with faculty to present course content and create a learning environment with it on the course home page). • Instructional Technologist (works with instructional designer) • Video Production Liaison (works with faculty member in production of videos, and liaise with video production team)

5 x Video Production Team:  Production Manager •  Camera operators/equipment technicians • Audio-technician

In this case study, videos were produced at a high quality, using a full video design team. The final costs were calculated using records from the institutions, though the report authors made some estimates due to lack of detail on some aspects of human resource inputs.

Screen Shot 2014-05-20 at 8.59.34 AM

‘MOOCs: Realities and Expectations’ (p 144). One of the two data tables accompanying case study #11. Table 7 gives the range of hours spent on MOOC design (p 144)

Lecture Videos: Costs and Student Engagement
One of the primary drivers of costs in MOOC development (for platforms such as Coursera, FutureLearn, etc.) is video production. The more complex the video, for instance addition of graphics, multiple cameras used for shooting, post-filming editing, the higher the costs. Low-tech efforts,  where there might be one camera person, or even the faculty member self-recording on his or her laptop requires far fewer resources.  Some institutions seek a higher quality finished product, which in turn demands a high level of production using a team of video professionals. Accordingly, the costs vary dramatically. ‘MOOCs: Expectation and Realities’ estimates high quality video production at $4,300 per hour of finished video (p 11).

One may be tempted to think that the higher the video quality, the better the learning outcomes. However a report published recently by EDUCAUSE, What Makes an Online Instructional Video Compelling? suggests that students engagement with videos relies upon several factors, including whether or not the video links to an assignment within the course. Furthermore, the average viewing time of videos is less than five minutes (Hibbert, 2014). What this suggests is that videos presenting content must be carefully and strategically planned for during the course development phase, and tied closely to the instructional strategy. Higher production costs does not necessarily mean higher student engagement or learning outcomes.

Closing Thoughts

The report discussed here, ‘MOOCs: Expectation and Realities’ is an important contribution to the MOOC discussion in higher education. In my opinion one of the greatest benefits of the report is the spotlight it puts on the resources required for developing a MOOC, in contrast to the reasons why institutions engage with MOOCs. When one examines closely the reasons, it appears that the amount of resources invested, in some cases is extreme. I agree with the authors in the point they make in the executive summary,

” [we]…conclude that most institutions are not yet making any rigorous attempt to assess whether MOOCs are more or less effective that other strategies to achieve these goals” (p 11).

I’ll add one more point to this, and that’s the need for a complete and comprehensive approach to course design, (applicable to any course) that involves from the beginning, a thorough needs analysis that determines the goals of the organization and how the [potential] course fits into it. It’s only after this analysis that the course design process can proceed.

References:

Hibbert, M. (2014). What Makes an Online Instructional Video Compelling?. EDUCAUSE Review Online. Retrieved from: http://www.educause.edu/ero/article/what-makes-online-instructional-video-compelling

Hollands, F. M., & Tirthali, D. (2014). MOOCs: expectations and reality. Full report. Center for Benefit- Cost Studies of Education, Teachers College, Columbia University, NY.  Retrieved from: http://cbcse.org/wordpress/wp- content/uploads/2014/05/MOOCs_Expectations_and_Reality.pdf

MOOC Design Tips: Maximizing the Value of Video Lectures

“Which kinds of videos lead to the best student learning outcomes in a MOOC?”
How Video Production Affects Student Engagement: An Empirical Study of MOOC Videos (Guo, Kim & Rubin, 2014)

Reel of FilmAn excellent question that design teams and instructors of MOOCs want to know—which kinds of videos lead to the best student learning outcomes in a MOOC?  According to a recent study conducted by researchers for the edX MOOC platform, this was the most pressing question posed by the course design teams working with its partner institutions. Given that most MOOCs offered through higher education institutions platforms such as edX, iVersity, or Coursera use video lectures as the primary content delivery source, it is a critical question that preoccupies many if not most MOOC instructional design teams. Adding to this need-to-know element is the fact that video production is most often the highest cost associated with MOOC production. MOOC video production can range from a few hundred dollars and run up to the thousands. This post suggests how institution can use resources effectively in the video production process with the primary goal of supporting students’ learning outcomes.

The report released by edX last week gives design teams some concrete data to examine. I’ve emphasized below the recommendations and practical application points from the paper for readers who might be part of a design team for MOOC, online course, or for those with an interest in video production for instructional videos.  There are limitations to the study outlined in the paper, though the depth of the analysis does provide data worthy of consideration. 

The report, the first of its kind according to the authors Guo, Kim & Rubin, analyzes students’ engagement* with lecture videos gathered from data extracted from over 6.9 million video watching sessions across four edX courses.  *Student engagement is defined in the study by:

  1. Engagement time: the length of time that a student spends of a video. This is the same metric used by YouTube. Though researchers acknowledged the limitation of engagement assessed from this one-dimensional perspective.
  2. Question/Problem Attempt:  Almost one-third of the videos across the four courses featured an assessment problem directly following the video, usually a multiple-choice question designed to check a student’s understanding of the video’s contents. “We record whether a student attempted the follow-up problem within 30 minutes after watching a video.”

Videos Types for MOOCs
Lectures are divided into two primary types for the study, [which mirrors most MOOCs]: 1) lecture videos for content delivery—presented via an instructor/professor (‘talking head’ is the term used in the paper), and 2) a tutorial/demonstration, a step-by-step problem solving walk-through, common in computer science courses, courses featuring mathematical concepts or science courses featuring lab demonstrations.

Video Production Format
For analysis purposes, researchers coded the videos examined in study using six primary video production formats, which I’ve summarized below, along with production styles not mentioned in the study.

1) Lecture-Style Video Formats:

  • Instructor(s) with/without Presentation Slides: Features instructor(s) lecturing, with or without PowerPoint slide presentation slides inserted throughout with instructor ‘voice over’ while slide is displayed
  • Office Setting: close-up shots of the instructor filmed at his or her office, typically instructor speaks directly to camera
  • Classroom Setting: video captured from a live classroom lecture
  • Production Studio Setting: instructor recorded in a studio with no audience, typically speaking to the camera

2) Tutorial/Demonstration Video Formats:

  • Video Screencast: of the instructor demonstrating a concept, i.e. writing code in a text editor, or command-line prompt (in the case of computer science courses), using spreadsheet or document
  • Instructor Drawing Freehand on a Digital Tablet, using a software program, which is a style popularized by Khan Academy videos (click here to view an example)

Other Formats not mentioned in the study:

  • Instructor interviewing another expert or guest speaker
  • Instructor delivering lecture in another setting related to the course (though not always), for example an ecologist giving lecture at the beach, an art historian in a museum, etc.
  • Panel Discussion of experts on specific course-related topic

Which format to use? The primary factor that determines which format to use are the objectives of the MOOC or course, and the course content. The course design team typically selects the video formats during the course design phase when the instructional strategy is created, for example: the formats of the video are chosen, the content chosen for each, related student activities or assessments selected, etc.

The second factor determining which format to employ is the amount of resources (dollars) available for video production. This determines right off the bat which tool, program or hardware will be used for the video production. Important to note, the amount of resources invested in video production does not scale to how much students’ learn or to MOOC completion rates. For example, I completed a course on Canvas Network, Statistics in Education for Mere Mortals (my course review here). The course featured video lectures and tutorials, all created by the instructor using low-budget technology. Lectures appeared to be filmed on the instructor’s laptop using a web cam, (power point slides were added, so there was some editing). Each module featured a tutorial, a screen cast where the instructor demonstrated application of various formulas to a data set. I found the professor, Lloyd Rieber, encouraging and personable; he also delivered the content concisely in lecture videos and tutorials. Interestingly, the course completion rate was over 10%, higher than typical MOOC completion rates that are usually lower than 7%.

Key Findings of Study

  • Shorter videos are more engaging. Student engagement levels drop sharply after 6 minutes
  • Engagement patterns differ between the two video formats; engagement higher with the lecture style videos (‘talking head’) which researchers suggest is due to more “intimate and personal feel”
  • Several MOOC instructors interviewed for study felt more comfortable with the classroom lecture format, however this format did not translate well online, even with much editing in production studio
  • For tutorial/demonstrations videos, the Khan-style format where instructor draws on tablet and narrates, was found to engage students more effectively than screen casts. A contributing factor—instructors ability to situate themselves “on the same level” as student
  • Video producers and edX design teams determined that pre-production planning had the largest impact on the engagement effect of the videos. Researchers used a data set within the study to test this idea

Practical Recommendations for Course Design Teams

  1. Identify type and format for each video lecture using course objectives and module breakdown as a guide, and budget. Plan each lecture for the MOOC format and its potential students. Consider copyright terms for images used in videos and slides. Plan ahead by selecting appropriate images, free from copyright during the planning phase
  2. Invest in pre-production planning phase. Segment course content into chunks, using six-minutes per video as a guideline. Identify purpose for each video lecture, and key content points to deliver within each.  Write script for each [lecture video format] and have instructor practice before filming—reduces filming and editing time

  3. For tutorial/demonstration videos introduce motion and continuous visual flow into tutorials, along with extemporaneous speaking so that students can follow along with the instructor’s thought process. Complete basic outline of video beforehand, not full script to be read word-for-word
  4. Provide more personal feel to videos. Try filming in an informal setting (such as the instructor’s  office) where he or she can make good eye contactit often costs less and might be more effective than a professional studio. Coach instructors to use humour, personal stories and convey enthusiasm where possible

Closing Thoughts
MOOCs are here to stay, which makes studies like this one valuable for helping educators be more effective through course design. This study brings us closer to finding the answer to the question which kinds of videos lead to the best student learning outcomes in a MOOC?  Yet it’s a start, there is still much more to be done in understanding how students learn in massive courses, and how institutions can be more effective with investment of its resources for increasing student learning outcomes.

Further Reading:

MOOC ‘Jam': Highlights from a Jam on Digital Pedagogy

This post includes takeaways from a ‘MOOC Jam’, a synchronous discussion online I participated in with a group of educators about digital pedagogy. 

MOOC_Jam_Image

‘Jam’ by John Wardell (Flickr)

“What is a Jam?  A Jam is an asynchronous, typed, online discussion designed to work around your schedule. The goal of a Jam is to gain perspective and solicit ideas that inform the community. After the Jam is over, you can read the exchange and the posted resources, which will remain available for several weeks.” MOOC Jam II, Digital Pedagogy   (The three threads of asynchronous discussions in this jam are: 1) Competencies for teaching online, 2) Developing Faculty Competencies, and 3) Learner Analytics for Faculty)

Screen Shot 2014-04-01 at 11.16.09 AM

Screen Shot of participants online during the Jam participating in the threaded discussions (only partial shot of participants)

This past Tuesday, I participated in a  MOOC Pedagogy Jam via the website Momentum, a platform created for stakeholders to discuss critical issues related to education, sponsored by the Bill and Melinda Gates Foundation. The purpose of the platform is to provide a space to host online events about topics related to online education, with the ultimate goal of the Jams ‘to gain perspective and solicit ideas that inform the community’.  I participated in the first MOOC Jam this past November. The topic, “Peer Review of a Framework for MOOCs” hosted by George Siemens, focused on the design of the MOOC Framework. Siemens, creator of the Framework, sought input from the community of participants.

The topic of this Jam, [which turned out to be more of a synchronous discussion] was digital pedagogy, divided into the three threaded discussions as mentioned above. Each discussion featured a moderator, responsible for responding to participants and furthering the discussion, and another moderator summarizing key themes of the discussion each hour. I chose to participate in ‘Competencies for teaching online: describing effective pedagogy’ given its description— “An exchange on how information is delivered to students, how they are engaged as active learners and community is built and how learning is assessed”.

Screen Shot 2014-04-03 at 3.55.00 PM

Screen shot of one the three threaded discussions of the Jam held on the Momentum platform

Digital Pedagogy: Themes and Highlights
Following are my insights from the discussion on digital pedagogy and I’ve included comments from other participants. (Jam II, Momentum, Digital Pedagogy).

The discussion was rich with ideas, insights and provided a glimpse into the issues and challenges with online instruction. Though the title of the Jam featured ‘MOOCs’, much input from contributors pertained to closed, online courses which created an interesting discussion by highlighting one of the primary challenges in online education—the application of appropriate pedagogical methods, which will vary depending upon the learners, the delivery method and goals of the course.

Themes:

1)  Part of the discussion was devoted to the contrast and challenges between learner-directed and instructor-directed learning. The fact that much discussion focused on this issue highlights one of the challenges with MOOCs; a MOOC, due to its scale and format lends itself to be learner-directed. It’s not surprising then that MOOCs attract learners that already know how to learn, are motivated and educated. Several Jam participants discussed methods to get learners involved in learning, how to encourage students to engage and participate [typically in the context of closed online classes].

“I’ve done something similar to engage students in action research with me. I was teaching Web Development and was not happy with the development framework we were using. So as a class we researched the pros and cons of various frameworks and decided as a class (with my approval) which one to use. This worked well – they had “buy in” as we used to say.  Beyond that, I set basic specifications as to what they were to include in their work product, but allowed them to choose the subject matter (content). I also had to give approval before they began coding.”

I see the above challenge highlighting two opportunities: 1) to provide support to students to learn how to be self-directed, and 2) to provide skill development for educators and course designers in how to be flexible and adapt instructional strategies by assessing learners, the learning context and creating appropriate learning experiences, implementing pedagogical methods that match the learning needs.

One Jam participant shared an initiative that his institution recently started for its students; a program designed to address much of what was discussed here.

“California State University, Monterey Bay, is creating an online training module for training in baseline skills in web technologies for collaboration and other soft skills, such as team working relationships. Selecting appropriate pedagogy again depends upon an analysis of the learners — goes back to careful and thrustful planning”

2)  Considerable discussion focused on how to get students to interact, collaborate and engage with peers in online classes, and what the instructors role is in facilitating group formation, participation and learner engagement. Though this theme is similar to the theme mentioned above, interesting thoughts on group formation and collaboration emerged—should it be encouraged, facilitated or left for students to form spontaneously? And if so, how? This relates to the motivation of the learner, which is quite different when students are in for-credit classes versus ‘free’ and open classes  [MOOCs] that are driven by interest and desire to learn—essentially self-directed.

But a key trade-off when you have non-static groups, as Michaelsen, Fink et al have looked at is that you lose the crucial accountability factor and or the time to form constructive group norms/roles etc. — this then leads to the ‘freeloaders’ issue that gives groupwork such a bad rep.  There is the challenge — in a MOOC context, can you establish stable, productive learning groups with accountability, positive norms, roles etc to really activate the engagement, peer learning and other benefits of group learning?”   

The comment above is interesting—is it really possible or desirable in a MOOC environment that the responsibility for group accountability and productivity rests with the instructor?

3)  The session wrapped up with discussion that focused on supporting learners, helping learners to learn in a MOOC format.  The question appears to be—how can this be accomplished, is it through course design, or while the course is live, accomplished via course facilitators?  Or do we need to teach students how to learn in a MOOC?

I think one of the goals for a MOOC is enabling learners to make connections, share, collaborate and learn from one another. Rather than thinking about self-directed or facilitator directed maybe we need to think about how we can create ways that encourage learners to support one another?”

I am very interested in how learners can and do support one another’s learning in MOOCs. Do you have some thoughts in mind about the answer to this question? What can we build into the design that supports and encourages peer-peer learning?”

Closing Thoughts
Discussions, similar to those within this Jam, create excellent opportunities to get the issues and challenges facing education, specifically online education, out in the open.  It also helps stakeholders identify what needs to be discussed and explored within their own institutions. There are commonalities across all institutions when it comes to online education, and ironically the very barriers affecting these issues, exist within institutions at all levels. Fortunately there is progress—many institutions are experimenting, collaborating and striving to adapt to cultural shifts, increase access, yet still provide high quality, relevant education. Are there similar discussions happening within your institution?