Udacity & San Jose U Halt Online Course Experiment: Lack of Instructional Strategy Undermines Courses

“We want to fail fast, learn from it and move on.”
Mohammad Qayoumi, San Jose State University President (San Jose Mercury News)

In his statement [above], San Jose State University’s President, Mohammad Qayoumi was referring to the dismal results from his university’s online course experiment with MOOC provider Udacity. The majority of students enrolled in one of the three courses in the pilot project failed; more students flunked the online courses than students in the same face-to-face courses at San Jose University [SJSU].

Qayoumi’s statement is unsettling. Quayomi appears to gloss over the fact the failure applies to students; students in pursuit of higher education. In his statement the university president sounds more like the CEO of a corporation than the leader of a public institution. Yet students are not consumers of a product that may be inconvenienced by a faulty design. In fairness to Qayoumi, his statement may have been taken out of context; he may have elaborated on adverse effects experienced by students. An alternative viewpoint to consider is that Qayoumi appears eager to learn from the mistakes made and move forward.

What is more pressing than Qayoumi’s comment however, is the problems within the three pilot courses as revealed by Thrun in a frank interview with EdSurge [Corcoran, 2013]. The problems were significant and numerous. Several students did not have access to computers to log onto the course site to complete their course work, there were “clerical” errors within the curriculum, poor communication with students, lack of clear expectations and lack of specific deadlines for assignments. It’s likely that the problems were exacerbated with the time crunch; Udacity and SJSU working independently [likely not collaboratively] to meet tight deadlines, resulting in a disjointed curriculum.

Background
This pilot project between Udacity and SJSU was initiated by Governor Jerry Brown last year when Brown contacted Sebastian Thrun co-founder of Udacity, and asked for his help. The CA state governor was looking for ways to improve access to impacted classes for college students in public universities in the state of California. The deal was struck between the two where Udacity would support three classes created by professors at SJSU—Developmental Math (entry-level math, Algebra review), College Algebra and Elementary Statistics. In the remedial math class, only 29% passed, versus 80% for in the face-to-face class, and 44% versus 74% passed in the Algebra course, and 51% students passed versus 74% in the same in-class Statistics course.

What was Missing?  A Course Design Strategy
It is not only disappointing, but surprising at how basic fundamental course design errors were made in the pilot courses; errors [and omissions] which could have been avoided had there been, i) instructional design support, ii) a team approach to course development, and iii) more time allocated to course development. Granted there was a host of issues that the program faced, not to mention the skill deficiencies of the remedial students. The program’s failure is not due to one specific omission or error, however I suggest that poor course design was a significant factor.  I’ll outline why below. By doing so, I hope to provide insight that may be helpful to readers who are currently, or planning to develop their own online courses. Perhaps even institutional leaders considering similar programs to SJSU may find something of value here.

In my experience with online course design, I’ve discovered that in order for online for-credit courses to be successful, a comprehensive instructional plan and strategy are essential. An instructional strategy is the blueprint for the course development process.  In addition to a strategy, a team approach to course design and implementation is required. There are many elements to the online classroom environment that require expertise unique to this educational delivery method. Professors are the subject matter experts, but are not necessarily the experts in web page layout and design for the course site, media selection and development, use of open educational resources, etc. Nor might they have the skills in online class instruction that are required to create a sense of community, presence, and a culture of learning in the online classroom.

Lessons Learned from Udacity & San Jose University. 
Thrun provided candid responses to EdSurge when discussing the program, sharing the downfalls and mistakes made. Readers will appreciate his honesty, and willingness to share and be open about the program’s challenges. This is a positive move; it allows other institutions and educators to learn from their mistakes. Below I present four fundamental principles that course designers and educators should consider when developing their own online for-credit course in light of what was done, or not done within the SJSU/Udacity pilot. After each I include Thrun’s feedback as outlined in the EdSurge interview.

1. Analyze learner and learner contexts. This is the first phase in the process of designing curriculum, yet likely the most neglected. However this step is essential; it shapes the direction of the course, and in some cases reveals that the course should not be developed in the first place as planned. The learner analysis phase involves examining students skill level [technical and educational], cultural background, attitudes and motivations for learning, etc. Another consideration with online learning—how will students access content and instruction?, i.e. what platform will be used [learning context]. Where Udacity/SJSU went wrong – they failed to analyze the learning context – how learners would access the course. And they failed to address the level of skill required for course completion: technical and prerequisite knowledge. As an outsider looking in, it appears that delivering a remedial course to this group of students may not have been the best course of action.

“As the class progressed, Udacity also realized that many of the students simply couldn’t get to a computer regularly enough. For some students, says Thrun, “there were none in the home, [and] even in school they couldn’t get the hours needed to make progress … It was actually a big deal”.

“When students did get to the online programs, even navigating the computer systems could be daunting. One of the questions that tutors were frequently asked was how to do exponential notation on a computer”.

“…many students lacked even elementary-school-level mathematics knowledge…”

2. Establish clear and concise assignment guidelines and expectations. This is critical to student success. Students require concise, detailed and thorough instructions for all assignments, activities, forum discussions, exams, etc.  This is often the most overlooked step in online course design. Where Udacity/SJSU went wrong: they did not provide enough guidance for students; students were likely confused, disorganized and frustrated by not knowing what assignments were due, or what was expected. Time management is always a challenge for online students – without deadlines in a for-credit course, students get behind and become overwhelmed, more so towards the end of the course. 

“Then the unanticipated problems started to crop up. When the courses started, two of the three classes didn’t give students precise deadlines for assignments. “We communicated our expectations poorly,” concedes Thrun”.

3. Establish upfront the channels and avenues of communication. Create instructor presence and a sense of community where students will be more likely to feel comfortable asking for help.  Students need to know at the start of the course what academic support is available, and how to access it. Essential to successful learning in the online environment is building a sense of community where students feel connected, and know where to go for academic help and technical support. Where Udacity/SJSU went wrong: they did not communicate that support was available, which was a tremendous oversight given that one of the courses was serving remedial students.

“Initially many students were unaware of the online tutors (who are real people) who were available online to help, 12 hours a day. But over the weeks, it became clear that the tutoring services were crucial”.

4. Allow adequate time for course development. Where Udacity/SJSU went wrong: The deal between Udacity and SJSU was complex and required extensive negotiation taking up much time. The time allocated to course development was less than adequate. The courses were not completed before the program launch, but were designed while the courses were in session.

“The professors creating the curriculum for the program didn’t have much time; they were still writing curriculum when the courses began. 

We had a whole bunch of clerical mistakes. In most cases we heard about it, and fixed it on the fly. It happens in the classroom as well.”

Conclusion
Despite the disappointing results from the San Jose pilot there may be value in the experiment if constructive dialogue results.  However, I am concerned about the students, the students that failed in the pilot project, especially those in the remedial courses. I am also concerned that the focus in the soon-to-be published follow-up report will focus on the decision-making process of the institution, and ‘who didn’t do what’ at the leadership level rather than on the students.  Also disturbing is the apparent lack of expertise in the online learning pedagogy, course design and student support systems in this program. If this serious shortcoming is not recognized as such, online learning in general might be regarded as ineffective and/or inappropriate for college students. There are numerous successful online and blended programs in higher education institutions throughout the United States and beyond. But the results of this complicated, high-profile pilot project may hinder the dialogue about online education and its role in higher education given the lack of expertise and course design errors inherent to it. However,  I do look forward to reading the results of the forthcoming analysis and report from SJSU and hope that it provides constructive dialogue for the decision makers involved.

Resources:

• Udacity’s Lessons Learned, Betsy Corcoran, EdSurge
• San Jose and Udacity: The Obvious Problems and Value of Report, Phil Hill, e-Literate
• MOOC Mashup: San Jose State University –Udacity experiment with online-online courses fizzles, By Kell Fujimoto and Elizabeth Cara, San Jose Mercury News
• San Jose Pauses Udacity, WSJ Blogs

Posts on Instructional Design:

• “Start Here”: Instructional Design Models for Online Course Design, Online Learning Insights
• How Course Design Puts the Focus on Learning, Not Teaching, Online Learning Insights
• Online Course Design with ‘X’ in Mind, Online Learning Insights