How Course Design Puts the Focus on Learning Not Teaching

This is the second post in a series of four on instructional design for online courses. The goal of this series is to introduce a fresh, innovative approach to course design.

Instructional design, also known as learning design, appears to be making a comeback, which is most fortunate given that I am writing a series on this very topic. Massive open online courses [MOOCs] that mimic the classroom model where the learner is passive and the instructor is not, highlights the need for fresh, new approach to course design. And it’s not just MOOCs that need help, but numerous courses currently offered online; many are in need an overhaul to create an environment focused on learning, rather than one that focuses on instruction. My aim here is to provide readers with a course design framework, information and tools for designing online courses. These tools and skills appear to be in demand by educators that are trying to adapt their courses and teaching methods to an online or blended format.  Some educators also find when entering into the MOOC world of instruction, they are in need of a different skill set from what is required for face-to-face teaching.

In the first post of this series, Why Online Courses [Really] Need Instructional Design I wrote about the dire need for instructional design: how it is needed to guide the process of learning for students, and guide the teaching of the course as well. A courses’ structure is a reflection of the design teams’ [or designer] belief in how people learn. The teams’ learning perspective translates into the choices made for instructional tools, the delivery methods, application activities and assessment methods for the course. We can see this in traditional higher education, which for the most part ascribes to the cognitive theory of learning, a theory that rests on the assumptions that learning is an internal process; a function of the learners memory. It’s an instructor-focused paradigm, where the teacher describes concepts for the learner via instruction, and in some cases relies upon visual representations for reinforcement. (Roblyer & Doering, 2010). This model still is the primary model in colleges and universities—professors lecturing to a class of [passive] students. Even though considerable research proves that students learn better when active in the learning process [in keeping with the constructivist theory], the cognitive theory prevails. George Siemens, co-founder of the first MOOC noted in a recent blog post:

The current MOOC providers have adopted a regressive pedagogy: small-scale learning chunks reminiscent of the heady days of cognitivism and military training. Ah, the 1960′s. What a great time to be a learner.” gsiemens, March 10, 2013

Before I move on and provide further examples, I’ll review four learning orientations—beliefs in how people learn. I wrote about each theory in-depth in a previous post, A Tale of Two MOOCs: Divided by Pedagogy, though here I’ve put together an image that summarizes the core principles of each.

When examining the image [above], one can see how learning theories influence the methods of instruction for a course. This diagram is not meant to be an implementation tool for course design, but an information source. In subsequent posts of the series I’ll review how learning theory is incorporated into the design of a course during the initial phase, the analysis phase [following the analysis is the development phase and then implementation].

Image shows four perspectives on learning based upon theoretical principles. Instructional methods associated with each, adjacent to respective quadrant. Orange quadrants represent a student focused learning model, blue instructor focused.

In another article published recently, The Pedagogy of MOOCs, the author suggests that xMOOCs offered through platforms such as Udacity, are a step backward for education. Rather than leveraging the technology and the thousands of students to advance learning, where students become contributors to the course, the MOOC applies instructor-centered teaching and assumes the learner is passive – an empty vessel.

Closing Thoughts
The crux of my post, the point I would like to leave with readers, is that a fresh, new perspective is needed for teaching and course design—course design is about creating environments to help students learn. As new courses are developed or transformed from face-to-face to an online format, the focus should not be on the technology, the platform, the video lectures, the forums, the instructor, but on the students —what methods will facilitate their learning? How will they learn? Stay tuned for my next post in this series.

Resources:

• The Pedagogy of MOOCs, Paul Stacy
• How would MOOCs should be designed and structured differently? Learner Weblog
• Opinion: The Case for Learning Designers, by Peps McCrea
• Group work advice for MOOC providers, George Siemens
• Salomon, G. (1991). From theory to practice: the international science classroom — a technology-intensive, exploratory, team-based and interdisciplinary high school project. Educational Technology, 31 (3), 41-44.

Why Online Courses [Really] Need an Instructional Design Strategy

In this post I examine and define instructional design, and share why it’s essential to the development of online courses.

“Design brings forth what would not come naturally“  Klaus Krippendorff

Developing an online course that is engaging, promotes interaction, motivates learners, and above all facilitates learning is easier said than done.  It’s even more challenging when trying to modify a face-to-face course for the online format. I know because I worked at a four-year college as lead curriculum designer. My job was to collaborate with faculty to transition their face-to-face courses to a 100% online course which was delivered via the schools’ learning management platform, Moodle. During the process of converting more than fifteen courses to 100% online courses, I was able to recognize and appreciate fully the principles and process of instructional design.

The Design Process for Online Courses, by Giulia Forsythe (Flickr)

I have over ten years experience as an instructional designer, which includes creating management development courses and employee training programs. I’ve also developed courses for K-12, including math enrichment curriculum and education programs for parents. I almost always adhere to instructional design principles and a framework for all course design projects, though I’ve found online courses to be the most dependent on sound design principles. In 100% online classes, not only is one building a course, but also a virtual classroom and community.

In this first post of a four-part series on instructional design, my aim is to provide an overview of what instructional design is, and why it’s absolutely essential to online course development. In subsequent posts I’ll cover learning theories and their role in course design, models of instructional design, and I’ll present a framework for online course design that I’m currently working on.

What is Instructional Design?
Though I’m an instructional designer and call myself such, I realize the term is vague. My definition in one sentence— “Instructional design is creating an environment for learning by structuring content and creating activities that engage students and facilitate meaningful learning”.  Also, critical to the concept of instructional design is knowing that it supports the process of learning rather than the process of teaching. Skilled instructional designers are not the subject matter experts, but collaborate with the experts to create environments where students can participate in rich, meaningful learning experiences. Of the many formal definitions of instructional design, I find the following the most accurate:

Instructional Design is defined as “a systematic process that is employed to develop education and training programs in a consistent and reliable fashion” (Reiser & Dempsey, 2007). In addition, Instructional Design models or theories may be thought of as frameworks for developing modules or lessons that 1) increase and/or enhance the possibility of learning and 2) encourage the engagement of learners so that they learn faster and gain deeper levels of understanding (www.nwlink.com).

Why Instructional Design?
Some critics of instructional design suggest that with abundance information available on the web, anyone can learn just about anything on their own terms and customize learning to their own needs. Instructional design is not at all relevant many argue, the models are inflexible and outdated. Yet I disagree. Even though we have access to unlimited content and can learn just about anything on the web, I suggest that this is all the more reason that structure is needed to guide the learner, frame the experience, even for students seeking a self-directed learning program.

Principles of Instructional Design
In subsequent posts I’ll be writing in detail how readers can apply instructional design principles when developing their own online courses. Below I provide an overview of the  principles that establish a foundation for course development. I’ve summarized the principles into three phases:

I. Analyze: Integral to the design process is analyzing the learners, learning contexts and the purpose for the instruction. Though this step is often overlooked, this phase shapes the course. Included in the learner analysis is examining students’ skill level [technical and education], cultural background, attitudes and motivations for learning, etc.  Another consideration—how will students access content and instruction, i.e. what platform will be used [learning context]. Finally instructional analysis includes identifying the learning objectives [institution established or student created] and the purpose of the instruction i.e. credit, certificate or personal interest, etc.

II. Develop/Select: This phase is time intensive and where most of the work is done. Goals for the course are identified and articulated, which may be specific or general depending upon results from the analysis. This phase includes identifying and selecting content sources on the web and/or developing new content, as well as developing an instructional strategy. The instructional strategy includes selecting the delivery methods or sources for instruction, developing/selecting instructional materials, selecting the applications and methods to support group interaction and tools. Selecting or developing tools or methods for student assessment (as applicable) and course evaluation.

Critical in this phase is developing thorough and detailed instructions for students, and ensuring a user-friendly interface with the delivery platform.

III. Implement: This phase is about putting course into practice with students. Garnering feedback from students is critical to revising and updating the initial design of the course. Conducting formative and summative evaluation is necessary to modify the course interface, materials, content and/or instruction, to better meet the needs of learners. Though revisions and updating to online courses is an ongoing process, as learning and teaching are dynamic, always changing and evolving.

Closing Thoughts
In higher education [and K-12] the design principles for instruction and learning in a traditional face-to-face setting are specific to the four walls of a classroom. But online education has no walls, and uses a delivery method that shifts the model of course development entirely. Instructional design principles and models are essential to deal with the complexities that are inherent with the Web as a delivery method. Instructional design methods are essential to online course development so that the focus is on student learning, not on the technology or the platform. In my next post I’ll review instructional design models, and examine the merits of each in the context of online course development.  Update: Post two in this series, How Course Design Puts the Focus on Learning Not Teaching.

Resources

• Instructional Design, Big Dog & Little Dog’s Performance Juxtaposition
• Instructional Design in e-Learning, elearnspace
• Dick, W., and Carey, L. & Carey, J. (2004). The Systematic Design of Instruction. Allyn & Bacon; (6th ed.)
• Reiser, R. A. & Dempsey, J. V. (2007). Trends and Issues in Instructional Design (2nd ed.). Upper Saddle River, NJ: Pearson Education, Inc.
• Krippendorff, K. (2006). The Semantic Turn: A New Foundation for Design. Taylor & Francis Group

Teaching Tips From a Master MOOC-Maker

Teach a MOOC … what are you, crazy? … However, what I learned was that teaching a MOOC has been the purest and most fun form of teaching and learning that I’ve ever done. Maria H. Andersen, Ph.D. (Canvas Network, Sloan Consortium Streamed Session, April 9, 2013)

I attended the Sloan Symposium last week as a virtual attendee and participated in several  sessions including Designing a MOOC for Canvas. The above quote comes from the sessions’ leader, Maria Andersen who currently works at Canvas and teaches her own MOOC on Social media through the Canvas Network. The session was insightful, and several readers may find the tips shared helpful, which is the purpose of this post. Strategies shared in the session are applicable to online course design in general and are not exclusive to Canvas.

I’ll include the highlights of the session—an insiders look at MOOCs based on Andersen’s experience supporting thirty MOOCs in her role with Canvas as Director of Learning, and the methods she shared for creating activities that drive learning and sustain student interest. I have no doubt that many readers will find what Anderson has to say instructive and helpful, even more so for to those considering developing a MOOC, and/or planning to teach one in the future.

Andersen has extensive teaching experience, both in face-to-face and online venues, though she much prefers teaching MOOC students, describing them as eager to learn, motivated, and willing to dig deep into content areas of interest for the sake of personal development. In contrast, Andersen describes teaching students of for-credit courses as painful and tedious (Andersen, Designing a MOOC for Canvas). Though I have differing views of traditional students than Andersen, I liked what she had to say about the roles of teachers and learners, and the design methods for online courses.

Design Strategies and Insiders Tips for Teaching a MOOC

Role of the Instructor: Andersen shared sage advice for MOOC instructors. First, that the role of the instructor is profoundly different in a MOOC in comparison to traditional for-credit course. The instructor’s role is one of curator and creator – creator of a learning environment that encourages contributing and sharing. Instructors also need to accept that fact that he or she may not be the only expert in the class. One or more enrolled participants may hold advanced degrees or be an expert in the course subject – her advice, embrace them, use them to guide other learners, perhaps as a teaching assistant or discussion moderator. A UC Irvine professor teaching with Coursera did not follow this advice. He attempted to silence the experts in course, instructing them not to participate in discussion forums. The professor ended up quitting half way through the course. 

Andersen suggests leveraging international perspectives

Other advice included leveraging the international perspectives of students in the class to provide learning depth and diversity. Anderson shared examples from her course where students contributed resources from Asia that gave cultural insight and diverse perspectives on course topics. She encourages her students to contribute and share in the course through blog posts and comments, which she encourages by frequently mentioning or showcasing student work or comments in class messages created for students. Including relevant resources on the course home page that students find and crediting the student is another effective method. In the MOOC e-Learning and Digital Cultures, the professors discussed and read clips of students blog posts during the live bi-weekly Google+ hangout sessions. As a student I found this effective, I felt instructors were involved and present in the course.

Profile of Learners: Anderson shared that the majority of learners in Canvas Network have college degrees, many holding advanced ones—nothing we don’t already know, but the rest of information was original, for instance that most learners attention span, the time students devote to the course is between two to six hours per week, the average falling around four. The ideal length of a massive course appears to be four weeks, anything longer than that becomes a challenge for students to sustain. Though it is possible, but requires specific strategies to keep the course going and learner engaged, for example a featured guest discussing a topic on Google hangout, or engaging in a discussion within a discussion forum.

In the learning community there is much discussion about the rigor of xMOOCs or lack there-of considering the length and time invested by students.  Andersens’ view—less hours invested per week does not mean less rigor but more. With a well-designed course, one that encourages learners to learn actively and create content, she finds learners are intensely focused and engaged.

Tips on Course Design

Excellent advice on course design from Andersen in this section—the principles here also apply to  small, for-credit online classes. Where does one start with MOOC course design if adapting content from a face-to-face class? Create a new syllabus. Do NOT use the same syllabus as used in a face-to-face course. Developing new materials and methods is key to a successful MOOC, or at least adapting content and methods that will be applicable to the different learner and learning contexts.

Include a ‘start here’ page that guides students through the early stages of the course

Below I’ve included key points on course development that Andersen presented as it applies to MOOCs, which is also applicable to small, online for-credit courses.

• Include a ‘start here’ page on the course site for students to go to when starting the course—similar to an orientation. Students in the first week of an online class are overwhelmed, and don’t know what to do first. An orientation page helps tremendously. Including a welcome video from the course instructor is much appreciated by students.
• Learning activities and assessments should drive learning with action, students should have to do something.
• Activities should get students to apply the content and share. An example from Andersen’s social media course— one activity was for students to get 50 new Twitter followers over a two-week time period. Then they had to describe in writing how they went about doing so either in a class forum or blog post.
• Assessments should provide another opportunity to learn—not to assess whether they have learned. Assessments should benefit the students not the instructor.
• Assessments [ideally] should drive students to find information and provide opportunity for further learning. Allow students to take assessments as many times as needed to achieve a goal score. An example from Andersen’s class—she developed a quiz that asked specific and detailed questions about a social media platforms user policy [Facebook]. She designed the quiz so that students would not be able to pass the test unless they read the policy in-depth, or even had to research the answers on Google. Either way they would learn something in the process.

Onwards to a MOOC

The advice Andersen provided about MOOCs was helpful. I hope that readers interested in teaching MOOCs found something of value, yet I don’t want to overlook the fact that the role of MOOCs is still unclear, the future of xMOOCs is fuzzy. For that reason, considering the other dimensions that surround MOOCs is recommended, and using what is here as a guide. In an ideal world, [sigh] MOOCs should be developed and taught with a clear purpose that includes expanding and sharing learning with others beyond the walls of the institution.

Four Good Reasons Why Students Need Instructor Feedback in Online Courses

In this post I describe why instructor feedback in an online course is essential for students that have yet to master a specific skill set and knowledge, and why MOOCs won’t cut it.

In a world of MOOC news within higher education, what is getting lost are the other modes of online learning that include closed online classes, offered for-credit at colleges and universities. These courses rely on cognitive or constructive pedagogy, with determined learning objectives and are delivered 100% online to small classes of thirty or fewer students. In this model, the instructor can support and interact with students, provide feedback, and encourage critical thinking. This type of personalized instruction can’t be provided in a massive open online course [MOOC], which leads me to suggest that the MOOC model [as it stands now] is not the right modality for most lower-level college courses required for a college degree.

What is Instructor Feedback?
I want to clarify before going further what instructor feedback means in the context of online learning classes. Instructor feedback is constructive and specific information that is provided by the instructor to the student on his or her course work [artifacts or other] and/or class contributions in relation to the course objectives and expectations. Feedback can be provided in a variety of mediums including, written, recorded voice, chat, video or other. In my next post I’ll address how to give effective feedback to students using various methods.

College Students Need Feedback
College students benefit greatly from instructor feedback, including when it’s provided in a small online learning community where interaction exists between students and instructor and students and students. In a Massive Open Online Course, [or even a F2F class of 100+ students]  it’s impossible to provide the required learning conditions for this type of interaction. It worries me that colleges and universities appear to be moving towards the MOOC model for delivering some or all courses (as in the case of SUNY or California’s public higher education institutions); courses that don’t provide for a student-to-instructor ratio that supports personalized learning. The MOOC model cannot provide the type of learning experiences needed for freshman or junior college students that is required for courses that include writing composition, communications, literature analysis, and other humanities courses. One could even argue that this is the case for some courses in math and sciences. Though I am an advocate of MOOCs, since they provide an excellent learning experience in numerous circumstances, the model which relies on the premise of massive, is not an effective one for every learner in every learning situation.

Why MOOCs Won’t Work for College Students
I am not suggesting that college students can never learn in a MOOC setting, but I do suggest that certain courses not only require a low student-to-instructor ratio, and that colleges need to provide students with the skills in how-to-learn in a networked environment where learning is pulled and not pushed. I’ll use my 18-year old daughter, a high school senior here as an example. She will be a college freshman this September, and though she is a good student, with several Advanced Placement courses under her belt, she in no way would be able to learn successfully in a MOOC in her freshman year. Not that she can’t learn, or won’t be able to at some point, but she is a product of the public school system where students are told what to do, when to do it, how to do it. Furthermore, high school students are not prompted to think outside the box, to create a networked learning environment, or to be a self-directed learner.  My daughter is an expert test taker, she says she has figured out the system, can produce what the teachers are looking for, and thus can get good grades. She is similar to many high school students. How can we expect these students to be successful in a massive course with little guidance let alone feedback?

Four Reasons why Students Need Instructor Feedback and Peer Support
Below are four reasons that support the position that college students need instructor feedback in small, closed, online classes.  I should mention that I concede that small (under forty students) face-to-face classes can accomplish all but point #3, though this point is an essential component for students working within a digital world and can only be accomplished in an online class.

1. Prompt feedback allows students to assess existing knowledge, reflect on what they have learned, what they still need to learn, and receive suggestions for improvement [Chickering and Gamson, 1997].  The principles in the classic article Seven Principles for Good Practice in Undergraduate Education, still apply today; the tools and methods may change but the principles should not.
2. Students want feedback from instructors. Research suggests that students want feedback, and personalized feedback that includes suggestions for improvement, with explanations (Getzlaf et al, 2009).  In an online course, detailed and specific feedback on assignments or class contributions through discussion forums, live chats etc., is even more critical given the medium.
3. Students require guidance in how-to-learn in an open and online environment; instructor feedback and peer interaction in closed online courses can develop digital communication skills, including how to seek information, create personal learning environments and collaborate virtually with peers.
4. Over 60% of newly admitted college students in the US are not ready for college level work. They lack the basic writing and math skills required for the college courses they are enrolled in (SREB, 2010).  For this reason alone, college students need instructor guidance and support in completing lower level college classes, and in some cases remedial classes. Instructors can also guide students to find open resources, and provide support and encouragement to students who might be discouraged or frustrated.

Concluding Thoughts
Bottom line – the MOOC model cannot provide the level of feedback and support needed for many college students.  Instructors skilled in online teaching for small classes can provide constructive feedback to students, create a learning community that promotes interaction, and most importantly, teaches students how-to-learn in an online environment. I’ll be watching closely what happens within higher education with regards to MOOCs, and hope that administrators consider carefully a strategy that supports student development in the first and second year of college. In my next post, I’ll provide practical how-to instruction on tools and methods for giving meaningful feedback to students in an online class.

Resources:

• Effective Instructor Feedback: Perceptions of Online Graduate Students, (July, 2009), Getzlaf et al,  The Journal of Educators Online, V 6, no 2
• Collaborative Learning in Closed Online Classes vs. MOOCs, Online Learning Insights
• Meaningful Feedback in the Online Learning Environment (2012), by jalinskens67
• Student Interactions in Online Discussion Forum: Empirical Research from ‘Media Richness Theory’ Perspective, (2010), Balaji & Chakrabarti, Journal of Interactive Online Learning, v 9, issue 10
• Beyond the Rhetoric: Improving College Readiness Through Coherent State Policy, (2010), Special Report by The National Center Public Policy and Higher Education and The Southern Regional Education Board

Groups, Clay Shirky and Online Education

This post explores the significance of student groups in online learning courses—the value and influence on institutions in light of the principles outlined in Shirky’s book “Here Comes Everybody”.

Image: penguingroup.com

I just finished reading Clay Shirky’s book Here Comes Everybody, The Power of Organizing Without the Organization. You may wonder how valuable such a read would be given the book was written in 2008, yet reading the book with five years of Internet advancements under our belt, was strangely thought-provoking.  More so when considering the influence and power that groups can, and have wielded in the realm of online education, specifically in courses attracting massive numbers. Here I’ll share the potential that groups hold for learning within online courses, the three principles needed for successful groups, and how student groups are subtly influencing the paradigm shift in education.

Clay Shirky: Author
Shirky is a professor, journalist, author; he studies and writes about the effects of the Internet on society from a cultural and economic viewpoint. Several recent articles written by Shirky have caused some educators to bristle. In a blog post from 2012, Napster, Udacity, and the Academy Shirky compares higher education to the music industry, suggesting that the MOOC model, or some version of it, will replace higher education as we know it. Yet my focus here is not on Shirkys’ views on education, but his insights on group formation which is particularly relevant in light of learning online in massive courses.

Three Principles of Effective Groups
Shirky discusses how society is transforming, with citizens newfound ability to form en masse with the advent of the Internet and the low barriers to group formation. Now, in 2013 there are even fewer barriers to group formation than at the time of Clay’s writing. Now groups can come together with a choice of platforms. Though all groups develop for different reasons, there are elements common to effective groups which Shirky outlines in his book (p 261):

1. The Promise is what brings the group together, around a topic of interest with a basic desire to participate. Implied in the promise is that each member will participate and contribute.
2. The Tool is the platform that will help people approach the problem/topic together. Usually a leader [organizer] emerges within a group, or soon-to-be group and chooses the platform for group communication and collaboration i.e. Facebook, Google + Community, etc. He or she will invite members, and/or approve members joining, etc.
3. The Bargain is the guidelines or rules for participating. Group norms are established, i.e. what is acceptable for communication and not, contributions, etc. The organizer essentially established the tone, which influences the culture of the group. I have seen examples of this in virtually every large online group I have participated in; the organizer dictates through actions and tone, which influences the effectiveness of group collaboration and even participation.

Groups that Collaborate
To collaborate effectively groups need the three components as mentioned. Shirky uses Wikipedia as an example of a large group that demonstrates the three principles. With its tightly knit core group and a commitment to a promise, to create a database of free content maintained by a massive community of contributors, Wikipedia has control built-in by its group norms [the bargain]. An example of the bargain in action, is when one or more of the community of contributors overrides an article written by someone that tries to sabotage its integrity. Enough members care about Wikipedia, which is why it continues to thrive (p 140).

Groups, Learning and MOOCs
Here is where things get interesting. In small, online closed courses, group collaboration is under the control of the instructor—groups actions are structured, guided, even graded as part of a student’s final grade. Group work in this instance can be effective, as instructors can teach students how to participate and collaborate effectively.

But in MOOCs learning through group interactions is not within the instructors control, yet there is even greater potential for rich learning to happen with its large, diverse body of learners. Though MOOC organizers can guide participants and be catalysts for group formation by suggesting participants share, connect and collaborate outside of the MOOC platform. [It's also helpful that instructors suggest tools to facilitate group work. Though this may seem obvious, the instructor cannot assume that everyone is familiar social tools as a vehicle for learning]. Still, learning within a group in this context is dependent upon the self-direction and motivation of its members.

The Power and Influence of Groups on the Institution
Yet for institutions that offer massive courses, there is a risk. When working with massive numbers of students, not only is group collaboration and learning not guaranteed, there is potential for groups to influence actions and decisions of instructors and institutions in ways that may not be expected, or even desired. Groups have the potential to sabotage the learning of others and the course over which the instructor has little control. Already we have seen the influence of groups within Coursera courses. For example with the Microeconomics course where groups of students were challenging the professor which prompted the professor to quit the course before it was over. Or the Fantasy-SciFi that was sabotaged by a group of students participating in the discussion forums that were working under anonymous profiles causing numerous to be vocal about their negative experience. And the Foundations of Online Education with the thousands of student complaints about the  structure and technology glitches, forced it to shut down.

Conclusion
We are just beginning to see the power of groups in online learning courses with massive numbers of students. There is great potential yet to be realized, for the development of new knowledge and problem solving with the collaborative efforts of students worldwide. The power of groups cannot by ignored, the influence they have is great, and the institutions that embrace it and acknowledge that they are no longer in control, will be better prepared to create conditions to harness its potential.

Resources

• A Group Is Its Own Worst Enemy, (2003), Clay Shirky
• Clay Shirky: Institutions vs. collaboration, (2005), Ted Talk
• Here Comes Everybody, Summary from Wikipedia

Professors, Pedagogy and MOOCs

In this post I’ve collected and commented on recent reports outlining several professors perspectives on the development of MOOCs, the teaching experience and how the instructional methods differ from traditional courses.

What is it like to teach 10,000 or more students at once, and does it really work? (Kolowich, 2013). Excellent question—and now that Massive Open Online Courses [MOOCs] offered through providers such as Coursera have been around for a several months, we are starting to get data and insights from the professors in the trenches—those that have developed and facilitated the courses with thousands of students. The Chronicle of Higher Education shared results of a survey it conducted in February with professors from higher education institutions involved in teaching a MOOC. A total of 103 professors responded to an online questionnaire that was sent out to 184 professors that had recently taught, or were in the process of teaching a MOOC (Kolowich, 2013). Results are intriguing, and in some cases startling, for example the responses to a question on peer grading [see image below]:

Screen Shot from The Chronicle’s survey results on peer grading. http://www.chronicle.com/the-professors-behind-the-mooc

What I found fascinating was the change in professors’ attitudes towards MOOCs after they had finished teaching their own massive online course. Before teaching a MOOC, 29.7% respondents reported being ‘very enthusiastic’ about fully online courses, yet after the teaching experience, 55.9% reported being ‘very enthusiastic’. Hmm. Even the somewhat skeptical group changed their views after their MOOC experience. To read more results from the survey, click here.

Screen shot from “Voices from the Survey”, The Chronicle

Voices from the Survey
Included in The Chronicle’s coverage on the survey results, is a section that shares professors views on online learning, pedagogy and grading.  It is an interesting read. To the right is one of the professor’s comments, which addresses a topic that may be worthy of further research. From the last two  MOOCs I have completed with Coursera, I too have the impression that many active participants hold at least an undergraduate degree. Perhaps after identifying the demographic profile of the average student in a MOOC, course content and methods could be adapted accordingly.  Click here to browse through the feedback from The Chronicle’s report.

MOOC Development at Duke University

Duke University launched its first course with Coursera in June 2012, Bioelectricity: A Quantitative Approach, and wrote an extensive report upon its conclusion that summarizes the course development and instructional experience. The report is thorough, well prepared, and provides details about the experience from the developer and instructor’s perspective, including the number of hours invested developing and teaching the course (Dukespace.lib).

Included also is extensive data on enrollment rates, student participation patterns, demographics on active students, including their educational background. The data from this course supports what Professor Llewellyn stated in The Chronicle’s survey as mentioned above—that the active students (those students that participated in the survey at least), possessed an advanced level of education as the screen shot shows below right:

Screen shot from Duke University report on development of Bioelectricity: A Quantitative Approach offered through Coursera in June 2012

The course involved over 600 hours in development and delivery, and the professor of the course, Dr. Roger Barr, invested 420 hours of the 600. The report also speaks of Dr. Barr’s extensive involvement in the discussion forums, as he devoted many hours to answering questions, reading posts and monitoring feedback.

Final Thoughts
What is becoming apparent after reviewing the reports mentioned in this post, and in conversations with several of my peers through Google+, etc, the time involved in teaching a MOOC is extensive, requiring an instructor’s undivided attention. I’ve heard of one or more professors teaching a MOOC while assuming his or her full teaching responsibilities at the university, which put tremendous pressure on the individual. Also apparent is the volume of upfront work that is required—an extensive amount of time is devoted to preparing the content, materials, instructions for students, and the course home page. There is still much more to learn about MOOCs. It is too early at this point to determine how this format will be most effective for learners, and which instructional design methods work best, but we are seeing emergent patterns and results, which are taking us in the right direction.

Further Reading

• The Professors Who Make the MOOCs, (2013), Steve Kolowich, The Chronicle
• MOOC Completion Rates: The Data, (2013) Katy Jordan
• Who Owns a MOOC?, (2013), Ry Rivard, Inside Higher Ed
• Duke University’s First MOOC review, (2013) Bioelectricity: A Quantitative Approach
• The Professor’s Big Stage, (2013), Thomas L. Friedman, New York Times

Why and When Peer Grading is Effective for Open and Online Learning

Is peer grading an effective assessment method for open and online learning? What about in MOOCs where student feedback may be the only means of determining a pass or fail in a course? This posts examine peer grading and suggests what conditions must be present in order for peer grading to be effective.

After I wrote the outline for this post I came across this essay, by history professor Jonathon Rees, Why Peer Grading Can’t Work. The title was in stark contrast to my views on peer grading, but I  incorporated Rees’ argument here as it is worth consideration. Rees is also author of a blog I follow, More or Less Bunk where he writes about current issues within Higher Education often with a slice of sarcasm. Our views on online learning couldn’t be more dissimilar, yet I appreciate Professor Rees’ perspective and enjoy reading his posts. In his essay Rees shares his views on peer grading, which he experienced while taking a world history course as a student through Coursera. Rees’ central argument is that peer grading can’t and won’t be effective in grading written work produced within MOOCs, as the majority of students-as-graders are not able to provide quality feedback that can help students develop their writing and critical thinking skills.

In this post I’ll examine the conditions that need to be present for peer grading to work,   factors that can sabotage the process, and I’ll address the points put forth in Rees’ essay.  I’ll also explore briefly what is at the root of the differing views on peer grading, which I suggest is based on differing perspectives on learning philosophies [which I wrote about it in my last post, The Tale of Two MOOCs: Divided by Pedagogy].

Conflicting Views on How People Learn
At the root of the dissension over peer grading is the conflicting view on how people learn. One of Rees’ comments within the essay “Professors in the trenches tend to hold their monopoly on evaluating their students’ work dearly, since it helps them control the classroom better by reinforcing their power and expertise,” supports a cognitive and instructor-focused learning orientation. The concept of peer review, which leaves for the most part the instructor out of the equation, aligns with the social constructivist learning orientation. There is strong support in constructivist theories for the peer review which is grounded in student-centered learning where students learn as much from the review process itself as from the final grade on an assignment.

A paper on peer review published in 2007 described how the idea of peer review is embedded in the philosophies of learning theorists. The authors call out Vygotsky and his beliefs that learning occurs in, and is mediated by, social interaction. Authors do present the downsides of the peer review process, though at the conclusion of their research they determined that students involved in peer review perform better academically than peers graded only by their instructors (Lu & Bol, 2007).

Peer Grading @ Coursera
When developing a course there are numerous assessment methods instructors can choose from, yet the choice of the tool and method depends upon the learning conditions within the course that include—the planned learning outcomes and the learning environment. An analysis of these conditions determine the best assessment strategy [which may include several methods in one course] for the course and its objectives. Peer grading worked well in the Digital Cultures I completed recently with Coursera in consideration of the learning context—the environment, topic and goals of the course. Also the course was not-for-credit and only five weeks in length.

In Rees’ history course, peer grading was used to evaluate essays, which appeared to be the primary method for assessment. Given the topic of Rees course, essays could be considered an appropriate assessment mechanism given the number of students and the availability of the software available that can facilitate peer grading [see link at the end of this post on Calibrated Peer Review]. Rees admitted the guidelines were clearly outlined as to how to grade, and that the grades he received were accurate, yet it was the quality of comments that he felt was lacking,

For me at least, the primary problem with peer grading lay in the comments. While I received five comments on my first essay, for every subsequent essay I received number grades with no comments from a minimum of two peers and as many as four…Every time I did get a comment, no peer ever wrote more than three sentences. And why should they? Comments were anonymous so the hardest part of the evaluative obligation lacked adequate incentive and accountability. (Rees, 2013)

What can Go Wrong: The Loafers and Others
What Rees shares here is a good example of what can go wrong with peer grading in anonymity in a massive course. Though the algorithms within a peer grading system may work, the problem lies in the uncontrollable learning conditions inherent to a massive course. One example is outlined in Lu & Bol’s paper I quoted earlier, which is the phenomenon of social loafing or hide-in-the crowd behaviours associated with anonymity. Students that fell into this group were physically and cognitively lazy, not contributing to the process as required. This phenomenon was referenced in several other research studies within the paper. I suggest another group be added to the mix besides the loafers— students that cannot provide feedback due to the lack of necessary skills, whether it be education background or language.

When Peer Grading is Effective
Peer grading has tremendous value for a variety of learning situations in higher education, though it requires a specific set of learning conditions to be present in order for it to work as intended. Listed below are a list of conditions needed to ensure that peer grading  is effective: 

1) When learners are at a similar skill level.
2) When assignments are low stakes [i.e. when a course is taken for professional development of personal interest as was the Digital Cultures course].
3) Where credit is not granted.
4) When learners are mature, self-directed and motivated.
5) When learners have experience in learning and navigating within a networked setting [if the review is completed in an open and online setting].
6)  Learners have a developed set of communication skills.

The break down in peer grading occurs when the learning environment cannot provide the conditions as mentioned above. Also, there are other factors that can sabotage its effectiveness, including an assignment that requires a high level of critical thinking skills, or when there are students in the mix that are non-participative, or have intentions that don’t align with the course. In my Coursera experience for example, with Digital Cultures, one of the artifacts I was to evaluate in the peer grading process [which was a website] was a marketing pitch. This happened to at least one other student, according to the course Twitter stream.

Closing
Peer grading has great value. It has proven to be effective in variety of education settings. It can work well in MOOCs that are not for credit, when the assignment lends itself to a peer review, such as the digital artifact we graded in our Digital Cultures course. It can also be very effective in small, closed online classes where students are at similar skill level and receive instruction and guidance in how to grade within the process. Yet there are times when it won’t work, this is where I agree with Professor Rees, the situations where students do need detailed and constructive feedback from an instructor, or mentor that is qualified. Furthermore, there are many students that need remedial support in writing and communications skills, some require support in how to learn online, and how to be responsible for their own learning.

Further fine tuning is needed to address some of these issues within MOOCs. I see the opportunity for two or three tracks within a MOOC for students wanting to participate at varying levels, which would address some of the issues with peer grading by addressing some of the required learning conditions as mentioned. Another suggestion is to offer resources for skill development for students requiring help with their writing skills within the course itself, i.e. links to Purdue’s writing center.  Perhaps, addressing students that receive a grade below a certain level in the peer review after the course closes with suggestions for additional writing skill development would be helpful. There are many options still to be explored. Time will tell.

Update: Response post from Professor Rees on More or Less Bunk, here.

References:

• Peer Grading Can’t Work, (2013), Rees, J. Inside Higher Ed
• Pedagogical Foundations, Coursera
• A Comparison of Anonymous Versus Identifiable e-Peer Review on College Student Writing Performance and the Extent of Critical Feedback (2007), Lu & Bol, Journal of Interactive Online Learning
•  Calibrated Peer Review, The Regents of the University of California
• SWoRD™ Peer Review, Panther Learning