Four-Step Strategy to Create Active Learning in any Learning Space—Online, F2F or Blended

Embed from Getty Images

In a recent article The 3 Instructional Shifts That Will Redefine the College Professor, the ‘dynamic classroom’ was listed as number one. The dynamic classroom is where faculty “do practically anything other than lecture” (Craig, 2015).  A meager definition but the idea is right on—active learning, where students apply concepts through discussion, debate, writing, hands-on experiments, etc. produces better learning results. Numerous studies back up the claim; a recent paper shows student performance increased by just under half a standard deviation with active learning compared with lecturing (Freeman et al., 2014). In this post I share a four-step strategy that instructors can use to make learning active for online, blended or face-to-face learning spaces. Readers will find instructive examples and resources on active learning in the photo gallery (below) as well as in the list of resources at the end of this post.

Definition
Active learning is not a theory but a teaching method that supports learning. The method uses techniques such as writing reflections, discussion, problem solving—activities that promote analysis, synthesis and evaluation that guide students towards achieving learning objectives. Typically tools are used to support the activity, for example handouts, whiteboard, chalkboard, smart phone apps, platforms such as Google Drive or Twitter. The choice of activity and tool are (or should be) determined by the learning goal, as well as other factors that include, time available, location (in-class or online), class size and others that are specific to the students, such as their skill level and access to tools.

Another way to define active learning is to consider the opposite—passive learning where students are recipients of knowledge, are expected to record and absorb knowledge  delivered by an expert—a faculty member or textbook (McManus, 2001). Passive learning aligns with behaviorist theories where the student is viewed as an empty vessel waiting to be filled. In contrast active learning aligns with the constructivist perspective of learning. The constructivist view embraces the idea that knowledge is actively constructed by the learner and integrated with his or her existing knowledge and experience.

Active Learning Photo and Resource Gallery
Click on an image below to open up the photo gallery. Each image is captioned with a link to further information about the method featured.

Students work in small groups and use the white board in one of the school’s active classrooms (Queen’s University, Canada). Further details: http://queensu.ca/activelearningspaces/classrooms/ellis-319-flexibility

An example of student’s active learning assignment from Professor Laura Gibbs ‘Mythology and Folklore’ course at the University of Oklahoma. Assignment instructions posted here: http://onlinecourselady.pbworks.com/w/page/99190910/projectand link to the student’s Pinterst board: https://www.pinterest.com/ckzgirl/mythology-and-folklore-portfolio/

Eric Mazur, Harvard professor pioneered the idea of the active lecture using the method of ‘peer instruction’ described and demonstrated in this brief video: https://youtu.be/wont2v_LZ1E

Engineering students at Clemson University work in small groups problem solving in the MyEngineeringLab classroom. More details at http://www.masteringengineeringcommunity.com/training-tips-guidance/case-studies/case-study-using-technology-and-active-learning-to-get-engineering-students-off-to-a-good-start/

Screen shot of a Twitter Chat discussion. Twitter is used in both online and face-to-face classes as a tool to support active learning. A professor at UT Dallas documented an experiment using Twitter to engage students during lecture, described here: https://www.youtube.com/watch?v=6WPVWDkF7U8. Other ways to use Twitter, learn more http://www.onlinecolleges.net/50-ways-to-use-twitter-in-the-college-classroom/ (image from Dr Z. Reflects Blog http://drzreflects.blogspot.com/2014/03/r-u-following-education-twitter-chats.html

Nursing students work on concept board activity in small groups during a class on Mental Health at University of Wisconsin-Madison. More details at http://news.wisc.edu/23299 .

Four-Step Strategy for Making Learning Active
Below is a four-step framework for re-designing a lesson or unit to make learning active. Integrating active learning requires a skill set that goes beyond skills required for facilitating the traditional lecture format; numerous institutions provide professional development for faculty and instructors looking to incorporate active methods given the time and skills required. This framework below is a starting point; it features the bare bones principles of making learning active. An important note, the lecture method still has a role; lectures are an effective method to deliver information, yet it’s using lectures (and active learning) strategically as a method along with others that creates a comprehensive instructional strategy (“150 Teaching Methods”).

To illustrate the framework an active learning scenario featuring a group of nursing students in a face-to-face (F2F) course is used and described in the ‘application’ section of each step.

1. Identify learning objectives for the lesson/unit. Considering the learning objective or goal ensures the activity aligns with the course objectives and the lesson itself. If the unit/lesson doesn’t have a specific goal, but the course has overall learning outcomes, create one by considering the question—what should students be able to do or know after the lesson that will support them reaching a (given) course objective? A useful tool for identifying and writing learning objectives is a Bloom’s Taxonomy resource from UNCC ‘Writing Objectives Using Bloom’s Taxonomy’.

Application: Using the nursing students scenario the two goals for the learning unit on eating disorders are: 1) identify symptoms associated with disordered eating, and 2) determine a patient’s symptoms on the eating disorder continuum. These goals support one of the course objectives  • Analyze patient behaviors to determine presence of disordered eating on the eating disorder continuum, and helps students gain the knowledge and skills needed to meet this learning outcome.

2. Identify core concepts students need to learn. List key concepts— frameworks, formulas, theoretical principles etc. students need to learn. Determine the breadth and depth of knowledge required—the level at which students need to know the concepts, e.g. familiarity or mastery. Consider students’ current level of knowledge on the topic; this  helps determine how concepts will need to be presented to students before they engage in the activity.

Application: In the scenario nursing students need to recognize symptoms of disordered eating and determine which patient behaviors are normal and ones associated with eating disorders. Students need to know characteristics of eating disorders and be familiar with the ‘eating disorder continuum’.  Faculty determined that knowledge of the students is varied, thus assigned a textbook reading prior to class and selected a short video to show prior to the activity that illustrated key concepts.

3. Consider options—select activity and tools. This step has two phases. First determine types of activities that could work for the concepts given the learning context—class size, learning space, time constraints, etc. The activity might be a debate, mind map, or small discussion. Next identify the tools needed to support the activity, while also considering factors such as students skill level and access (to the tool). If skill level could be an issue, consider providing instructions, tutorials for students to learn the tool and build learning time in accordingly. If the learning curve is deemed too high given the time allocated, consider an alternative tool.

Nursing students work on concept boards in small groups in School of Nursing class at University of Wisconsin-Madison (2014). Picture by Jeff Miller.

Application:
In the nursing face-to-face class of 150 students a concept board activity was chosen. Students worked in small groups after watching video about a woman struggling with binge eating. The activity was introduced by professor with questions to guide the discussion. The groups collaborated using a white board to create their concept board.

This activity can be applied to online setting—student groups can create a concept board using Google Draw program on Google Drive. Students can work on the concept map asynchronously or synchronously.

4. Articulate activity instructions in detail. Instructions that highlight the goal of the activity and its purpose frame the activity and prepare students for learning. Instructions for F2F and online environments need to be specific, clear and detailed and should include three components, 1) goals of activity, its purpose and expected outcomes, 2) details of the assignment: requirements, due dates, concepts to incorporate, format for product, etc. and 3) execution details: group or individual assignment, how groups will work together, e.g. group etiquette, collaboration strategies,  tools and platforms to use (more so for online classes).

Closing Thoughts
Active learning is a proven method that supports learning. Yet it’s one of several that supports knowledge building and creation that engages and motivates students. Below is a list of resources for educators looking to incorporate active learning strategies into online, blended or F2F course.

Resources

• How Can You Incorporate Active Learning in the Classroom?, Chris O’Neal and Tershia Pinder-Grover, University of Michigan
• Strategies to Incorporate Active Learning into Online Teaching, Diane Austin
• Online Teaching Activity Index, Illinois Online Network
• Active Learning, Cornell University, Center for Teaching Excellence
• Active Learning Tutorials, University of Minnesota
• Active Learning Classrooms, University of Minnesota
• Best Practices for Teaching with Twitter, Jason Rhode, Ph.D.
• The Eight Minute Lecture Keeps Students Engaged, Illysa Izenberg, Faculty Focus
• Active Learning (list of articles and other resources for active learning), Michigan State University

References

• Craig, R. (2015, August 4). The 3 instructional shifts that will redefine the college professor. EdSurge. Retrieved from https://www.edsurge.com/news/2015-08-04-the-3-instructional-shifts-that-will-redefine-the-college-professor
• Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. (2014). Active learning increases student performance in science, engineering, and mathematics. PNAS, 111(23). doi:10.1073/pnas.1319030111
• Herr, N. (n.d.). Passive vs. Active Learning. Retrieved from http://www.csun.edu/science/ref/pedagogy/active-passive/active-passive-learning.html
• McManus, D. The two paradigms of education and the peer review of teaching, (2001). NAGT Journal of Geoscience Education, v 49 n 6., pp. 423-434.

Are you Ready to Teach Online? Readiness Surveys Aim to Help Faculty Prepare

“…the following questions will help you determine what you need to do to succeed at online learning. Post-survey feedback will also provide you with information on what you can expect from an online course”  — The University of North Carolina at Chapel Hill, Online Learning Readiness Questionnaire for Students

There are several excellent self-scoring assessments on the Web for students to assess their readiness for online learning as the one mentioned in the opening, yet few for instructors and faculty planning to teach online. The purpose of such questionnaires is to help students identify what skills they’ll need to be successful; technical and ‘soft skills’ such as self-direction, time management, etc. This post aims to help instructors prepare for teaching online by reviewing readiness tools and questionnaires that help to assess and determine what skills they will need to make the transition from teaching face-to-face to an online learning environment.

Shared below are two surveys and key findings from two papers on ‘readiness’ for online learning and teaching; both are examined briefly. One paper discusses student, instructor and institution readiness and argues that a successful online learning program must include a systematic process of planning, designing and creating environments where learning is actively fostered and supported (Mercado, 2008, pg. 18.2). In the same paper, Mercado stresses that teachers must also “possess personal attributes to perform online teaching and administration of the online environment successfully“. And most instructive are the tables within the paper with questions that focus on attitudes that aim to assess instructors’ perceptions towards online teaching (screen shot below). It’s the attitudes that are most important given that a student-centric mindset is required for successful online instructors, one that differs significantly from traditional, face-to-face instruction.

“Teaching in an online course involves more than replicating classroom strategies in a different form. It “requires a different approach—one that focuses less on the amount of time students spend together in a particular place, and more on facilitating a distance community and on activities designed for students working individually” (University of Washington, 2004).”

Screenshot above: Excerpt from survey that assesses teachers attitude towards online teaching as part of readiness. Designed to encourage teachers to reflect on their teaching style and strategies, circumstances, abilities, motivation and time management. (Mercado, 2008, pg.18.6)

Though there is little research that supports the thesis that readiness questionnaires lead to better learning outcomes for students, there is consensus that readiness assessments at the very least emphasize for students and instructors the expectations and demands of learning and teaching in an online space, encouraging the assessment-taker to take the necessary steps and actions to prepare in an effort to reduce the learning curves associated with technology and teaching approaches (Gascoigne & Parnell, 2014).

Categories and Barriers to Readiness
The assessments available on the web specific to instructor-readiness are few—two are listed below. Both categorize the required skills into three areas; 1) technology, 2) pedagogy and 3) administrative skills. Though as already mentioned, the paper by Mercado includes attitudes and perceptions of online learning as one component of a readiness which highlights the different mindset required for teaching online. It’s the mindset of a traditional education model that puts up the biggest barriers to instructors transitioning to teaching online. An extreme example of an instructor unable to adapt to a student-focused approach is from a MOOC on the Coursera platform where a faculty member quit midway through “because of disagreements over how to best conduct this course, I’ve agreed to disengage from it” (Kolowich, 2013).

I suggest that the required skills fall into three categories. Though the two surveys included below are good, they do not encompass all of the skills I suggest are required. In lieu of a survey that includes all three areas, I recommend consulting the “Teacher Attitudes” survey questions in Mercado’s paper in conjunction with one of the surveys below pages (18.5 to 18.7).

1.  Technology and Social Media Skills: Technology skills are fundamental, and though social media skills not an essential, they enhance the instructor’s ability to connect with students. Skills include: ♦ basic computer skills ♦  proficiency with software applications ♦ installing/updating software and plug-ins ♦ internet search literacy ♦ proficiency with features and functions within the LMS including uploading files, grading tools and grade book ♦ LMS tools for asynchronous/synchronous communication ♦ familiarity with platforms for communication/engagement outside of LMS, e.g. Pinterest, Twitter, Google+

2.  Administrative and Organization Skills: Includes skills such as time management e.g. ability and willingness to respond to student questions with immediacy e.g. within  24 hours ♦ provide constructive feedback on student assignments in timely manner ♦ proficiency with grade book and ability to submit grades by required ♦ monitor/follow-up with academic integrity issues

3. Pedagogical Skills and Teaching Approach: ♦ student focused learning model ♦ instructor focus on supporting and guiding learning not delivering content and instruction ♦ providing constructive feedback ♦ establishing and sustaining online presence

‘Readiness for Teaching Online’ Surveys

1. Faculty Self Assessment: Preparing for Online Teaching from Penn State University is free to use under the Creative Commons license. It’s excellent. One is required to input a name and email address to access the survey, though apparently it is not stored on the school’s server, and you to get a very detailed, comprehensive report emailed upon completion based upon your responses (check your spam folder; email is from PSU Online). CUNY published on its faculty website an example of a feedback report of the Penn State Self-Assessment—you can see the example by clicking here.

There are thirty questions ranked within three categories: technical, administrative and pedagogical competencies. The survey assumes the instructor has teaching experience—it includes questions about familiarity with LMS features and teaching online for instance, but it does highlight for instructors new to online teaching, the skills and expectations required.

Screenshot of Penn State’s Faculty Teaching Readiness Questionnaire https://weblearning.psu.edu/FacultySelfAssessment/#

2. Faculty Online Teaching Readiness Survey from the University of Toledo, twenty-question self-scoring survey. This survey is not nearly as comprehensive as Penn State’s, but it does provide a snapshot of skills required and provides in the feedback for each response, a detailed description of the skill with links to resources for further learning and/or information. Note, the selection buttons are misleading on this quiz, when completing it select the radio button that is above the answer you want.

Resources

• Kolowich, Steve (2013). Professor Leaves a MOOC in Mid-Course in Dispute Over Teaching. The Chronicle of Higher Education
• Gascoigne C. & Parnell, J (2014). Distance education readiness assessments: An overview and application. Online Journal of Distance Learning Administration, Vol. XVII, (IV)
• Mercado, Cecilia (2008), Readiness assessment tool for an e-learning environment implementation, Fifth International Conference on e-Learning for Knowledge-Based Society, Bangkok, Thailand (PDF)
• Faculty Self-Assessment: Preparing for Online Teaching, CUNY Hybrid Initiative
• Online Learning Readiness Questionnaire for Students, The University of North Chapel Hill
• Readiness Assessment for Online Learning for Students, Penn State University

Need-to-Know-News: Micro-Credentialing Movement in Higher Ed & Active Learning Trumps Lectures

This ‘Need-to-Know’ blog post series features noteworthy stories that speak of need-to-know developments within higher education and K-12 that have the potential to influence, challenge and/or transform traditional education as we know it.

1) The Micro-credential Movement in Higher Ed
The latest trend in higher education is micro-credentialing, the non-traditional education path where students gain skill sets in a specific area and receive a credential. Case in point, Udacity announced this week a new nanodegree (Udacity defines nanodegrees as ‘curriculums designed to help you become job-ready)’— the Android nanodegree in partnership with Google. Another example—Penn State’s College of Business also launched this week an online bootcamp course, ‘Supply Chain Leadership Academy’, to educate “supply chain leaders of tomorrow in leadership and best practices in holistic supply chain management”.

The micro-credentialing trend is driven by business entities that have a real (or perceived) workforce skill gap, where jobs can’t be filled due to lack of qualified applicants. Google reports it has thousands of jobs to fill given a dearth of qualified applicants. The Linux Foundation, also offering a certificate course in partnership with edX, reports it has over 50,000 open jobs.

MOOC providers and select higher education institutions are leveraging the apparent skills gap, using their platforms to build their online program offerings with credentialing options for a fee. A good idea. The target market is not traditional higher education students, but non-traditional students that are already in the workforce and are looking to further their careers and/or switch career paths. Alison.com is a platform offered credentialing in specific skill sets long before MOOC providers began doing so. Though Alison’s business model is different from MOOC providers such as Coursera or edX. Students aren’t the revenue source but advertisers, featured on the platform, are.

Sampling of micro-credential programs and associated fees:

• edX’s Linux System Administration Essentials course, “This Linux course is for those just starting their career in IT as well as professionals with experience on other operating systems who want to add Linux to their portfolio”. Fee: $399
• Stanford Online, Professional Certificates, “Our professional certificates offer short, focused courses that give you tools and techniques you can apply right away“. Fee: $1295 per online course; required number of courses vary by certificate.
• Udacity’s Nanodegree – “All the course content is free online, but the $200 per month pays for the non-scaleable parts of the degree: project grading, feedback, instructor mentorship, assistance and a final certification”. Option to receive reimbursement of 50% of tuition upon completion.
  
• Digital Literacy & IT Skills Diploma Courses, Alison.com. Free with option to pay nominal fee for paper certificate delivered via mail.
• Coursera’s Specializations – “Master a skill with a targeted sequence of courses”. Fee: $95 per course, with a fee for the ‘capstone project’, e.g. Business Foundations Specialization = $595 for four courses and capstone project.

Screenshot of recent email from Coursera announcing upcoming Specialization certificates. ‘Specializations’ consist of a two or more courses on focused area.

Insight: This non-traditional student population, which micro-credentials target, is an emerging market and such options are a boon to working or unemployed adults seeking skill development. It’s a positive development in higher education. Employers appear receptive to micro-credentials. However, micro-credentialing is favorable provided the programs provide quality learning resulting in tangible, applicable skill sets. The majority of the credentials require not only a financial investment, but a significant investment of students’ time and energy. It’s buyer-beware; credentials do not guarantee a job, though the courses backed by business entities likely have higher placement rates than those without a business affiliation.

•  Google, In search of more Android Developers, Creates Online ‘Nanodegree’ with Udacity, Forbes
• Google Partners with Udacity to Launch Android Development Nanodegree, TechCrunch
• Developing Strategic Supply Chain Leaders, Corpu
• Professional Education, edX
• The Employment Situation, May 2015, Bureau of Labor Statistics, U.S. Department of Labor

2) The Case for Active Learning over Lectures*
This is not new news, but worthy of review—evidence that performance of students engaging in classes that primarily offer active learning is improved over classes involving primarily lectures. A significant study on active learning was released last year; it provides compelling evidence on active learning benefits specific to STEM subjects in higher education (Freeman, et al., 2014). Researchers conducted a meta-analyses of 225 studies in published and unpublished literature that documented student performance in courses with at least some active learning versus traditional lecturing.  Though intuitively we might know that active learning is more effective for learning, there’s now solid evidence to back it up:

The data reported here indicate that active learning increases examination performance by just under half a SD and that lecturing increases failure rates by 55%. The heterogeneity analyses indicate that (i) these increases in achievement hold across all of the STEM disciplines and occur in all class sizes, course types, and course levels; and (ii) active learning is particularly beneficial in small classes and at increasing performance on concept inventories.

Implications: Is the lecture dead? Absolutely not, but to increase student learning, retention and success, involving students in active application of concepts should be the norm not the exception. However, implementing active learning is challenging for many educations, and especially for online courses, yet it can be done with deliberate, thoughtful development of a course learning strategy. Below are links with suggestions and examples of active learning applications. One of my favorite examples of active learning, is an online literature instructor Laura Gibbs, who creates assignments using online platforms—blogging platforms, Pinterest, etc. where students engage with content, each other and the Internet community.

• Anatomy of an Online Course, Laura Gibbs
• OU Digital Tools, blog by Laura Gibbs
• Teaching Tactics that Encourage Active Learning, The Critical Thinking Community
• A Manifesto for Active Learning, ProfHacker
• Show What You Know Using Web and Mobile Apps, Learning in Hand
• Online Teaching Activity Index, Illinois Online Network [list of ideas for learning activities]

References:

• Freeman, S., Eddy, S.L., McDonough, M., Smith, M.K., Okorafor, N., Jordt, H., and Wenderoth, M.P., (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences (PNAS), 111 (23), 8410-8415.
• Weiner, Maryelleb. (2015). More evidence that active learning trumps lecturing. Faculty Focus Higher Ed Teaching Strategies from Magna Publications.

Feature Image: by GotCredit on Flickr

How ‘Good’ is Your Online Course? Five Steps to Assess Course Quality

The view that online education is “just as good as” face-to-face instruction was not widely held in 2003: 42.8% of chief academic officers reported that they considered the learning outcomes for online instruction to be inferior to face-to-face instruction. The view of online quality has improved over time. However results for 2013 revealed a partial retreat in faculty perceptions of online learning providing quality learning experiences. The 2014 results indicate that the retreat continues—there’s an increase in faculty that perceive online education as inferior. — Grade Level: Tracking Online Education in the United States, 2015

One of the main criticisms of online courses is they are of poor quality as revealed in the annual Babson study mentioned in the opening. Positive perception of online learning by faculty has declined in 2013 and 2014 (Allen & Seaman, 2015). Face-to-face courses appear to be the hallmark for quality when it comes to higher education. Yet this doesn’t seem fitting considering the ongoing and often heated public dialogue about the quality of higher education programs with little consensus on what quality is. In this blog post I suggest that online educators can and should tackle the quality issue in their own courses, and that they do so by assessing their course holistically. A holistic approach encompasses elements such as students’ perspectives, results over a period of time, artifacts created during learning, and the instructor’s course experience.

I also review recent research on quality assessment specific to online courses. I also examine existing frameworks and rubrics for online course assessment and explain why, even if an institution follows such standards, these are starting points. I outline five-steps that instructors can follow to assess whether a course is ‘good’—an assessment for quality that considers foundational elements, student perspectives, course artifacts, student and instructor learning experiences.

What is Course Quality?
Up until a few years ago ‘quality’ in higher education was measured by a course’s content, pedagogy and learning outcomes (Bremer, 2012). This approach has changed to a process-oriented system where a combination of activities contributing to the education experience are considered. Activities that include: student needs, use of data and information for decision-making, department contributions, as well as improved learning outcomes (Thair, Garnett, & King, 2006). This holistic approach of evaluating education experiences is often applied to the development and assessment of online learning. For example, Online Learning Consortium’s Five Pillars of Quality Online Education (below) and Quality Matters (QM) rubric.

“Five Pillars of Quality Online Education, the building blocks which provide the support for successful online learning”.  http://www.onlinelearningconsortium.org

Why Assessing Quality is Difficult in Online Education
Yet there are challenges associated with setting universal quality standards for online education, and though a starting point, a thorough quality assessment requires ongoing consideration of numerous elements, some that occur over a period of time.  Key challenges with assessing quality through set standards are outlined in ‘What is online course quality‘ and include: 1) the lack of authoritative body (able and willing) to address minimum level of standards across all states with their accrediting bodies, 2) the challenge of creating a comprehensive, evaluative tool to address complexities of online courses, and 3) the implementation process itself given the significant resources that would be required to implement an institution-wide evaluation process (Thompson, n.d.).

Limitations of Quality Assessments
There are other limitations. Some assessments are inherently limiting with a prescriptive set of standards that may not fit all contexts.  Another is the tendency to establish a minimum level of quality, ‘baseline standards’ which limits innovation and creativity (Misut & Pribilova, 2015).  Most course assessments are done at a point-in-time and are unable to capture dimensions over the life of a course and post-course; dimensions that include student perceptions collected as formative feedback (mid-way through course) and end-of-course feedback surveys. Furthermore, quality assessments frequently focus on course/instructional design and fail to include learning experiences of the instructor and students.

What’s involved In a Good Course Assessment?
A holistic assessment goes beyond course design; it acknowledges the nuances that make a course unique, including input and contributions from students, developments in the field of study, and current events. Most valuable are students perceptions of their learning and of the course experience. A good course assessment considers the course over a period of time, and considers interactions between instructor and students, students and students, all of which create artifacts that can be studied and analyzed (Thompson, 2005).  Artifacts might include, emails or forum posts of student questions,  dialogue within forums, feedback from group interaction, end-of-course student surveys, LMS reports on student interaction patterns, student assignment results, and more.  Course artifacts give valuable clues to a course’s quality, more so when collected from two or more course iterations and analyzed collectively.

Figure from paper describing the Online Course Criticism model based on the concept of educational criticism which suggests a holistic review of a course to assess quality (Thompson, 2005)

Other elements to consider:

♦  Student behaviours including questions asked in forums, emails, interactive patterns within LMS, interaction with resources, participation patterns within discussion forums,  social platforms designated to course, etc. ♦  Student perceptions evaluated through questionnaires, formative course feedback, post-course questionnaires, one-on-one interactions  ♦  Knowledge creation/transfer by students evaluated through assignment analysis, course artifacts, post-course surveys  ♦  Course design as per rubric/assessment tool  ♦  Use of current technology tools and platforms  ♦  Course data and artifacts from two or more sessions analyzed and compared  ♦  Quantity and type of interaction between students and instructor

Five-Steps to Assessing Online Course Quality

1) Asses Using a Rubric or Other Tool to Consider Basic Course Elements
Assess course using the tool or framework employed by your institution e.g. Quality Matters rubric. If your institution does not have a tool in place I recommend the rubric created by California State University Chico which covers six domains. The rubric (embedded below) is free to use and download under the Creative Commons Attribution 3.0 United States License.

* Thanks to a reader’s comment – there is an updated version of the Chico rubric which is a checklist format with additional dimensions. It is similar to the Quality Matters rubric. I prefer the version embedded here — its more approachable given it’s less lengthy and rigid. Link here to the updated version.

2) Analyze Course from a Student Perspective
This is perhaps the most difficult yet useful element for improving course quality. There are a variety of ways to consider students’ perspectives, several already mentioned. Other recommendations—take an online course as a student (e.g. a MOOC) in a topic you aren’t familiar with. This provides an eye-opening view of how it feels to be an online student. Another method is to ask a colleague from another department to review your course and provide constructive feedback.

3) Assess Course Artifacts, Materials, & Feedback
Another useful exercise is analyzing course artifacts. Analyzing results from student feedback via a questionnaire midway through course is helpful. If a course is offered more than once, compare data from course iterations collectively.  Consider, is student feedback incorporated into subsequent course re-runs? What about student-generated content? All artifacts and materials associated with a course are valuable material for assessing a course’s quality.

4) Consider Level and Type of Student-to-Student and Student-to-Instructor Interactions
Interaction is critical to an online course; students that feel connected, establish themselves as individuals within an online course are likely to have higher levels of motivation and learning satisfaction over those that don’t. Consider the forums, the interactive assignments where students can participate, the social exchanges within course-associated platforms, and other places for interaction. An example of assignments that encourage student feedback and involvement, leading to high levels of engagement can be found on this online instructor’s (Laura Gibbs) course site here. Also consider the Community of Inquiry model for the types of interactions in an online course that lead to positive learning experiences.

5)  Results: Are Students Learning?
Evidence of learning  is the most important assessment dimension, yet nearly impossible for a standardized quality assessment tool to evaluate.  One could argue that before and after quizzes within a course can evaluate learning. I suggest that the instructor is able to assess at a deeper level whether or not learning occurred, can determine the level of critical thinking. This can be done only when assignments demand that students demonstrate what they know and are required to apply course concepts.  Assignments that draw out students thinking by demonstration either through dialogue or written work allow the instructor evaluate learning effectively. There’s no formula for this fifth step, this is an example of customized course evaluation. But I suggest instructors evaluate student artifacts from one course to another and to consider what students learned and how well  they articulated what they learned. There may be opportunity for revising assignments, activities or other course dimensions.

Conclusion
Assessing quality in online courses is complex as we’ve seen here, yet addressing quality is critical to advance the positive perception of online education for one, but more importantly to provide learning and teaching experiences that are rewarding, rich and meaningful. Quality assessment can start one course at a time, and who better to do this than the course instructor?

References/Resources*

• Allen, E., & Seaman, J. (2015). Grade Level: Tracking Online Education in the United States (Rep.). Babson Survey Research Group. Retrieved from http://www.onlinelearningsurvey.com/reports/gradelevel.pdf
  
• Bremer, C. (2012). Enhancing e-learning quality through the application of the AKUE procedure model. Journal of Computer Assisted Learning, 28 (1) (2012), pp. 15–26 http://dx.doi.org/10.1111/j.1365-2729.2011.00444.x
• California State University Chico (2009). Rubric for Online Instruction. Retrieved from http://www.csuchico.edu%2Feoi%2Fdocuments%2Frubricpdf  *
• Gibbs Laura (2015). Project Comments: Week 14 and 15. Retrieved from onlinecourselady.pbworks.com/w/page/95336159/storybookcomments14 *
• Lederman, D., & Fritschler, A. L. (2010, September 9). Setting Quality Standards in Higher Ed | InsideHigherEd. Retrieved from https://www.insidehighered.com/views/2010/09/09/fritschler
• Misut, M., & Pribilova, K. (2015). Measuring quality in context of e-learning. Procedia – Social and Behavioral Sciences, 177, 312-319. doi:10.1016/j.sbspro.2015.02.347. Retrieved from http://www.sciencedirect.com/science/article/pii/S1877042815017012
• Thompson, K. (n.d.) What is online course quality? Retrieved from  http://ofcoursesonline.com/wp-content/uploads/2008/06/thompson_onlinecoursequality.pdf
• Thompson, K. (2005). Constructing educational criticism of online courses: A model for implementation by practitioners. Unpublished doctoral dissertation. University of Central Florida: Orlando, FL. Retrieved http://etd.fcla.edu/CF/CFE0000657/thompson_kelvin_200508_EdD.pdf
• University of West Georgia. (n.d.) Rubric for Online Instruction. Retrieved from http://www.westga.edu/~distance/webct1/Rubric/Rubric.html *
• University of Central Florida, & American Association of State Colleges and Universities. (n.d.). BlendKit Course: BlendKit Reader: Chapter 5. Retrieved from http://blended.online.ucf.edu/blendkit-course-blendkit-reader-chapter-5/ *

Does it Take More or Less Time to Facilitate and Develop an Online Course? Finally, Some Answers

How much time does it take to teach an online course? Does teaching online take more or less time than teaching face-to-face? How much time does it take instructors to develop an online course? — Instructor Time Requirements to Develop and Teach Online Courses (Freeman, 2015)

A study released in March of this year set out to answer these burning questions that the majority of online educators would like answers to. There’s considerable anecdotal evidence that favors both sides—it takes more time versus less time to facilitate an online course when using a face-to-face course as benchmark. The purpose of this study was to nail down the facts—to measure the perceptions of and actual time spent developing and teaching online courses. The findings are significant for institutions and educators involved in online education for several reasons. Professional development for one. The report reveals areas where survey-respondents struggled during the course development phase, and where the majority of time was spent when facilitating (the conclusions are surprising). Secondly, results may be helpful for institutions when considering compensation and work-allocation models. Institutions can use the results as benchmark, at the very least the study may act as catalyst for constructive conversations about compensation and support for online course development and facilitation. And finally, it may help online instructors gain insight into their own teaching experiences by considering the experiences of  other educators that have experience with face-to-face and online courses.

This post highlights the findings and suggests factors for educators to consider when it comes to, 1) the time spent developing online versus face-to-face courses, and 2) how much time is invested in online facilitation, and how it compares to face-to-face instructions.

Survey Details
To put the results into context—the survey gathered data from 68 instructors from a total of 165 solicited from three universities across eight academic disciplines. Each respondent had developed an average of 2.13 online courses and had experience teaching an average of 2.03 online courses, and had been teaching at the university level for an average of 14.2 years (Freeman, 2015).

1) Course Development Time: Pedagogical Learning Curve Steepest
Survey results confirmed that developing online courses is indeed more time consuming than developing face-to-face courses. Though the time required declines when the same instructor develops a second or third online course. Twenty-nine percent of respondents indicated they spend over 100 hours (median of 70 hours) to develop their (first) online course. This significant number of hours is likely due to the fact that 59% of respondents developed over 90% of the course without any assistance, which included developing content, assessments, assignments, and time associated with course design. The other 41% received course design support from instructional designer(s) and/or used ready-made content available through textbook publishers. Also significant is the technological learning curve which was found to be shorter than the pedagogical learning curve, in other words instructors required more time to determine how to implement pedagogical methods, how to create learning experiences and deliver content appropriate for the online format than they did learning about the features and nuances of the technology used to deliver the course. The learning curve is described as the time it takes to “get used to” the course elements [platform, tech features] and/or the method of teaching.

(Freeman, 2015)

Implications:
Developing a quality online course is complex due to the fact that technology adds yet another layer to course design and one that requires a unique skill set. In addition there is an interdependent relationship between technology and pedagogy specific to online courses—for instance the features of a LMS platform will determine and shape the course and the teaching methods. Using the discussion forum as an example—the flexibility of the forum feature—how easy it is to set up by the course designer for group assignments, and how it can be used by students for a group assignment whether it can facilitate the communication and collaboration that is required for the assignment will dictate how effectively the ‘method’ is executed in the course.

Online course design requires a breadth of skills that includes technical knowledge, not only familiarity with LMS features, but also outside tools including social media platforms that can enhance student learning.  Knowledge of user-focused design, or web design principles is also critical in delivering an intuitive, learning experience for students (How Five Web Design Principles Can Boost Student Learning). Second are the pedagogical methods, in other words how learning is sequenced, framed and presented to students.  This array of skills required is far beyond the scope of most faculty, who are experts in their field of study, not necessarily course design. Realistically creating an online course requires at least two or more individuals with specific skills sets working together to develop an engaging, intuitive and quality learning experience.

The onus is on institutions to provide not only professional development for faculty in course design principles and strategies, but to provide support in the technical and pedagogical aspects.

2) BIG Time Commitment Facilitating First Online Course — Levels Off After 2nd Time, But Grading Involves More Time Investment
Though respondents in the survey originally perceived that teaching online took more time than teaching face-to-face, by the third time facilitating respondents reported that it took them about the same amount of time as it did a similar face-to-face course.

There is supporting evidence to the earlier finding that teaching an online course the second and third time becomes about as time-consuming as teaching a face-to-face course the second and third time.  The factors that still remain more time-consuming for online teaching compared with face-to-face teaching, even after teaching the course three times, are Instructor-Student Interaction and Grading & Assessment, the two specific factors  that can not be prepared in advance for online courses (unlike Content Development and Pre-Semester Setup).

Implications:
Sixty-nine percent of survey respondents indicated that it took ‘much more’ and ‘more’ time to facilitate an online class for the first time. Yet by the third time, it dropped to 25% in this same categories (table 4 below), which does support the learning curve theory. These findings suggest that acknowledging that more of the instructor’s time will be required the first and even the second time facilitating a course, is important for both the instructor and the institution. Though it does also suggest that professional development is needed for instructors—development focused on facilitation skills that will support skills specific to the uniqueness of online instruction. Such training can potentially reduce the learning curve for instructors, as well as reinforce the building of effective skills, best practices, and efficient use of time.

Annotated screenshot that shows two-thirds of  respondents by the third time facilitating online indicate that it took ‘somewhat more’, ‘more’ or ‘much more’ time to grade and assess students in an online course than face-to-face (Freeman, 2015).

A startling (and significant) finding of this study is the time dedicated to grading and assessing online students. It appears that the time dedicated to grading students’ work actually increased from the first to third time of facilitating an online course (table 4). Two-thirds of the respondents indicated by the third time it took ‘somewhat more’, ‘more’ or ‘much more’ time to grade and assess students in an online course than face-to-face.  I find these results encouraging since an instructor’s feedback of students’ work is a critical component that can motivate students, deepen their knowledge and push them to think critically (Getzlaf et al., 2009). Implications are that skill development in this area are needed and will benefit not only students but can help instructors to provide feedback more efficiently. There are several technology tools and applications that can help instructors achieve efficiency and to make the most of giving feedback using online tools that deliver meaningful, quality feedback for students (Morrison, 2014). Again professional development is needed in the area of grading and assessment to support instructors in their efforts.

Conclusion
By no means is this study the definitive answer on the time requirements for developing and facilitating online courses, but it is an excellent starting point for conversations about ‘time’ needed to create quality online learning experiences.

References

• Freeman, L. A. (2015). Instructor time requirements to develop and teach online courses. Online Journal of Distance Learning Administration, 18(1). Retrieved from http://www.westga.edu/~distance/ojdla/spring181/freeman181.html
• Getzlaf, B., Perry, B., Toffner, G., Lamarche, K., & Edwards, M. (2009). Effective instructor feedback: Perceptions of online graduate students. The Journal of Educators Online, 6(2). Retrieved from http://www.thejeo.com/Archives/Volume6Number2/GetzlafetalPaper.pdf
  
• Morrison, D. A. (2014, February 28). Best methods and tools for online educators to give Students helpful and meaningful feedback [Web log post]. Retrieved from https://onlinelearninginsights.wordpress.com/2014/02/28/best-methods-and-tools-for-online-educators-to-give-students-helpful-and-meaningful-feedback/

How Five Web Design Principles Boost Student Learning in an Online Course

“Our team realized quickly that we needed to do a better job cross referencing material on our course site. For example if we mention syllabus, we must link to it. Some students we have learned want a great deal of guidance” MOOC instructor, Karen Head (2013)

In the quote above, without realizing it, the instructor was referring to the concept of ‘user experience’. And it’s not guidance students wanted so much as an intuitive learning experience. Creating a user-friendly course site begins with incorporating web design principles. Even the most basic of principles customized to online course design reduces barriers associated with virtual learning by minimizing distractions, highlighting concepts and making resources readily accessible. Embedding a link into the phrase ‘assignment guidelines’ for instance, when the assignment is referred to within a course page, is an example of making resources readily available (if the assignment guidelines are within the syllabus, refer students to the page number). This reduces the amount of time students spend searching and frees up time for learning.

The challenge of designing online courses is not only pedagogical, but also technical, which is the category that ‘usability’ falls under. We are at the point with online learning where pedagogy and technology are interdependent; where a well-designed, user-friendly course with a clear learning path needs to adhere to technical principles as well as pedagogical ones. Technology is a new form of pedagogy. The course site design, how content is presented, is an aspect of online pedagogy. In this post I cover five principles of web design that are essential to online course design.

Retail sites frequently adhere to best practices for web design given customers (users) are more likely to spend time and money on an attractive, intuitive website. I suggest educators use similar web design principles to support their students.

Before we examine the principles, defining user experience (UX) is in order. There are numerous definitions of user experience but the one below specific to web design, incorporates key elements of the entire experience:

“User experience (UX) is about how a person feels about using a system. User experience highlights the experiential, affective, meaningful and valuable aspects of human-computer interaction (HCI) …. It also covers a person’s perceptions of the practical aspects such as utility, ease of use and efficiency of the system. User experience is subjective in nature, because it is about an individual’s performance, feelings and thoughts about the system. User experience is dynamic, because it changes over time as the circumstances change…” All About UX

Five Principles of Web Design Applicable to Online Course Design

1. Design for the user
This seems obvious—design a course from the student’s perspective, yet it’s an atypical approach. When designing a web page for a course site, always ask ‘how will this look to the student’? Anyone involved in online course design needs to take an online course as a student. Completing at least one week of course work in a MOOC for example, gives one an entirely different perspective on course design—guaranteed. Design the course from the student’s perspective—always.

2. Consistency
For online course, consistency is probably the most under-utilized principle. Specifically in terms of how resources are titled, labeled and/or placed within the course site. I’ve taken many courses where the same resource, an article for instance, is referred to by two different names—in the syllabus it’s titled one way, and in the course site another. Confusing. Same goes for assignments, calling an assignment by slightly different names, even by one word suggests there are two assignments, not one. Another, posting the same document in two different locations within the site suggests there are two different documents. The time students spend searching, checking, comparing etc. is valuable learning time that is spent on logistics. Consistency is key.

3. White Space
Effective use of white space emphasizes key concepts, improves comprehension (up to 20%) and reduces cognitive overload (Lin, 2004). White space is the part of a web page that is left blank or unmarked. It’s the (white) space between columns, text, images, and margins on the page. This space provides visual relief to the reader and improves readability. Avoid using big blocks of text. Break it up with a graphic, or block of white space or increased line space. See examples below.

Example of text with little white space.

Same text as above but with increased line spacing. Effective use of white space improves readability.

4. Simplicity
In keeping with the idea of white space is simplicity. A cluttered page with three or more colors of font, sizes of font and images placed sporadically throughout that are of different type and size creates a chaotic-looking virtual classroom. It’s far easier to study and focus on learning in a physical classroom that is organized with minimal distractions. The same goes for an online classroom. Keep it simple, two colors of font, same size and style throughout, organized and consistent pages creates a Zen-like classroom where students can focus on course content and application of concepts. Learning is enhanced greatly.

“The way information is organized and presented to students affects not only the usability of information, but the usability of the course itself” (Young, 2014)

Tabs grouped by category (Coursera)

5. Use Tabs Effectively
Imagine opening a file drawer that is full of file folders with inaccurate or missing labels. The same principles of file labels apply to web sites except rather than alphabetized listing, it’s an order that makes sense to the student. For example the ‘start here’ tab should be at the top of the menu not third or fourth down the list (which happens more often than you would think). Tabs should be two words (max 3 words), and with descriptive language, ‘Start Here’, ‘Week One, or ‘Student Support’ for example.  Use sub-tabs if possible, and if not group tabs into categories (screenshot right). Also avoid CAP LETTERS for titles of tabs. CAPITAL LETTERS can appear loud and abrasive on a website (there are exceptions as the screenshot above right demonstrates).

Conclusion
Developing an online course is a multidimensional process. Usability is one dimension often neglected; understandably so given that most educators approach online course design with little expertise in web design. Yet a little goes a long way—by implementing just the basic web design principles, educators can create an intuitive learning path that gives students the boost they need to invest more time in learning, not searching.

References/Resources

• Head, K. (2013, September 6). Lessons learned from a freshman composition MOOC. The Chronicle of Higher Education. Retrieved from http://chronicle.com/blogs/wiredcampus/author/khead
• Introduction to web accessibility. (n.d.). Retrieved from http://webaim.org/articles/visual/colorblind
• Lin, D. Y. M. (2004). “Evaluating older adults’ retention in hypertext perusal: impacts of presentation media as a function of text topology.” Computers in Human Behavior, 20.
• Misfud, J. (2011, October 17). 14 Guidelines For Web Site Tabs Usability – Usability Geek. Retrieved from http://usabilitygeek.com/14-guidelines-for-web-site-tabs-usability/
• Morris, I. (2015, February 16). White Space – A perfect Option For Improving Usability In Web Designs – Usability Geek. Retrieved from http://usabilitygeek.com/white-spaces-improving-usability-web-designs/
• Young, H. N. (2014). Putting the U in MOOCs: The importance of usability in course design. In The invasion of the MOOCs: The promise and perils of Massive Open Online Courses. Retrieved from http://www.parlorpress.com/invasion_of_the_moocs