Ha•bit: noun: a usual way of behaving : something that a person does often in a regular and repeated way — Merriam-Webster
“Hooked” is about how to build habit-forming products…habit-forming digital products that is. I included “Hooked” on my must-read list to see if any of the principles discussed might apply to education—Learning Management Systems (LMSs) for instance or other ed-tech applications. Given our culture’s fixation with mobile devices, surely there might be some lessons to make digital education applications more compelling. Can educators create platforms or applications that ‘hook’ students into learning, where learning behaviors become a habit? If so, how? Hooked provides some answers—see below.
‘The Hook’ Model
Author Nir Eyal, entrepreneur and product designer describes the book’s topic as “behavioral design”. Behavioral design when applied to product development incorporates concepts of user experience, behavioral economics and neuroscience. Nir describes it as the intersection of psychology, technology, and business. His recipe for creating habit-forming products begins with ‘The Hook’ model.
A trigger, either internal or external, leads to a behavior—it’s the spark that prompts action. Habit-forming products start by alerting users with external triggers such as an email, a website link, or an app icon on a smart phone, with the aim of prompting repeat engagement until a habit is formed. The trigger also tells the user of what to do next—to act.
The action needs to be seamless; the user should be able to act with ease —without barriers. Action according to Nir, relies not only on ease of use but on principles of human behavior. It’s based on the premise that the user seeks one of three things: 1) pleasure in order to avoid pain, 2) hope to avoid fear, or 3) social acceptance to avoid rejection. Jack Dorsey founder of Twitter builds on this premise with his platform which is designed to solve a problem (communication, knowledge building), while addressing desires and emotions of its users (social acceptance via ‘likes’, retweets, etc.) (pg 39).
What distinguishes the Hook model from the traditional feedback loop (embodied by the familiar model B.F. Skinner’s model where rewards are used to support behavior change through positive reinforcement) is the variability of the reward which creates a desire for feedback, motivating the user to seek it out. Traditional feedback loops are predictable; they don’t create desire according to Nir. Yet when there’s uncertainty to the reward or variability to the type of reward—the user’s interest is piqued Think of the variability of the reward structure with slot machines; they’re unpredictable. In an education context, Nir describes how Codeacademy uses variable rewards with symbols that benchmark students progress along with variable feedback that uses rewards to fulfill the student’s desire for acceptance and validation (pg. 89).
The investment occurs when the user puts something into the product or service such as time, data, effort, social capital or money (pg. 7). The more users invest in the product or service, the more they value it—supporting the idea that labor leads to love. This investment concept is applicable to education—online courses for example where students contribute to course content (investment of time), complete course work (more time) and engage with peers (even more time).
Case Study: The Hook Model in Action
After reading the case study of the Bible app, YouVersion in “Hooked” I could see the application of the Hook model, its relevance to learning contexts. The app provides a selection of bible study programs users can choose from based upon their needs. The app sends reminders and encouraging messages when readings or homework is due. When a message is avoided or missed, a red icon appears over the app, another cue. If more than two readings are missed, users receive a supportive message suggesting they consider a different (less challenging) plan. There’s also a virtual community, where encouragement from its members is another source of ‘triggers’. Rewards come in several forms. When a reading assignment is done for instance, the user gets a message “Day Complete” with a check mark on the app’s calendar. YouVersion is a success story. It’s the #1 downloaded Bible app with over 200,000,000 downloads.
How the Hook Model Can be Applied to Learning
What if learning did become a habit, where students check into their online class daily, share relevant content with classmates or engage in group assignments willingly? The case study of Youversion is instructive, suggesting that the model concepts are applicable to learning scenarios, specifically to learning platforms and applications. Learning applications created thoughtfully and purposefully can support behavior changes that result in seamless learning, with few barriers and built-in rewards that provide variety and freshness that also leverage the learning community. Yet creating learning that follows the Hook model requires a different mindset, and commitment to create compelling learning with integrity and care that protects students, content and the process of learning.
What will 2016 hold for education? There’s no shortage of articles and reports with predictions describing what to expect for the coming year. It’s tempting to be dismissive—scanning the headlines knowing that predictions are far from a sure bet. Yet for educators, considering trends across industries in conjunction with current developments in education is constructive, strategic and provides an edge; it gives insight, helps us prepare and be proactive. In this post I share my analysis of current trends and developments within higher ed and k-12 and outline what to expect in 2016.
There’s a spate of articles on the Web across all sectors: education, business, consumer and design, all describing what to watch for—micro-credentialing, wearable technology, mobile, augmented reality and a host of others. Yet how are these trends applicable or relevant to educators? I analyzed numerous sources, some specific to education and many not, to determine what will affect the education sector in 2016. I consulted New Media consortium’s collaborative Wiki for the 2016 NMC Horizon report, Mary Meeker’s annual Internet Trends report (2015), and Fast Company’s Future of Work Trend Report along with several articles and reports from this past year*.
I identified three themes: 1) Alternative credentialing, 2) Experimentation in new teaching modelsand learning spaces, and 3) Student-driven personalized learning. Two other themes are worth noting, Gamification and Augmented Reality. Yet I don’t see these as influencing education for the short or medium term given the challenges with implementation, and with augmented reality, the uncertainty of its effects on users’ health and cognitive state.
Alternative Credentialing and Pathways to Higher Education
I don’t buy the argument that MOOCs haven’t disrupted higher education. MOOCs have led to significant discussions about alternative learning pathways and institutions have responded with education programs that not only provide a variety of learning options, but embody alternative credentialing. Alternative credentialing can be described as alternate methods of assessment for learning (with the traditional degree as the metric), and recognition of that learning in credentials other than a degree. Badges was one of the first alternatives. Now we have programs by MOOC providers such as Udacity with their Nano-degrees, Coursera with its Specializations, and edX’s Professional Certificates. What’s new this past year is the increase of alternative programs offered by higher education institutions, such as Bootcamp programs, MOOCs for credit, and mini degrees as in MIT’s MicroMasters.
Drivers of Alternative Credentialing
Student demand: With increased Web-connectivity, students have access to learning platforms, informal learning using social media platforms, and learning-specific apps. Access via mobile devices continues to grow; connectivity via smartphones has increased in the US from 18% in 2009 to 64% in 2014 (Meeker, 2015) and in other nations (Pew Research).
Increase in non-traditional students. A huge market exists—adults in the work force who are looking for opportunities to learn new skills to improve their career options.
Government support: Governments seek opportunities to lower costs of education and increase access which translates into funding for alternative education pathways. This quest often involves grants and funding programs for digital learning, flexible degree pathways that may involve recognition of work experience in competency-based programs.
Department of Education (DOE) launches the Educational Quality through Innovative Partnerships (EQUIP) experiment (DOE’s Fact sheet)
The American Council of Education recently launched a significant program ‘The Alternative Credit Project‘ that helps students complete lower-division courses online from accredited providers
Degree within Reach a program in Colorado that offers ‘reverse transfer’ allowing students to combine credits from 4 and 2-year institutions for an Associate’s degree
2. Experimentation in NewTeaching Modelsand Learning Spaces
There are claims that the education system is ‘broken’, a term that is not constructive or accurate. A more fitting description is one that outlines how the traditional education model is transforming in response to digital technology and culture. As a result there are a variety of new models; school models in the K-12 sector that aim to adapt to the changing culture and improve a system that is not serving students adequately, as well higher education institutions who are reinventing their learning spaces—a more subtle approach to changing the traditional learning model from one that is instructor-focused and passive to one that is student-centered and active.
Examples of new models: Sal Khan’s Lab School, a school to ‘investigate and explore new methods of learning and teaching’, Mark Zuckerberg’s The Primary School geared to low-income children where health care and education are combined under one roof. In higher education there’s Purdue University’s IMPACT program, which includes new classrooms and active learning spaces that support blended learning. Others, Vanderbilt University with their emphasis on creating new learning spaces, and University of Central Florida’s large-scale program that is increasing the number of students it serves while lowering costs by offering students F2F courses along with an ever-widening menu of online and blended courses.
Drivers of New Teaching Models and Learning Spaces
Under-performing K-12 schools and poor performances in international tests via OECD PISA testing
Pressure on higher education institutions to reduce costs, increase access to under-served groups, and improve performance
Our digital culture where students have 24/7 access to information, can learn anytime and anywhere, in conjunction with institutions that are struggling to leverage the culture shift
Developments in New Teaching Models and Learning Spaces
Arizona State University’s Global Freshman Academy “to offer full university freshman level courses for credit” via edX
3. Student-driven Personalized Learning
Personalized learning is one of the top buzzwords in education now; it suggests a host of different learning methods that are typically institution-driven. Yet I suggest that personalized learning is, and will continue to be learner-driven where learners control their learning and become not just consumers of content but active creators of content, building knowledge through collaboration and connectivity via smart phone apps.
Students will be in control not only of when they learn, but will demand that they contribute to their learning through discussions and collaboration, creating content while doing so. This student-driven phenomenon suggests that schools and higher education institutions will need to respond by creating learning programs that acknowledge that the learner is seeking this kind of personalized learning experience.
Drivers of Personalized Learning
Learners…because of their ownership of mobile devices with Web access
Learners…given the abundance of phone apps that allow them to create content and collaborate
Learners…communicating within messaging apps, which Meeker suggests will evolve into major communication hubs (slide #53)
Developments in Personalized Learning
Universities providing guidance to faculty on how to incorporate the strategy of student-generated content, Santa Clara University
Though we can’t predict exactly what will happen in 2016, we can make informed decisions and be strategic for the upcoming year. Nothing is certain in the future except change as the saying goes, yet being proactive rather than reactive will put educators in the best position for a successful 2016.
Data has this marvelous capacity to show patterns of human behavior, tell stories and even predict what we are going to do next. It’s the premise of “Dataclysm”—the stories data tells about what we value, how we think and act. I chose the book as one of my 2015 reads because of the big mountain of data that education institutions are collecting; I wanted to get a glimpse into how data predicts behavior, to learn about privacy boundaries, and was hoping to get a glimpse into how data might help us design, develop and deliver better learning experiences for students. A tall order. Not surprisingly I didn’t find answers; but I did learn about the power that data holds and discovered a good report by EDUCAUSE that does have some of the answers I was looking for.
The biggest takeaway from “Dataclysm” is the incredible potential data holds, which can translate to education sector. On its own data has no value, but with the right software data can inform, support, predict and help. Most organizations including education institutions collect mounds of data. Some is put to good use though according to the EDUCAUSE report the majority of data is used to satisfy credentialing or reporting requirements rather than to address strategic questions. And much of the data collected is not used at all (Bichsel, 2012). Education data is of abundance. Students generate a significant chunk. Every time a student logs-on to the LMS, school portal, or uses school software, printers, e-books, etc. data is collected. Every click, key stroke, time on web pages, links clicked are recorded.
I definitely think it’s good. … All of this data — everything in the book and generally anything you read online about people’s behavior on sites — is aggregated and anonymous. Nobody’s looking at your personal account. But when you put all this stuff together, you’re able to look at people in a way that people have never been able to look at people before. — Christian Rudder, Author of “Dataclysm: Who We Are” NPR Interview
Rudder, author of the book and quote above, is also co-founder of the dating site OKCupid. He gets most of the content for his book from data on his site though he also draws from Twitter and Facebook. Rudder describes how he takes data, without identifying details such as user names, and analyzes it to create narratives that describe human behaviors. The book is full of stories the data tells about race, gender and politics, which at times was disturbing. Not the writing, which is witty and entertaining, but the results of his analyses. Rudder calls his work more of a ‘sociological experiment’, examining human behavior, values, even biases by looking at (online) actions, words, choices, link clicks, and ratings.
‘Dataclysm’ was interesting—not instructive but insightful. Since finishing the book I’ve recognized how, what many label as disruptive services, are data-driven. Uber for instance, the new taxi service. Its business model rests entirely upon big data. Uber uses complex algorithms to aggregate data into actionable info that quite literally drives the business (Marr, 2014). Another—a new email program by Google, SmartReply, can write email responses for us by using machine learning to ‘work on a data set that they cannot read’ (Corrado, 2015). Whatever that means. But the gist is, its BIG data behind it.
Big Data and Education
EDUCUASE defines analytics as “a tool used in addressing strategic problems or questions”. Analytics are typically applied to institutional data and learning or academic data. Yet it appears that the potential of big data in education is yet to be tapped. The field is broad, complex and there are numerous barriers as described by Bichsel.
One of the major barriers to analytics in higher education is cost. Many institutions view analytics as an expensive endeavor rather than as an investment. Much of the concern around affordability centers on the perceived need for expensive tools or data collection methods. What is needed most, however, is investment in analytics professionals who can contribute to the entire process, from defining the key questions to
developing data models to designing and delivering alerts, dashboards, recommendations and reports.
Though there are many institutions working extensively in learning analytics with the goal of helping students succeed and improving outcomes. One is University of Michigan who have helped create a standard that ensures third-party vendors (e.g. LMS providers) provide institutions with access to data generated by their students—not to withhold the data which can be critical for schools looking to use it to support and inform student success (Mathewson, 2015). Another is Purdue University who has done extensive work in academic analytics with its LMS program Course Signals (Research on Course Signals, n.d.).
Rudder states in his book that we are on the ‘brink of a revolution—a data revolution‘. I think he may be right. The education sector may take some time to figure it out, but for those that get it right, it will be revolutionary.
I recently listened to a lecture ‘The importance of place’ from a Great Courses series on Cultural and Human Geography. The professor discussed the process of ‘place-making’ that occurs when humans interact with, and modify a physical space to make it their own. The result is a distinctive place with a unique culture shaped by the people within. This got me thinking about ‘place’ in terms of online education. Do students view an online class as a ‘place’? If they do, how can educators make an online space into a place for learning?
Place in context of human geography has two elements—the physical characteristics of the natural environment and the human influences—ideas, interactions and interventions. There are several books on the concept of place including “Place: A Short Introduction” by Tim Cresswell, author and professor of human geography. Cresswell describes place not only as a location, but as a physical space where humans produce and consume meaning (2004). “Space and Place: The Perspective of Experience” by Yi-Fu Tuan, one of the first scholars in the discipline of human geography, describes place and space as a relationship, that includes the feelings and thoughts humans associate with each (2001).
Why Online Students Need a Place
When examining face-to-face education settings we get a sense of how place influences learning. Inherent to education is the idea of going to a place to learn—a place to study, to meet with classmates, to lecture, to engage in discussion, to do research, etc. And a place’s physical aspects—the decor of a room, the temperature, WiFi availability, space constraints, available resources, even furniture, can influence students motivation, their effectiveness or even willingness to learning. As do the human aspects associated with a learning place—interactions, discussions, institutional processes, etc. In my experience it’s these human aspects that are most often associated with online education. Yet a sense of place that addresses the physical aspects in an online course is more elusive. Elusive—but necessary.
How Do Educators Make a Place Online
Few resources exist for educators wanting to learn about place-making in an online class. The closest I’ve found is the Community of Inquiry (CoI) model; a framework for describing presence online. The CoI model encompasses three dimensions of presence for online learning spaces: teaching, social and cognitive presence. The model suggests that effective education experiences occur when the three dimensions coincide. This is a good starting point for place-making. It addresses the human aspect of place-making through the three dimensions. Yet as mentioned, addressing the physical dimensions of place-making online is more challenging.
Example 1: Creating Dynamic Online Course Sites with Digital Tools
Effective use of Images and media can contribute to place-making. An experienced online educator Laura Gibbs (@onlinecrslady) maintains a dynamic course site for each of her online classes (Mythology and Folklore) using embedded images and content representing students’ course work from various digital platforms. Not only do students get to see their work featured on the course site, but the course becomes a dynamic space for learning. When students log-on to the course site, the most recent student assignment is featured (screenshot below), which students can access via an active link. Gibbs’ use of media tools (Rotate Content, inoreader widgets, etc.) and platforms (Twitter, Pinterest, Blogger) support place-making by creating a learning space that has meaning and purpose for students. Students shape the space by their contributions, interactions and involvement—making it a ‘place’ for their learning.
Using Twitter as a tool to contribute to place-making in an online course.
Example 2: The Student Lounge
Some online courses feature a ‘student lounge‘, usually a discussion forum or chat room within the course site labeled as such. In my experience these lounges typically aren’t used, usually because there’s little invitation or reason to get involved—it’s just there. Yet there are ways instructors can make these spaces inviting and dynamic, to construct and lay the foundation for place-making. As we saw in the previous example, it takes a focused effort by the instructor to support place-making. Below are ideas for creating a student lounge from high school teacher Heather Wolpert-Gowan outlined in a blog post:
It is not hard to create a virtual student lounge in an online classroom. Think about what a student union provides and go from there. Make sure that students go there first, familiarize themselves with the space, and are encouraged to return to it time and again to recharge their social batteries. It should be a place where everyone can hang out and share, [by] posting, uploading and downloading. You can also create a “water cooler” thread that serves as just a fun way to build your community, introducing students to each other as members of a Virtual Learning Community (VLC.)…For instance, you might want to set up a space or thread that:
Links to resources or even a gift shop of materials and supplies that students can buy or access in order to supplement their learning
Allows students can share thoughts, writings, and videos
Serves as an “information desk” so students can ask questions about the classroom and/or the content
Acts as an “arcade” of links to content-related games and entertainment
Promotes and celebrates learners in the community for their academic or personal accomplishments both online and offline
The idea of ‘place’ in online education is an interesting one; it’s another way thinking about and approaching online education. The idea of place-making in online courses could be one way to engage and motivate students, to make learning relevant, meaningful and effective in our digital world. If you have ideas for place-making in online courses, readers would be interested—please share!
Cresswell, T. (2004). Place: A short introduction. Malden, MA: Blackwell Publishing.
Robbins, P. (2014). The importance of place. Lecture presented in Understanding cultural and human geography. Chantilly, VA: The Great Courses.
Tuan, Y. (2001). Space and place: The perspectives of experience. Minneapolis, MN. University of Minnesota Press.
This ‘Need-to-Know’ blog post series features noteworthy stories that speak of need-to-know developments within higher education and K-12 that have the potential to influence, challenge and/or transform traditional education as we know it.
1) The Micro-credential Movement in Higher Ed
The latest trend in higher education is micro-credentialing, the non-traditional education path where students gain skill sets in a specific area and receive a credential. Case in point, Udacity announced this week a new nanodegree (Udacity defines nanodegrees as ‘curriculums designed to help you become job-ready)’— the Android nanodegree in partnership with Google. Another example—Penn State’s College of Business also launched this week an online bootcamp course, ‘Supply Chain Leadership Academy’, to educate “supply chain leaders of tomorrow in leadership and best practices in holistic supply chain management”.
The micro-credentialing trend is driven by business entities that have a real (or perceived) workforce skill gap, where jobs can’t be filled due to lack of qualified applicants. Google reports it has thousands of jobs to fill given a dearth of qualified applicants. The Linux Foundation, also offering a certificate course in partnership with edX, reports it has over 50,000 open jobs.
MOOC providers and select higher education institutions are leveraging the apparent skills gap, using their platforms to build their online program offerings with credentialing options for a fee. A good idea. The target market is not traditional higher education students, but non-traditional students that are already in the workforce and are looking to further their careers and/or switch career paths. Alison.com is a platform offered credentialing in specific skill sets long before MOOC providers began doing so. Though Alison’s business model is different from MOOC providers such as Coursera or edX. Students aren’t the revenue source but advertisers, featured on the platform, are.
Sampling of micro-credential programs and associated fees:
edX’s Linux System Administration Essentials course, “This Linux course is for those just starting their career in IT as well as professionals with experience on other operating systems who want to add Linux to their portfolio”. Fee: $399
Stanford Online, Professional Certificates, “Our professional certificates offer short, focused courses that give you tools and techniques you can apply right away“. Fee: $1295 per online course; required number of courses vary by certificate.
Udacity’s Nanodegree – “All the course content is free online, but the $200 per month pays for the non-scaleable parts of the degree: project grading, feedback, instructor mentorship, assistance and a final certification”. Option to receive reimbursement of 50% of tuition upon completion.
Insight: This non-traditional student population, which micro-credentials target, is an emerging market and such options are a boon to working or unemployed adults seeking skill development. It’s a positive development in higher education. Employers appear receptive to micro-credentials. However, micro-credentialing is favorable provided the programs provide quality learning resulting in tangible, applicable skill sets. The majority of the credentials require not only a financial investment, but a significant investment of students’ time and energy. It’s buyer-beware; credentials do not guarantee a job, though the courses backed by business entities likely have higher placement rates than those without a business affiliation.
2) The Case for Active Learning over Lectures*
This is not new news, but worthy of review—evidence that performance of students engaging in classes that primarily offer active learning is improved over classes involving primarily lectures. A significant study on active learning was released last year; it provides compelling evidence on active learning benefits specific to STEM subjects in higher education (Freeman, et al., 2014). Researchers conducted a meta-analyses of 225 studies in published and unpublished literature that documented student performance in courses with at least some active learning versus traditional lecturing. Though intuitively we might know that active learning is more effective for learning, there’s now solid evidence to back it up:
The data reported here indicate that active learning increases examination performance by just under half a SD and that lecturing increases failure rates by 55%. The heterogeneity analyses indicate that (i) these increases in achievement hold across all of the STEM disciplines and occur in all class sizes, course types, and course levels; and (ii) active learning is particularly beneficial in small classes and at increasing performance on concept inventories.
Implications: Is the lecture dead? Absolutely not, but to increase student learning, retention and success, involving students in active application of concepts should be the norm not the exception. However, implementing active learning is challenging for many educations, and especially for online courses, yet it can be done with deliberate, thoughtful development of a course learning strategy. Below are links with suggestions and examples of active learning applications. One of my favorite examples of active learning, is an online literature instructor Laura Gibbs, who creates assignments using online platforms—blogging platforms, Pinterest, etc. where students engage with content, each other and the Internet community.
The view that online education is “just as good as” face-to-face instruction was not widely held in 2003: 42.8% of chief academic officers reported that they considered the learning outcomes for online instruction to be inferior to face-to-face instruction. The view of online quality has improved over time. However results for 2013 revealed a partial retreat in faculty perceptions of online learning providing quality learning experiences. The 2014 results indicate that the retreat continues—there’s an increase in faculty that perceive online education as inferior. — Grade Level: Tracking Online Education in the United States, 2015
One of the main criticisms of online courses is they are of poor quality as revealed in the annual Babson study mentioned in the opening. Positive perception of online learning by faculty has declined in 2013 and 2014 (Allen & Seaman, 2015). Face-to-face courses appear to be the hallmark for quality when it comes to higher education. Yet this doesn’t seem fitting considering the ongoing and often heated public dialogue about the quality of higher education programs with little consensus on what quality is. In this blog post I suggest that online educators can and should tackle the quality issue in their own courses, and that they do so by assessing their course holistically. A holistic approach encompasses elements such as students’ perspectives, results over a period of time, artifacts created during learning, and the instructor’s course experience.
I also review recent research on quality assessment specific to online courses. I also examine existing frameworks and rubrics for online course assessment and explain why, even if an institution follows such standards, these are starting points. I outline five-steps that instructors can follow to assess whether a course is ‘good’—an assessment for quality that considers foundational elements, student perspectives, course artifacts, student and instructor learning experiences.
What is Course Quality?
Up until a few years ago ‘quality’ in higher education was measured by a course’s content, pedagogy and learning outcomes (Bremer, 2012). This approach has changed to a process-oriented system where a combination of activities contributing to the education experience are considered. Activities that include: student needs, use of data and information for decision-making, department contributions, as well as improved learning outcomes (Thair, Garnett, & King, 2006). This holistic approach of evaluating education experiences is often applied to the development and assessment of online learning. For example, Online Learning Consortium’s Five Pillars of Quality Online Education (below) and Quality Matters (QM) rubric.
Why Assessing Quality is Difficult in Online Education
Yet there are challenges associated with setting universal quality standards for online education, and though a starting point, a thorough quality assessment requires ongoing consideration of numerous elements, some that occur over a period of time. Key challenges with assessing quality through set standards are outlined in ‘What is online course quality‘ and include: 1) the lack of authoritative body (able and willing) to address minimum level of standards across all states with their accrediting bodies, 2) the challenge of creating a comprehensive, evaluative tool to address complexities of online courses, and 3) the implementation process itself given the significant resources that would be required to implement an institution-wide evaluation process (Thompson, n.d.).
Limitations of Quality Assessments
There are other limitations. Some assessments are inherently limiting with a prescriptive set of standards that may not fit all contexts. Another is the tendency to establish a minimum level of quality, ‘baseline standards’ which limits innovation and creativity (Misut & Pribilova, 2015). Most course assessments are done at a point-in-time and are unable to capture dimensions over the life of a course and post-course; dimensions that include student perceptions collected as formative feedback (mid-way through course) and end-of-course feedback surveys. Furthermore, quality assessments frequently focus on course/instructional design and fail to include learning experiences of the instructor and students.
What’s involved In a Good Course Assessment?
A holistic assessment goes beyond course design; it acknowledges the nuances that make a course unique, including input and contributions from students, developments in the field of study, and current events. Most valuable are students perceptions of their learning and of the course experience. A good course assessment considers the course over a period of time, and considers interactions between instructor and students, students and students, all of which create artifacts that can be studied and analyzed (Thompson, 2005). Artifacts might include, emails or forum posts of student questions, dialogue within forums, feedback from group interaction, end-of-course student surveys, LMS reports on student interaction patterns, student assignment results, and more. Course artifacts give valuable clues to a course’s quality, more so when collected from two or more course iterations and analyzed collectively.
Other elements to consider:
♦Student behaviours including questions asked in forums, emails, interactive patterns within LMS, interaction with resources, participation patterns within discussion forums, social platforms designated to course, etc. ♦Student perceptions evaluated through questionnaires, formative course feedback, post-course questionnaires, one-on-one interactions ♦ Knowledge creation/transfer by students evaluated through assignment analysis, course artifacts, post-course surveys ♦Course design as per rubric/assessment tool ♦ Use of current technology tools and platforms ♦Course data and artifacts from two or more sessions analyzed and compared ♦ Quantity and type of interaction between students and instructor
Five-Steps to Assessing Online Course Quality
1)Asses Using a Rubric or Other Tool to Consider Basic Course Elements
Assess course using the tool or framework employed by your institution e.g. Quality Matters rubric. If your institution does not have a tool in place I recommend the rubric created by California State University Chico which covers six domains. The rubric (embedded below) is free to use and download under the Creative Commons Attribution 3.0 United States License.
* Thanks to a reader’s comment – there is an updated version of the Chico rubric which is a checklist format with additional dimensions. It is similar to the Quality Matters rubric. I prefer the version embedded here — its more approachable given it’s less lengthy and rigid. Link here to the updated version.
2) Analyze Course from a Student Perspective
This is perhaps the most difficult yet useful element for improving course quality. There are a variety of ways to consider students’ perspectives, several already mentioned. Other recommendations—take an online course as a student (e.g. a MOOC) in a topic you aren’t familiar with. This provides an eye-opening view of how it feels to be an online student. Another method is to ask a colleague from another department to review your course and provide constructive feedback.
3) Assess Course Artifacts, Materials, & Feedback
Another useful exercise is analyzing course artifacts. Analyzing results from student feedback via a questionnaire midway through course is helpful. If a course is offered more than once, compare data from course iterations collectively. Consider, is student feedback incorporated into subsequent course re-runs? What about student-generated content? All artifacts and materials associated with a course are valuable material for assessing a course’s quality.
4) Consider Level and Type of Student-to-Student and Student-to-InstructorInteractions
Interaction is critical to an online course; students that feel connected, establish themselves as individuals within an online course are likely to have higher levels of motivation and learning satisfaction over those that don’t. Consider the forums, the interactive assignments where students can participate, the social exchanges within course-associated platforms, and other places for interaction. An example of assignments that encourage student feedback and involvement, leading to high levels of engagement can be found on this online instructor’s (Laura Gibbs) course site here. Also consider the Community of Inquiry model for the types of interactions in an online course that lead to positive learning experiences.
5) Results: Are Students Learning?
Evidence of learning is the most important assessment dimension, yet nearly impossible for a standardized quality assessment tool to evaluate. One could argue that before and after quizzes within a course can evaluate learning. I suggest that the instructor is able to assess at a deeper level whether or not learning occurred, can determine the level of critical thinking. This can be done only when assignments demand that students demonstrate what they know and are required to apply course concepts. Assignments that draw out students thinking by demonstration either through dialogue or written work allow the instructor evaluate learning effectively. There’s no formula for this fifth step, this is an example of customized course evaluation. But I suggest instructors evaluate student artifacts from one course to another and to consider what students learned and how well they articulated what they learned. There may be opportunity for revising assignments, activities or other course dimensions.
Assessing quality in online courses is complex as we’ve seen here, yet addressing quality is critical to advance the positive perception of online education for one, but more importantly to provide learning and teaching experiences that are rewarding, rich and meaningful. Quality assessment can start one course at a time, and who better to do this than the course instructor?