How Learning Analytics Can Make Instructors More Effective in an Online Course

This is the first in a three-part series on learner analytics, cutting edge insight for the course instructor; how to assess student behaviours in an online course using the LMS data collection tools in order to provide more effective course design and instruction.

Most course instructors strive to create a class where students are engaged with the content, appear eager to learn and participate. The indicators of student engagement in a face-to-face class are straightforward enough, attendance, participation in class discussions, and/or visits to the instructor during his or her office hours.

Measuring student engagement in an online course is more complex. However with the current learning management systems [LMS] such as Moodle and Blackboard now in use in virtually all education institutions, there is a treasure trove of data on student behaviour. This data has the potential to tell a story about a student’s engagement, even predict student success within a course. Each click or ‘view’ of a web page or resource on the course homepage is recorded in the activity database along with the time spent on each. The LMS platform becomes not only a resource provider and virtual space for students, but a source of information for instructors about student behaviour and actions.

Consider the potential if course instructors could access and interpret the data collected on the actions of students in a few simple steps. The good news, this is not only possible but takes minimal time on the instructor side, yet reaps big rewards in terms of getting feedback on what is, and is not working within a course. Online instructors that can assess patterns of student behaviours and interactions with course content and learning activities, can be responsive and adjust their teaching style accordingly.

Correlation between Engagement and Student Performance
Before exploring further, identifying the purpose of measuring student engagement in terms of data analysis is necessary to frame the discussion. Several studies have determined that a strong relationship exists between students’ LMS usage and academic performance. California State University, Chico identified that the more time students spent on learning tasks within the LMS [‘dwell time’] along with a high number of visits to the course home page, was associated with higher student grades (Whitmer, Fernandes, & Allen, 2012). Another study conducted by scholars at Central Queensland University which used a sample population of 92,799 undergraduate online students, reported a statistically significant correlation between the number of student views on the course home page and students final grade (Beer, Clark and Jones, 2010). The more ‘views’ or visits to the course home page, the higher the final grade.

Academic Analytics
The amount of data stored in educational institutions is gargantuan, and the new term for data collection is Big Data. According to McKinsey Global Institute, the education sector ranks as one of the economy’s  top ten in terms of the amount of data stored. The question becomes, what do we do with it.  At the institutional level, there are numerous opportunities for data analysis where schools can identify many patterns, gaps in student performance is just one example. Arizona State is an example of an institution that uses data analytics extensively, and has done so for several years with sophisticated analysis programs.

However in this post we are looking at the micro level, how the course instructor can affect his or her instruction using the information stored within the course to improve instruction and support students.  I’ve outlined below a few practical suggestions to get started, the basics to analytics.

Practical Applications for Course Instructors
Within virtually all learning management platforms there are reporting features that course instructors can access to display student data. Below are questions instructors may have about the students within their course that the data can answer through various reports that can be generated.

  • Which course resources/tools are being used most frequently? Video clips, posted documents, etc.
  • How often are students logging onto the course?
  • When did the student review the assignment instructions? Submit an assignment?
  • Which discussions boards generate the most traffic – have more students views? This is different from the number of discussion board postings, as many students may view [and read] the posts but not contribute.
  • When was the last time students logged onto the course? How many times per week are students logging on?
  • What are the patterns of performance in online tests? By question?

Learning to Use the Reports
Learning how to use student data is not complicated once you know where to access the information. I’ve included a selection of brief videos below [average time of each is three minutes] all created by course instructors from various institutions that demonstrate how to access student reports in Moodle and Blackboard. In my next post I’ll delve into what student engagement can tell you about your course design, how to adapt instruction to be more effective and how to troubleshoot student problems based upon my experience with analysis of the online courses at my workplace.

Click here for part two of this series, How Course Instructors can Improve Students Engagement with Analytics.

Resources: How To Videos

Photo credit: Big Data, metaroll’s photostream, Flickr

Extreme LMS Makeover – Unconventional Approaches…

“Leaving the LMS: Checking out of Hotel California”
Campus Technology Forum 2012, Session T07, May 1

Do you ever wonder what it would be like to have a seamless, intuitive learning management platform that  does what you tell it? Starting your course fresh with a new system? Yet the barriers appear steep and deep. Well two universities did just that – one, Western University shared their story (title above) in a session at Campus Technology’s Forum 2012 that wrapped up today in Long Beach, CA, and the other, Temple University, in “Making Education Social: the FoxMIS Community Site”.

Both sessions were relevant and timely for those that work with traditional LMS platforms such as Moodle, or Blackboard. Front line educators from both institutions shared their stories of doing the unthinkable — ditched the LMS, fired them, dropped them, whatever you want to call it. And, the school didn’t fall apart, or implode, in fact,  the instigators of the program volunteered their time to tell other like-mined educators how they did it.

Temple University logo (no text, "T"...

1) FoxMIS Community Platform, Fox School of Business, Temple University, created jointly by Dr. David Schuff and Dr. Munir Mandviwalla.

The Problem
Schuff and Mandviwalla identified several ‘problems’ with courses in their LMS before making the change: 1) operating like silos (sounds familiar), 2) not connected with the larger community, 3) students had little ownership in the learning process, 4) challenges in creating communities around ideas. It didn’t stop there….the  students, who are spontaneous, often work ad-hoc, are event based, collaborative, social, and digital consumers. These mismatched needs led to the development of this model….

The Solution: WordPress and BuddyPress
The concept of Socialize the Enterprise emerged, where the goal was to integrate the learning, teaching, professional development, socialization all while building a community. How did they do it?

  • Image representing BuddyPress as depicted in C...Built a platform with WordPress.org (open source blogging software) and BuddyPress (open source software for self-hosted WordPress social network)
  • Added custom plug-ins for Grade book and e-portfolio
  • Developed custom themes and functions for e-portfolios and courses

I encourage you to visit the site – it’s impressive, the developers managed to create a community ‘feel’, yet the site functions practically as a tool for faculty, staff, administrators, prospective students and  employers.

2)  Leaving the LMS: Check out of the Hotel California, Western University of Health Sciences, Dr. Gerald Thrush, (Assistant Dean, Pre-clinical education) and Dr. Matt Curtan (Technical Support Supervisor).

Western University was using a Learning Management Platform that they found non-responsive, clunky and heavy with features and tools they did not need or use. To complicate matters, there are over eight different schools within the University.

The Problem:
The school had three primary needs, 1) Content sharing – the need for high storage capacity and ease of file transfer, 2) Score recording and reporting (including historical data to be pulled from ALL testing, advisor notes, records etc. 3) Low stakes quizzing capabilities.  Faculty and administrators were not interested in the bells and whistles, what they called ‘feature creep’.

The Solution: Microsoft’s SharePoint
Dr. Curtan and Dr. Thrush started with a backwards approach, with the end in mind.  After exploring several options, SharePoint, had all the features they needed, which included:

  • Unlimited space
  • Intuitive interface
    • Web browser
    • Windows Explorer
  • Active Directory Built in
  • Academic Progress Portal: APP
  • Web-based data warehouse

Though they did experience significant challenges, and not all colleges have adopted the program (though many now are looking to change to SharePoint), the new system was implemented in five weeks in two of the colleges.  Impressive? Absolutely. Is this for everyone, definitely not.

Conclusion
I’m not suggesting that we all abandon our LMS platforms tomorrow, but I hope to plant a seed. Change is possible – solutions to an unmet need may even be within your own LMS. But, I believe it begins by identifying what is needed to best meet the needs of your instruction, students and content.

Other Resources:
Re-imagining the role of technology in higher education

Fox MIS Community Project
Ideapressa WordPress as a LMS