Thank you for Subscribing to CIO Applications Weekly Brief
The New Learning Analytics Driving Individualized Learning Outcomes
Len Peters, Vice President & CIO, New York University
While 90% of NYU classes are in-person, the majority also extend into a digital space, including fully online courses. So since Fall 2017, our Teaching and Learning with Technology unit, headed by Associate Vice President of NYU IT, Ben Maddox has been thoughtfully growing our learning analytics program. We use data that our Enterprise Data Management team, headed by Associate Vice President of NYU IT, Angela Chen, mines from our Learning Management System
(LMS) to see how students use class materials.
We provide near real-time feedback for professors on what teaching methods are actually working on an individualized basis. We conduct clickstream, network, and text analysis of how, when, and how often students access and interact with study materials. We then use data visualization software and a reporting tool with embedded analytics to provide professors with an interactive dashboard showing what each student is reading, watching, and working on at what time and in what order. We have around 20 integrated learning analytic tools we are incorporating into our dashboard to provide instructors continuous insight. When we link this with how students are performing in the class, professors can see what is most effective in helping students learn to determine how to best support them individually.
What is different about learning analytics here at NYU is the scale. We’re not only focusing on depth of analysis. We’re going for breadth: we are in the process of making learning analytics available to any faculty who are interested in using them for instructional improvements across all schools and departments in all locations. Since our proof of concept in Spring 2017, the cohort each semester has increased in size and scope.
Our role at NYU IT is to explore new ways to leverage learning data to help improve outcomes on an individual level
The Spring 2019 cohort had approximately 50 faculty participants. This summer will integrate 17 new courses into the phased rollout. The current target for Fall 2019 is to expand to 100 faculty members and more than 100 courses. We have already provided faculty with insights into the learning experiences of thousands of students, which is to multiply in the near future.
In order to personalize the learning process, faculty are our most critically important collaborators. Given the rich diversity of offerings at NYU, we came up with solutions that fit ourlarge and complex research university. The initial proof of concept included five professors whose feedback made it possible for us to improve the way we collect, analyze and present the data before expanding. It also led to the initial Institutional Review Board (IRB) approved research on faculty impressions of our learning analytics tool, which is supported by a rigorous data governance practice. We’re working within the university culture by harnessing the research mindset that the professors at NYU have already cultivated and enabling them to pose and respond to questions about their class. Partnering with the pedagogical experts at NYU Learning Analytics Research Network (LEARN), we empower students and teachers to take control of the story the data is telling.
Our determination to bring data analytics to scale has also presented us with implementation challenges similar to any big data problem. Due to the high number of students, faculty, and courses, we have an immense volume of data. To ensure these analytics tools can be effectively leveraged, we needed to ensure our teachers can quickly access this data anywhere in the world in real time, so they can see how their students’ progress. By employing big data strategies to our academic challenges – for example switching to cloud technology – size, scalability and growth ceased to be a concern.
The main goal of any educator is for the student to come away with the strongest possible comprehension of the subject matter. Our role at NYU IT is to explore new ways to leverage learning data to help improve outcomes on an individual level. Our data analytics program is already showing tremendous potential as faculty are getting answers and insights to big teaching questions that they couldn’t before.
One of the most exciting things about learning analytics at NYU is that, as our program matures and becomes increasingly integral to everyday teaching and learning, the findings will become “smarter” and more sophisticated. From a technological perspective, in the next few years, we plan to shift to more open models of data collection, as faculty hope to get data about learning experiences from teaching and learning tools beyond the LMS. As the data become more voluminous, we will start to consider how we can apply AI capabilities to the dataset to enable faculty and advisors to see and predict patterns in learning.
These educational technologies are still in their early stages, and the full scope of their power from a pedagogical perspective will take years to realize. In the meantime, our focus continues to be how we can use technology responsibly to help faculty personalize student’s learning and support a top-tier research university now, and build solid foundations to be ahead of the curve in the years and decades to come.
Additional Contributors: Angela Chen, Associate Vice President and Chief Data Architect, NYU, and Ben Maddox, Associate Vice President, Teaching & Learning Technologies, Chief Instructional Technology Officer, NYU.