Project researcher Caitlin Kirby said that by looking at how students in these programs engage with Khanmigo, researchers hope to understand what factors make the AI most effective in learning. 

This research comes at a time when colleges across the country are determining how — and if at all — to integrate artificial intelligence programs in academia. The ongoing controversy regarding AI is not unfamiliar to Kirby, who said she often encounters people who are concerned about its use in the classroom. But, Kirby said, it's important to understand how it can be a powerful tool for learning. That's what this research on Khanmigo seeks to accomplish.

"We will look at students in this program who had these kind of activities, or this kind of support with Khanmigo," Kirby said. "Did that make those students use it more? Or make it more effective for them to use it?"

Khanmigo was developed by Khan Academy, a non-profit education organization that creates online educational tools and courses for students. Instead of generative AI programs like ChatGPT, which immediately provides answers when prompted, Khanmigo responds by asking students questions and providing them with assistance on concepts in an array of areas such as math, science and writing.

Kirby said that in addition to the learning benefits, Khanmigo is also just a more accurate program. 

“Khanmigo is better than ChatGPT," she said. "I know ChatGPT has gotten better with numbers, but Khanmigo can actually do math, so it's more tailored in that way."

A large portion of the work Kirby is doing will be evaluating whether the program has a significant impact on student performance and is a worthwhile program for the university to have. She's also involved in research on the use of Khanmigo in introductory math classes, specifically looking at how instructor interaction with Khanmigo in class impacts students' usage of the program. 

“We're hoping to see that if an instructor is integrating it in course activities, that students are going to be using it more often and maybe using it with better outcomes,” Kirby said.

One of the key components of this research in mathematics courses is collaboration with instructors. In order to assist with the evaluation of Khanmigo, mathematics professor Jane Zimmerman is using the program in two of her Math 103 classes — but with two different approaches. 

“One (class), I offered it to them and then didn't talk about it anymore,” Zimmerman said. “The other one is the one that I'm making it really a part of class and encouraging the use and suggesting ways for them to use it.”

This contrast will allow Kirby to evaluate how the instructor's integration of Khanmigo affects students’ usage and outcomes. 

“That will tell us if we're going to continue a program like this, what kind of support do we need for students so that they learn how to use it for their learning,” Kirby said. 

Both Kirby and Zimmerman are able to view how often students use the program throughout the year. Zimmerman, however, does not contribute to the data analysis, since her students are the participants.  

On top of the research being conducted, Kirby also sees real potential for the assistance Khanmigo can provide MSU students. 

“I think in the case of some of the larger classes that we have at MSU … it's a very easy way to get that huge number of students in touch with a really personalized learning experience that some of those larger classes maybe wouldn't be able to do otherwise,” Kirby said. 

Another program involved with the Khanmigo pilot program is MSU’s Resource Center for Persons with Disabilities (RCPD). AI is not an accommodation offered by RCPD, but Khanmigo is being offered to students in order for researchers to study how the tutoring program is used and how much of an impact it has on performance. 

RCPD Assistant Director of Assistive Technology Innovation Tyler Smeltekop does not have communication with students who are a part of the pilot program, but he does enable student participation by introducing them to Khanmigo. 

In an email to The State News, Smeltekop said that by providing programs like Khanmigo to students, MSU is providing more students with the ability to take charge of their own learning. 

“When studying, a student may identify an area in which they need to improve, but working alone means it's on them to construct their notes and available resources to resolve their inquiry,” Smeltekop said. “While this is an effective method for some, it's not the only way to learn and study.”

By providing AI tutoring programs, Smeltekop said, there is a lot of potential for students with varying needs to get access to the ways of learning that work best for them. 

“AI tutors, like Khanmigo, allow students with disabilities to have a greater range of options for engaging with and learning from their course materials and pedagogical concepts,” Smeltekop wrote. “Continuing to scale beneficial technologies will create a learning and studying ecosystem for students that is adaptive to individual students' needs.”