by BLACK ENTERPRISE Editors
The results could open a new front in the evolving discussion over how to use AI in schools — and how closely humans must watch it when it’s interacting with kids, The 74 reports.
A tutor powered by AI, paired with a human helper and individual-level data on a student’s proficiency, can outperform a human alone, with near-flawless results, a new study suggests.
The results could open a new front in the evolving discussion over how to use AI in schools — and how closely humans must watch it when it’s interacting with kids, The 74 reports.
In a randomized controlled trial involving 165 British secondary school students, ages 13-15, the ed-tech startup Eedi.com put a small group of expert human tutors in charge of a large language model, or LLM, offered by Google’s LearnLM. As it tutored students on math problems via Eedi’s platform, it drafted replies when students needed help. Before the messages went out, the human tutors had a chance to revise each one until they felt comfortable sending it themselves.
Students didn’t know whether they were talking to a human or a chatbot, but they had longer conversations, on average, with the “supervised” AI/human combination than simply with a human tutor, said Bibi Groot, Eedi’s chief impact officer.
The AI, researchers concluded, was “a reliable source” of instruction. Human tutors approved about three out of four drafted messages with few to no edits.
Eedi Labs
The results suggest that “pedagogically fine-tuned” AI could play a role in delivering effective, individualized tutoring at scale, researchers said. Interestingly, students who received support from the AI were more likely to solve new kinds of problems on subsequent topics.
That guided the AI’s strategy about whether students needed an extra push or just more support — something an “out-of-the-box, vanilla LLM” can’t do, she said.
“They don’t know anything about what the teacher is teaching in the classroom,” Groot said. “They don’t know what misconceptions or what topics the students are struggling with and what they’ve already mastered, so they’re not able to dynamically change how they address the topic, as a human tutor would.”
Human tutors, she said, generally have “a really good sense of where the student struggles, because they have some sort of ongoing relation with a student most of the time. An LLM tutor generally doesn’t.”
All the same, even master tutors typically don’t go into a session knowing a student’s comprehensive history in a course, including their misconceptions about the material. “All of that is too much information for a human tutor to read up on and deal with while they’re having one conversation” with a student, Groot said.
And they’re under pressure to respond quickly, “so that the student is not left waiting. And that’s quite an intensive experience for tutors that leads to a bit of cognitive overload,” she said. The AI doesn’t suffer from that. It needs less than a millisecond to read all of those contexts and come up with that first question.”
The new findings could encourage schools to use AI as a kind of “front line” tutor, with humans intervening when a student is “derailing the conversation, or they have such a persistent misconception that the AI can’t deal with it,” said Groot. “We think that would be an interesting way to collaborate between the AI and the human, because there is still a really important role for a human tutor. But our human tutors just cannot have conversations with thousands of students at once.”
Robin Lake, director of the Center on Reinventing Public Education at Arizona State University, said the study is important in and of itself, but also in the context of broader findings elsewhere suggesting that, with proper training and guidance, “AI can be an incredibly powerful tool — and certainly has a potential to take tutoring to scale in ways that we’ve never seen before.”
Under controlled circumstances, she said, it’s also “outperforming humans — that’s really important.”
Lake noted a June study from Harvard researchers that examined results from 194 undergraduates in a large physics class. They presented identical material in class and via an AI tutor and found that students learned “significantly more in less time” using the tutor. They also felt more engaged and motivated about the material.
But one of its limitations, she said, is that it relied on 13- to 15-year-olds. “So immediately I have a lot of questions about whether the findings are applicable for younger students, especially using a chat-based model,” which may not be a good one for such students.
She also noted that there are many questions around student persistence with AI tutors, including what happens when students get frustrated or aren’t sufficiently engaged in the work.
“I still mostly think that entirely AI tutoring programs are biased towards students who want to do the work or are interested in learning,” Cohen said, “and it’s pretty easy to see that students who aren’t bought in or are frustrated are going to give up more readily with an AI tutor.”
She noted that her 12-year-old daughter has experienced problems persisting in an AI-powered math tutoring program. “She gets frustrated if she can’t get the answer, and then she doesn’t want to do it anymore, so I think we need to figure out that piece of it.”
This story was produced by The 74 and reviewed and distributed by Stacker.
RELATED CONTENT: RZA Says AI Is Transforming His Creative Process In Film and Music
Source: Black Enterprise
