In this episode of Teaching Channel Talks, host Dr. Wendy Amato welcomes Kristen DiCerbo, Ph.D., Chief Learning Officer at Khan Academy, for an engaging discussion about Khanmigo, the AI-powered tutor and teaching assistant. Discover how this AI tool leverages large language models to help students get unstuck and assist teachers with administrative tasks and lesson planning. In this conversation, Dr. DiCerbo explains the science behind Khanmigo, emphasizing its role in activating prior knowledge and sparking curiosity in students while supporting teachers with customizable lesson hooks and plans.

Resources for Continued Learning

Khanmigo is currently free for teacher use, learn more about this AI-powered digital tool and explore how Khanmigo can be your ultimate teaching assistant!

Learn more about how the AI behind Khanmigo was developed in Kristen's article: How We Built AI Tutoring Tools

In this episode, Kristen talks about using prompt engineering to craft high-quality lesson plans using Khanmigo. Learn more about how Kristen and her team trained Khanmigo's AI to understand key pedagogical principles, curriculum standards, and the diverse needs of learners in her article: Prompt Engineering a Lesson Plan: Harnessing AI for Effective Lesson Planning

During the episode, Kristen and Wendy talk about Khanmigo's integration with iteach's teacher preparation program. Learn more about how these organizations are ensuring new teachers with how to use AI ethically and effectively in their classroom from day one by visiting iteach.

Welcome to Teaching Channel Talks. I'm your host, Wendy Amato. And without any delay, let me welcome Khan Academy's Chief Learning Officer, Kristen DeServo. Kristen, welcome. You've been doing great things. I'm glad to have you here. Hello. Thanks so much for having me. I'm excited to talk about what we've been up to.

There is news about Khanmigo. Should we jump right in and have you tell us what it's about? Let's do it.

Khan Academy, as many people know, has been a site for instruction and practice for many years now. And in March of 2023, we launched what we call Khanmigo, which is an AI powered It's called tutor for students and assistant for teachers, and it uses these large language models that so many people have been talking about to be able to help students get unstuck and help teachers both save some time and help them with some of those administrative tasks.

Let me spotlight the care that you're putting on these two Academy's Khanmigo. It's students. and teachers. And I think people often consider one population or the other, but you all have been very intentional in designing something that works well for both groups and works well together.

We have, and we do think that's a little bit of the special sauce of Khan Academy, which is thinking about both of those folks.

And when we first started brainstorming what we might be able to do with the, these new models that are producing language, we had ideas in both spaces and. As we started talking to teachers and showing them things, the teachers themselves had so many ideas about what we might be able to use this for, and they would say, could it help me write a rubric?

Could it help me think about how to unpack standards into learning objectives? And every time they would ask, we'd say, let's see if we can. But that work with the actual folks that are using the tool is really important to us in being able to create something that's useful to the people who we want to use it.

When you talk about special sauce, help me understand what is behind Khanmigo that makes it more than just a chat GPT.

One of the things that my role at Khan Academy is charged with is bringing learning science and education research into the offerings that we create. So that means taking what we know about how people learn and how teachers teach, And building that into how Khanmigo acts.

So for example, we know when you think about what a lesson hook is supposed to be, that thing that a teacher does in the, just the beginning of the classroom. We know that from learning science, when you're learning something new, you want to, what we call activate prior knowledge. And so that means you're bringing what the student might already know about this topic or the prerequisite skills that are related to this topic to the forefront.

So that's active in their brains as they're learning this new thing, because that then helps them connect the new information to existing information. So we want to activate prior knowledge in that warmup activity. And we want to spark their curiosity. When we think about. Why students want to learn something.

It's often because it can be because they're thinking, Oh, what is this? What is this interesting question that got posed about how the world works? Or why does that work that way? So when we started designing the teacher assistant that would generate potential lesson hooks. Those were two things where we said, Hey, this is what this lesson hook might be.

So we give instructions to these large language models to produce results that do those things. How do we activate prior knowledge and spark curiosity? And so that's the piece that if you just go to. GPT and say, I want a lesson hook for sixth grade fraction, multiplication of fractions, whatever that is the model doesn't have all, it doesn't know to look at all of that learning science to be able to bring that in.

So we prompt it to do that. So it will give better results for teachers.

I love that. I also like that your role as a teacher is not being erased by using Khan Academy's Khanmigo. Talk to me about how the teacher continues to have opportunities to tweak and adjust and be involved.

It's not the expert that's telling the teacher what's to do. It's the assistant that's helping the teacher. So if you're a teacher using it, you should think of it. Like your helper like your assistant, and so it's going to offer a suggested, if we keep going with the lesson hook example, some suggested lesson hooks, you might say, you know what this one, I want to make sure that this is really targeting things that kids in my class are interested in.

So could you rewrite this and Make it about the sports team that they're really interested in right now, or we're also studying space and science. Could you include that in this math example that we're doing? And so you, as the teacher, get to go back and forth to craft really what it is that you want from the this output.

And as opposed to it saying, here's what to do, it's not an instruction manual, it's your assistant.

I think every educator would be happy to have an assistant, especially one that has been informed by what we know works in teaching and learning. I'm grateful that your expertise has helped to inform the behind the scenes of the tools.

Can we talk a little bit more about the specifics of the tools? What do I do as a teacher when I get this recommendation for a lesson plan or a bell ringer or a hook? Where do I go when I see that I want to make changes?

So the this is all in the interface on the Khan Academy platform. So if you are using Khanmigo, you, for instance, on that lesson plan, when you click on you'll see a menu with a bunch of different things you can do.

So if you want to do the lesson plan, you click on lesson plan, it'll ask you what grade, it'll ask you just a couple of questions to help understand what it is you want a lesson plan about those kinds of things. And then it'll start generating it and you'll have what looks like an editable document.

And and then you'll have a chat window with Khanmigo over on the side. And so Khanmigo will generate the lesson plan in that editable window. And then you can have the chat with Khanmigo, or you can highlight things yourself in the document. And you can make changes to it, or you can request that Khanmigo makes changes to it.

And then it will do that as well. So the interface is set up so that you, as a teacher, can Have that ability to change things yourself or ha and have the conversation with Khanmigo at the same time, which is another thing that's a little bit different than a chat GPT tool, is that ability to, it's not just the chat, but also your own ability as the teacher to edit and change things as you want to.

And then when you're done. You actually can export it to Google Docs, for example, or you can print it out in any of those ways so you can get it out of the Khan Academy platform and share it with others or do whatever you want to do with those artifacts.

I love that you describe having a conversation with it because it really is conversational.

I asked a question about a word choice in one of the results that were produced, and not only was my question acknowledged, but there was like, Oh, I see why you might feel this way. And here's another way to go at it. It was so comforting. It didn't make me feel like I was to be intimidated by technology.

It was truly an assistant experience.

That idea of the tone that Khanmigo has. It comes really from initially, we always try to make Khan Academy, even back to the days of Khan Academy videos, that the videos for students aren't a professor lecturing at you. It comes from Sal. Sal Khan, the founder of Khan Academy, initially started Khan Academy as tutoring his cousin.

And so he was like the friendly older cousin. To that, and that's always the tone we want to have with students is for students. We're like a slightly more knowledgeable friend or older relative Not the expert that's going to lecture at you and tell you how to do things So when we started thinking about what tone we want kanbigo to have with in terms of a teacher assistant We similarly one didn't want to be the expert because it's not the expert as we've been talking about Also, we just want it to be supportive and feel like it's a friend in your corner.

You think about my sister who's a teacher talks sometimes about just, the teacher next door, , who she sometimes just, they're just talking over coffee or whatever. And so that kind of tone is what we are definitely looking for as we started.

The tone is collegial and warm and engaging.

I think that's a home run for you all. I'd love to ask a little bit, if you don't mind, about the filters and the criteria that are behind the AI. I'm thinking about a rubric for lesson plans, for example.

Yes. So one of the things as we started creating activities was how do we know if the output is good?

How do we evaluate the output from the model? And when we were started making the lesson plan activity, we Start you can just ask. You could ask chat GPT. Can you make me a lesson plan for fourth grade on this topic, and it'll spit you out a lesson plan. Won't necessarily be a good one. How do we know if it's good?

It turns out there are published rubrics about what makes a good lesson plan, and it lists the performance criteria. And then what does it look like to be developing, et cetera, and teachers that are in teacher preparation programs often have their lesson plans graded with similar rubrics. So we selected a rubric that we thought was a good rubric.

And as we were developing the lesson plan activity, we kept grading the output on based on this rubric. Until we got it to where the output was consistently at at three or four on that lesson planning rubric. Our initial efforts were not great and it took a good bit of work to be able to get it to that place.

But the idea is that, again, we're taking known things about what makes good teaching, what makes good teaching tools and using those to help us build the assistance for the teachers.

Speaking of teacher preparation or teaching how to make a good lesson plan, you've had an interesting collaboration with iTeach as a teacher preparation organization.

How did that differ from some of the other work that you've been doing? What was special about it?

One of the things that we've found in working with school districts. So we often will go into a school district. And of course, there's many teachers who have not used a I in a school district is that we found that those teachers need some professional learning to learn how to interact with an AI and to learn how to help support their students in interacting with an AI tool.

And so these are practicing teachers who we find once they have that training, they're much more effective and they're more comfortable with using the tool and what that looks like. But if you think about, we have all of these new teachers who are thinking about entering the profession. And if we could get to them with this new technology early, it really could make them comfortable using this technology that seems like it's here to stay.

So they're getting a jump on thinking about how they might use this and support their journey on learning to be a great teacher. From the very beginning. So we're really excited about being able to integrate the ConEgo tools into the courses that students are taking as they prepare to teach, because then this AI isn't something that's bolted on afterwards, but it just becomes part of their workflow and part of their expected workflow that helps them improve and become great teachers.

I've seen some of those iteach assignments and it is successful in helping people as they prepare to become teachers. It's successful in helping them to focus on those things that matter most in lesson planning or preparing to meet the needs of students. I'm going to switch gears a little bit, Kristen, and ask you if you can share some insights about the behind the scenes work, the science behind Khan Academy and Khan Miko.

Let's nerd out a little bit as fellow educators and get into the science of learning.

Absolutely. So I'll talk a little bit, aside from the teacher tools, we haven't talked a lot about the student tutoring and as we started thinking about designing the tutoring experience for students. We really dove into the literature on what do good human tutors do.

And so I will I will say as part of this there's a blog post that I've written that we'll put in the show, we can put in the show notes that talks about some of the learning science behind the tutoring. But there's some key researchers that have been working on what we've called intelligent tutoring systems for decades, but they never quite had this conversational technology that would allow for this back and forth that now we can have, but they did do a ton of research about what good tutors do.

So you can think about, there's literally descriptions of tutor moves. So when does a tutor summarize what the student has said? When does the tutor provide a hint? When does the tutor ask a question? And Press for more information from the student versus when they're, giving them a little bit of information.

All of that is really many, many people who are tutoring just do that naturally or but they're not really thinking through. Oh, this is the time to summarize. This is the time to do this. But if you look at what they do, you can identify those activities. So we then, as we were thinking about designing Khanmigo, we set, took some of those academic articles that describe what are tutor moves and can feed them into Khanmigo and say, these are, this is what good tutors do, these are the kinds of things.

And if you say, for example, to to the large language model, what are the tutor moves recommended in Khanmigo? Gracer person in Magliano that the, which is one of those academic journal articles, it will tell you what those are. And then it can, you can start thinking about how you can design a tutor to do those things.

What kind of feedback have you had about the tutoring aspect of Khanmigo?

We, a lot of good feedback from students and teachers, because what we, the use case for this is really when you think about if a student. is learning a new thing, it shouldn't be so easy that they can do it totally independently, because that probably means they already know how to do it.

It's still good to practice it every once in a while, but they're probably not learning something new. On the other hand, there's also the place where something is way too frustrating, and then they're just going to give up. When you're learning something new, you want to do it, get it to be right at that point where the student can do it with just a little bit of support.

But if you're a teacher in a classroom of 25 30 kids, How do you give that little bit of support to every kid that needs it right in that time?This, Khanmigo now, is that little bit of support. So I'm working on this problem, I'm trying to answer these questions, and I'm just not sure about this step. I can't figure out what to do next.

Instead of having to raise my hand and wait for the teacher who's working with six other kids and needs to get to all of them, I can ask how you go now. Same thing if I'm at home working on my homework. It's the same kind of interaction that I could have with a tutor, but now I can have it anytime or when my tutors and my teachers busy with someone else and where those folks are.

So that's a little bit of what we intended it to do. And we hear teachers and students. Phrasing it for just those things. It helps me get unstuck. It helps my students get unstuck so I can focus on the small group that I have pulled over here is what teachers often say. So that's what we've designed it for.

And luckily not luckily, but luck and a good bit of work. We've gotten teachers and students recognizing that's a really nice benefit for them

As a member of the Khanmigo community myself. I am now engaged in the invitations to participate in how to set up your classroom or where to get more information.

Can you talk about some of that wraparound support that is being provided for educators? They're really not alone when they decide that this is going to improve teaching and learning in their space. What is the support like?

So many folks know Khan Academy as the place where individual students go and find answers and get help with their homework, which is great.

We call those grassroots students. Then teachers can always access our content for free, can always create a class, enter students into their class, work through that for free. Those teachers can also have free access to Khanmigo. That students that are in those classes that we call grassroots classes, we don't, students are not able to access Khanmigo for that through that yet because we need Students to access Khanmigo either need parent permission or district permission.

Just because of the AI, safety, and ethics, we want to be really clear who's approving the use of this tool. But if you're a teacher in a classroom, if, and say you want to use Khan Academy though, practice with your students and want to set that up. We have actually a Khan for Educators course that is just like any other Khan Academy course that teachers can go through to learn how to set up and do the nuts and bolts and think about how it might be integrated into your classroom workflow and instructional practice.

And then we have the districts program. So school districts, Can contract with Khan Academy and when they do that, they get things like automatic rostering and teacher professional development. And in those places, then those districts and teachers in those districts are able to then participate in some of our synchronous professional development.

Which includes both Khan Academy and Khanmigo, and thinking about how to use AI in their classrooms, and of course, how to get set up and get tools using.

Are there any myths that you'd like to dispel, or any misconceptions that we should clear up?

You mentioned one that I think we need to be clear about, which is this is not going to be replacing a teacher.

I think that we need to all be clear that this is an assistant who can help a teacher, but it is in no way doing the majority of things that teachers do, including building relationships with students. That is so important as part of what the teacher does and AI tools can do.

That's one of the things I appreciate most in having learned quite a bit from you this goal is to make sure that teachers can use their time for those things that only the teacher can do.

And to hear you say relationships, gosh, that's right at the heart of teaching and learning also.

It absolutely is. When we, there, and there's research to support it. Kids who know there's someone in that school building who cares about them are, have better graduation rates. They have better post secondary school attendance.

It's so important and we should be making sure teachers have time to, to build that.

I'm grateful for the conversation we've had today. Thank you, Kristen. Thank you for having me. To our fellow educators, thank you all for joining us. If you'd like to explore topics that Dr. Kristin DeCervo and I discussed today, please check out the show notes at teachingchannel.com slash podcast, and be sure to subscribe on whatever listening app you use. It will help others to find us and to learn how to have their own teaching assistant. I'll see you again soon for another episode. Thanks for listening.