In this episode, Sharona and Boz explore what assessment might look like in a world increasingly shaped by AI. Starting with a recent article from faculty at Middlebury College challenging institutions to recenter learning rather than ranking students, the conversation moves into a provocative discussion of oral exams, authentic assessment, and the growing limitations of traditional testing. The hosts unpack a history professor’s experiment with 71 oral final exams in 12 days, reflecting on the power of conversation-based assessment to deepen feedback, strengthen trust, and reveal genuine student understanding in ways that written exams often cannot. Along the way, they connect these ideas to their own classroom experiences, the challenges AI poses for validating student work, and the need for assessments that emphasize creativity, revision, human interaction, and meaningful thinking over rote production. Ultimately, the episode argues that the future of grading reform may depend not only on changing how we grade, but on fundamentally reimagining how we assess learning itself.

Links

Please note - any books linked here are likely Amazon Associates links. Clicking on them and purchasing through them helps support the show. Thanks for your support!

• Recentering learning when we talk about grades
• What I learned from giving 71 oral exams in 12 days
• Finding Meaningful Moments in a Merger

Resources

The Center for Grading Reform - seeking to advance education in the United States by supporting effective grading reform at all levels through conferences, educational workshops, professional development, research and scholarship, influencing public policy, and community building.

The Grading Conference - an annual, online conference exploring Alternative Grading in Higher Education & K-12.

Some great resources to educate yourself about Alternative Grading:

• The Grading for Growth Blog
• The Grading Conference
• The Intentional Academia Blog

Recommended Books on Alternative Grading:

• Grading for Growth, by Robert Talbert and David Clark
• Specifications Grading, by Linda Nilsen
• Undoing the Grade, by Jesse Stommel

Follow us on Bluesky, Facebook and Instagram - @thegradingpod. To leave us a comment, please go to our website: www.thegradingpod.com and leave a comment on this episode's page.

If you would like to be considered to be a guest on this show, please reach out using the Contact Us form on our website, www.thegradingpod.com.

All content of this podcast and website are solely the opinions of the hosts and guests and do not necessarily represent the views of California State University Los Angeles or the Los Angeles Unified School District.

Music

Country Rock performed by Lite Saturation, licensed under a Attribution-NonCommercial-NoDerivatives 4.0 International License.

148 - Let's do oral

===

Sharona: So they suggested an exercise for faculty, if you have not yet done the alt-grading, and this could be something that you could ask if you are someone doing alt-grading and you're looking for an entry point to someone else who's complaining about things. At the end of the semester, once grades are submitted, take a look at your roster and ask yourself if the grade next to each student's name matches how well they met your learning expectations. Be self-reflective. Do you feel confident that every student who got an A or a B or a C is at that level? And if not, maybe you wanna rethink some things.

Boz: That goes all the way back to some of the conversations we had with Dr. Thomas Guskey about don't ever give up your professional integrity to an algorithm.

Welcome to The Grading Podcast, where we'll take a critical lens to the methods of assessing students' learning, from traditional grading to alternative methods of grading. We'll look at how grades impact our classrooms and our students' success. I'm Robert Bosley, a high school math teacher, instructional coach, intervention specialist, and instructional designer in the Los Angeles Unified School District and with Cal State LA.

Sharona: And I'm Sharona Krinsky, a math instructor at Cal State Los Angeles, faculty coach, and instructional designer. Whether you work in higher ed or K-12, whatever your discipline is, whether you are a teacher, a coach, or an administrator, this podcast is for you. Each week, you will get the practical, detailed information you need to be able to actually implement effective grading practices in your class and at your institution.

Boz: Hello, and welcome back to The Grading Podcast. I'm Robert Bosley, one of your two co-hosts, and with me, as always, Sharona Krinsky. How you doing today, Sharona?

Sharona: I am doing well. It's ... Unfortunately, we've got a little bit of that June gloom in May going on outside, 'cause it's really gray. But other than that, I am really enjoying the fact that, as of the date of this recording, I have completed the teaching portion of my semester. I'm giving a final on Monday and then grades, but I made it through. And It was a interesting journey with this pre-calculus class, so

Boz: But wow you guys have your final first day of finals week .

Sharona: Yep. My particular final is the first day of finals week. My students did a whole bunch of checkpoints yesterday. I was out for a medical for about a week and a half, so I had to make some real changes to the grading system at the last minute because I had talked about wanting to do a portfolio, and I had set it up and things, but I just couldn't handle it. So that's the nice thing about this grading system, is I think we did an episode on being nice to yourself too or something. Yeah. And I definitely took advantage of that, and I think my students were happy that I did.

Boz: How is I know we talked briefly about it last time, but are you still recovering well from your medical procedure? Is everything going okay?

Sharona: Yes, I am recovering. The amount of painkillers I have is dramatically decreasing, which is really great. And I did go back to the classroom this week, so yes, day by day I feel a lot better.

Boz: Oh, good. Glad to hear it and glad to be back here today. W-we both, this is one of our favorite times of the week, is when we get to sit down and record.

Sharona: Absolutely, 'cause we get to talk about the things A, that we choose to talk about, and B, that we are enjoying talking about, so I'm really excited about that. I'm excited about the conference coming up. Just a quick plug on that. Register for the conference if you haven't yet. If we have any math people around, we're doing stuff at MathFest as well, so I'm really excited. It's gonna be a fun summer.

Boz: Yeah, in fact we're hoping to have a couple of the organizers on with us in the near future, that it's always fun when you get us together, and we get to talk about grading rather than organizing the conference. 'Cause- Yes ... the 10 of us don't get enough time just to sit and, be in awe of each other, 'cause everyone on the committee is just a phenomenal educator, and, there are so many different viewpoints and just experiences within the committee, and it's just fun when we get together and talk about grading instead of work in the conference

Sharona: Yeah, 'cause usually we're talking about dates and deadlines and Zoom links and who's organizing this, and coming up with systems for that. And you're right there's absolutely no grading discussions- ... usually among the 10 people who are so committed to grading that we run a conference, so that's interesting. We should host them all out here. Get them all out to LA and ho-host them for a day.

Boz: Oh, that would be fun.

Sharona: How are you doing? Did you have a good week?

Boz: Yeah. Not bad. It's, I think I've talked about this a couple times, but I'm still adjusting in my new role 'Cause testing has been going on for the last, week or two at my schools, and even though I was never the testing coordinator at Santee, I was always so heavily involved that it's one of the busiest times of my year. I'm not, a- and even though this is my second testing season now, it still just feels weird not to be, proctoring in a room or running the CAASPP site codes and getting all the sessions set up. So it's still odd.

Sharona: And it's a little bit lighter for you 'cause a lot of your teachers are involved in the testing- Oh, yeah.

It's a, it's- ... so they're not available.

Boz: Yeah. So you're like- it's a lot lighter, especially with one of my schools, 'cause not only is their 11th grade the testing class, but their 9th and 10th grade students are also doing one of the kind of interim assessments. So, three quarters of the school is testing, and the seniors are doing senior things. There's no one for me to go and even visit or so it is odd. Or observe or

Sharona: any of that.

Boz: Yeah. All right. So what are we talking about today?

Sharona: Couple of episodes ago, I think it was maybe Matt Townsley who told me about Google Alerts- ... which I knew existed, but I had never set them up. But I figured out how to set them up, so I'm now getting fed articles about grading quite a bit. And I'll be honest, I, it- it's been really hard in some ways because there is a plethora of articles that basically I think were kicked off by that Harvard article that we've already talked about on two episodes.

Boz: Yeah. We went into a lot of detail talking about the too many A's at Harvard in episode 138, and then back again on episode 142 with Dr. Stephanie Valentine kind of talking about some of Harvard's plans of what they're gonna do about that. So yeah, we've had two whole episodes in the very recent past about this.

Boz: Yeah, and this article just came out. This was April 23rd of this year. So this is a very recent article.

Sharona: And the title of the article is "Recentering Learning When We Talk About Grades." So right there, I was like, "Yes, learning."

Boz: Yeah that, one of the, one of the two articles that we were talking about on episodes 1 42 was the whole i- that last line. The conversation shouldn't be about is there too many A's. The conversation should be about the problem is the grades themselves. And this is doing the same thing. It's saying "Let's stop talking about grades. Let's start talking about learning."

Sharona: Exactly. And what I like about not only this article, but how Middlebury is approaching it, is their own newspaper's editorial board, they put forward a call for a more nuanced conversation about grading standards beyond letter grades or percentage cutoffs. And their a- strategic plan at Middlebury specifically calls for action. They have a an action item that says, "Conduct a review of grading practices, including the causes and consequences of grade inflation, and develop recommendations to support academic rigor." Now, we can have all the rigor conversations in those. I don't wanna go down that route today. But their strategic plan is calling for this, and the editorial board said, "Hey, let's make it more nuanced than just letter grades or percentage cutoffs."

Boz: Yeah, and in fact, when they were looking at this, and I'm gonna read this word directly 'cause I love this. But they said, "Throughout these conversations, we've always tried to ground our discussions around a couple of key questions." And these are the same key questions that you and I talk about all the time. Question one, what are grades for? And do our grading practices help accomplish our educational goals? So what's the purpose of grades, and is your current system accomplishing that purpose?

Sharona: And the we that is asking those questions is a group, an informal group of faculty that has been having these conversations for the past decade And they compare notes on how they approach grades across disciplines. They discuss alternative grading practices. So that's one of the things that I'm really excited about. We're up to 23 institutions that have registered for the grading conference this year. That's 23 institutions that I'm hopeful can form a group of faculty, a core group at their institution, to guide and support these conversations and do them with the principles that we believe in, because that's exciting. That's how you get this going.

Boz: Yeah.

Sharona: So this group, they actually pose these questions in a public forum, and that's exciting.

Boz: After doing that, what did they say came out of this?

Sharona: I think what I like about this article is they say that we have some common beliefs, but the goals, and this is what we said in that episode with Stephanie Valentine, that none of the goals of grades, facilitating or communicating learning, which is typically what - their grade purposes are for, either to reward students for doing well within a course or to communicate competency outside the course. They say none of them are helped by placing a cap on the number of A's. And they do talk about the same pedagogical problems that we discussed in that article. But then the, what they said, and this is where I really like it, "We feel strongly that it's not a faculty member's job to rank students against one another, particularly within a course." I can't emphasize how much I agree with that statement.

Boz: Yeah, they've actually got a couple of statements in that same paragraph that one came out of talking about some of the grade inflation limits with the Harvard article that I agree 1,000% with everything they said. In fact we need to try to reach out to some of this group and see if we can get them on here 'cause, like I said, everything they wrote is just yes, and yes.

Sharona: Exactly. And then they go into the research. There's a deeper critique of grading. They mention the Butler article. They mention that research suggests that grades are a poor extrinsic motivator. And then they say this: "Effective learning requires creativity and risk-taking, an emphasis on an iterative process and learning from mistakes." And actually, some of the other Little Ivies like Reed and Brown and Sarah Lawrence, they've all adopted unconventional approaches to communicating learning to students. So again, what we're starting to see is these smaller colleges that still very high standards, very competitive admissions, high sort of credibility in the greater world to have a degree from these colleges, they are having a lot of success with the alternative thing

Boz: You were mentioning about the risk-taking and creativity. You didn't finish that sentence because I think the next part of it is incredibly important, and that was something that Dr. Stephanie Valentine was also bringing up, that when we center our pedagogy in grades and then traditional grading, that it runs counter to that whole process of risk-taking and learning from your mistakes and, that iterative process. Which is what I found was, when I f- finally stopped doing all the band-aids and did go into alternative grading, that's what I found was my problem that I didn't understand. But even though a core belief of mine was math stands for mistakes allow thinking to happen, my grading was contradictory to that. And they're saying the exact same thing here.

Sharona: And it just continues to reinforce all of these things that have been said. So I was very happy to see this discourse. This article goes on for several more paragraphs, but I did one thing that they challenged people to do. So they suggested an exercise for faculty if you have not yet done the alt-grading, and this could be something that you could ask if you are someone doing alt-grading and you're looking for an entry point to someone else who's complaining about things. At the end of the semester, once grades are submitted, take a look at your roster and ask yourself if the grade next to each student's name matches how well they met your learning expectations. Be self-reflective. Do you feel confident that every student who got an A or a B or a C is at that level? And if not, maybe you wanna rethink some things.

Boz: That goes, all the way back to some of the conversations we had with Dr. Thomas Guskey about don't ever give up your professional integrity to an algorithm. No matter how you have your grades set up, whether it's traditional, whether it's alternative, never give up your professional integrity to an algorithm. And yeah, go through those grades. And, you and I a- actually have this written in our syllabus. We promise never to lower someone's grade, but we'll definitely raise it. But yeah, go through and you look at that student and you go, "Yeah, that, that grade doesn't match," do something about it.

Sharona: Exactly.

Boz: And I know that's something, we've also talked to Joe Zeccola quite a bit about. That's something that, that he is unapologetically about. It's yeah, he'll go through, and if the grades don't match he's not afraid whatsoever to change that grade So it does match the learning, 'cause that's, it's, that's the way it's supposed to be.

Sharona: And I am sensitive to this. I know that I, in past episodes, we pooh-poohed this grade inflation thing, although we've pooh-poohed it in terms of alt-grading, 'cause people accuse alt-grading of being the reason we have grade inflation. I actually think we have it in in points and percentages. But I think that, I do think that we're seeing it in weird places. On the one hand, like you said, we'll raise the grade if they do better than what our rubric shows. But we also... it's a real thing that we're getting higher levels of graduation and increasing levels of needed remediation. That other conversation, I don't wanna have that conversation today, but there's been a lot of articles recently about increasing graduation rates and yet gaps in scores in math and English. So I do think we need to look at that at some point. But for alt-grading itself, I will look and I will see if it matches. This whole thing just calls for some introspection on the part of faculty. But none of that, in my opinion, leads to grade caps. No. I just think that I am so opposed to that because that says that the, that none of the learning matters.

Boz: Nope.

Sharona: All that matters is how this group of 30 students or 300 students do against each other.

Boz: Yep.

Sharona: So to me, grade caps just take a big old bomb and destroy the whole system.

Boz: Absolutely. All right. But, looking at this article I love the kind of last sentence or two of this. So I'm gonna read that directly, but I'm gonna pose a question after that, okay? And I wanna see- Okay ... what you do with that question. So the last line of this is, "While we welcome ongoing conversations about grading at Middleburg, we hope we can focus on our shared institutional and educational goals to facilitate high levels of learning and academic achievement so all students might be able to meet and exceed our expectations." So that's definitely the conversation, and we've brought this up before. That's the conversation we should have. We shouldn't be talking about all this grading reform. We should be talking about learning. However, learning is also starting to look a little bit different, and this is something that you've brought up a lot about AI really being this either catalyst for massive change or catalyst for the downfall of higher education.

Sharona: Yeah.

Boz: So I am- Yes,

Sharona: I have ...

Boz: I am curious, with you trying to focus on learning in your classrooms rather than grades, what have you been doing because of some of these recent changes in our climate and just the world?

Sharona: Yes. So- With this semester, I really did two things, one of which I ended up having to drop because of all the medical issues that I had. But I wanna say the two things I had planned, one of them is I pulled all of my a- assessments, which are measures of learning, back into the classroom and onto paper, which one of the things we talked about over, in the pod over the last couple years is that we were so excited because we had transferred all of our assessments into online formats, and we'd done a lot of work to build out assessments that were a little sort of more cheat-proof in a way, that it was easier for students to just do them rather than get help doing them.

Boz: Part of the other reason we were so excited, or at least I was so excited about doing that, and I'm still more resistant about losing all of it, is I got so much more instructional time with my students by not having to spend so much of it on assessments in class.

Sharona: I don't trust that anymore. I am not comfortable at all with timed assessment measures that are supposed to be done without outside help being done outside my eyes. At this point you can screenshot anything, throw it into any of the AIs, and if it's math, if it's a math problem that is closed-ended and has a right answer, it can do it in a split second. And it's so frictionless-

Boz: Yeah ...

Sharona: that I just I can't trust it right now. So that was one thing I was doing, is I was like, "I'm just gonna have to generate paper." And I would generate paper and then scan it and upload it into the system so we had a digital record. Pain in the neck, but I had confidence.

Boz: Yeah. And this was something that a while back, and I should have looked this up, but this was something when we talked to, Dr. Robert Talber, he was, he ended up doing the same thing about a semester or so quicker than you did. But yeah, he was making similar kind of adjustments.

Sharona: But the one thing I was going to do, and I did not end up doing, but I was going to let them work on stuff outside but I used AI to custom design their portfolio assignments, and I had them use AI to help them generate a topic for their portfolio. So they needed to come up with a topic that would be specific enough and relevant enough that they could test all the different types of math we were learning and try to explore that topic using a variety of mathematical models. And then I would print it out and give it to them, and they would have to work on it. And the other thing that they're having to do is they're having to do it by hand because I'm finding that at least in mathematics where it is not easy to do the work In typing form, you have to learn weird character combinations to make math work in typing form. I want them to be able to do it by hand, and their hand skills are atrocious.

So that was the plan. I did not end up doing the portfolio, but I did end up doing a lot of paper assessments- ... with a lot of handwriting on them. And what was the cool thing about it was I saw the growth. At the beginning of the semester, most of their paper, papers were almost blank. And then as they retook things and took new things over the semester, like towards the end, many of them were now writing a lot more. They were writing it more accurately. They were writing it more clearly. So that was good. But I had to pull it all in because I just couldn't trust that what I was going to get from an outside assignment was gonna be valid.

Boz: Yeah, a year ago, we didn't have these same concerns. A lot of the AI things would do weird things with the math. It ain't that anymore. No. No. And it's not just the paid services. It has gotten ridiculously, like you said, what, what did, what word did you use? Frictionless? Frictionless?

Sharona: Frictionless. Yeah. And but the thing is also, there's research coming out that when you outsource a lot of the stuff to AI, and I'm seeing it with myself, like I did these AI-generated portfolio assignments. I couldn't tell you what they say. I have no idea what they said. I did them, but I didn't do the thinking work.

Boz: Exactly.

Sharona: So there's no retention.

Boz: So that's one way. That's what you did. A- and I am resistant to that as, long as I can be and as much as I can be, I'm going to be.

Sharona: Yeah, good luck with that.

Boz: I know. I know. But we did come across another, recent article that talks about the way they did this, and this is a very different solution. I don't-- I can see it maybe working in some of our math. I don't know if it would completely, but it's a really creative and interesting one. This article came out April 20th, so just a few days before the one we were talking about a second ago, and it's titled "What I Learned From Giving 71 Oral Exams in 12 Days."

Sharona: Yes. So this is a instructor at Cal Poly Pomona who wrote this, and he teaches history. So he says most of the classes that he teaches, he has are lower division history One was mostly history majors and the other ones were fulfilling a GE. And I think this is really important because one of the conversations I had recently with a leader at my university is, and having had two kids who just finished college, both of my kids have now graduated. Yay.

Boz: Congratulations.

Sharona: Ugh. The G- some of their GE courses are complete BS, like complete and total BS. He took a nutrition one that was basically a semester-long infomercial. It was bad. And I'm like, "This is not okay." The students are not gonna put up with this for very long. So he talks about that he chose to do oral exams for finals, and he set up 20 in-person slots and 51 Zoom slots across 12 days, and they are half-hour time slots it looks like. So he spent about 36 hours, 35 hours, 35 and a half hours on this. And then he did an anonymous survey asking for feedback. So I love this, that he's taking into mind sort of our own principles that even though this professor's not necessarily an alt-grading professor, he doesn't mention anything about his grading scheme, but this alternative assessment and then asking for feedback yay, kudos.

Boz: Exactly.

Sharona: So some things were interesting, and he says, "In total, 71 oral exams were conversation boot camp-" "... which tested his own mental stamina and ability to think on his feet." He says, "Afterwards, I emerged better at the gift of gab. Psychologically, Blue Book Mountain looks more daunting than a day's calendar with eight appointments." So I thought that was just really fun.

Boz: Yeah. I, because I don't have the ability to do this, I love when people are able to write with this kind of imagery in a, academic setting and make it a fun read.

Sharona: Yeah, Blue Book Mountain.

Boz: This is really interesting. And you're right, he doesn't the author doesn't say really much about his grading, so we don't know if traditional or alternative, but the idea of alternative assessment, and the fact that he went to an oral assessment. And we've talked about this with quite a few different guests, especially those that got into alternative grading by first doing alternative assessment, talking about how much more you can learn from just talking to your students. Not just, what they do or don't know, but how deeply they know it, how left, you know- Is there, there misconceptions? I- is it just a little bit off, or are they out in left field when you're playing football? So I love this idea. And the person, admits right away that, yeah, this seemed mentally exhausting. - I'm not saying this is something that everyone should be doing, especially if you're in a large classroom setting, but it's a really interesting idea. And he goes a little bit further into this, 'cause he didn't just give these oral exams. What else did he do with these exams?

Sharona: And I wanted to comment before I answer that question. That I have said that grading is the most personal relationship that a student and instructor actually has.

Boz: Yep.

Sharona: So in these exams, he had that experience because he saw grimaces, he heard despondent voices, he saw smiles that stretched ear to ear. So I do think that one of the aspects of having that interpersonal reaction during an assessment session, I think is actually going to build, has the potential to build trust and the relationship between the student and the instructor. So he definitely describes some of the things he did for this exam. So for example, he said it resembles a tutorial, right?

Boz: Yeah.

Sharona: So a month before the exam, he had a student who said he was interested in ancient Assyria. Now, the instructor had removed the Assyrians from his lectures, so for that student's exam, he actually gave the historical context as part of writing the exam, and then the student went off, analyzed several Assyrian primary sources, and then came in to take the exam.

Boz: Yeah, see, and that's one of the, that's one of the things that I was trying to point out with not just that he gave these oral exams, but he was able to tailor them. This is definitely something that if you're doing oral exams, you can do really easily compared to other types of assessments. With a history class, come on, where's a better place than that to do this?

Sharona: And then he gave them a curveball question that I really loved, which was, "What is a question you don't know the answer to but wish you did?" So the students had to come up with a question that they don't know the answer to, and some of his exams turned into a press conference- where he played the historical character, and the student had to be a reporter poking at the questions and asking and trying to discover. The student then had to either analyze the response or follow up with a question. So again, very interactive, very engaging, very moving. One thing that worries me, again, we just have to really look at what technology is doing, is I've seen some advertisements recently with an AI that literally can be on your Zoom and will analyze the question as it is asked to you and then throw up a script to send back. There's all kinds of ways to cheat the system, and I really think that it's going to come down to convincing students that what they are learning has value and purpose for them as human beings, not just for a job or this or that, and it's gonna have to be, there's gonna have to be a relationship of trust because otherwise we're just gonna really be difficult.

Boz: But now I wanna go back to what you were talking about with those curveball question. 'Cause what a great example of doing exactly what I was just saying. Not just learning what your student might or might not know, but l- just how deep their knowledge or even their misconceptions do go. Yeah. 'Cause th- yeah, that kind of question and where you can go from there.

Sharona: That's what I was hoping to do with my portfolios is because there were basically four function families that I was teaching this semester, and those families represent different rates of change in the world. One, one is a constant rate of change, one is a variable rate of change, and the, or three of them are variable, but they vary in different ways. And there, there's a level of importance if you're going into a field that depends on this mathematics for you as a person out in the world to be able to recognize and think about the data that you're looking at or a problem set and recognizing which of these families might be the most useful, most effective. You can't-- It's too messy in the real world to just depend on computers. This is where human creativity comes into it. But convincing my students in the levels that they're learning about these families that this is what's important is hard, and that's why being able to personalize it to them, being able to give them a topic that they themselves are interested in, especially a messy topic that's not gonna fit into some pure, easy formulas, that's where I was going with that portfolio.

But unfortunately I didn't quite get to it.

Boz: Yeah.

Sharona: It didn't quite work. But that's okay. When I teach it again, I will. So towards the very end, he says, "Oral exams are also organic. Several of them went down unforeseen paths. Stimulating intellectual discussions make universities special places." Complaints, gossiping, and mundanities fill most conversations, but a good chitchat that makes labor joyful and life pleasurable, that's what he got out. He got to have history conversations with his students.

Boz: Almost from our first episodes when we talked about some of the, the mistakes and some of the the facts that we did not get this right the first time, what was the reason that I never left it was the conversations I had with my students at the end of the semester. So I completely understand and agree with what he's saying about having these great conversations about something that he obviously loves because he's an educator in history, so he's gotta enjoy history q- pretty much. A- and the, we were talking about this at the beginning of the episode here. We were excited to have, the, at least some if not most of the other gr- grading conference organizers on the podcast to talk about grading, 'cause- ... that's something we just enjoy talking about. So yeah this idea of, these oral exams becoming more than just exams.

Sharona: Yeah.

Boz: And I love something that he had talked about earlier in the article that w- with the written exams, he didn't know how many of his students were actually reading the comments on his exams or papers, didn't know if students were actually using that to deepen their understanding or better their understanding. But something that he saw with oral exams is that students could hear what they did and didn't know. They could hear his feedback. They could hear where they needed improvements before, the grades so that instantaneous feedback and the feedback where we know the students are actually getting it, and if they didn't, they could ask questions about the feedback. That's one of the things that we've talked a lot about this, about one of those pillars of alternative grading being these feedback loops, and the importance of feedback in those, and it being timely and actionable, but also needing to teach our students not just where to find it on your LMS system, but how to use your personal style of feedback. Everyone has slightly different styles, and just giving your students feedback and assuming they know how to use it and what to do with it is one of the reasons our students don't use our feedback Here, he was able to give feedback in real time, be able to see if the students were understanding that feedback so they're getting something out of it. So I love that concept or that aspect of this whole process.

Sharona: And to be honest, with the class that I had this semester, I encountered the most resistance to feedback that I've ever encountered. And it wasn't like-- It wasn't they were like, "Oh yeah, I looked at the feedback and I reject it." It was despite numerous times showing them where to find it, numerous times talking through it, they just never really engaged with it. I have students who had three different checkpoints on the same learning outcome all marked as revisable, and they were just taking more and taking more because they weren't sitting down and saying, "Oh, I have this thing that's due for a revision." So there was some sort of a disconnect in this particular class, and I think that I, again, reli- I think in this case, I relied too much on digital. I was handing back the papers and saying, "Go look in the system for your feedback." And I think I maybe should have been marking their papers as revisable and, like not copying the comments down, but by not putting the markings on the piece of paper that they received, it didn't trigger them into the feedback loop. And I wonder if this, at least this set of students that I had this semester, I needed to be more personal with. I needed to have more of that verbal and physical engagement because the digital side, they just, they tuned it out.

Boz: Yeah.

Sharona: They tuned it out so much.

Boz: Yeah. Like we said in our feedback episodes, a feedback loop does not exist if your students aren't engaging and using the feedback. And that's something that this professor saw with the oral feedback, that not only did the students engage, he knew the students were engaging with it. Whereas before, like he said, he had no idea if students were actually reading feedback on his papers or assessments.

Sharona: And he was sick of getting the AI-generated essays. That's- it's just becoming a more and more- Yeah ... of a bigger

Boz: problem. But that was the whole, thing, the catalyst that had him go to oral was, yeah he was sick of getting these AI ones. He had long ago done away with like Scantrons and things like that. So that was the catalyst. But I really do think that, some of these other aspects, despite the workload, despite the intensity of it and the mental exhaustion that he must have had doing this, I'm sure he's probably gonna try to find ways to streamline it, probably try to find ways to make it easier on him. But I bet you he doesn't go away from this. This sounds like something that's going to be part of his pedagogical setup in his classroom for a very long time.

Sharona: Yeah.

Boz: And I love that. I absolutely love that. Am I saying this is what everyone should do? Absolutely not. If you've got a class of 300 chemistry lecture, are you gonna be able to do 20-minute conversations as exams? Not a chance.

Sharona: No. No. Would you- But this was one way to do it for one person. And the reality is, he was probably gonna spend at least 20 minutes per student grading their finals if he was doing long-form essay finals.

Boz: Yeah. Yeah, y- you're right. You're right there.

Sharona: I do wanna point out one other thing in this article that he mentioned that I did I also love about some of this, is he points out that his structure was more accommodating. So he had both in person and Zoom, he had multiple days only one student needed a disability accommodation. A lot of students commented that they appreciated that the oral exam was 20 minutes, their individual exam, compared to taking a two-hour, multiple choice history test. They felt that being on Zoom, some of them, it eased their anxieties.

Boz: Oh, and I love what he also said about that 20-minute thing, since you brought that up. Did you see at the bottom of the section Tailoring Exams he would tell the students after 20 minutes, "You currently have this grade." "You currently have a B," or whatever. "Do you wanna quit now or do you wanna continue for another five minutes to try to increase it, but also risk lowering?" We've had several episodes about giving student choice, but h- how fun is that? You get to, okay your 20-minute exam's up. Hey, you've got a B right now. You wanna go an extra five minutes, see if you can raise that up to a B+ or an A, but know that you're risking taking it down to a B-?

Sharona: See, I'm fine with the trying to increase it. I would never do the risk to lower.

Boz: I think that, that gives, that little bit of kind of gamification into it. Yeah. And-

Sharona: I don't like that. That feels like a gotcha. If they've got the B, I'm okay with them trying to raise it, but I don't wanna ever-

Boz: Because this isn't decreasing it afterwards. This is in the process. This is going, okay, yeah, in the next five minutes you might be able to show me that you actually know more, but you could also show me that you do have a gap that I didn't see before. So- Th- that's why I don't mind that at all.

Sharona: Yeah, we're gonna have to agree to disagree on that one. I wanted to, as we begin to get close to wrapping this up, bring in one more piece of literature. This is from also in April and it was published on the Grading for Growth blog. It's an article that's labeled Finding Meaningful Moments in a Merger. I'm not gonna talk about the whole thing, but basically this faculty member's institution is merging with another one, so they're teaching their current classes for the last time, and some of them are smaller. And some of the things that she got to do, so she teaches writing, and in her honors senior seminar, she allowed some of her students-- They asked to create a digital anthology on readings on pop culture that were inspired by a podcast. So they are now curating, designing websites, curating them, writing articles, and she gets to observe and look as they're working in class and brainstorming and inspiring the assignment. And she can do her assessments in that same personalized way of observing and interacting. And it's still all writing. It's all writing. It's-- But they have to-- Like, it's so much more writing that's more relevant to them than an essay of a particular type. She allows her students to do choose your own adventure style prompts 'cause she can accommodate those in a small class. So she's really enjoying this, again, the interactive piece. And I really think that is where this is gonna have to go. With AI invading all of our spaces, we're gonna have to really rethink what it means to learn our topics and our contents, and it's gonna have to be judged in person. I just-

Boz: And I hate to say this 'cause I hate talking bad about our personal group of educators. But in my experience, when we start talking about a- alternative assessments, and we've seen these two great examples, these oral exams and this history class, that project that you were just talking about, and that sounds like a, an article we might need to get more into. But that gives so much more in-depth understanding to the educator about what the student actually knows. Math educators, at least in my experience, are the most resistant to any kind of alternative form of gra-- not grading, but of assessment.

Sharona: Yeah.

Boz: What do you mean? I have to give it to them on a test. I can't-" just talk to the student a- and yeah, they might be able to explain it perfectly, but they can't do it when I give them on paper. So I, I don't know why that is, and I don't know if it's just 'cause I deal with more math teachers than most any other kind, but we just seem to be so resistant to that. And you're right, I think that might be, you keep talking about the fact that education needs to evolve, or especially higher education really face the, threat of not existing anymore. These are the kind of a- adjustments that could be part of that.

Sharona: And I think that, there are areas where this is going on. I really wish I had the opportunity to sit in or take or teach an AP pre-calculus class, like a high school using the actual AP curriculum, because they've done, from what I can tell, quite a deal of redesign of bringing in this problem-solving and this different lens. So there's some smart people out there that are trying to do this. It's just getting it everywhere all the time, all the things is a real challenge. So anyways, I just was thinking about this as we were talking about this oral exam thing and this professor getting this opportunity to really blow up her classes because they're small. Let's be clear, the reason she felt she could do this is her classes got very small, and that's another thing we're gonna have to think about. That goes back to that two Sigma problem- Yeah ... that Bloom did. That how do we make higher education behave like having your own tutor?

Boz: Yep, and yes, the answer might be lowering class sizes, but I don't think that's the only answer. I, I think there's ways to do this and we've seen some really creative every single year we have at least one session in the grading conference about doing this in, large settings. They're always extremely popular 'cause it's a huge issue, but we always, in fact, one of our keynotes last year was about doing this in large classes.

So-

Sharona: And we have several sessions about it-

Boz: Yeah ...

Sharona: doing large classes this semester, this time.

Boz: All right. We are coming up on time, so as we start to wrap this up is there anything else that you wanted to bring up real quickly before we do cut this?

Sharona: Just if you haven't registered for the grading conference yet, please register. There's still time. If you would like to get a group of at least eight educators together, you guys can do better doing an institutional registration. So you can send us an email, [email protected] to do that. And if you wanna be on the pod, we've gotten some recent inquiries, and we're gonna have some guests coming up. So send in a message on our website.

Boz: If you personally have been doing any kind of alternative assessments, things that aren't your traditional pen and paper or keyboard Let us know. I'd love to hear some of these other ones. Like I said, this oral exam was a really cool one. That project , that you were just bringing up with your portfolio, even though it didn't actually end up happening because of, some personal reasons with you, or the one that you were just talking about in the article. I would love that we've got so much creativity in this community. I would love to hear some of the other ways that people assess learning. But until then, you've been listening to The Grading Podcast with Boz and Sharona, and we'll see you next week.

Sharona: Please share your thoughts and comments about this episode by commenting on this episode's page on our website, www.thegradingpod.com. Or you can share with us publicly on Twitter, Facebook, or Instagram. If you would like to suggest a future topic for the show or would like to be considered as a potential guest for the show, please use the Contact Us form on our website. The Grading Podcast is created and produced by Robert Bosley and Sharona Krinsky. The full transcript of this episode is available on our website.

Boz: The views expressed here are those of the host and our guest. These views are not necessarily endorsed by the Cal State System or by the Los Angeles Unified School District.