We're heading into the holiday break, so over the next few weeks we'll be replaying some previous episodes while adding some thoughts as we conclude the Fall semester of 2023. So enjoy your holiday break!

In this episode, we describe the four decisions that make up the "Grading Architecture" of an alternatively graded course.

• How will you assess your learning targets?
• What type of proficiency scale(s) will you use?
• How will students show proficiency for a specific learning target? (I.e. how much evidence and of what kind to they need to provide)
• How will you wrap up into a final grade?

After introducing the four decisions, we explore some examples of common options used for each of the decisions. This includes:

• What types of evidence of learning will you accept?
• Exploring three common types of proficiency scales - 2 level, 3 level and 4 level scales
• Describing different methods for determining proficiency on a specific target (the Guskey method, N times, and decaying average)
• Evaluating options for rolling up grades, including the simple count method, the bucket method, and the "good to great" method with multiple levels of proficiency.

Links

Please note - any books linked here are likely Amazon Associates links. Clicking on them and purchasing through them helps support the show. Thanks for your support!

• Implementing Mastery Learning, by Thomas Guskey

Resources

The Grading Conference - an annual, online conference exploring Alternative Grading in Higher Education & K-12.

Some great resources to educate yourself about Alternative Grading:

• The Grading for Growth Blog
• The Grading Conference
• The Intentional Academia Blog

Recommended Books on Alternative Grading (Please note - any books linked here are likely Amazon Associates links. Purchasing through them helps support the show. Thanks for your support!):

• Grading for Growth, by Robert Talbert and David Clark
• Specifications Grading, by Linda Nilsen
• Undoing the Grade, by Jesse Stommel
• Grading for Equity, by Joe Feldman

The Grading Podcast publishes every week on Tuesday at 4 AM Pacific time, so be sure to subscribe and get notified of each new episode. You can follow us on Twitter, Facebook and Instagram - @thegradingpod. To leave us a comment, please go to our website: www.thegradingpod.com and leave a comment on this episode's page.

If you would like to be considered to be a guest on this show, please reach out using the Contact Us form on our website, www.thegradingpod.com.

All content of this podcast and website are solely the opinions of the hosts and guests and do not necessarily represent the views of California State University Los Angeles or the Los Angeles Unified School District.

Music

Country Rock performed by Lite Saturation

Country Rock by Lite Saturation is licensed under a Attribution-NonCommercial-NoDerivatives 4.0 International License.

[00:00:00] Sharona: A factor that really needs to be thought about is the grade tracking of all this stuff.

[00:00:05] Bosley: Yeah, that's actually a really big factor and one that can be overlooked when you're new to this and first trying to put this together. We've talked about some of the big mistakes. One of them is just making things too complex. Well, even if you've got a simplified version, you have got to have a way to track it.

[00:00:29] Welcome to the Grading Podcast. Where we'll take a critical lens to the methods of assessing students learning from traditional grading to Alternative methods of grading. We'll look at how grades impact our classrooms and our students success I'm Robert Bosley a high school math teacher, instructional coach,, intervention specialist and instructional designer in the Los Angeles Unified School District and with Cal State LA.

[00:00:54] Sharona: And I'm Sharona Krinsky a math instructor at Cal State Los Angeles, faculty coach and instructional designer. Whether you work in higher ed or k12, whatever your discipline is, whether you are a teacher a coach or an administrator This podcast is for you each week You will get the practical detailed information you need to be able to actually implement effective grading practices in your class and at your institution.

[00:01:22] Bosley: Hello and welcome back to the podcast. I'm Robert Bosley, one of your two co hosts and here with me, Sharona Krinsky, your other co host. How are you doing today Sharona?

[00:01:31] Sharona: Well, I am so close to the end of the semester that I can taste it. I am so ready to go on break. How about you?

[00:01:40] Bosley: Yeah, I'm the same way. I'm wrapping up grades now. I'm about to finalize one of my classes and then a little bit of grading on another and I'm done.

[00:01:51] Sharona: Exactly. Well, we are going to go on holiday break ourselves. This is our first time getting a real longer break since we started the podcast. So we wanted to let everyone know that we're going to be doing some repeat episodes over the next few weeks. And I think it's a good time for that.

[00:02:09] Bosley: Yeah, but we didn't just pull some random ones. We were actually pretty intentional in which ones were repeating. This episode is going to be repeating our grading architecture one. Why did we choose this one Sharona?

[00:02:22] Sharona: It's fascinating to me that we have this course that we've been doing alternative grading in since 2018. And every semester we revisit the grading architecture or some aspect of it. Last semester it really was the calendar, but this next semester we're going to be looking, really taking a look at those four decisions again. So I felt it was an appropriate time for everyone who's wrapping up to take a minute and really go back and think, did that do what you wanted it to do?

[00:02:53] Bosley: Yeah, I think that's a common practice and a good sign of any good teacher. We're always looking to better ourselves, to better our pedagogy, to better our classes, and this is just part of it. So we hope you guys all are enjoying your breaks and have great holiday season and enjoy revisiting one of our early episodes.

[00:03:15] Sharona: We'll see you guys soon.

[00:03:16] Bosley: On today's episode, we're going to be looking at what we call the grading architecture, which, combined with the four pillars, is kind of the nuts and bolts of how you set up your course for alternative grading. So, Sharona, what do we mean when we say grading architecture? What is that?

[00:03:38] Sharona: Well, thanks for the question, Boz. I really like this question because I think, as far as I know, I'm the one who started using the term grading architecture. What we mean by grading architecture is the decisions that you make, the structure of the grade for your course.

[00:03:56] Bosley: Yes, since we're not doing points and percentages and averages, how then do you come up with the end term grade?

[00:04:06] Sharona: Exactly. And we have four decisions that we think people need to make to build a grading architecture. But before we get there, we have a fundamental question to ask about the course and about that end of term grade, which is: what is the purpose of the end of term grade?

[00:04:26] We've talked about on the pod that we're assuming that most of us are in a multi level grading environment. That you have to give some form of an end of term grade, whether it's pass fail, A, B, C, D, no credit, whatever it is, it's a multi level grade. What's the purpose of it, in your opinion, of that end of term grade?

[00:04:45] Bosley: That's actually a question that we start a lot of our trainings off with. What is the meaning and what is the purpose? I think the standard answer we get , it's supposed to communicate some sort of level of proficiency or mastery of the course content which is a really generic answer.

[00:05:05] But in this context we need to go actually a little bit deeper than that. Don't we?

[00:05:10] Sharona: I think so and I don't think there's one answer. I think this is another one of those situations where it's not what's the purpose of an end of term grade? But what's the purpose of an end of term grade for this specific class, in this specific institutional context, in the placement of this class in a greater curriculum. So..

[00:05:33] Bosley: Yeah, like so many other things when we're talking about alternative grading and all the different flavors of it or the different degrees of freedom is really looking at your course. What are the end goals? Is your course in a sequence of courses? Is there a prereq? Is it a kind of a capstone of a sequence? Is it an elective? Like what is the purpose of that course?

[00:05:59] Where does it fit into the overall makeup of a student's, not just current education, but future education as well.

[00:06:09] Sharona: Exactly. So, I teach, primarily ,three different classes. I teach a general education statistics class, I teach linear algebra, and I teach history of math, which is a senior, junior senior level elective.

[00:06:22] And I would say the purpose of the end of term grade in those three classes is somewhat different. Although, at the end of the day, most of them, for my purposes, in my institutional context, an end of term grade communicates what proportion of the material in the class has been achieved to a proficiency level.

[00:06:42] So I'm still at the moment, although this may change, an A in my class means proficiency at 90 percent of the material, a B is 80 percent of the material, give or take, a C is 70 percent of the material. So that's kind of the context that I'm in, that I come through. I know that some of the engineering folks we work with a little bit different.

[00:07:03] A C means mastery of a core base content set. That these are the things they absolutely need to be successful in the next class. A B means that not only is the student mastered that core competency material but there's some extra material that they've gotten. And an A means they've gone even more than that.

[00:07:22] Bosley: So in that context, and I would actually argue with you about your linear algebra, but an A actually means that the student Is set up to likely be successful in the next course in the sequence, regardless of which classes we're talking about. Whereas, the statistics, the gen ed statistics class, and the history of math, because it's not in a sequence, the A does mean something a little bit different, even though they're both based on the level of material that the student has shown proficiency in.

[00:08:00] Sharona: Yeah, I would agree with that. And when I think specifically about the statistics course, yes, I want my students to have gotten mastery of 90 percent of the material, but what I am assessing, in the statistics course, is based on the fact that it's a gen ed course that is not a prerequisite.

[00:08:18] So I have more freedom to assess a little bit more of the thinking skills and a little bit less of the strict procedural skills and the course is designed for that.

[00:08:29] Bosley: Yeah. Yeah. This isn't a intro to stats that a computer science major would take as a prereq to some other higher level statistics or analysis kind of course.

[00:08:41] And if it was we would have a different purpose than what we have in that math 1090 class that we both teach in.

[00:08:48] Sharona: Well, and I think that that first question of what's the purpose of the grade actually comes even before you design your learning outcomes. So in a previous episode, we talked about designing learning outcomes, but I choose my learning outcomes in part based on the goal of the course.

[00:09:05] Bosley: Well, let's back up a little bit because we just did kind of jump into the topic of this episode, but I want to make it clear. So when we talk about grading architecture, there are four decisions that make up your grading architecture that you need to go through. However before you can start that, this is only done after your learning targets are, at least the first draft or the first go at it, is done.

[00:09:35] Like this is something you don't jump into before you do your learning targets.

[00:09:40] Sharona: Exactly. And if we want to extend the analogy, I would say that what is the purpose of your grade is sort of the overall, what kind of a house are you building? Is it an apartment building? Is it a house? And then the learning targets are your foundation.

[00:09:56] Bosley: Yeah.

[00:09:57] Sharona: That's where you go. And now we have to make some additional decisions.

[00:10:02] Bosley: Alright, so what is that first decision? And these are actually kind of linear, even though you'll go back and forth in it. We're going to present these in what I think is kind of a necessary linear progression of what questions you ask.

[00:10:18] Sharona: Exactly. So you would try to answer number one first before you go to number two, before you go to number three, before you go to number four. So.

[00:10:27] Bosley: What is that first question?

[00:10:28] Sharona: First one is, since you already have your list of learning targets, how are you going to assess them? How are you going to obtain your evidence of learning for that particular learning target?

[00:10:39] We have an episode coming up shortly where we're really going to dive a lot more deeply into the different kinds of learning outcomes. But essentially, at the end of the day, we're looking for evidence of learning. Are you going to get it from quizzes? Are you going to get it from portfolios, from projects, from tests?

[00:10:53] Is it going to come from conversations with students? How are you going to get that evidence of learning?

[00:10:59] Bosley: Yeah, and again, because our courses can look very different, this can look very different, and you can have multiple ways of doing this assessments. Whether it's going to be projects, whether it's going to be traditional tests and quizzes. Is it going to be portfolios? You need to decide how you're going to assess the learning targets that you've come up with, because that's going to then turn around and help determine some of the next questions.

[00:11:30] Sharona: And in other future episodes, we'll talk a lot about aligning those assessments to your targets, because you really want good evidence for what the student's proficiency is in a specific learning target or learning outcome.

[00:11:43] Bosley: This episode is meant as an overview of the grading architecture, the nuts and bolts of how you do this.

[00:11:49] We will have a single episode on each one of these four questions. And going deeper into why you need to do this, how you can do it, and hopefully even having some more guests on, looking at different ways that people have done it.

[00:12:06] Sharona: So the second decision that we need to look at is what type of a scale will you use in looking at your evidence of learning?

[00:12:16] Bosley: Yeah, well once you've determined how you're going to collect it, a student has done one of these. How do you assess that assignment? How do you assess that project? Now in a traditional way, this is where points come in. This is where the points start to come in, you make it out of ten points and you take off points again, that has all kinds of Issues that we brought up in our episode on what's wrong with traditional grading So if we're not going to use points What are we going to do? How do we do it?

[00:12:52] Sharona: So typically in a lot of the courses that we teach, there are three primary scales. There's what we would call a two point scale, meaning there's a student is proficient or student is not yet proficient. So they've met the expectations or they've not yet met the expectations. That's actually the one that a lot of people use when they're more in a specs graded world.

[00:13:17] You might have a whole list of these learning targets and you have to get each one of the targets on a single assessment to a "yes, you've met it" spot. The second one of these scales is a three level scale and that's the primary one I use. Which is, you've got the top of the scale, which is, you've met expectations.

[00:13:39] You're proficient, or you're complete, or you're done, or you're an emoji, because I use a lot of emojis, but it's one of those things. So you've met it, you've not quite met it, and you either need to revise or retake it. And then the third one is, you're not there yet, or insufficient level of evidence shown.

[00:13:57] You either don't have enough information, or you're really far away from the target, and you're not yet there. So, satisfactory, getting there, and not yet, whatever that means. And then the third one is a four level scale. And that one is typically used when you have two levels of proficiency and two levels of not yet there.

[00:14:19] So you have a meets expectations or meets proficiency and an exceeds level. So it's like a good and a great. And then you have a not quite there and not yet there. And that's a four level scale.

[00:14:34] Bosley: Now, we're calling these proficiency scales, some people might think of them as rubrics, and they are similar, there are some subtle differences, which when we get into this episode that's dedicated just to this question I'm sure we'll explore kind of the subtle differences between proficiency scales and rubrics but for those of you that are listening, if you've never heard proficiency scale and you're thinking to yourself, wow, this sounds a lot like a rubric.

[00:15:00] Yeah, they are very similar and some will even use these interchangeably.

[00:15:05] Sharona: And the reason that we're moving more to this language is a lot of the LMS systems assign a rubric to an assignment. And so we're assigning a proficiency scale to a target, a learning target, and not to an entire assignment.

[00:15:22] So a rubric might be an assembly of multiple proficiency scales because it's on multiple learning targets. That's why we're trying to distinguish this language from itself.

[00:15:33] The other thing I wanted to remind people, Boz, is that one of the big problems with traditional grading is a hundred point scale means a hundred levels of discrimination of quality of work. And research has shown that human beings can't discriminate that finely.

[00:15:50] Bosley: Yeah, we talked quite a bit about that in our what's wrong with traditional grading. That the human brain, with any kind of professionals, we don't separate that many different degrees of precision.

[00:16:06] Sharona: Exactly. So what I would encourage people is don't go more than five levels and that fifth one is really a four level one with an added zero. Zero meaning no evidence provided.

[00:16:20] Bosley: Yeah. And we're not saying that you can't, but if you're going to go, you need to have a good reason why. Even the AP Lit and Langs that used to have a nine level rubric on their writing prompts have even gone down to the five level, which is a zero through four. No evidence. And then a not yet, almost, proficient, and exceeds. So. Not that you can't, but if you're going to go more than five, which is really the zero through four, you need to have real rationale as to why you need those extra levels.

[00:16:56] Sharona: So, Boz, the third question, that I want to kick over to you, is, we've talked about how will evidence of learning be found?

[00:17:04] How will you assess learning targets? What type of proficiency scale? Now, how will the students actually show, ultimately, that they've achieved that learning target.

[00:17:14] Bosley: And what's interesting about this particular part of the architecture is that it's the one that's completely missing in traditional grading.

[00:17:23] I have found especially with new practitioners, this is the hardest one to kind of hammer out because with traditional grading, you put points or percentages on your assignments, and then your end grade is made up of some sort of collection of all of those points or percentages, whether it's a straight average or a weighted average, but you don't have this middle step.

[00:17:46] So this one is unique to grading systems that are not traditional, and that is because we're looking at individual learning targets, how do you know when a student has achieved, proficiency or mastery of that learning target.

[00:18:04] Sharona: And so the first one is your favorite, so I'm going to let you discuss it.

[00:18:08] Bosley: Yeah, the first one is taking the whole body of evidence of however many assessments or assignments you've had on that learning target, and looking at the overall pattern. Definitely putting more weight towards the latter assessments or assignments than the beginning ones, but looking at the whole breadth of evidence that you have.

[00:18:30] And Dr. Thomas Guskey and Ken O'Connor and several others, they've written quite a bit about this. In fact I like to refer to this as the Guskey method. But yeah, looking at and using your professional judgment over any analytical tool or any algorithm and just looking at that pattern.

[00:18:52] Sharona: It's almost like creating a narrative of the student that results in a final grade. So, in one of those first episodes, we talked about Ken O'Connor's parachute packing problem. And there were the three students, one of whom started off strong and just petered out throughout the semester.

[00:19:09] Another student started very weak and just got stronger and stronger and stronger. And another student who was scattered all over the board. And you can come up with a final end grade for each of those students by looking at those patterns, and telling that narrative story.

[00:19:24] Bosley: Yeah. And also, we gave that example to showcase the problem with using an average, because all three of those students mathematically had the same average, even though they have very different stories.

[00:19:36] And if you look at the overall pattern of those three students, it's very easy to see which student deserves the higher grade on that learning target.

[00:19:46] Sharona: Exactly. Now the second method, which is the one that we primarily use, is called N times. Which means that on a certain number of assessments, the students must show proficiency.

[00:19:59] Bosley: Yeah, so let's take our gen ed statistics class as an example. Our students have five attempts to show mastery of any one of the learning targets. We define it as you have to get it twice. Don't care which two times you do it.

[00:20:16] You get it twice, you've shown us that you've proficient enough in that learning target that we can count that learning target towards your grade. So whether it's the first and second attempt, whether it's the fourth and fifth, if it's the second and fourth, doesn't matter.

[00:20:32] It's just a predetermined amount. That is key. It is a predetermined amount of attempts, out of the attempts they have, how many times do they have to get it on that single assignment?

[00:20:44] Sharona: And this is definitely an interesting balance. I know a lot of instructors who've tried one time and they felt they got a lot of false positives.

[00:20:53] Students who just accidentally got it. We tend to use two times for that reason. I know some people who've tried to do three times and then when you do the math, if you're in a 15 week semester, they have to show it three times. You have say, 25 learning outcomes. You're going to have probably five or six assessments.

[00:21:17] So, 25 learning outcomes times 6, that's 150 of them per student that you have to make available. Now you've got a class of 25 or 75 students, it very quickly just gets unwieldy. So most of the time we see 1 or 2, or sometimes you have to show it on different types. Like, maybe you have to show it once on an in class assessment, and once on an untimed assessment, or once in a portfolio place. We've seen that as well.

[00:21:44] Bosley: This also goes back to what I said earlier of why you can't make these decisions until you've got your learning targets. That doesn't mean as soon as you start trying to define your grading architecture that your learning targets are set in stone and you can't change them.

[00:22:00] Of course you can go back and revise, add, subtract, but for your grading architecture to make any sense, you need to have an idea of how many learning targets you have. You need to know kind of what those are. If I only have five or six learning targets, then maybe requiring it three or four times isn't unreasonable, but like you said, if I've got 25 of them, unless every single assessment I do is measuring three or four, five, six of these learning targets at a time. Yeah, I just buried myself in grading.

[00:22:36] Sharona: And that's why we say you have to at least start these grading architecture decisions in order. Because in order to know you're going to do it twice, you've got to know what they're doing it on. I mean, if they're doing big, massive portfolio projects.

[00:22:50] There's only time for so many of those in a semester, so maybe you can't require it twice. Or maybe you're giving daily quizzes, and therefore maybe three, four times is reasonable, because you've got a class that meets five days a week in a K 12 setting. It might be much more reasonable. So these decisions are somewhat dependent.

[00:23:07] Again, you're going to iterate your way through these decisions.

[00:23:10] Bosley: Yeah, you'll go back and forth and make revisions.

[00:23:13] Sharona: And It occurs to me that this also comes down a little bit to philosophy, because one question I often get asked is, if a student shows mastery twice by week five of the semester, and I'm not requiring them to do it again on the final, how am I confident that they still know how to do it?

[00:23:27] And the answer I give is, I'm not confident, but I'm confident that once they've done it once, they can get it again pretty quickly. I mean, the common joke in our field is, when does somebody actually learn calculus? And the answer is the first time they teach it.

[00:23:40] Bosley: Yeah.

[00:23:40] Sharona: So, if we're not holding even ourselves accountable for really learning this material in a way that's going to stick forever and ever and ever, I personally am okay if they got it once, and they need it again, they'll go back and get it again. And I'm okay, even if that's in week five and they don't have it by week 15.

[00:23:59] Bosley: But again, that goes back to what you started this episode off with was what is the purpose of your course? And what is the purpose of the grade of your course? If I'm looking at calculus, knowing it's a foundational course to pretty much any other math course, whether you're math, science or engineering. I want to make sure that those skills are a little bit more hammered in because I know they're going to see it again and again, in physics, they're going to see it in strengths and materials or whatever.

[00:24:29] Whereas our stats course, our gen ed stats course, it's not a prereq for anything. The purpose of this course and the meaning of those grades are different, so that is going to change. I would not do the same grading architecture for a Calc 1 class that I do in this gen ed statistics class.

[00:24:53] Sharona: I completely agree with you. I completely agree. And there's a third common one. Having made the mistake of using this one, I don't like it at all. And this is our podcast, so I get to say that. I do put it in there in part, though, because one of the things we want to talk about in a minute is the way that the tools we have access to is going to influence some of this.

[00:25:14] But the third one's called a decaying average. So this is where you take your proficiency scales, which I don't even like putting numbers on scales at all. I like language and emojis. But you can put numbers on them and then you can do math on them. And so the decaying average basically says the stuff earlier, that's assessed earlier is weighted less.

[00:25:33] And the stuff later is weighted more. So basically as a student. continues to provide evidence, the later evidence is more valuable.

[00:25:41] Bosley: Yeah, which sounds similar to the Guskey method, but with one big difference. In a decaying average, let's say I've been doing well on different assessments throughout the semester, and then the last time you assess it, I woke up. My dog bit me. I don't have any milk for my cinnamon toast crunch. I missed the train. By the time I get to your class to take this test, I am in no state of mind to be able to do this. And I bomb it.

[00:26:11] In the Guskey method, as the professional, you can look at that pattern and you can recognize that was just an outlier, but with the decaying average, that day happens at the first assessment? No problem. That day happens on the last assessment and it will tank the grade.

[00:26:29] Sharona: And as I said, this is one of the ones that's built into a lot of the learning management systems. We call those LMSs. There's some other ones where it's like the highest two grades you can take those two but simple is good, simpler, the better.

[00:26:46] And we have entire episodes coming up on simplification of these systems. So those are the three most common ones. Now, again, I would say that the Guskey method is probably a little bit more of an umbrella form where a lot of people.. By the way, none of these methods, are we saying that only the instructor has to decide.

[00:27:05] Bosley: That's an interesting point. That's right.

[00:27:06] Sharona: Because when you think about some of the more ungrading versions of this stuff, the Guskey method where the student is looking at the overall body of evidence is what I would sort of argue is the fundamental to an ungraded method where they're still assigning a final grade.

[00:27:24] So you can do a lot of these wrap ups, these roll ups of how to achieve proficiency or in a learning target and it can be the student looking at their own work. You could even do it with N times because you can hand out an answer key and have a student self assess their answer against the key and decide whether or not they think it's a proficient answer. So this can go definitely in multiple ways. This isn't just the instructor as sort of God on high.

[00:27:52] Bosley: And what's even, I think, more interesting is not only can the students do it for themselves, but there's a lot of realistic and practical settings where the students can do it for each other. That peer review, that peer editing and assessment.

[00:28:08] Sharona: And it was occurring to me that even the things I don't like, like decaying average is not my favorite. However, what if you are a licensing prep course, say in surveying. We're working with an engineering professor who does the surveying course. It's an option that the final exam is the most important, most weighted thing because it's the thing that happens right before they go to take their licensing exam.

[00:28:32] So it's essentially extra incentivized in the grade in part because you want them to feel the pressure for making sure they're ready because they're going straight into the licensing exam. It's a thought.

[00:28:44] Bosley: Yeah, and it goes back to that original thing. What's the purpose of the grade, of the course and what's the meaning of the grade?

[00:28:51] Yeah, if you're doing some sort of licensing prep or your classes typically were the last class they take before some sort of licensing test, then there might be an argument, there might be a rationale or a reason to want to do a decaying average. So even though it's not one that you and I personally like to do, there are definitely cases where it's probably the most appropriate one to do.

[00:29:17] Sharona: Yeah, and I'm thinking about like a community college context where failing the course is a lot less expensive than failing the licensing exam. So, it's really looking at the full multi dimensionality of the whole thing.

[00:29:33] Okay, so we've done the first three decisions. How are we going to assess an individual learning target? What type of proficiency scale are you going to use? How will your students demonstrate? Like, what evidence do you need for a specific learning target? Now we come to the fourth decision, the final decision, is how is this all going to roll up into a final grade?

[00:29:53] Bosley: In traditional grading this decision is usually pretty mindless, but it's there. You define your A as 90 percent to 100 percent and your B as 80 to 89.9 or whatever. So it's the same kind of idea, but because we don't have points, we don't have percentages, we don't have weight categories.

[00:30:17] How do we then take this collection of learning targets, which ones the students did and didn't get, and determine that in grade, whether it's pass, fail, A, B, C, F, how do we do that?

[00:30:31] Sharona: So we're going to give three main different ones.

[00:30:35] Bosley: We like the number three here.

[00:30:36] Sharona: We do like the number three. Although, I will say, I would add to this one, the ungrading one, which is just in consultation with the student and against a set of final rules, you can do that one. The one that I use the most often is a simple count. So if I have 28 learning outcomes in my linear algebra class, 26 or more of them at a proficiency level is an A.

[00:30:58] 25 is an A minus, 24 is a B plus. So I just go straight down the line with mine. Now why 26 out of 28? Well, honestly, I'm kind of defaulting back to a little bit of a traditional grading system and saying, well, that's 90 percent of the content, give or take. So these are some of the things that maybe I could use a little more examing on?

[00:31:17] But I do like the simple count. This ignores a lot of the nuance, but it does simplify the communication and it gets a lot of buy-in from students.

[00:31:26] Bosley: Yeah, so it doesn't matter which of the learning targets they do and don't, it's just the number of learning targets they get. So like you said, I do a course that's got 15 in it.

[00:31:38] Last semester I did one that had seven in it. So yeah, the A was, with the 15 one, the A is 13. No, that's not quite 90%, but that's because looking at the learning targets that I had in there and what I thought was efficient for defining those letter grades.

[00:32:00] Like, what did an A mean for me in that course? What did I think that A should message? Taking those answers and then looking at my learning targets and going, yeah, they can miss two of these 15 and still have an A.

[00:32:13] Sharona: Exactly. And even in my linear algebra where it's just, I don't care. It's any 26 of them. The reality is in order to get 26 of them, you have to know most of the content and even getting, I think it was 19 maybe to pass, in order to get any of them, you needed the foundational ones.

[00:32:31] So it was fine.

[00:32:32] Bosley: All right, so that's one method. That's just the simple count. What's another way that we could do this?

[00:32:38] Sharona: Another way that this is often done, we've seen it done a lot, and this makes a lot of sense in a lot of contexts, is the bucket method. So you're somehow going to break your learning outcomes up into groups.

[00:32:49] And I've flirted with this at various times. But you could have a distinction between, say, content outcomes and discipline practice outcomes. Those could be buckets. You could bucket things by, well, these learning outcomes are absolutely critical to success in the next course, so they form a bucket, and that's a mandatory bucket.

[00:33:09] And then you can have an expansion bucket of it. And this stuff is really great to have, and if you get it, you're going to get a higher grade. That's another kind of thing you can do.

[00:33:17] Bosley: So, looking at kind of an example of that one, and we've worked with a group of professionals that do it this way, where they have a set of learning targets that they've actually gone and talked to the professors in those later sequence courses, because these are courses that are beginning of a sequence of courses, and asked them, "What is it that our students need to know going into your class to be successful?"

[00:33:44] Okay, those are those skills, so in my class that I'm teaching, my C is defined as you have to have all these foundational ones. And then my B is, you have to have all these foundation and a couple of these extra ones. And then an A is all of these foundations and maybe four or five of these extras.

[00:34:03] So defining your buckets as foundational in some sort of expansion or additional. So that's another way of doing it. A third way.

[00:34:13] Sharona: Well, before you go to that, I was going to say, you've also worked though with some people who do more survey courses, right? And don't they have a different type of bucket?

[00:34:20] Bosley: Yeah. And that's what I was going to bring up and that's breaking it up into content buckets. So again, like you said, I I've seen this a lot with, especially the survey science classes in the high school or the freshman level college ones. I actually did this with last time I taught Algebra 2, where you break up content.

[00:34:41] So, my Algebra 2 book, it's like I had one for graphing. I had one for solving equations, and then so on and so on. And then my C was defined as you had to have so many from this bucket, from this bucket, from this bucket, from this bucket, my B, you had to have so many from..

[00:35:02] And it just define each of those letter grades by how many of the learning targets in each bucket. Again, I see this a lot with the science, especially the survey high school ones or freshman college where they have such a huge breadth of content that they have to cover. They have to cover evolution and genetics and biodomes and all this other stuff.

[00:35:25] So they break those content standards or learning targets into those buckets. And then the C is, okay, I've got three of them in the genetics bucket, so they have to have at least one of those. And I've got five in the biodome, so they have to have at least three of those. And just breaking it up by content that way.

[00:35:47] Sharona: And I think if I ever got to teach Calculus 2 again, I think I would do that because in my mind, Calculus 2, at least the way it's taught at my university, has three content buckets. It has advanced integration techniques, is what I call it. It has sequences and series, and it has motion and space. I would want my students to have at least some proportion of all of those, especially the later it goes on into the semester, the harder other classes get and they can start to slack.

[00:36:16] And I don't want them all to always skip motion in space because that's going to hurt them when they go into their physics class or when they go into some of their other classes.

[00:36:24] Bosley: Yeah. And again, that Calc two class is definitely a course in a sequence They're going to need some of that information, so you're in later courses to be successful.

[00:36:36] So you're right, just having students skip out or blow off one of those later buckets, like what did you say? Motion and something?

[00:36:46] Sharona: Motion in space.

[00:36:47] Bosley: Motion in space could be very much a depriment to them in a physics course later down the road.

[00:36:57] Sharona: Exactly. So then the third way that we typically talk about, though, wrapping up the letter grade is by actually using multiple levels of proficiency on the proficiency scale.

[00:37:09] Bosley: Yeah. So for this one, you really do have to have at least two different levels of proficiency in your proficiency scale when you're talking about your second decision. This method is looking at and defining the A by getting so many goods and so many greats, or so many goods and a maximum of so many non proficient scores.

[00:37:34] So maybe the C is, if I've got ten of these things, you've got to get at least seven threes, or seven goods. And then my B is, maybe you need. eight, but out of those eight, two of them have got to be at the great level and none of them can be at the one level. So you're defining your letter grades based on those different levels of proficiency on the assessments.

[00:38:02] Sharona: And one of the things that we see as people make these decisions and a factor that really needs to be thought about is the grade tracking of all this stuff.

[00:38:15] Bosley: That's actually a really big factor and one that can be overlooked when you're new to this and first trying to put this together. We've talked about some of the big mistakes.

[00:38:27] One of them is just making things too complex. Even if you've got a simplified version, you've got to have a way to track it.

[00:38:35] Sharona: And one thing that I worry about with some of these rubric scores and other kinds of things is whether or not the technological tools that you have access to support them.

[00:38:44] So even something as simple as, well, you need seven at a good level, and then for a B you need eight at a good level of which two have to be great and none can be not yet or whatever it is. Well, trying to check just that on 75 students becomes very hard.

[00:39:02] Bosley: Yeah, and if we're talking about the, K 12 world, we're not talking about 75 students. We're talking 150 to 200 students, and we're talking about likely having to do that two to four times a semester because we have to do interim grades.

[00:39:18] Sharona: Exactly.

[00:39:18] Bosley: In my district, we do it every five weeks in a 20 week semester, so I'm turning in grades eight times in the course of a year, four times in the course of a semester.

[00:39:29] Sharona: So we've already talked about the fact that we're going to have to have a Hacking the Gradebook episode, but this is an area where I'm going to strongly encourage you, the simpler you can keep it the first time, the better. The simple count is the most straightforward way to go. It's very easy to communicate to students.

[00:39:47] Or maybe a count within buckets. Most of the systems that I'm aware of that support this type of grading at all support the simple count or the simple count in buckets. Canvas I know does both of those. I think Schoology probably does those.

[00:40:02] Bosley: Actually, not as much. It does the counts. It's harder to do the buckets. I mean, I've heard of someone saying that they found a way to kind of hack the Schoology LMS to be able to do it. I haven't seen it actually yet. So.

[00:40:17] Sharona: And a lot of the LMS systems don't support it at all. So there are a ton of people out there who've developed technical tools to support grade tracking.

[00:40:26] If you're thinking of doing this with your course, I highly recommend you get in touch with people within the community who might have tackled this exact problem. Because I think all of us at some point have tried Excel, Word, Mail Merge, Google Sheets, Jupyter Notebooks, Python scripts, you name it, we've tried it.

[00:40:44] Bosley: It's great if you can use your LMS system. Or it's great if you have a lot of these technical skills, I've been using Excel for 20 plus years and Google sheets, I wouldn't call myself a master at it, but I'm pretty dadgum proficient at these. But you don't have to be, I mean you can find someone that is, or like you said, we've got a ton of these in our community.

[00:41:12] If you go to slack, if you go to the grading conference, there was just blew me away. Not last year, but the 2022, we had a session on different tools that people had developed and the level of sophistication and ease that it would be to take that and just copy and paste it and use it for yourself.

[00:41:36] So these tools are out there. If you're proficient with some of them and you want to create your own great, if you're not, don't run away. There's still steal.

[00:41:46] Sharona: Yeah. One of our favorite phrases around the alternative grading commuting is stealing the hubcaps. Yep. So we steal everybody's hubcaps.

[00:41:54] Bosley: Oh, my master teacher, when I did my student teaching, love her to death, Miss Whistler. Yeah, she used to tell me that the best teachers are great thieves. Like, anything you see that's good, steal it.

[00:42:06] Sharona: Exactly. Exactly. So, we've had the four pillars. Clearly defined learning outcomes, assessment of mastery, eventual mastery, and helpful feedback.

[00:42:16] We've now had the four decisions of grading architecture, which is how to assess your learning targets, what type of proficiency scale will you use, how will a student show evidence of learning, and how will you wrap it up. These eight things are going to interplay throughout all of these alternative grading conversations.

[00:42:39] Bosley: Yeah, the four pillars is kind of the philosophy behind it, and the grading architecture is kind of the nuts and bolts, but they absolutely go hand in hand. We've already seen how, you'll go back and forth from that first pillar, which again, there's a reason why that's the first pillar is clearly defined learning target. Like this is the key thing. Like you cannot do anything else with before you get those.

[00:43:06] And it's again, one of the three biggest mistakes that I see new practitioners doing is trying to shortcut that step. But yeah, these two things are going to go back and forth with each other. And we're going to be going into much deeper conversations about not just the grading architecture that we've talked about today, but also those four pillars and see how those kind of interact with each other.

[00:43:34] Sharona: So I'm really excited to see what we've got coming up, I think this is going to be a fun journey.

[00:43:39] Please share your thoughts and comments about this episode by commenting on this episode's page on our website, www.thegradingpod.com. Or you can share with us publicly on Twitter, Facebook, or Instagram.

[00:43:53] If you would like to suggest a future topic for the show or would like to be considered as a potential guest for the show, please use the Contact Us form on our website. The Grading podcast is created and produced by Robert Bosley and Sharona Krinsky. The full transcript of this episode is available on our website.