30 - Listener Request - A Deep Dive into the Weeds of our Grading Architecture

Episode 30 • 6th February 2024 • The Grading Podcast • Sharona Krinsky and Robert Bosley

instructors and: 2000

I'm crazy, but I'm not that crazy. And then I heard Uri Treisman speak at the CSU Innovation and Teaching Symposium at Cal Poly that year. That was also the year that I accidentally ran into Robert Talbert at the same symposium. We had already been connected for a while online, but I didn't know he was giving a flipped learning workshop that day at PolyTeach, and he and I sat and talked for a couple of hours.

And I walked away and said, All right. I've got no choice. I've got to try to convince the powers that be, in particular shout out to Dr. Sylvia Heubach, who I managed to convince.

Bosley: Welcome to the Grading Podcast, where we'll take a critical lens to the methods of assessing students learning. From traditional grading to alternative methods of grading. We'll look at how grades impact our classrooms and our students success. I'm Robert Bosley, a high school math teacher, instructional coach, intervention specialist, and instructional designer in the Los Angeles Unified School District and with Cal State LA.

Sharona: And I'm Sharona Krinsky, a math instructor at Cal State Los Angeles, faculty coach, and instructional designer.

Whether you work in higher ed or K 12, whatever your discipline is, whether you are a teacher, a coach, or an administrator, this podcast is for you. Each week, you will get the practical, detailed information you need to be able to actually implement effective grading practices in your class and at your institution.

Bosley: Hello and welcome back to the podcast. I'm one of your two co hosts, Robert Bosley. And with me as always, Sharona Krinsky. Good morning, Sharona. How are you? And I understand you've got a pretty fun announcement to make.

Sharona: I do. I am doing very well. It's been a little bit of a crazy week. Our semester just started. We were on strike and then we weren't, but the one thing that we have that's really exciting is we now have registration open for the fifth grading conference. Which I can't believe I'm saying the word fifth in there.

Bosley: I was just about to say the same thing. It's hard to believe that we've been in doing that conference now for this will be our fifth annual.

Sharona: It is. It's our fifth annual. It's definitely continuing to grow. The conference started sort of in the STEM fields and it has now branched out.

Bosley: Well, no, to be honest, that first year, it was really a math focus.

Sharona: True, true. It was math.

Bosley: And then we expanded it to, or I shouldn't say we, cause I wasn't part of the organizing committee that first year, you guys decided to then of course, expand it to STEM. And then this year, what are we doing this year with it?

Sharona: So this year we are intentionally going to all higher education fields. So we did that to some degree last year, a little bit organically, but now we are definitely doing more outreach and doing more programming to intentionally hit the gamut of all higher education fields.

And I want to clarify, everyone is welcome. We've definitely had K 12 educators attend, but the organizing committee is all higher education educators, which you are one of those, because you bridge the two worlds. And most of the presentations will be from the perspective of higher education, but I do think it is relevant to K 12 teachers as well. And everybody is welcome to attend.

Bosley: Yeah. And that's one of the nice things about this conference for those that have never attended is it is an online conference. It is completely remote. So we were able to keep costs really low. We're not paying for conference rooms and everything.

So the conference fee is only $50 and there's options to even go cheaper than that. So we want to make it to where as many people can come that want to. You're right, most of the workshop presentations are a little higher ed focused, but it's still great experience.

It's great connections and a lot of our keynotes and stuff. Those sessions really are applicable in both K 12 and higher ed a lot of the times.

Sharona: So I have a feeling that a lot of people that listen to this podcast may already have some familiarity with the grading conference. So there is something else I want to mention, which is our call for papers is going to be coming up soon.

If you want to present something about your journey or your class or an experience you've had with alternative grading, watch for that. Go ahead and submit. There is no additional fee to present or anything like that.

It's going to also be three days for the first time. This year, we are expanding from two days of programming to three days of programming. And the vast majority of the conference sessions are recorded and available to registered people within maybe a week of the conference.

So check it out. It's at www.thegradingconference.com. We're very creative around here with our naming conventions. And we would love to have you register and attend. So, Boz, what are we going to be talking about today?

Bosley: Well I'm really excited about this one. I've been looking forward to doing this one.

We had some other things calendared already, so we hadn't been able to get to it. But a little while back, we got an email from one of our listeners, and I didn't have permission to say his name, so I don't want to say their name yet.

Sharona: Can we say where they're from?

Bosley: One of our listeners from New York city and they really had two recommendations on their emails.

So one of them is to invite people to come on with questions or, if they're new practitioners, any kind of roadblocks that they've come across, which we've talked about once before in one of our early episodes, but yeah, we would absolutely love to have people contacting us at our website, either with voicemails or emails with questions or issues.

And with permission, we'd love to play some of those voice messages and answer them. On one of the episodes. Where can they go to contact us with that, Sharona?

Sharona: So you go to www.thegradingpod.Com and there is a contact us form. If you want to leave us a voice message, you can just record it as a voice memo on your phone and attach it. There's an attachment button on the contact us form on the website. And we would love to respond to those.

Bosley: Yeah. Take a couple of minutes out of each of our episodes and respond to some of those, that would be a lot of fun. So please, if you have any questions or any roadblocks, if you're thinking about making the switch and something's holding you up, let us see if we could help you get past that speed bump. But then the other thing that this listener was asking for is saying that we've talked a little bit about how we got to where we're at, what brought us into alternative grading.

And I'll mentioned a little bit about some of the things that we've done in the past, but this listener was asking if we could actually do an episode where we kind of talk about the evolution, both of our journeys and of our actual grading architecture of our different courses that we do this in. So I think that would be a lot of fun to kind of talk about.

So if you're listening from New York city, sorry, it's taken us so long to do this, but we are excited to get to it.

Sharona: Well, and thank you for taking the time and the detail of your email. We, we greatly appreciate it. So do you want to go first? Do you want me to go first?

Bosley: So because I think mine is a little bit more turns and ups and downs than yours was, because mine wasn't as a direct path. Yeah, I think I'll start a little bit with before we met and started collaborating together on that first statistics class.

My journey didn't really start with alternative grading. It started with, actually really kind of started with PLCs and looking at some of the DuFors' materials and going to one of their conferences.

Sharona: For our higher education listeners, what's a PLC?

Bosley: Sorry. Professional learning communities.

Sharona: Thank you.

Bosley: But that's kind of where my reform started. And in the process of that, I was introduced to Joe Feldman's grading for equity and some of the articles out there that we have talked about on this podcast quite a bit, but one of them that really had an impact was the case against the zero.

Sharona: Is that one by Dr. Guskey?

Bosley: No, he he's got case against the percentage. And if I'm flipping those, I apologize to both, but I believe his is case against the percentage and I'm drawing a blank on the author of case against the zero. We will definitely link these all of these into the show notes though.

But anyways that's what really started my personal reform. And then about a year after that, I was sent to this conference for Illuminate, which is a LMS system. My school wasn't using it as an LMS system, but we were using it as a testing platform. And me and Joe Zeccola, this is actually where we kind of met, even though we had been working together, this is where we became friends and started realizing how much in common we had as educators and as education philosophies.

While there, we learned how to use Illuminate to actually eliminate the zero from a grade book. So even though it wasn't our LMS system, we could try to use the grade book. I mean, we had to hand transfer everything from there to our actual LMS and grade book at the grade time.

But when we came back from that conference, Joe and I, that became our first reform. And we started training our our fellow teachers on it. Really looking at the harm that zeros do mathematically to a grade. And the unfair punishment that a zero can do. And it was nice having Illuminate. It allowed us to show a missing assignment, but not put it in as a 0%. And the ease of doing that is what helped us convince most of our colleagues, to, Oh, this is just a, I can make this custom mark and set it at a 40 percent or a 50 percent.

So that was my first real thing was taking one of the aspects that we've talked a lot about on this podcast. One of the issues with traditional grading in trying to address just that issue.

And I'd done other things along the way. Trying to like drop lowest grade and homework or replace lowest test with your final, if it was higher.

I've done some of those other things that make traditional a little bit more equitable, but really getting rid of that zero and coming up with a mathematical way to account for that was my first big trip into what would eventually lead me to getting rid of points and percentages and averages all together.

But yeah, that was my first one and again, like I said, that's the one that got me and Joe together. That's where our paths really crossed.

And then him and I ended up doing a lot of things together when he was still at Santee. So what was kind of your first into this?

Sharona: I first started teaching when I was in grad school, which was in the nineties. And back then my mom was already a math educator and had been talking a lot about cooperative learning and all of the things and what is now active learning. So I had tried a lot of things in my classrooms to try to have a better experience, let's say.

And it was fine. Like I got some active learning. I got some things like that, but it wasn't great. And I did the same things. If your final grade is higher and yada, yada, yada. And then I stepped away from teaching for a long time.

came back to the classroom in: 2015

I have three courses I have not taught. I don't even know what goes in them. How am I going to form a syllabus? How am I going to lay my schedule out? So I went searching online. Just to find existing courses and materials. I was looking primarily for active stuff, like inquiry based calculus, things like that.

And I ran across a couple of blogs. So I ran across Kate Owens, who had been blogging about this, and Josh Bowman. So I kind of stumbled into the whole thing, and of course thought, this is fantastic! Let me go ahead and design three courses in three weeks to use a completely new grading system.

No idea what I'm doing. Off of a blog post. So there weren't conferences and books. I mean, there was a book, but I didn't know about it yet. Specifications grading. Actually, I think it came out right around that time. And I just jumped off the deep end. Literally.

I went whole hog in three new courses. With no real sense of what I was getting into, and I think I was teaching actually four classes, even though, no, it was three classes in a workshop, actually.

And honestly, looking back on it, I'm kind of impressed that I got as much of it as I managed to shoehorn in, in such a short time. So, for example, I went to rubrics, never having used a rubric in a math class. I had a well, five level, including the nothing done. So one through four, I had reassessments and I had a grading architecture that was based on multiple levels of success.

So I had a four level rubric with two levels of mastery, two levels of not mastery. And you had to have so many completed and so many of them had to be at a four and you couldn't have any below a three in order to get an A. So.

Bosley: So, so you were using that wrap up part of the architecture, which was based on the rubric scores, which is one of the ways that we talked about in our, what was that, episode five?

Sharona: The grading architecture.

Bosley: Designing your architecture.

Sharona: So just to be really specific, there were 47 course standards in my pre calculus class. You needed to have fours on 42 of them and the other five should not be below a three.

Bosley: Okay, that, that sounds like, like that was a pretty high bar.

Sharona: Right. You also had to have an A on your homework grade and you had to have a pass on two discussion team problems.

Bosley: So you had a nice complex?

and you are one of my former: 1040

Bosley: See, that sounds like a class I would have never, I would have never gotten an A in that class. Like, there's no way I could have done that kind of homework.

Sharona: You wouldn't have passed because you also, for a 51 homework assignments with a score of 80 percent or better.

Bosley: Yeah. I don't know. Which class was this, calc 2?

Sharona: No, this is pre calc.

Bosley: Oh yeah. There's no way. I would have never gotten that much homework done. But it is interesting that you were talking about rubrics, and the fact that math teachers..

Sharona: Or what we call proficiency scales now.

Bosley: Yeah. But as math educators, I don't think we're as familiar, or at least as familiar as a lot of our colleagues when it comes to rubrics, but I remember the first time that I was really using and forced to use rubrics and thinking about what I'm actually grading and what I'm actually taking points away for was in the K 12 world. If you're a K 12 er and you're in one of the Common Core states, I'm in California, we are, so we have these tests that we call SBAC. I don't know if they're called SBAC in all of the Common Core states, but it's the end of the year state exam. Well, we have a practice one, there's a couple of different versions of practice ones, we call them IABs or ICAs.

Well, the first year those came out, my school gave them. And me, at the time, was the department chair. I was tasked with leading a lot of this stuff. And part of that ICA is there's a lot of it that has to be hand scored. But it was based on rubrics.

So we had these, and learning myself and having to teach my then department of a pretty good size, how to grade a math problem using a rubric and how to really focus in on, okay, here's what that proficiency scale for a one says. Here's what it says for a two. Yes, the student made a lot of little mistakes, but look at the language of the proficiency scale.

Does those mistakes really knock it from a two down to a one or a one down to a zero. So that was really my first time learning how to use a rubric. Which of course then does become a big part of my grading architecture now and the letting go of feeling like if I'm grading someone, if someone's doing a some sort of assessment for me on a learning target that has nothing to do with sign mistakes, and they make a sign mistake, are they really showing me that they don't know how to do, you know, do they really not know how to do simplification of rational functions? So that was another one of those like side contributors that helped me get to where my grading architecture is now.

Sharona: So I'm looking at my syllabus cause I have it. I'm glad that I have it. I can look at it. It's amazing to see the things that I did manage to do in my intentionality. And then the things that I feel I would not do again.

I did have a proficiency scale. It was a four level scale, which I think is fine. I don't use it anymore, but I don't have any problem with two levels of meeting and exceeding expectations and two levels of not. The challenge that I had here was reassessments were available, but there were some times when they were also automatic.

So a standard might show up multiple times on things that they had to take. So I had some, I still had quizzes in midterms and they weren't perfectly aligned. The student's score on the particular learning outcome was the average of their two most recent.

So you could be penalized if you had a late problem, if you didn't do well on something you had done well on.

Bosley: Yeah. You had one of those days where you just came in on assessment day and you were in a car wreck on the way up there. You got some bad news. You just, for whatever reason, you weren't in the headspace of taking that assessment and you messed up on things you actually have already proven. You really know how to do, you just had a bad day.

Sharona: Right, so they could take additional reassessments in my office, in my office hours. And what this did was, especially in the later midterm, if they did poorly on the midterm, it tripped a bunch of reassessments on my part and they'd have to reassess multiple times to push that bad one out to being the third most recent.

So it wasn't like you could just do well, do poorly and do well and you were good. You had to do well, do poorly and then do well two more times to push that, that one that you did poorly on really late, all the way to the third time.

Bosley: So you basically would have to get two in a row that were good. So if I went one good, one bad, and then another good, but then another bad, like I, that would, doesn't matter how many times I was getting it good.

If I didn't get it twice in a row, I was in trouble. Is that what you're saying?

Sharona: Yep. Yep. I had midterm, side quizzes, and I had the final. Which was mandatory and cumulative. So it made the final..

Bosley: You still had that, you still had that high stake final and..

Sharona: Very high stakes final.

Bosley: And again, if I was in your class and I had done great on your assessments all the way up, but that final, just from the stress, cause I don't know. Maybe I had an English final right before that one and then was all worried about that final and then came in and just screwed up yours, but.

Sharona: And then tracking this was a nightmare. So this was before I had a mastery grade book. This was absolutely Excel with complicated averaging formulas and a lot of handwork because there was no way to tell Excel to take the two most recent and average it, at least not that I'm aware of.

So that had to be done by hand. So this became a tracking nightmare, and it became a reassessment nightmare, and the level of attainment needed to get these A's and these B's, I'm sure I adjusted it during the semester, but looking just at the syllabus, I would have been like, I'm out. Like this bar is so high.

Also keeping in mind, this is a pre calculus course. It's really designed primarily for engineering students who come in to our university not considered ready for calculus, but who don't have a full year available to do a full year pre calculus. So they are cramming in this pre calculus at a very fast pace.

So this is a 16 week semester with 47 course standards. Now some of those course standards, to be fair, are the mathematical practices standards. They're not all calculus.

Bosley: It still sounds like a lot.

Sharona: It was a lot. So that was my first attempt and I thought, this is good. Like as a mathematician, I'm like, everything's balanced.

There's all this structure. This is good. Right. Looking back on it now, I was like, wow, I'm going to get those edge cases of people who don't do the homework or maybe they have, they don't have all 57 homework assignments at an 80 percent or better. Maybe they have 49 of them at a hundred percent.

Well, that's not passing. So I'm going to take a student who got 45 course standards complete and 49 out of 57 homeworks perfect and fail them? Looking at it now, this is ridiculous. Now, I'm pretty sure I already had my out, which is I could always raise the grade. I've always had the out of I could make a better grade. But I'm looking at it going, if I were a student, this would be very discouraging.

Bosley: So that that's interesting. Cause that brings up like three points that we have made repeatedly on this podcast. First one being, keep your grading architecture simple. I mean, that's the biggest advice. I think it was Dr. Kate Owens that came on and said her three biggest pieces of advice to anyone was keep it simple, keep it simple and keep it simple.

So that this is a prime example of you had both requirements of numbers, but also a proficiency scores. Some had to be at least fours, others could be no less than twos. You had this another level of, okay, here's also the homework that's going on top of that. Plus you had these other things.

So it definitely sounds fairly complex and really three things can happen when you have a grading architecture that is that complex. First, it can end up burying you at the end with grading. I don't know if that happened to you in this one, but it's definitely,

Sharona: Abso-fricking-lutely..

Bosley: Did it? Just, you didn't say that that happened and just by listening to the description, I could tell. Yeah, that was a real possibility. Second possibility is, just like you said, a student looking at this and going, hell no, just dropping the class. And then the third one was the other thing you just brought up. Those weird edge cases. Oh, no one could get mastery on this stuff without doing the homework.

So of course they have to do the homework and they should do it well. Well, then what, like you said, what happens to the person that did really well on all the assessments, did great on most of the homework, but just didn't quite do enough. Do you really fail that student?

Sharona: And then on top of it, like this is, okay, this feels awful. I'm reading these syllabi and I'm like, so I'm going to read you a sentence that's in this syllabus. And you're going to just reach through the computer, cause we're in the virtual studio, and hit me. Ready? It says, "in general, if you are not sick enough to see a doctor on the day of a quiz or a midterm, then you are not sick enough to miss the exam."

Oh. Ow. Oh.

Bosley: Yes, because there is nothing more important in your life than my class.

Sharona: Oh lordy. Ouch. And I'm actually admitting this on air.

Bosley: Yeah. Okay. One of our, still to this day, one of our most popular keynotes at the grading conference was when Dr. Robert Talbert went in and opened up some of his old stuff and said just how bad some of his first iterations of alternative grading was.

Sharona: So I want everyone to be as nice to me, as they were to him. Okay, we ready to move on to iteration two because I've got the next one. Ready?

Bosley: Well, there is a couple other things though. I wanted to point out about this one. You said you were using a four level rubric, actually five. If you included the zero which you don't anymore, but, the big reason you were is you were actually using that as part of your wrap up. I mean, that was part of the architecture of your wrap up which again, I like the simplicity of a two or three level rubric, but you could have as many levels as you need on those proficiency scales as you're grading architecture are like, if there is a reason for it, you should have it.

You had a reason to have the three and the four at the time. So that's why you should have it. You've gotten rid of that reason in your linear algebra class. So it makes sense now that you only use a three level rubric instead of a four level.

Sharona: Well, and that's something I've definitely been grappling with.

Now that I've gotten to teach a more senior level class, like history of math is that linear algebra, calculus, I know there's a push for creativity, but the bulk of the class is really relatively well set within some broad parameters. That there's things that there's specific technical skills and technical knowledge that we're trying to convey to students so that they are ready for the next class.

And I personally think that there's a limit in terms of how well you can get a problem correct. Like a lot of the problems that I'm doing or that I'm giving to students in calculus and linear algebra have an answer that, within certain parameters, is correct and not correct mathematically. And so in those courses, I'm not comfortable saying, well, you managed to do it in three steps, so that's better than someone else who managed to do it in four steps or something like that.

So I am not comfortable on good to great. But something like history of math, where there's so much more flexibility and someone could really do this amazing, almost research paper level project versus some other project, then maybe yes, or an engineering design class or an art class where there really is a distinction between multiple levels of success, I would put it in.

Absolutely.

Bosley: Yeah.

Sharona: Okay. So that was iteration one and keep in mind, that was the pre calculus course. If I look at probably a better comparison to the next semester would be maybe one of the calculus courses. And just to pull that one up real quick,

Bosley: Well, as you're pulling that one up, I kind of want to talk about what, even though I didn't know at the time, what really was my first attempt at something that had the ideas of alternative grading.

Sharona: OK, go for it.

Bosley: And that was, I've talked about this before, this is actually where we met, was through this program called SLAM. I was one of the two original instructors. You came on, I think year three of it, or maybe year four.

harona: I came in the fall of: 2016

Bosley: The idea of this was in the fall semester, there was a co taught class, a dual enrollment between Dr. Kristen Webster and myself but the spring semester, I still had the same students, I kind of had an open rein of what to do that second semester. The college part, Dr. Webster wasn't part of that anymore. It's the dual enrollment was just the fall.

So the first year we basically tried to turn it into an AP stats class, which was an absolute disaster. I actually, one of those original students now works at my school. I'd love to actually have her on and just talk about how horrible that first year, second semester.

But the second year, the second semester of the second year, we tried to do this research project. I had my students doing a, what was very similar to a college grad level research project. A literature review, problem statement, like actually designing a way to collect the data.

And basically these groups, and then that's the whole semester. That was the culminating project for the entire semester. And as we went, students would turn in chapters and I'd give feedback on it. Turn it back to him saying, hey, here's some feedback if you want to make it better, but this is this is good enough if you want to move to your next chapter. Or handing it back going, here's some real revisions. Here's some suggested things. And no you need to give this back to me in a week because you can't go on to the next one. So this idea of, without knowing it, I was doing a type of specs based grading.

Now because I was still using points and percentages and I still had to give points, they had these weird assignments, these like check in assignments and stuff so I could give them points and things all throughout the weeks and then the end project. Once they did turn it in after they'd had approval from each of the different chapters, I did go in and give points for the different chapters and stuff. So it was traditionally graded, but it had, without me knowing it, it had kind of the foundation of a specs-based graded class.

Sharona: It's amazing that we all sort of started to converge on this from various angles and various points.

Bosley: Yeah.

Sharona: In part, because it was such a different class that you kind of had to think of a way to do it.

Bosley: That and we had, it was year two. So we had already seen some success with doing things in a non traditional way. This was the idea of let's do a dual enrollment math class where they're doing the exact same thing they're doing in the college level.

These weren't the high level students. These were students that were specifically chosen because they wanted to go to college, but likely would have had to have taken those remedial courses that no longer exist but that used to be a huge barrier to our students here in California, in the UCs and Cal State systems.

Sharona: Exactly.

Bosley: So, yeah, we were taking students that were not your best students, not your best math students. Again, college bound, but not STEM. Most of them would admit that they, math was their least favorite, and we were showing that, no. You don't have to do all this silly remedial stuff and there is alternative pathways.

So yeah, it had given me kind of permission mentally to say, you know what? The heck with the normal playbook, let's try something different.

in the spring of: 2017

the stats redesign during the: 2017

Fours and threes and all the same policies and all that kind of stuff. From that semester to the fall, I made significant revisions to the calculus 2. So instead of having just this massive number of learning outcomes, I actually went up, I went from 38 to 40, but 10 of those were the mathematical practice standards and those were now articulated and separated out.

So I went from all of them sort of being mixed together to basically a bucketing system. And so I made it more complex, basically. A lot of the principles from the original one were there, but I made it a little more complex because I started separating them. So I had now three kinds of standards, I had basic content standards, I had advanced content standards, and I had mathematical practice standards.

So, for example, to get an A, you needed to have fours on 19 of the 21 basic standards, seven of the nine advanced standards, nine of the 10 practice standards and nothing be below a three. So that came out to 26, 35. Is that right? 35 of the 40 standards had to be fours, but it couldn't be any 35. It had to be a specific spread.

So the A was the same or harder. The C was I would say significantly easier. Because in the previous semester, you needed 30 out of 38 to get threes. For the C you needed now 26 out of 40. So the A stayed high..

Bosley: Did you still have, but did you still have like specific from the different buckets? Like you had to have so many of the basics. Okay.

Sharona: Yes. You needed 16 of the 21 basics, 3 of the 9 advanced and 7 of the 10 process or practice standards. You still needed certain percentages of homework grades. I think I made those a little bit easier. You still needed pass, no pass. And now, though, I had, instead of calling them midterms, I had Uber quizzes.

So quizzes and Uber quizzes. I started making different names for that. And the final exam now became choose your own adventure. So you didn't have to take a cumulative final that tested everything. And I still had continued mastery. I believe at this point, though, I went to a decaying average. So now instead of having to have the two most recent, the most recent one was 65 percent of the weight, all of them before that were like 35 percent and something like that.

And again, I was using Excel spreadsheets to track all this. I was using mail merges to generate grade reports for the students. So it was still pretty complex. In fact, it was probably more complex now.

Bosley: It's interesting because another thing that we've talked a lot about on the podcast is the importance of utilizing the tools you have available that and it sounds like you were using a lot of tools that were outside tools, like you were having to use your own generated Excel programs.

At this point,

Sharona: we were still on Moodle. So I don't even know if Moodle had a grade book. Like, I couldn't tell you today. I'm sure it did. I'm sure it does. No clue. No clue. So then I went from that, and I want to skip through this a little bit because I want to get to the stats class. Again, the next semester, so now I'm starting my redesigns in the statistics, but I really haven't made that huge jump.

So it doesn't change a lot, other than I cut from 40 to 35 standards. So now I'm starting to cut my number of learning outcomes, but I still have these buckets. I did away with advanced and basic. So I went to advanced and basic the one semester and then I thought, Nope, we're getting rid of that. I've now started the simplification process.

So I got to like my most complex and then I started simplifying still using mail merges. I'm still using decaying averages. I never really, I don't think, told the students what those were. And the other thing, I don't know if you remember, because you helped me with this is getting, once I finally switched to canvas, getting canvas to display the correct things required some advanced math.

I think we did that more on the statistics class, though. When you do a decaying average, you have to set a bar for canvas above which it'll mark it as mastered. And the way decaying averages work, we had to figure out looking at all kinds of different scenarios on a four level rubric, what things would trigger mastery.

at's where I was as of spring: 2018

Bosley: That's when we redesigned and tried to do what we were calling mastery grading then with this SLAM statistics class.

So, you were talking about the fact that you got tasked to redesign this course at Cal State. Part of the way you redesigned it was from things that we were actually learning in this dual enrollment program called SLAM. So thank you College Bridge. A lot of who knows where our past would have necessarily gone had they not been there to really bring us the two of us together on this.

Sharona: Exactly.

Bosley: But yeah, we started to try to redesign and we're like, okay, let's bring in this alternative form of grading that we just called mastery grading back then.

Sharona: Exactly. And this came about, and I think I've talked about this in one of our other episodes, because I wasn't going to do this with the stats redesign.

instructors and: 2000

That was also the year that I accidentally ran into Robert Talbert at the same symposium. We had already been connected for a while online. But I didn't know he was giving a flipped learning workshop that day at Poly teach. And he and I sat and talked for a couple of hours and I walked away and said, all right, I've got no choice.

I've got to try to convince the powers that be, particular shout out to Dr. Silvia Heubach, who I managed to convince, and said, all right, we're going to do this. We're going to launch this redesigned course in an alternatively graded format. That was interesting.

Bosley: Yeah. So let's talk about, so when we started this what was some of our grading architecture of that first iteration, because out of all of the classes, I think this is the one that at least looks like it has changed the most from first iteration to current iteration.

Sharona: Right. So the first thing to understand is I went just now and looked at the old syllabus. And the syllabus says nothing. Literally nothing. It says, why are you in this course? When does it meet? What are the quote student learning outcomes, which were the ones given by the department? Okay. They actually aren't terrible in that they're all action verbs, but yeah, they're not good.

And then there's an academic honesty statement and a student responsibility. And that's all, that's it. So I'm thinking that we put all of this on canvas probably? And I'd have to pull up the old one, but we decided to basically have four learning outcomes if I recall correctly.

Bosley: No, no, no, no. Well, first we didn't really call them learning outcomes.

Sharona: No, we had buckets.

Bosley: We had, yeah, we were going to break all of our, actually it wouldn't even skills. We were breaking the content into three things, three buckets, the basics of statistics, descriptive statistics and inferential statistics. Those were our three buckets. Those aren't learning art outcomes or learning targets, those were our buckets.

And basically everything that we wanted to do because we were still using like traditional points based grading on assignments and assessments. But the first two or three chapters or two or three units, we're going into the basic statistics. Our next two units, we're going into the descriptive where we talked about what, the data summaries and five points, five points.

Sharona: No, it was worse than that. It was statistical inference, data production and data analysis. Those were the three. I just pulled it up.

Bosley: Okay.

Sharona: Whatever the heck that means.

Bosley: Yeah. We had, we, we had those three things and still not really well defined learning targets.

We had units and things were going into those and they were very much broken up in a timeline. So like all the inferential stuff was at the end and that was the third unit or the third bucket. So, yeah, we had those and then how did we. What were we doing inside of those? Those kind of buckets.

Sharona: Oh, my gosh.

So I just pulled up the grading tools that we built there. We have elaborate. We have. Well, we had. No, it's worse than that. We have these elaborate spreadsheets where each faculty member had a spreadsheet that had a tab for every student. And you had to, it was, essentially it was a weird conglomeration of specs and standards.

So like, we graded everything, at least the one I'm looking at, at a 2, a 1, or a 0, but on a whole bunch of details. So we had like, for instance, conceptual knowledge, right? Identifies the problem as a proportion problem and uses P to state the null and alternative. And the student got a one out of two. And then we would add these things up.

And so there were like one, two, three, four, five, six, seven line items in this one thing, okay? Add it up. That gave a number, internally, the students never saw this. And then we would convert that number to a four level score, E, M N or WB. Like the math behind this, I literally, I could recreate it if I tried, but it was insane.

And so all the students really saw is they saw the four level rubric grade, but they had no idea how it was computed, they didn't understand what they did right, they did wrong, what a nightmare.

Bosley: Okay, so I'm actually looking and I just realized we're talking about two different classes.

Sharona: Well, which one are you talking about?

Bosley: Before you did this at, before you did the redesign at Cal State, we actually did this with the SLAM class, with just the SLAM class. And we had, we did have, these three different groups that, so I don't know if it was the semester before, or the year before, but yeah, we had a slightly different grade book and we did have decaying average, and we had to mess around and figure out, it took us a while to figure out, okay, we're doing a decaying average, where do we, and how do we want these three different things, because we do, we want decay and average.

And then we averaged those three groups to get the final grade.

nking maybe you're looking at: 2019

This. We really, in a way, it was worse than the stuff I did in the calculus because..

Bosley: Well, for for one reason it didn't have the specific learning targets.

Sharona: Yes.

Bosley: Like we still could look at the grade book and I could still say to you, if you were my student, okay, Sharona, you've not done well in the inferential statistics.

I still, I couldn't tell you, okay, you still need to improve on hypothesis testing or you've not reached yet on understanding and demonstrating that you understand what a confident interval is, how to calculate it and what it tells us.

We could do that now. Like that, I mean, that's the conversations we have. We couldn't do that then because we still had these huge buckets without well defined learning targets, which it sounded like your calc and your pre calc classes at least had that. Which does make it infinitely better than what we did the first time in the statistics class.

Sharona: Yeah, I think because we didn't, we were designing a course that kind of didn't exist. The statistics course is not an intro stats class. It is a, what we call a quantitative reasoning with statistics. And our goal for the course was to create a critical consumer of data. So I looking at it now, I'm going to say that without intentionally doing it, it was more of a specifications graded course, but a poorly designed one. Because we were not using, so first of all, the four pillars didn't exist yet as a conceptual construct.

It came out later after we had done this redesign. And we knew we wanted to grade on evidence of learning. But you're right. We did not have clearly defined learning outcomes. And that's, I think where this thing really went off the rail.

Bosley: Which is why we try to emphasize as much as possible before you do any of this, that's where it has to start.

If you don't, that doesn't mean that you might not go back and revise them some, but you have to, when you redesign a course, you have to start with the learning targets. And no, it is not just a simple pull from your state standards. If you're a K 12 er or your college student learning outcomes, if you're a higher ed, you really got to examine that.

You really got to spend some time looking at those and designing those. Otherwise, no matter what else you do, it is going to collapse.

I don't know if we did it in: 2017

Bosley: Yeah, I think it was the next iteration. It was the next year.

Sharona: It was the next year. So we started with this weird, I can't even describe it to you, nightmare of a situation with, by the way, 15 or 20 instructors that semester, nobody knows what's going on. I'm trying to push this mastery grading thing.

And it was awful. It was awful. And it still resulted in very high pass rates relative to what had existed when we were in a remediation structure.

Bosley: Yeah.

Sharona: And the instructors did not revolt. So, yeah, looking back, that's the issue, is we didn't have the learning outcomes. They were not clearly defined. Because I'm looking at this list of things they had to do right. It's just all stuff.

Bosley: So lessons learned from your pre calc and your calc classes, keep it simple, stupid. And if you're going to add some of that complexity and again, the bucket method is probably, at least from my experience, especially with my K 12 world, is one of the most popular grading architectural structures.

A lot of people want to do that, especially at the beginning. And I don't think there's anything wrong with that, but be prepared for those edge cases. Be prepared for those oh, wait, I didn't plan for this. And if I go strictly by my architecture, this kid's going to get an F or a D, even though he's mastered majority of the material.

But because of where the two or three things he didn't fell, my grading architecture says I should give this a really low score and don't do it. Like Guskey always says, do not ever give up your professional opinions up to an algorithm or some sort of grading architecture. Use your professional judgment.

And then the lessons learned from the stats class is if your foundation isn't the learning outcomes, it's going to fail.

y clearly defined. So fall of: 2018

And some of our choices are being dictated by what's available on Canvas. So we're starting to play with the tools available and now we have 20 learning outcomes.

That went better. But we found that we were rushing the inferential statistics. So we were equally weighted between descriptive and inferential statistics in the grading, but we needed more time on the descriptive in terms of teaching. And so we were rushing the inferential and that was hurting the students.

So we took a hard look at those learning outcomes and said, well, wait a minute. If this course is about creating critical consumers of data, should we be equally weighted between descriptive and inferential statistics? And we said no. Because even on the inferential statistics side, it's more about them being able to understand and interpret results of inferential statistics than being able to set up and perform inferential statistics tests.

Bosley: Yeah, the understanding of what it means and being able to read it is what we wanted instead of really concerning ourselves with, can a student set up and run a chi squared test or an ANOVA.

rona: Exactly. So for fall of: 2019

ts. And we settled on that in: 2019

Bosley: Yeah, and actually more of our change has been refining the language of some of our learning targets. It's been refining the language of our proficiency scores, defining exactly at what level are we looking for with precision? So you're right.

The grading architecture itself has been fairly stable since then. Now, like we said, we are looking at, we've been doing this long enough that we are looking at some, possibly some changes to that. But yeah, as of right now. Most of our revisions really has been to language of the learning targets and proficiency scales.

Sharona: Well, and the other two things is we've been enhancing our assessments to really uncover the evidence of learning we're looking for. We have been working to scaffold proper practice problems and the structure, like we've been continuing to iterate the backwards design of the course. So that from the moment we start teaching, we have a clear path all the way through when they start assessing a learning target. So that's been happening, but..

Bosley: And we've been finding fun tools to use like CheckIt.

Sharona: Yes. Yes, for sure.

Bosley: Which if you don't know what CheckIt is, go back a few episodes and listen to

Sharona: So I and the other thing that we've really changed is in a lot of those previous iterations, we were using some form of a decaying average.

I which, the way that it's set up, that is one of the things Canvas supports. You can set the weights. The issue I have with the decaying average is that at some level, mistakes are penalized. They never quite completely go away. So my preference is either two highest, two most recent or something like that.

Bosley: Yeah. But not just that, not only are mistakes still penalized. If you make that mistake late, it's devastating.

Sharona: And again, that depends on the course context. If you are preparing for a high stakes final situation or licensing exam, maybe that matters even then, though, given the consequences that we're beginning to see, that happen with bad grades for students.

I know in our context, students who fail may lose their housing and it's the only housing they have. They may become even more food insecure. The consequences are just so horrible that it's really rough to say that any one grade should take someone's housing away, for example. So that's hard, but so I'm not a fan.

Again, that's where personal value systems and there is no, again, there's no one true way. There's no one right way to do this clearly since we did about 18 wrong ways.

Bosley: But like you just said, that's a personal belief that you hold very strong, which alternative grading, when you really examine it, gives you that kind of freedom to bring in those values and those morals.

I mean, that's the same thing that Joe said. The way he has his designed and set up, it's really true to what he really values as an educator and as a writer. And that's what alternative grading does it really allows you, if you take the time to examine it and the time to examine your own beliefs, you really can make it to where it fits you as an educator so much better than anything traditional.

We are kind of running up on time. I do want to make one more plug because the part of the reason that this listener requested this episode was that they mentioned they can find tons and tons of articles about the why of alternative grading, but not as much on the how.

So we've plugged this before. We've had both of these authors on before, but if you are looking for some literature to read that really does focus more on the how to, go check out Grading for Growth. A good chunk of the book is case studies about different courses and exactly how they have things set up very concrete, very logistical, so if you're looking for a more how to book go check that out. We'll make sure we link that in and again we had an episode, one of our early episodes, was talking with both authors. So check that out.

Sharona: Also another place to go, on TheGradingConference.Com website we have links to repositories of different STEM fields, and in there are actual syllabi and detailed materials. Some of them are getting old. You'll find some of my old ones that I just described that I probably should maybe pull those down.

I don't know. But, and I should be adding some new ones, but there are detailed syllabi in those repositories linked on the resources page at TheGradingConference.com..

Bosley: All right. With that being said, thank you for joining us as we take a trip down memory lane, which fun, even though not always a great trip, but thank you guys and we'll see you next time.

Sharona: Thanks everyone.

Please share your thoughts and comments about this episode by commenting on this episode's page on our website www.TheGradingPod.Com or you can share with us publicly on Twitter, Facebook, or Instagram. If you would like to suggest a future topic for the show or would like to be considered as a potential guest for the show, please use the contact us form on our website.

The Grading Podcast is created and produced by Robert Bosley and Sharona Krinsky. The full transcript of this episode is available on our

website.

Bosley: The views expressed here are those of the host and our guest. These views are not necessarily endorsed by the Cal State system or by the Los Angeles Unified School District.

Share Episode

Shownotes

Transcripts

Follow

Links

Chapters

Video

More from YouTube