Let's discuss the role of homework (or other effort pieces of learning) in an Alternatively Graded course in more depth. In this episode, Sharona and Boz look at a recent research study about the inclusion or exclusion of a "homeworK' grade in the grading architecture of a high school geometry class and the impact it had on student completion of homework as well as their final grades. We distinguish between "product", "progress", and "process" grades and discuss their potential roles in different types of courses.

Links

Please note - any books linked here are likely Amazon Associates links. Clicking on them and purchasing through them helps support the show. Thanks for your support!

• What Is Standards-Based Grading, and How Does It Work? by Matthew Stone
• Assessing the impact of standards-based grading policy changes on student performance and practice work completion in secondary mathematics by Huey, Silvey, Vaughan and Fisher
• Episode 31 -The Role of “Effort” or “Work” in an Alternative Grading Architecture and the Relationship with Due Dates

Resources

The Center for Grading Reform - seeking to advance education in the United States by supporting effective grading reform at all levels through conferences, educational workshops, professional development, research and scholarship, influencing public policy, and community building.

The Grading Conference - an annual, online conference exploring Alternative Grading in Higher Education & K-12.

Some great resources to educate yourself about Alternative Grading:

• The Grading for Growth Blog
• The Grading Conference
• The Intentional Academia Blog

Recommended Books on Alternative Grading:

• Grading for Growth, by Robert Talbert and David Clark
• Specifications Grading, by Linda Nilsen
• Undoing the Grade, by Jesse Stommel

Follow us on Bluesky, Facebook and Instagram - @thegradingpod. To leave us a comment, please go to our website: www.thegradingpod.com and leave a comment on this episode's page.

If you would like to be considered to be a guest on this show, please reach out using the Contact Us form on our website, www.thegradingpod.com.

All content of this podcast and website are solely the opinions of the hosts and guests and do not necessarily represent the views of California State University Los Angeles or the Los Angeles Unified School District.

Music

Country Rock performed by Lite Saturation, licensed under a Attribution-NonCommercial-NoDerivatives 4.0 International License.

Sharona: So what I want to do is I want to pose that out to our listening community. Number one, what are some other learning targets we could create? Thinking about product, progress, process. And are there certain things that are better handled at a sort of a policy level, either in your course or elsewhere?

Boz: Welcome to the Grading Podcast, where we'll take a critical lens to the methods of assessing students learning from traditional grading to alternative methods of grading. We'll look at how grades impact our classrooms and our students success. I'm Robert Bosley, a high school math teacher, instructional coach, intervention specialist, and instructional designer in the Los Angeles Unified School District and with Cal State LA.

Sharona: And I'm Sharona Krinsky, a math instructor at Cal State Los Angeles, faculty coach, and instructional designer. Whether you work in higher ed or K 12, whatever your discipline is, whether you are a teacher, a coach, or an administrator. This podcast is for you each week. You will get the practical detailed information you need to be able to actually implement effective grading practices in your class and at your institution.

Boz: Hello and welcome back to the podcast. I'm Robert Bosley, one of your two co hosts and with me as always, Sharona Krinsky. How are you doing today, Sharona?

Sharona: I am doing okay. I'm a little, getting a little bit of a slow start today. So I'm going to be counting on you to keep me on task. How are you doing today?

Boz: Well, it's, it's an interesting time for me. So, we've been kind of teasing this for a couple of episodes, but yesterday at the time of this recording yesterday was my final day at Santee after, this was my 20th school year. So the school opened in 05 and I started in 05, but Friday, I said goodbye to my family at Santee.

So I'm a little bit emotional to say the least.

Sharona: Well, and I had a question for you. How much longer have you been in California than you have been at Santee?

Boz: About 14 hours.

Sharona: Okay. . So you got the.

Boz: I was literally in a moving van 14 hours before I started Santee.

Sharona: Wow. Wow. Sounds like you had some pretty nice send offs yesterday though, from your school.

Boz: I did. I had a lot of really cool things. So I think one of the funniest and most unique ones is anyone that knows me knows that I always wear cowboy boots. I don't really wear other shoes.

I've grew up on horses and I'm used to cowboy boots. That's all I wear. So one of the things that the staff did was like a boots for Bosley Day. So a lot of the staff came in boots and they did some other things to mimic the way I'd normally look like. Those that have radios, cause we have to carry radios at my school, they all wore it on their right hip pocket because that's where I wear it.

Or a lot of them had pens stuck between buttons on the shirt, cause that's what I do. So it was really flattering. It was really fun. It was really cute. It was a nice gesture that I was not expecting that I thought was absolutely adorable. And it was funny. I was telling my oldest daughter, Scarlett, about it. And she was both laughing and crying. It was really cute.

Sharona: Well, and I love how personal it was. Like it wasn't, Oh, we're having another goodbye party cause someone's leaving. People planned ahead to do something that was specific to you, and of course I love the alliteration of boots for Bosley.

So I think that's fantastic. What are you doing next? Why'd you leave?

Boz: So I will be a secondary instructional coach with the education transformation office, which is a division at LAUSD that specifically works with our schools that, they're called priority schools, but they're schools that have been identified as having some academic issues, whether it's testing, grades, graduation, all of these different things get combined into identifying priority schools.

So I will be working with a couple of different schools as an instructional coach and assisting with some of their data dives and instructional practices to try to improve student performances.

Sharona: That sounds amazing. Hopefully you'll get to do as much of your equitable grading stuff as possible.

Boz: Yeah, that is something that I'm hoping to bring in one of the schools that I will be working with. I noticed a really interesting piece of data that I think the equitable grading practices might play a key role in, and that is they have a really high graduation rate, but a much lower A through G compliance rate.

So what that means is the A through G are the things that, in California, that the Cal States and UCs look at for college admissions. There's slight differences between graduation and A through Gs, but the biggest difference is for high school graduation, Ds will get it done for graduation, but not for A through G.

So I have not done a deep data dive yet. That's going to be one of my first chosen task. I'm sure there'll be tasks that I have to do before that that aren't completely my choice. But my first chosen project will be really doing a data dive to look at why there's such a difference. Is it the course offerings or is it high D rates?

And if it is high D rates, then yeah, that's going to be my first place I go with alternative grading and what we call in LAUSD EGI or equitable grading.

Sharona: Right. Because for example, in mathematics specifically, I believe the A through G requirements require three years of high school math, correct?

Boz: Three years and through algebra two. So that might be one of the, when I said course offerings, is it the students are getting the three units of math or the 30 units of math, the three courses, but that aren't through Algebra 2 or aren't above Algebra 2.

Sharona: And how many years of math does LA Unified require to graduate? Because I think the state requirement is only two years.

Boz: The state requirement is two, but LAUSD's graduation is actually three.

Sharona: So that's another possibility. They're definitely taking the three years if they're graduating, but they're not necessarily taking the right courses or they may not be getting the C's or better that they need.

Boz: Yeah. And that's what, like I said, that's going to be the data dive that I need to do to find out which one of those two cases is it.

Sharona: Exactly. Well, I'm sure that everyone listening to the podcast is excited to go on this journey with you through your new job. I know that I'm excited to see what happens because I think you're going to do a lot of good.

Boz: So, I hope so.

Sharona: Well, you've got a whole cheering section here on the pod. So speaking of grading, What are we talking about this week?

Boz: Well, we've been talking a lot about some of the changes that this year has brought for you and some of the challenges, and I know in particular, you have one of the groups that you're coordinating that you really are being challenged by and are finding it difficult to really get going with this course.

And I know part of what you've been doing to try to figure out where to go with this, like what this massive hill, what do you tackle first? And I believe to help you do that, you have been doing, you know, going back to the readings. Going back to looking at research. So like any good reflective person will do when you're coming up to a challenge, you go back to the research, what does the research say? Where should we really tackle this?

Sharona: Well, and I was just scanning through my Amazon history recently, and I have three books. Maybe more, in the last month, that I've purchased on Amazon. And that doesn't even include books that I had purchased before that, but that arrived this month. But I don't restrict myself to books.

I also do some Google searches and I do some article searches. And yeah, I ran across one that is very recent, actually. It came out I believe in September,

Sharona: And the title is "what is standards based grading and how does it work?" And, you know, I've read probably 80 million of these articles, but when a new one comes out, I at least like to look at it.

And in this particular article, they're talking about a school district in North Dakota and how they did this big conversion. And we're reading through this and it's interesting and there's a million different topics. So we will link this article in the show notes. But there was one that really struck me because although I believe in the equitable piece of this, I am internally conflicted.

And this is this question of requiring homework, and making it part of the grade, in a mathematics class.

And you're like, well, I'm reading that study too. We both had clicked on that link. Now I do have to admit this is behind a paywall. So we are going to link the public cite of this and does give you some of it, but if you want the full article, you'll need to go through either a university system or one of your other if you have access to the system.

So we'll try to describe it, but it is unfortunately behind the paywall of academic publishing.

Boz: Yeah. Which luckily for both of us having Cal state LA emails allowed us to get to it without actually having to pay, but it is behind a wall. So, yeah.

Sharona: Right. So.

Boz: The data collection is also really recent.

Boz: Townsley, Townsley is linked a few times. Another one of our favorite guests.

Sharona: Right, exactly. And Marzano's in here. So like this is a who's who of links to grading practice research in this introduction. So again, kudos to these authors for that.

Boz: Yeah. The introduction, or what some might call the lit review, is a very well done, very well balanced review.

Sharona: This particular study was looking at a suburban grades eight and nine school in the North central region of the United States. It looks like the district has about 7, 000 students, 7, 300 students, of which 1, 761 are low income and only 700 are English learners. So this is a more high performing school district or a high, it's, it's not quite the same demographics as where you and I teach, let's put it that way.

Boz: Yeah.

Boz: And the difference between these two cohorts besides just the year is that's when they made some grading reform policies. So the first cohort was basically the traditional grading. You're saying no.

Sharona: No it wasn't. They actually had done a lot of their standards based grading work prior to this time. If you look in section 3.3, they implemented elements of standards based grading several years prior. So they had aligned to standards, they had reassessments and they had separate reporting of certain grades, but the one thing that they kept for Cohort 1 is they did have the homework grade as part of the bigger overall grade.

And that's actually the distinction between cohort one and cohort two. Cohort one, 15 percent of the final weighted grade came from their process skills, cohort two, the process skills were completely left out. So we should probably define process skills and these different things before we go on.

Boz: Okay. Yeah, go ahead.

Sharona: One of the things that this article lays out is that there's really three types of grades that are typically conflated. And for this article, when they look at standards based grading, they separate learning, which is what they call a product skill or a product grade improvement, which is a progress grade, and process, which is work habits.

So they have these three different types of things that are typically, in a traditional grading system, mushed together. And they're saying in standards based grading, you would want to separate those. And then once you separate these, what do you do with them? And we've talked about this before, right?

That we have our product learning outcomes, which is usually going to be your content often. You have your progress, which might be growth or improvement from the beginning to the end. And then you have your process, your work habits, your attendance, your timeliness, all of those other things.

Boz: Yeah.

Sharona: So the big change between cohort one and cohort two of this study was that in cohort one, 85 percent of their final grade was based on the product category and 15 percent was on the process category. So homework and other work process skills included in the final grade. And with cohort two, a hundred percent of the grade was on the product and the process was reported separately.

Boz: Okay.

Sharona: So, so they did a good job setting some baselines and showing that the cohorts were very similar mathematically, which was one of my first questions.

Boz: Yeah. The looking at demographics, looking at what did they call them? Map scores or. Right. I'm guessing.

Sharona: There's some sort of map scores, which is a standardized assessment that they used to check that they were similarly situated in terms of their prior math skills and knowledge.

Boz: Yeah. So, so they've done a really good job of setting up comparable groups and showing all of that information. So this is a really well done study.

Sharona: And the size of the cohorts was decent. I think it was like 151, 152 students in each cohort. So it was not tiny.

Boz: Yeah.

Sharona: That being said, demographically, it's definitely not representative of our students, so that's just.

Boz: No, and that's an interesting observation. A lot of times we hear people using that as an excuse not to even look at the research to say, oh, yeah, but those students aren't our students, which drives me nuts, but.

Sharona: Right.

Boz: But it is an interesting observation.

Sharona: And the thing to note is these are grades eight and nine students in geometry. So they are already high performing students. And they indicate that in this article.

These are high performers. These are not mixed level performers, really. So what happened when they did this? What did they see?

Boz: Well, one of the big things that they saw is even though the performances in both groups were still really high, that cohort two, the one that had didn't have the homework in the main grade, performances were lower than cohort one's. Pretty much across the board on all eight units data was collected on.

Which, by the way, I'd missed this the first time, but those eight units were not all eight units of study. They did, because obviously cohort two was the year the pandemic started, they did take out from the data and from the research, in both cohorts, all the units that happened after COVID and the remote learning and everything started.

So these eight units that were reported were the ones that were still done pre pandemic conversion of, and just the huge impact that that had. But yeah, all of them did some units more than others.

And they do go in and show that those differences were sometimes statistically significant and sometimes not statistically significant.

Sharona: But more often than not, they were statistically significant. And even more important, they were statistically significant both before and after reassessment opportunities.

Boz: Yes.

Sharona: So both cohorts had reassessment opportunities and they still had significant differences. between the two.

Boz: So this is trying to hint at, or at least the interpretation I was taking from this, is that taking that homework grading out of the main grade did have an effect on the students performance.

Sharona: And I think it's really important that we hold ourselves to the standard of, as alternative grading practitioners, that just because we don't like the data doesn't mean we ignore the data. I'm not happy, and yet I'm also not surprised that taking homework out of the final grade decreases not only its completion, but also decreases their success, in the content, in the product.

Boz: Yeah, because I don't think there is a single practitioner of alternative grading or traditional grading that would say that you don't need practice to be successful.

Sharona: Right? But how that balances with the equity component then gets tricky. Right?

Boz: Exactly.

Sharona: I was going to go on to the survey results. Are we ready to go there?

Boz: Oh, sure.

Sharona: So in part of what they did, they did do the analysis of the outcomes, but then for cohort two, they designed a survey to check on what were the students self reporting of their completion of homework.

Boz: And not just their self reporting of completion, but their self reported beliefs of some of the homework and purpose and usefulness of it.

Sharona: And also their beliefs of what would they have done if the grading had been different because that's where I think some of the interesting things are. For example when asked about, and they had really good completion rates of the survey, by the way. It was optional, it was anonymous, but 93 percent of the students responded. Which, shock. Because we can't even get like 1 percent of students to respond to a survey. Well, they reflected upon their completion, 23 percent reported completing less work, 27 percent reported doing more work, and 50 percent reported completing about the same.

So if those are accurate statements, which again, this is self reported data, three quarters of the students did the same or more.

Boz: Yeah. Which is really interesting.

Sharona: Cause then that kind of contradicts the, Oh, they did less work and therefore they had poorer grades.

Boz: Exactly.

Sharona: I'm like, Hmm, I don't know about that. And when they were asked how much they typically completed before an assessment, 75 percent of them reporting more than half or almost all, 16 percent reported half and 7 percent reported less than half.

And of course, almost all of them said if the practice work impacted their grade directly, 89% said they would do more.

Boz: Even though they also said.

Sharona: That they were doing the same.

Boz: They didn't, they didn't do less because , it wasn't, which is a really interesting result. I'm not quite sure I understand how that works, but Right.

Sharona: And then a lot of students had a lot of things that it would be interesting for people to read. Statements, "completion grades should be provided stating you have to learn something before you apply it". And that's why homework should be a completion grade. So this brings up something that there's a difference also between having to get the homework right to get the grade on it versus having to do it to get the grade on it. Right?

Because whether or not you include a homework grade in your final grade, I think what many of us object to is having the homework having to be correct as it counts towards the final grade. Cause those are two different things, also.

Boz: I don't think any of us deny that practice is needed, but whether or not that practice should be graded for completion or correction, because I am definitely in the mindset that mistakes allow thinking to happen. But if you're punishing for those mistakes, are you really viewing that as growth mindset and are you really viewing those mistakes as potential places to learn and grow?

Sharona: Exactly. And so, at the end, right before they get to the conclusion, there's a couple of really interesting paragraphs, in my opinion. And one thing I wanted to pause and say, this particular study was specifically done on mathematics, and I think that this idea of having to do homework is highly applicable, first of all, of course, in almost all the STEM fields that I know of. And I am not as familiar with homework questions the way we do them in STEM in other disciplines. But I would suspect that this practice work is critical across the board.

Boz: Yeah. You know, practice is how we get better at something. We try, we fail, we try again.

Sharona: But I wanted to read these two paragraphs, probably not all at once. I'll read the first one and then we can talk about it. But these are both very interesting because what this comes down to is how do you account for effort?

Boz: Yeah.

Sharona: And it says despite research findings from over 100 years of research indicating that teacher's value effort is a key contributor to generating grades, the element of standards based grading that has received the most attention and development relates to academic achievement. So a lot of times when you work to redesign your course, you spend the vast majority of your time developing your product learning outcomes. Your content ones, maybe if you have practice skills, something like that.

But the piece of it that is kind of ignored is, oh, and then they'll just do homework. It's placement, it's effort. Like we've done a lot more. Our triple P, we've thought about student effort in preparing for class, in participating in class and in practicing after class. And we have a thought process on how to tell students this is more important in this point, and this is more important in this other point, but you get to make choices. I don't see that in most other courses, certainly not any that have not done alternative grading.

Boz: Yeah, I would completely agree with that. That if you're using traditional grading and your weighted categories, you don't give a lot of thought to, okay, what's the purpose of this homework assignment?

Is this assignment that you're doing outside of class, is the purpose of it for the students to practice a skill that they've been introduced to? Is the purpose of this to actually get introduced to it? Is it to set something up? Is it to prepare students for an in class discussion where you're really going to build upon the individual knowledge?

What's the purpose of the homework? Instead, it's just work outside of the classroom, so it's homework, so it goes into this one category.

Sharona: But I would argue that even in classes that have switched to alternative grading, and I'm thinking particularly the types of standards based grading that we're seeing at the high school level at this point. Because the number that have converted at the college, although growing, is still pretty small.

And a lot of their learning outcomes are still written that way. But we know from the practice standards, perseverance is something we're actually trying to teach in a math class. Problem solving the process of learning and research supports this. So I haven't seen that much of those.

Boz: No and, I agree. And we've talked about these in some of our learning outcome episodes about the difference between those discrete skills and those bigger overarching. Whether it's from common core and it's looking at the anchor standards or the eight mathematical practices or the science and engineering next gens. All of these are those bigger overarching ones, and yet those are ones that are much harder to really evaluate in an assessment, especially if you're doing a traditional quiz or test type assessment.

Sharona: Exactly. And so this article is essentially pointing out that studies have shown that teachers tend to incorporate effort in final grade computations, both in traditional grading and standards based grading system, regardless of guidance provided by administration. So, administrators who require mathematic teachers to transition to a grading system entirely focused on product performance measurements are also asking teachers to implement assessment policies that are in direct conflict with their own belief systems.

Boz: And this really, when I was reading this, all I could think of, in the back of my head was, you know, the point that Dr. T. J. Hitchman brought up, I should have looked up the episode that was bad podcasting host on us, but he brought up this point that the less a student is intrinsically motivated to learn the subject, the more structure the class needs and vice versa. This kind of inverse proportion or inverse relationship between intrinsic motivation for the class and the structure needed.

Sharona: So that would be episode 32, our interview with TJ Hitchman.

Boz: I keep bringing this up. I've brought this up several times. I should have that freaking episode memorized by now.

Sharona: Yes, you should. And we will link it in the show notes. But yeah, agree because another thing that the lit review, in this particular article we're reading now mentioned, is that intrinsic motivation in mathematics is often set in grades one through four. And that it seems to be tied in multiple ways back and forth with how well they're doing in math at that age. So when you get to college, when you get to high school, which is the ones that we work with, we may or may not be able to affect intrinsic motivation too much in the course of any one course or a sequence. So we do need to think about that and one place where I haven't, and we'll talk about this a little later, is in that progress category, so I want to go there eventually, but not yet.

Boz: Yeah, and, but how do we deal with that structure in the class that we work with together? We do it with the Triple P. It is one of our learning targets. It is geared towards all the stuff that is done to prepare students for assessment of mastery of the other specific standard based learning targets, but that structure, and when I was reading over this, if you're talking about taking all of that out of the grade, I could see why some students maybe wouldn't do it. But with putting a little bit of structure in place and still giving a lot of choice and still having it being a small part of our grading structure, but still a part, gives the structure that the lower motivated students need. Which really got me also thinking, and I think it comes back to some of the questions you were asking before. What other types of learning targets, that might not be skills related to the content, could and should be in a course?

Sharona: Exactly. I mean, for example, the one that's coming to mind right now is we have this mathematical practice standard of perseverance in solving problems. That is one of the eight mathematical practice standards in Common Core. It's one that I have never used because I hadn't figured out how to measure that.

Does perseverance just mean doing the problems and completing them? And it occurs to me that perseverance is a progress skill. You're building your muscle. You started a certain level of perseverance and you want to grow it, but there's no bar of enough. Is there?

Boz: Yeah. I, I, I don't know. Cause I'm not sure how you assess someone's perseverance.

Sharona: I don't know, but it would be fascinating to try to figure it out. Cause what comes to mind is that one gosh darn rollercoaster problem in my son's physics class. We had this god awful roller coaster problem, and it was on a homework where you have infinitely many tries, and the homework does count towards your grade, so he had tried it eight times and he just, he was like, I'm out. I'm done.

I got it correct on the 42nd try. And he looked at me, and he's like, I can't believe you did that. I can't believe. But what was interesting to me is each time, because I'm an advanced learner and I'm very reflective, I learned and thought about something new about the problem each time I did it. I didn't just try random things and threw spaghetti against the wall each time because it would give some feedback and it would say, consider this, then consider this.

Have you thought about this? And so each time I did the problem and I got some of that auto feedback, cause this was auto graded of all things. Right? 42 times.

So it occurs to me that maybe getting perseverance and problem solving as a progress skill and maybe this is a place where a student, this is my tiny little bit of ungrading. Maybe my student starts to assess themselves on how they're doing on their perseverance, with guidance. I don't know.

Boz: See in the out of all the because like I said, I've never had this learning target specifically in my grading architecture because I don't know how to assess it.

But you're right. Maybe this is assessed through students reflections upon their own skills and going back to one of Hattie's high leverage student moves of reflection. And really, I know Joe has talked about he does all kinds of reflections with his students. So maybe that's how you do it.

Sharona: I think there's opportunities for all of us to continue to reflect on the product learning outcomes, progress learning outcomes, process learning outcomes. At the end of the day, I still feel very, very firmly that if you're going to put something in your grade, it has to be tied to a learning outcome. I just feel super strongly about that.

Boz: And I feel just as strong about the fact that if it's important to you and it's important to your style of pedagogy and what you want your students to come out of your class with, it should be a learning target.

Sharona: But you have to just say, what exactly are you hoping they're going to learn? I mean, I remember speaking to somebody in health management who was like, If you don't turn things in in time in health management, you run afoul of government regulation. I'm like, well then, yeah, you should have a learning outcome that says, I can meet government regulations or whatever the thing is because this is not just some, Oh, ho hum, it's people who turns things in on time are more successful and better people and, or some sort of a moral thing. It's no, like if you don't comply with government regulation, you can get shut down. You can lose your job. Like this is real, real stuff. People could die. Like, you know,?

So I just think that we need to really, really think back to those learning targets and then think about the role. I mean, one of the things that also was very interesting that happened this semester, and I've talked about a little bit, is they took that one triple P when they did this traditional grading "pilot" redesign on statistics and they made one learning outcome.

So we have one learning outcome out of 15. What did we decide that was 7 percent of the total learning outcomes. And that was one, and by the way, you can get any grade you want in the course without it because you only need 14 to get an A out of 15.

Boz: And more importantly, you don't have to do all of it. There's choice in there because that triple P is broken down into preparation, participation, and practice, there is a choose your own path, which again, student choice is another one of those Hattie identified high leverage moves giving students choice in their own progress through the class. Gives them self efficacy, improves students' learning.

So giving them that choice and going, okay, here's all the things that are available. Here's how much of it you need to get to at least this amount, how you do it is up to you. Whether you spend more time on the prep, whether you spend more time on the practice, like that's, that's up to you.

Boz: I've already got several students. We're in, what, about to start week 11 of a 16.

Sharona: I think we're coming into week 12, actually.

Sharona: And they're still doing work.

Boz: And yeah, they're still doing work. So not only do I have students that have already gotten the triple P I've got students that are 50 percent over the required amount.

Sharona: And yet it's one, one learning outcome. When they went to traditional, they took that one learning outcome, they made it 35 percent of the weighted grade. They split it. So, 15 percent is on preparation, 10 percent is on participation, 10 percent is on practice. And although some of it is completion, when you don't complete it, it hurts your grade. When you don't get it right, it hurts your grade. Every tiny little thing out of this massive set of things that we did, but we didn't want to take them out because then people could say, well, that's the reason they didn't succeed in the traditional grading is you needed all this preparation and participation and practice, even though on the mastery grading side of it or the alternative graded side, we don't require they do any of it.

They just got to do enough of the whole thing. So I feel I'm just like, Oh, so I am getting so many things now. Things like I got a request yesterday too late for me to deal with it. Can I have the second quiz reopened? I just realized I had to pay my internet bill. And now I've paid it. And can I have an extension?

Well, by the time I saw that, which was this morning, I have instructors already releasing feedback and the answer keys are about to go up. So I can't. Cause the answer keys are out or going to be momentarily.

And I feel terrible for this student because this wasn't a student who doesn't want to do the work. This wasn't a student who is, it's just slacking off. This is a student who couldn't pay his internet bill.

Boz: Yeah.

Sharona: So this is what I'm grappling with there. And I do think that that homework grade or that participation grade or whatever it is, that structure needs to be in place. Because the goal is the vast majority of students probably need to do this stuff. And then an occasional student does not, and we want to accommodate that student as well without hurting the vast majority of students who really do need it. So I think that's a little tricky.

Boz: But I think that was my big takeaway from this, well, two things, from this research. And again, compliments and kudos to the group that did this.

This was a really well designed, mathematically sound research. I really do appreciate that, even though the results maybe aren't saying what we would hope them to say as alternative graders, I do think it does point to that whole thing that Dr. T. J. Hitchman was saying, and that yes, 7th and 8th grade, 8th grade, 9th grade students might need some more structure, but it does bring me back to, and with how we kind of all got here with looking at these courses that you're redesigning, what other things should you have in your learning targets?

Because I really do believe, if it's important, it needs to be a learning target. And like you said, if it's going to be part of the grade, it's got to be part of the learning target. So we kind of have this agreement of how all three of those fall in line to each other.

Sharona: I'm looking at it two ways. First of all, yes, if it's going to be part of the grade, it should be part of the learning outcome. And are there things that could be handled at a course policy level or at a lower level? So for example, I'm looking at precalculus and I'm beginning to reconceptualize it. as a modeling class. That we're really learning the basis of the mathematics that leads to modeling our real world.

It's also a service class. So I have biology majors in there. I have engineering majors in there. I have math majors. I have chemistry majors. One option is to create a, a sort of more global learning target. You know, I can use functions to model the real world, something like that. But then I could allow student choice to say a student raises their hand and says, I am on a biology major, I would like to focus on biology based examples through my stuff. And I could give them assessments that utilize biology. Maybe it emphasizes exponential functions over some of the periodic functions, because that's more useful to biology.

And then another engineering student could raise their hand and say, I'm an engineering, I'm a mechanical engineer, so I need more right triangle trigonometry or something like, or like, or me as the instructor, I would know that, and I could give versions of the assessments related to that.

So, the learning outcome doesn't vary, but the ways in which I assessment could be personalized to students, and there could be student autonomy. That's one option. Or do I differentiate my course that there's different learning outcomes? So maybe not in math, but let's say you're doing a general survey science course that has buckets for life science, physical sciences, and earth science.

And you want to have some number of core learning outcomes, and then you have some other ones to get A's and B's, could you allow students to choose which bucket they get their extras from? So everybody gets the core, but then an engineering student says, I'm going to raise my hand for the engineering track, and you're like, okay, then you need to get the rest of the physical sciences ones.

Boz: See, and it's an interesting thought process, an interesting question to ask. I know, and I, again, I'm coming from a very specific viewpoint. I've never ventured into the kind of like ungrading or contract grading worlds. I'm coming from mathematics where I do see most of the things I teach are in a sequence.

So I do believe that the learning targets, I don't see how I can change those within a single course. That's what I believe as an educator. It's what the students need going forward. I like the idea of what you were saying, though, with maybe doing those changes in those customization in the architecture itself. Not in the learning targets, but in how, okay, this is what you need to get a C everyone needs to do that. Okay. You want a biology focus? Okay. To get the B or the A, you've got to do these different buckets. Engineering? Okay. You've got, so I, that I think is a really interesting idea or the other one where maybe a couple of my learning targets are a little bit broader, like the modeling learning targets where I, as the educator, I'm not specifically coming up with what models to use.

But the students are choosing, okay, I want to, I did really well in the exponential unit, I see how that might be involved in my future wantings of education, so I'm going to model something using exponential. Whereas, oh, I'm an engineer. I can kind of see how modeling with a linear system or something like.

So having a couple of those learning targets where the students have different choices of how they meet that. Kind of like what your history of math students did where they had to do so many projects, but they got to choose which learning targets to address with those specific learning projects.

Sharona: And from there I, yes. And the history of math, again, because it was not in a sequence, that was a great opportunity. The other piece that I'm toying with is what gets handled at a policy level as opposed to a learning target.

That being said, do I make it a learning target? Which we have in statistics. We said precise use of tools. So that has provided an entree point for me with the instructors to say, we have a learning outcome of them using tools. So they have to have opportunities to use tools and they need to use them correctly because we want them to learn how to use them correctly.

Or do we just say we're using calculators and we're just going to do assessments that require them?

Boz: Well see that, and I think that is a unique question cause that's different than the question we were just talking about, which was focused on the students. This is an issue that I think is unique and specific to someone in your kind of situation where you're coordinating a single course with multiple instructors. Which is not unheard of.

I mean, when I say unique, I don't want to make it sound like there's 10 people in the country that deal with this. But in that kind of situation, again, if it's important, as an educator and as some as what you believe in your pedagogical philosophies, if it's important, it should be a learning target.

That's my view period and stop. So yeah, if I was in your shoes, and I'm glad I'm not, cause I would have already been fired. But if I was in your shoes, yeah, that would be one of my learning targets. When I redesigned the learning targets is proper and precise use of tools.

Sharona: You just like being the power behind the throne.

Boz: Do what?

Sharona: You like being the power behind the throne. Yeah, I mean, that's the thing. And this thinking of mine came from another friend of the pod that we've mentioned many times, did a whole episode on her last time of Emily Pitts Donahoe, who was coming up with some AI options for students in her class.

And I was like, okay, is this a policy thing or is this an intentional thing? Are there certain courses where you're like, Hey, this is a course where I really, really, really should be teaching AI use. Then maybe it becomes a learning outcome. Or in her case, there's different ways. Like some people could use AI and they could have some interesting things come out of that and some people don't want to use AI, but she's in a writing course.

Can she provide that flexibility at more of a course policy level as opposed to a learning outcome? I don't have an answer to that question. So what I want to do is I want to pose that out to our listening community. Number one, what are some other learning targets we could create? Thinking about product, progress, process, right?

And are there certain things that are better handled at a sort of a policy level, either in your course or elsewhere? Like, if I'm teaching my own course and I'm not coordinating, I have a policy of, yeah, just use whatever you want. And in most of my courses, we're essentially full open note. I quiz asynchronously.

And my one rule is don't represent someone else's work as your own. So don't let AI do your work for you. Don't let Chegg do your work for you. Do your own work. But other than that. Fine. Use Google, use Desmos, use whatever. That's a policy choice, not a learning outcome choice in that case.

Boz: Oh, I would, I would disagree with you 100%.

Sharona: Okay.

Boz: I would because one of our learning targets is the precise use of tools. And I would argue everything that you just said, whether it's Google, whether it's notes, whether it's whatever, those are all tools. Those are tools of learning and therefore it is fulfilling that learning target,

Sharona: I agree it can, I don't know that I would put that in every course.

Like, I think it's in my linear algebra, I think, I don't remember. It was not in my history of math class. I did not have a learning target on tools.

Boz: You also didn't have assessments that were quiz like that could be open notes. You had projects where it's expected you're doing research.

Sharona: Right. But I guess my point, my point was. I don't know that I would always put that learning target in a course, depending on the type of course, but I might allow the use of all the stuff.

Boz: I would, I would challenge you to come up with a exact situation where it wouldn't be a learning target, but you would still allow it on because again your history of math, that wasn't the case because you didn't have those quizzes, those types of quizzes.

Sharona: I can see if I ever taught some of the more advanced math classes that did have actual exams, but that I still had it be open note. I just don't know that I would care what they used. I can't swear to that. I'd have to really think about it.

But I can just see courses where tool use is not the purpose of the course. That's all I'm saying. You disagree. You think I would end up putting that in. All right. You know what? Listening audience, when Bosley and I disagree, 99 percent of the time, he's correct. So we'll just go with that. I very rarely am correct when we disagree. So.

Boz: I don't know if I'd say that.

Sharona: Anyway, just to wrap this up, cause we are coming to time. I definitely would love to hear from people. What other learning targets have you created? Could you create? Has this created any thoughts of progress that you haven't shared or that you haven't thought about? I do think perseverance is the first one I've come up with and now I'm going to need to go think about that, thanks very much about that. What about you Boz?

Boz: Yeah, I want to pose something kind of related to that. But I'm curious to hear what other structures have you put in place to help your students be successful or what structures did you inadvertently take out that ended up having a negative effect?

Cause I really am curious. And again, that's going back to this research that was basically saying, taking that forced structure of homework out did seem to have a negative effect. And just getting back to , I really do want to, and if someone's done the research already, I'd love to see it, but I'd really like to see this kind of inverse relationship between motivation and structure really play out and really dive into that.

So I'm curious what other structures have you put in place or taken away that have had effects?

Sharona: So I think that's going to wrap it up for this week. I'm excited for some of the interviews we have coming up.

Boz: Oh, we really do have some good ones coming up, don't we?

Sharona: We really do. So I'm very excited about that. And I'm very excited to meet some of the authors of the books that we've been raving about. And I'm excited, we're starting to really ramp up on next year's grading conference.

Boz: Yes, we are. So I want to thank everyone for listening and we'll see you next time.

Sharona: Please share your thoughts and comments about this episode by commenting on this episode's page on our website, www.TheGradingPod.Com. Or you can share with us publicly on Twitter, Facebook, or Instagram. If you would like to suggest a future topic for the show, or would like to be considered as a potential guest for the show, please use the contact us form on our website. The Grading Podcast is created and produced by Robert Bosley and Sharona Krinsky.

The full transcript of this episode is available on our website.

Boz: The views expressed here are those of the host and our guest. These views are not necessarily endorsed by the Cal State system or by the Los Angeles Unified School District.