This week, Andy and I are talking with Macke Raymond, the director of the Center for Research on Education Outcomes (CREDO) regarding CREDO’s As a Matter of Fact: National Charter School Study III, which is the third installment of a multi-decade study examining the academic progress of students enrolled in charter schools compared with those enrolled in traditional public schools.

Here is a link to our conversation where you can listen to the podcast. And for those of you who would prefer a video recording, we provide a link to YouTube as well.

This week some of the topics we discuss include the following:

• The heart of the matter, the meta message, the most important takeaway from the study: How are charters excelling? (00:02:13)
• The study methodology: How you can actually compare students’ performance in different schools? (00:05:13)
• Maintaining balance: Macke’s thoughts about how to frame the progress charter schools have made while also acknowledging areas needing improvement. (00:11:16)
• Macke’s thoughts regarding performance expectations, student learning, society in general, and CREDO’s future. (00:16:25)
• Focusing on other themes in the study: The performance of Native American students and students in the suburbs, as well as how virtual charter schools are performing. (00:18:51)
• Interest levels in the three studies from the media and policy makers and what this may portend for future policy making. (00:26:31)
• Macke’s “aha” and the people who get credit. (00:35:00)

Notes:

• “As a Matter of Fact: The National Charter School Study III,” is the latest CREDO study.
• The Methods and Data Overview describes how CREDO approached the study.

Hey, Andy.

Hey, Jed.

How are you?

I am doing well.

Excited that we're going to have our first guest here at Wonky Folk.

I know, and this is like a poor man's version of Smartless.

We have a mystery guest.

We'll get to that in a second.

So for folks who, this is your first time or you've never watched this podcast, or

you've been in the Taylor Swift concert and lost your memory, and so you need

to be refreshed, this is Wonky Folk.

And with Jed Wallace of Charter Folk and a leading charter consultant.

And I'm Andy Rotherham of Bellwether.

And today though, this will be a Wonky Folk that's going to go deep on charters.

And the reason is because there was a big study that just recently came

out, about charter schools, the third in a series and it's by CREDO

at Stanford and it's important.

And so Jed and I tried to figure out like who would be a really good person

to have on, who incidentally will also be our first guest ever on Wonky Folk.

And we were trying to figure out, who would be a great person to have on to

discuss this study and its implications.

And so we chose no another, and we were delighted to have an

accepted invitation from Macke Raymond, who is the leader.

We figured go straight to the go, straight to the source.

So who better than the lead investigator and the leader of CREDO who's been

on this issue in doing these studies for well more than a decade now.

So, Macke, thank you.

Welcome Macke.

With that kind of buildup, it's have to be downhill from here.

Now we're excited.

I wore my, I don't know if you can see on the screen, I wore my charter shirt that

I only wear for like, really important discussions of things like charter

schools that are generally really tortured conversations and that's why we're excited

that you're here, because you really do a nice job of sort of cutting through

the rhetoric on all sides and getting to the facts of what's actually going on.

Well, I'm delighted to be here.

So I'm playing straight man to Andy Rotherham.

That's a good one.

But Macke, delighted to have you here for sure.

I'm going to take the first question because I do think we're going to focus on

a lot on charter school issues, but what's great also are the implications for your

study as it relates to public education more broadly, all sorts of decision

points, about what society is doing in terms of our commitment and our policy

making as it relates to public education.

So we'll go in a lot of different directions, but I want to start with

kind of the heart of the matter and what are the takeaways from your

standpoint, the most important ones that we should be looking at, as it

relates to the performance of charter schools over a multi-decade timeframe?

There are very few studies that have such a longstanding look at things.

And also you've been through ups and downs.

You've been at your moment when the charter school world

was so frustrated with you.

And there have been also this moment of like, "Hey, can you please do more?"

And it's always a struggle to get the quality data and now

you've been able to do it.

So just very intrigued to hear what your thoughts are about the

most important thing we should be thinking about given that multi-decade

frame that you bring to the work.

Sure.

So great way to start.

Thank you.

So as I look across all of the data from all three of the studies, the

thing that really stands out for me is the fact that there are so many, many

schools in our sample that demonstrate improvement over time in a timeframe

where that wasn't the case for the nation.

And so when I think about what the charter proposition does for schools,

I think it creates the right kind of incentives for the adults in schools

to do the right thing by kids.

And that is, for me, the largest takeaway.

So, go ahead, Jed, please.

Well, I wonder there are so many different findings that

you have within your story.

This idea of a sector that's grown to serve millions of kids, to be

able to improve at that scope, it just seems pretty un unprecedented.

But there are all sorts of sub stories, there are concerns.

"Hey, we are doing as well with white kids.

Hey, we've got our challenges as it relates to special education."

What's the right way to present the study in its totality?

What's the right metaframe while also maintaining, or encouraging the

movement to keep enough integrity to keep focused on the things

that still aren't good enough yet.

And Macke, when you answer that's a really important question, particularly

that last part, like for charter proponents, but also just for folks.

I feel like some folks may be feeling like they're joining

a play in the third act here.

Just talk a little bit about the study, the method, how you do it with the virtual

twins, and talk about this study, what's the timeline we're looking at, thing

like help readers also understand what specifically we're talking about here.

Okay, I'll start by answering Andy's first and then Jed, I

may have to circle back to you and get the question refreshed.

So, I also have to say, as a scholar of education policy in the United States,

I want to put on the table that our team doesn't pick, as an advocate, any

particular policy or program to endorse.

We let the data tell us what the answers are.

And so when we set out to examine the performance of charter schools, we

wanted to know really what was the impact they had on student academic

learning and progress over time.

And so our entire method is focused on what do kids learn.

And in order to do that study there's a lot of concern that you don't get

to compare charter schools to district schools head to head like that because

they might not have the same kids.

So we developed a methodology where we created a very complicated matching

methodology, I'll spare you all the details, but what we end up with is

we end up with essentially twins, the only difference between the twins

is that one student is enrolled in a charter school and the other is the

other example is students who enroll in district schools that the charter school

students otherwise would've attended if they hadn't gone to the charter school.

So very, very tight geographic comparisons and very tight demographic comparisons.

Based on that, we've been doing this for a really long time and we've seen the

results go from really strong negative and statistically significant, which is

what we came up with in 2009, the first time we did the study, to now in the 2023

study, in both subjects reading and math, students in charter schools are learning

more than they are than their peers in traditional district schools are learning

in the space of a year, and we think that trajectory over the three studies

that we've done, is evidence of the fact that schools themselves are improving in

little bits and little bits and little bits, and over time, they actually have

created these greater gains for students.

That's an incredibly important finding.

It's not just a charter school finding, it's an education finding,

and we're really excited that we were able to see that in the data.

Jed's question.

Yeah, I started at C C S A in 2009, and we already knew from a variety of

other data sets that we had some serious quality issues in the movement and

two mandates get better in advocacy.

But the other one find a way for us to begin to nudge the entire

charter school sector toward greater performance and your study really put

in repose, just how important that was.

So my question for you now, going back to the one I've asked before, there are,

there's the meta story, this incredible story of a multi-million student sector

actually being able to improve itself.

But there are sub stories as well.

Where maybe it's not as good or maybe there are sub stories are even better

than the meta story, but, How do you think we both keep the focus on the, I

think, very positive meta story that's happened while also have the integrity

to keep pushing ourselves in those areas.

And everything gets so politicized, it seems to me like charter detractors,

all they want to do is focus on our problems and charter advocates

can err in the other direction.

What's the right way to keep the mix here?

Macke.

Well, I think I can give you the thumbnail answer by pointing

to the title of this third study.

We have named this study as a matter of fact, the National

Charter School Study three.

And the reason that we did that was because of all of the thousands and

thousands of times when we have heard some small slice of opinion or position

about schools that are in the charter sector that just don't hold up under

scrutiny, they don't hold up to the data.

And so what I would say is, along with Andy's t-shirt, it's not

just a completely vanilla story.

I mean, there's very, very strong positive news that we see in the study that we

have brought forward, and we really want to celebrate that, particularly

in this post covid world where the need for evidence-based strategies for

student success has never been higher.

This is good news for everybody, but we also want to be completely honest about

the fact that there are still pockets within the charter school landscape

where there are students that are not learning as much as they should.

There are pockets where schools are not performing as well as they

should and we need to be completely transparent about that, as researchers,

because that's what the data say.

Yeah, I want to talk about those pockets, I

appreciate the t-shirt call out.

I think something that will surprise people in our extensive market

research, people think people are coming to this podcast for

the education policy discussion.

It's actually our fashion choice, as it turns out, is what's

actually driving listenership.

Yeah.

Which was a surprise to us as well.

So you talk about those pockets, like dig in both in terms of like

types of schools, types of kids.

What are some things we should be attentive to?

Because I do think one of the things I've respected about these studies

since the stars, as you said, they don't flinch on the bad news, these aren't

exercises in messaging and so forth.

And so I think that's what makes them valuable and credible over time.

But talk about what we did see some places here where there's obviously

a lot of room for improvement.

Sure.

So let me just say, across the entire student population that we studied,

we saw that there were particular student groups that did not learn

as much as they would have had they gone to traditional public schools.

I would call out here Native American students, even though they constitute

a small percentage, the learning gains that they post are just not anywhere

near what they need to be in order to prepare those students for the life

that they will eventually inherit.

The same is true for students who have special education needs.

They are just not getting the kind of education in charter schools.

And this has been a persistent finding across the studies that

we've done, not just these national studies, but when we drill down

into particular areas, particular geographies, we see this as well.

So this is sort of a collective finding for the entire movement.

And then finally, the one that just continues to distress

are the student results.

For students that are enrolled in virtual charters, the learning that they get

there is so far dramatically under their peers in traditional district schools,

that we just have to call it out.

There's just no way around it.

Yeah, that's been an ongoing issue.

Talk about that, if a virtual school proponent were here to try to give

them their due and steelman their argument, they would say: "Well, like

there's a lot of variants and kids don't go to virtual schools because

they were doing well previously.

They go because they're struggling and they're looking for a fit.

And so they're trying this thing."

Talk about, why do you think that is?

And to the extent you agree with this, why do even the strongest arguments

from virtual supporters fall short?

Well, I want to say, I'm gonna be drawing on not just this study, but our

earlier study of virtual charter schools.

And there we partnered with Mathematica, and also with the Center

for Reinventing Public Education.

So we brought a larger body of knowledge to that study than we

were able to do in this study.

The results continue to be concerning because a lot of the

claims, just are not substantiated.

The claim that the students all come to virtual schools because

they have had X in the past and that X has actually changed over time.

When we first started talking about this, it was that they were bullied.

Then it was because they had some particular categorical learning

challenge, then it's now because they're in post-traumatic stress.

The argument continues to be it's a student-based problem and

therefore we should be off the hook.

And I just want to say, If you're a charter school, your job is to figure

out how to meet kids where they are and bring their learning forward, and

I don't care whether you're talking about a Native American population,

whether you're talking about Hispanics or E L L or Special Ed or white

kids in suburbia, I don't care.

Your job, as a charter school, the reason that you've been given a charter is to do

exactly what I just said, take kids from where they are and move them forward.

And so after all of the time that we've put into virtual education, the fact

that we have gotten results this time that are even worse than last time,

and this is before the pandemic hit.

I'm just boggled.

I'm boggled why this is a conversation we continue to have.

I think that my experience in this is that, if you can come up

with minimum performance expectations that apply to charter schools.

The vast majority, there are some that are in prisons and there are some that

are working with certain populations of alternative students where maybe that

we need to have something very specific and different, but across the vast swath

of schools, it is possible to identify minimum performance expectations that

would apply to absolutely all schools.

And if the, if the movement and, and, and policymakers have the fortitude to

simply enforce those, we will address, if we have disproportionate problem in

virtual schools, you will see that those schools will face a reckoning or improve.

And it's important for us to do that.

I guess my question for you, Macke, is where do you think societal opinion is on

the use of standardized tests, generally, are we wavering in our commitment?

Is it harder for you to get the data that you need to do these kinds of tests?

What's the future here?

Are we going to be able to sustain enough focus on student learning such

that your kind of studies can continue?

And what does that mean for society more broadly if the answer is no?

Society more broadly.

Let's talk about the future of CREDO.

Both of these things align in this question because the work that we've

done for the last 20 years, requires that we be able to access student level

data of a preponderant number of students in each of the places that we study.

So states necessarily need to be educating the majority of students.

In public schools, in environments that are tested on a regular basis.

A number of those assumptions are starting to not necessarily hold true.

So we see in the most recent enrollment data that there are a lot of students

who are electing not to be involved in the public sector for education.

We see a lot of legislative activity that suggests that we're going to

see more and more students taking advantage of alternative pathways to

education, whether it's homeschooling or micro schools or private schools

or whatever, through the ESA programs.

So that in addition to a sort of a, shall I say a softening of the commitment

to testing students on a regular basis that we see in a number of states.

We're not convinced that the record about what kids know is

going to continue into the future.

And I think that's a real threat to the kinds of performance expectations that

we need to be able to place on schools, especially now, and so I have the concern,

not just for my team, but for the nation as a whole that, while we have a active

conversation about whether testing is good or how to make testing better, I would

like to make sure that we don't lose the signal that we do get from testing until

we have something better to replace it.

So, yeah, I agree, and on the virtual school question, it seems

like this is the kind of schooling that works well for a real subset

of kids, but has been scaled quite a bit and is serving more kids than it

probably is going to serve optimally, and that may be one of the reasons

that we're seeing these results.

Because you do see virtual school can work for some kids, but the scale it's

at and then the bellwether charter deck a few years ago, this was a very clear

finding, both on virtual schools and also states that had a lot of virtual

schools, their achievement was getting depressed overall as a result of it.

So you do see this.

I want to go back though to this question on Native Americans,

because I think it's fascinating.

You have the tribes are actually starting their own charter schools

and a real area of concern has been native kids in district schools we're

more than 90% of them go to school.

Like I think people can think that the BIE schools and so

forth are a larger section there.

Most of these kids are in district schools and there's been a move that

they're not being well served there.

So how do you address that and put them in schools that are going to be

more culturally affirming, be better and yet we're seeing these results.

So talk a little bit more about that Mackey, like what are your

hypotheses for why we're seeing that?

How should advocates be thinking about that?

So I think it's a really important sector of concern for

the charter school community.

My sense is that the models that have been introduced in the Native

American serving schools that we see, are not culturally affirming.

I would also say they're not necessarily sustainable in the

environments in which they're placed.

And so I think that there's a fundamental mismatch between the schools that enroll

Native Americans in charter schools.

I should note that the schools that have Native American students, there are

very few where the Native Americans are themselves the majority of the students.

So we're still seeing the same problem that you just mentioned for the district

schools, where we're seeing small slices of the student population be

Native American and slotting into other charter schools, across the country.

And I think there it's very easy for students to get missed and to

not be the center of attention.

So the reservation and tribe based charter schools are trying

to do a different thing, which I applaud and I'm looking forward to

learning more about as time goes on.

That's great.

That's really interesting.

Talk about one other subset of kids, you know, a big push you hear in the charter

world politically, and something Jed and I have actually talked about on this podcast

is, how do you build a suburban political strategy and how do we keep charters right

now are still disproportionately sort of a function of urban politics and haven't

really penetrated the suburbs, as much as how do you build a broad based movement,

yet some of the results that you find seem to raise questions about that as a sort

of efficacy strategy in terms of actual, as you said earlier, actual teaching and

learning, leaving aside the politics.

So how should we think about those results?

So the suburban results that we see are literally in

comparison to suburban schools.

And so when we don't see a necessary advantage to going to a charter

school in that specific achievement context, that might not be a bad thing.

Let me just say, we would always want to see charter schools doing

strong academic progress for kids and when you look at most of the

suburban schools, you will see that.

What you'll see though is that they're not quite as strong as the schools that

they otherwise would've attended, and that's why we get the result that we do.

Having said that, there are a large number of cases that suit a charter school

presence in suburbia where you have huge schools, at the high school level, where

you have a particular run of schools from elementary to middle to high school

that may not actually provide a kind of flexibility that families are looking for.

Those are contexts in which charter schools can actually enter and be very

successful because they're offering choice, they're offering flexibility,

they're offering the opportunity for a different education environment,

and what we see, in places where the suburban education has been really

high flying, but really high stress, that's the place where the charter

conversation has become very vibrant.

That's interesting and I've always thought the customization and

sort of choice and options part is a key thing in the suburbs, it's almost

like a Maslow's hierarchy of schooling, and so if like your basic schooling

needs are being taken care of, then you start thinking about, "I might want

core knowledge, I might want arts."

And that sort of customization is a key selling.

And actually, I think over time makes public education more resilient because

instead of trying to keep people in who want something else, you're giving them

lots of choices and people feel more bounded invested in the entire project.

I think that's a great point.

Macke, can you talk about the composition of students

who are in virtual settings?

My intuition is that we would have a larger percentage of those

students be Caucasian than in other classroom based settings.

And that could be skewing the results as it relates to white kids.

But is my intuition wrong here or not?

No, you're right.

And for those of you who would like to really like totally geek out, we

have a website of all of the results of our study that are available

for you to go and poke around with.

And if you look at the demography of the charter school landscape, the

demography of virtual charter enrollees is substantially more White than

charters or their district schools.

Lower proportions of Black students by a great deal.

And lower proportions of students in poverty.

And so there is a sort of an evolving archetype of what a typical virtual

charter school student looks like.

I will warn that there are contexts across the country where that is not the case.

Yeah.

It's my sense that this is one of the responsibilities that is going to fall

on charter school advocates, which is to continue to advocate for this

kind of data being available and for schools to be held accountable to it.

I think we find ourselves in a challenge because I think there have been all

sorts of instances where data has not been used wisely and not used fairly.

And so anytime you end up defending something like that, you run the risk

of undercutting your own credibility.

But more for the long term, for the long term.

There's have to be some voice and Andy maybe you've got other ideas

about where else, we can get some spine to keep this kind of testing

data available in the landscape.

Jed, that's a great question and you would think like in general

people would want as much information and as much transparency, but

as we know, that's not the case.

Um, and so I do think we're under a lot of pressure here.

We're seeing even like post pandemic, it's been a struggle to get test score

data released in a lot of places.

And I think in some of those places, and we talked about a few weeks

ago, New Jersey, I think one of the reasons maybe that it told a pretty

good story about charter schools and a less good story about traditional

public schools and there's obviously politics around all of that.

So I think this is a big problem and it's something that everybody

needs to pay attention to because without good data, this just becomes

brute force sort of politics.

So Macke, I kinda want to ask you related to that though.

Like I follow these since the beginning.

I remember the 2009 study and you got a lot of grief because there's

certainly in the charter world, there was a sort of hear no evil

speak, no evil, see no evil faction.

That was like, what, you know, essentially kind of argued you were like turning your

guns on the wrong lines or something.

And I think what gives these study credibility again, as we said

earlier is how it arcs over time and the results have changed.

But how like it seems like we're still heavily anchored on, I still hear a

lot about the 2009 results when you listen to advocates, it's like they're

like frozen amber or something, and like this one and the last one for

that matter didn't seem to get quite as much attention, which is striking if

you just step back and are like, we've created this enormous cohort of schools.

If you think about sort of the history of public education, this is one

of the more ambitious experiments and innovations historically that

we have seen in this country.

And these results keep coming in and they're changing.

It's an evolving story.

You would think they'd be more interest and curiosity, and yet I just didn't

see a lot of attention, maybe it's the old adage, if it bleeds, it leads.

And so good news, or somewhat good news.

Didn't get the attention, but I'm just one observer and you

were at the epicenter of it.

How did you experience, like, is there a changing appetite around like, you

know, what kind of news, the level of information, like what's, what's going on?

So I want to make a distinction between folks who are

actually engaged in the crafting of policy and the public media.

And I think the statement that you just made applies pretty

broadly to the public media.

We have learned from our past studies that a lot of this is a slow burn that

particularly as we continue to talk about or place op-eds in particular places other

news vehicles start to pick up on it.

So I'm a little bit surprised at the media reaction this past week, but

I'm not ready to pack the tent yet.

On the policy leadership side though, I have to say, t's been a

very different reaction this time.

Most of our state partners require us to do briefings with them before

the report is released, and of course we would want to do that anyway.

And the conversations that we've had in those settings have been

much more constructive, I would say, than we've seen in the past.

We've been asked to be much more available to share the results

and to discuss without making any policy recommendations, discuss the

implications of what we've learned.

And so lots of folks in the policy arena are handing us off so that

they can create a community of folks who know the findings.

And I, I look at that as a very positive development.

Yeah, that's actually, there's like a robust set of folks doing that,

that's extremely encouraging just in terms of how issues evolve that and

is that bipartisan and is that, are you seeing like, say a little bit

more about that because like, that's probably the most encouraging thing we've

talked about in this whole discussion.

If that's happening in a number of places.

So, one of the things that's happened over the long term that we've

been doing this research and as you spoke about keeping these data relationships

going, is that I think we've actually worked ourselves into, I wouldn't call

it allies necessarily, but at least respected colleagues in a number of

places that you would not normally think we would have any kind of traction.

And so there are quiet conversations that are going on in places that you

wouldn't expect, and surprising to me, lack of conversations in some of

the places where you would expect it.

So we're tracking this sort of one day at a time as it comes and really

trying to make sure that the focus is on looking at what the evidence says

and what it portends for future decision making for K12 in the public space.

I like that.

Focus on what this portends.

Because I've had conversations with several advocates whose

state sectors are just nailing it.

And it could very well be that you can make a big deal about

the results that are here.

But they also know they are sobered by their own data about what's happened

to charter schools during Covid.

And a lot of those supports that charter schools were using to generate these

kinds of results were the exact supports that were most affected by the pandemic.

And so our folks are saying, don't make too much hay about

it right now, cuz our data might not look as good a year from now.

Do you have anything that you've seen as it relates to this data and heading into

Covid that would help us understand what might be a coming, because everybody's

got their guess on what's gonna happen with charter schools through the pandemic,

but do you have any inclinations there?

So we don't have the kind of longitudinal student level data that

we have used in these studies, but during the pandemic we actually did

do a fair amount of research about what was going on in charter schools.

And we zeroed in on three states, three very different states, I should say.

And what we found was that, many of the things that we had believed

before about what made charter schools different were evident in spades.

And what I really mean here is, what we saw was a much faster responsiveness of

school teams to the changing conditions of the schooling that they faced, that they

literally turned within a few days to an alternative learning model and insisted

on learning as they went along and continuous improvement as they went along.

That was something that we didn't see in any of the comparative

examples that we delved into.

I also think that there was a real commitment that they identified themselves

without being prompted to being first responders and took on that mindset, and

so when we talked to school leaders and when we talked to educators and we talked

to parents, we got this consistent message of these guys are frontline fighters.

And I would say that throughout the pandemic, I think universally

we're gonna see learning loss when we get those data and actually can

nail down what that looks like.

I think we're going to see learning loss no matter where we look.

But what I would say is I think both, the magnitude of that I

anticipate would be smaller.

And I think the resiliency of those teams to get back on track is probably higher.

Those are my predictions.

Interesting.

And you square out that we've got some very preliminary

data in the their findings.

And so that's what you're sort of grounding that, as

well as what you're saying.

Well, and through some of the formative assessment providers that we

partner with, we can see some of those interim results starting to come through.

So I don't want to get too far out in front of the data, because frankly there's

not enough of it to be able to say things with sort of with the kind of authority

that these other studies provide.

But the early indicators are pretty encouraging.

So let me ask you a question about just the funding of this project and

what of portends Macke,.You've been able to do this for a long time and it seems

like maybe the real risk was after 2009 people just like throw in the towel and

stuff and instead you've been able to keep these studies going, and as you

said, and we'll put in the show notes.

There's studies of particular states, there's studies of cities, it's a

treasure trove of data on charters.

And we've had over the last 20, 25 years, a problem in the sector of funders will

quickly lose interest in something, move on often before the results are even in.

I mean, like the Gates Small Schools Initiative is sort of Exhibit A, but

you also saw that around some, around teacher evaluation, some other things.

Small interval challenge.

Yeah, exactly.

You've been able to like, keep with this and keep it going, which is, you

know, enormously important because the learning does happen over time,

this is not sort of a point in time thing, it's an overtime thing.

So talk about like, how have you been able to do that and

what should we be doing more of?

What are the broader implications for the field, even beyond

charters and so forth of this kind of work you've been able to do?

So it's a wonderful question.

And when we put the results of the three studies together and we

saw the upward trend across all of these three studies that emerged.

For us, it wasn't a wow, look at that charter schools are really pushing it

here and they're doing really well.

Obviously we were happy to see that result and we are happy to

bring those findings forwards.

But for me, the real insight there was policy takes a really long time to mature.

And it takes an endurance that I don't know is well matched to the context.

And so you asked about the funders, I have to give credit to the legislatures that

continue to endorse the policy over time.

I give credit to the school teams that continue to battle on when they're faced

with the kind of political uncertainty that they've faced over time, and I really

appreciate the authorizers who have been diligent about applying new standards

in increments so that, the incentives work in the directions that they do.

So for me, what we've got here is we've got a collective mix of a lot

of different stakeholders who are in it for the long haul and are willing

to stand tough over the long haul.

And that, for me is the big insight of that trend line that we see.

It's ironic though, right?

Because it's happening on an issue that is particularly

contentious, has become partisan.

I Think like a naive hypothesis would be this would be the kind of

issue where that wouldn't happen.

It might happen on stuff that's a little, where there's a little bit more of a

different kind of foundation, different kind of political and policy arrangement.

So I think it's a super encouraging story because it is a little bit of, if you can

make it here, you can make it anywhere.

If we can keep, if we can keep this kind of subsystem kind of working in

terms of policy and learning on an issue like this, like that's hugely

encouraging on other issues that we think about, like standards, accountability,

even school finance, other, other issues where the politics are a

little bit different to start with.

Yeah, I think that's great.

I also don't want to minimize how much effort it took to keep the game going.

So, let me just say, a lot of feet on the line standing in the fire.

So, and guys, I'm really sorry, but I have my annual meeting

with the boss in four minutes.

That's alright.

It's a great note to end it on, the mixed metaphor of feet in the fire.

But no, that note you just ended on how much hard work it was.

I mean, just.

I know I speak for Jed when I say this to speak for us.

Extremely grateful to you for this work.

This is, this, this is really substantial.

My team knowing that the two of you are giving that kind of praise, my team's

going to get really excited to hear this.

Thank you.

I'll pass that on.

I think that multi-decade view is becoming one of the most important

and it's the only view that helps us suss out what's really going on.

So for everyone that's doing it, you know, we should offer special

thanks and Macke you, no one's doing it to the level that you are.

So thank you for that and thanks for being with us here.

All right, Andy, it's good to see you, Jed.

I'll see you next week, right?

Yeah, for sure.

For sure.

Look forward to seeing you in Austin.

Good luck with with your meeting here in a few minutes.

You can tell your boss that we either loved it or hated

it, depending on what will help you, what will help you the most.

We found sometimes, sometimes it's our praise or our criticism

sometimes helps people more.

I

I'll report back.

See you back.

Hey, that was great.

What a good first guest like really substantive and important issue.

Yeah, I've learned so much from Macke over all our decades of work together

that to see her continuing to unearth things that no one else is finding.

It's just really great.

And I don't sense her own personal desire to keep going waning at all.

It's just, can we keep the soci societal conditions together such that this really

important work can be brought to light, or these findings can be brought to light.

But I think it's encouraging.

Look, she's obviously, she's someone who's disposed towards

reform, I mean that's clear.

But the fact they've been able to do this work, shoot it straight over the

years, talk about the good and the bad, even in this study, it's not all good.

It's such an encouraging thing when there's so much pressure on

everybody to just pick a side.

And you're seeing people on charters who are like, they're not really against

charters, but the politics the way they are, they're definitely throwing their lot

in with people who are against charters.

And then likewise, you've got people in the choice community who have never

like really stopped to question a lot, that they're just always are for choice.

And to see someone like Macke who does bring real nuance to it, to see her be

able to produce this work and succeed at it like she has, it's really encouraging.

And this last report had 2 million virtual twins.

It's 4 million data sets.

They're students who's who are being looked at, right?

So I think that's a part of what the credibility is for her.

Hey, it's having done it over many, many years.

But also to have a scope of statistical undertaking that just

becomes very difficult to refute.

So hopefully, those policymakers who are getting those individual briefings from

her will take this stuff to heart and keep the resolve necessary to allow further

studies like this to happen going forward.

Yeah.

Well look, just speaking as a guy who only has two twins, I can't

even get my head around that.

That is a lot of twins.

Well this was great Jed.

I look forward to seeing you in a couple of weeks.

I hope you enjoy Austin.

I know you're off to there.

Yeah.

Looking forward to conference and looking forward to talking

on the other side of it.

So take good care to them, Andy.

You as well.

See you soon.

Okay.

Bye-bye.