In this week’s episode of All Things Marketing and Education, we’re joined by Katie Boody Adorno, CEO and Founder of Leanlab Education, a nonprofit research organization dedicated to studying and advancing transformational education innovations. Katie guides us into the fascinating world of co-designed education innovations and unpacks the power of community-building in the education space. She also explores topics such as educator voice, the role of research and development in K-12 education, and the effectiveness of EdTech.

Show Notes: Leoniconsultinggroup.com/52

The EdTech Marketer's Planner: leoniconsultinggroup.com/23

Access this episode's show notes, including links to the audio, a summary, and helpful resources.

Elana Leoni:

Hello and welcome to All Things Marketing and Education. My name is Elana Leoni and I've devoted my career to helping education brands build their brand awareness and engagement. Each week I sit down with educators, EdTech entrepreneurs and experts in educational marketing and community building. All of them will share their successes and failures using social media, inbound marketing, or content marketing and community building. I'm excited to guide you on your journey to transform your marketing efforts into something that provides consistent value and ultimately improves the lives of your audience.

Hi everyone. Welcome to another episode of All Things Marketing and Education. This week I am excited to be sitting down with Katie Boody Adorno. She is the CEO and founder of Leanlab Education. We're going to talk about what that is. It's really exciting, but in a nutshell, they are a nonprofit research organization with a mission to study and grow transformational education innovations.

She's going to talk specifically about what that actually means, but I love that she looks at transformational education innovations that have been co-designed with school communities and when I first met Katie, I started to go, "Gosh, why do we get along so well?" And it's fundamentally because we care about the same core concepts and one of which is co-design. And when we think about co-design, many of you that have attended our trainings on community or heard our Director of Joy, Porter Palmer talk, we always talk about when you do community and you're doing it well, you're never just doing your own thing and hoping people come along for the ride.

You are co-building and co-creating, and that is such a beautiful journey, and I know we're going to talk a lot more about that in the show. So today, as you can guess, we're going to be talking about things around educator voice, community, but specifically we're going to talk about the role of R&D in education and K-12 education and efficacy in EdTech as well. So we got a lot to talk about. I'm really excited to get started, but before we do, I'm going to talk a little bit about Katie because what I find about people in the education space is that they're so humble. They never really fully introduce themselves, and I am so impressed by their passion and their expertise. So here we go. Here's a little bit about Katie.

So Katie is a native of Kansas City, and I'm going to go a little bit, I guess what my friend Porter would say, "A bird walk," a little bit of a bird walk for me is that my mother's family is from Kansas City and I think I told you that Katie at one point, but I remember when I read your bio, Katie, you mentioned the hospitality of Kansas City Barbecue, and it reminded me of a time of many visits where I went to see my family in Kansas City and they were super awkward because I'm from California. My mom actually raised me vegetarian and so I would go to Kansas City and they would be like, "Okay, let's get some barbecue going," because it is a thing. It's a thing and I was like, "Well, I'm vegetarian," and they say, "It's okay, you can have chicken," and I just remember just laughing at that.

Katie Boody Adorno:

Thanks so much for having me here.

Elana Leoni:

What's an EDS again?

Katie Boody Adorno:

Yeah, I think it might be a Midwest thing too. It's an education specialist degree, so it's between a master's and a doctorate essentially, and it's an administration focused practitioner degree.

Elana Leoni:

Okay. I was like, gosh, I just don't... So many things I don't know. Well, Katie, I'd love to start this podcast around just a little bit of the why, because when we think about our careers in education, it's so much driven by the why and the passion and the purpose and you founding this organization is a beautiful journey, but why education for you? Why did you become a teacher? Why did you decide to get into this beautiful but challenging space?

Katie Boody Adorno:

Yeah, yeah, for sure. I mean, I think for me, well one, going back to my childhood, I was an avid learner. I just have always loved learning. I was the nerdy kid that made my own flashcards and slept on my science book and brought home a microscope and I just was a really curious child that was really academically motivated, but I also really struggled with the social and systemic aspects of school. I was from a bi-cultural biracial family in a white affluent suburb where we didn't quite fit in. My parents, whereas you mentioned were community organizers and social justice advocates working in primarily communities of color.

So I kind of always struggled to reconcile the disparities that are present in their education system and I was aware of those disparities from a very, very young age because of my parents' work, because of my identity, because of where I went to school and I didn't know it at the time, but I really struggled to reconcile that. So here I was a very curious kid that was very driven to learn, but I didn't really fit into the structure of the way school was built and that ultimately manifested in a challenging way.

I ended up leaving high school early. I ended up really struggling in high school, not academically, but socially and emotionally and dropping out for a time and then graduating early and then that led me to leave the Midwest and go off to college and I was fortunate enough to pick myself back up and recoup and end up graduating from a university and then coming back to Kansas City actually to teach. And I think it was that kind of boomerang moment that gave me a chance to really think through how might I contribute to making the system better and that's what kind of ultimately led me into my career path.

Elana Leoni:

I was talking with someone about the power of maybe not having an optimal system that welcomes and allows you to explore and be you and succeed in it versus a system that you actually felt worked for you and does it motivate you more to go back and change it if it didn't work for you, and I think ultimately I've come to, it does, because when I was working at the George Lucas Educational Foundation, I don't know if George Lucas would've founded a foundation if didn't... If he had a decent experience, he's like, "Cool, I'm probably going to focus on healthcare instead," but he didn't fit in you, and it took him a while to... It almost set him back to his path of work. He actually ended up going, and I think because of that, it was in spite of this system, I'm going to change it and it was this motivation.

Katie Boody Adorno:

Yeah, I think that's a huge part of it and like I said, I didn't even really realize that I would say that well into my career in education, did I connect the dots that I was part of what was fueling me was not only my experience as a teacher, but my experience, my formative experiences as a student and feeling like I wasn't really fed the way I was desiring to be fed and I saw that keep manifesting in different ways when I was in the classroom, when I was working more at a system level leadership and just kind of that curiosity was always a driving force around why is this so hard, why is our system failing students, why is our system not engaging students and what does it take to build a better way forward?

Elana Leoni:

And it's really clear, the equity piece, the access piece, it runs through all of your work and if you've never experienced it firsthand, it may not be such of a driving force. Maybe let's talk a little bit about what Leanlab does as a segue in there because that has a through point into that point, but what does Leanlab do generally in the world of the education ecosystem?

Katie Boody Adorno:

Yeah. We like to think of ourselves as a human-centered research and development lab. So what does that actually mean? That means that we have a community of school districts, individual schools and educators and administrators who care deeply about the future of education and then explicitly what we do is we work with education technology companies and we pair them with these either school systems or communities of educators to inform their product development and ultimately to evaluate how well their products actually perform aligned to student outcomes. So we have a team of school organizers who help onboard schools and prepare them to engage in these ongoing R&D activities and we have a research team that then actually helps uncover the insights from the schools to make the products better and to evaluate their performance.

Elana Leoni:

I mean, everything you said was music to my ears. It's probably music to a lot of EdTech companies' ears because frankly, it's quite hard to do what you do and that's why you exist. It's like internally in an EdTech company, they're not going to be like, "All right, let's turn 20% of our organization into an R&D lab." I would say every EdTech company I've worked with has wanted to integrate educator voice and want to track outcomes of student performance, but they don't really know how. And it's costly and it requires expertise. And everything you're saying is like, "Oh my gosh, you exist. This is great."

Katie Boody Adorno:

Yeah. Yeah. I mean, I think it can sound kind of heady and abstract, but the reality is it's hard to do logistically. I mean, EdTech companies do often have UX teams, larger ones have learning science teams and researchers on staff, but the reality of logistically working within the constraints of a school day, the academic year in safe and secure ways that protect student and teacher data, it becomes challenging to figure out. I really want to design with teacher and student voice, but how to do that authentically and securely and ethically is difficult to do.

Elana Leoni:

Yeah. So how do we do it? If you were going to be working with a company, or if they were, I know that the process might be different if they were going to start on their own slightly, but how do you collaborate with EdTech to say, "Okay, we want to make sure that educator feedback is given to you in a way that ensures that your product is actually doing what you hope to intend it to do." What are the baby steps to begin that feedback loop?

Katie Boody Adorno:

Yeah, we think of it as what we call the product research journey and the kind of research we do, we call co-design product research because it's focused on co-design, elevating the educator voice or stakeholder voice, student voice here, and it's product focused. So we're really looking at how does that product itself become better? So that means the first thing that we do is kind of assess the stage of development, and we'll work anywhere across that journey from something that's already highly used that we're really wondering about the impact to something that's concept stage. So the very first thing we do is assess the stage. If it's concept stage, we're going to say the first thing you do before you build anything, before you build a minimum viable product or a prototype, let's validate the assumptions you have that this is even needed.

So we'll start with if it's a concept stage with a focus group or a panel of educators to really give voice to the need or help refine the idea when it's at concept stage. If it's later stage, we actually start with something called a product audit. So it's exactly what it sounds like. We ask for access to the product and we will do an audit on it and from that audit, we'll get going under the hood as a user, we'll look at any other data that the company might have, and we'll assess what stage of research it's ready for.

Essentially we're looking for does this mean that this product is ready to be fully adopted by a school and we can work with school districts in our network to actually get it used on a daily, weekly, whatever's appropriate basis to understand how it works at a school system level, or is it a little bit earlier stage and we need to do some more product work where we're doing more rapid iterations on the product testing so that it can be used with fidelity over an elongated continuous period of time.

So that's the first thing we do and then from that, once we understand what the needs are of the product, we'll recruit from our school network. So we'll say, "Hey, okay, it looks like you're at an early stage. We want a diverse set. We want a diverse group of administrators and teachers from different geographies and different types of schools to give feedback on this feature set." Great, we'll go ahead. We'll put that together, we'll design the questions and we'll do that or if it's later stage and it's more complex, we'll go deeper there. "Hey, we need to recruit an entire system or school to onboard this. This is going to require a more sophisticated data share agreement and research design," so on and so forth. So we really try to customize it based on the stage and what's appropriate for the tools development.

Elana Leoni:

And when I talk to so many EdTech companies, when you say in the beginning, it might be at the concept phase, and it might be based on some assumptions that we just need to validate and regardless of the stage of the EdTech, there's assumptions built in that we always need to validate too, but because our industry is so unique, we have founders that have been in the classroom, and maybe that's why they created the product, or maybe they weren't an educator, but they saw an opportunity and a need and validated it with maybe a couple of different educators, but regardless, we go in with the bias.

We go in with the bias and go, "Oh, this is what I need in the classroom potentially. So it's going to work for everyone," and I love that you first start out and say, "Let's just look, I'm not biased in here. Let's look at the assumptions you've made and let's validate them with probably a diverse sample of different educators and their target audience," right?

Katie Boody Adorno:

Exactly. Yeah, that's exactly right.

Elana Leoni:

That's awesome. Okay, so if you are an EdTech company and you don't feel like you are in a stage to hire anyone, I'd love for you to just think of my current product, think of your product and say, "Well, what are the assumptions I made here? And how is the role of the educator evolving? And are those assumptions still true?" Because this space, I know you don't do it even though you were an educator in the classroom, but most of my team have been educators in the classrooms and at no point do we say we know how an educator is feeling right now because it's so fundamentally different and you can never make the assumption just because you've been an educator that you know what it's like to teach right now.

So I love that you bring in educators and so if you're an EdTech company listening, think of ways that you can also bring in educators consistently to show up. Okay. Off the soapbox, but so for the side of the EdTech companies where they're seeking out and trying to incorporate educator feedback, you obviously have a network of educators that you go out to and you can figure out a diverse sample and organize that feedback and make sense of it. But how does an EdTech company begin to do this? This sounds hard and time consuming.

Katie Boody Adorno:

Yeah. Yeah. I mean, well, one shameless plugs, you can always work with us, but it is at a cost. However, I mean, if you're doing this yourself, and many, like I said, many companies have, UX teams have research teams, and if they're earlier stage at least has a product team, you can start thinking about what are more efficient ways to keep educator voice close and one thing we like to think about or promote is like a teacher panel and making sure, but taking a step. Many companies do this. You have your early adopters that you keep close, you give them gift cards to give ongoing user testing, that's great. I would urge them to take it a step further and stop and consider that sample. To your point, you just brought up representative samples multiple times and diverse samples and I think that's often a part that in that when working fast and product development is often overlooked.

You're looking at whoever answered, whoever's available, who are your early users, who are your early adopters, and those typically tend to be edge cases and the problem with that is they might give you false positives. You're looking at highly tech savvy people on the end of the spectrum of the innovation curve and these may not be representative of the actual archetypes that exist in traditional school districts and to your other point around the shift, we've seen post COVID, it's a very different landscape. So I would urge companies to think through, step one, what is the way that I can have an ongoing group of teacher or administrator depending on who your end user is, participants that are co-designing with me as I go through product development.

As I'm talking to administrators in my sales cycle, what do they need to be true from an impact perspective and am I thinking about that? And then two, how am I baking in systems to keep those educators close for that product development and thinking about their representation? Is this all early career teachers that are highly tech savvy or that are working in high resource schools or have I thought about where they're coming from as I think about and does that align to my go-to market strategy and the ultimate, what's ultimately needed from my school partners as we look to scale?

So I think that's one concrete thing to do. The challenging part is it's logistically challenging. It can be. So thinking through hacks up, can I set up a monthly advisory group where I give these educators incentives, payment access, whatever it might be for it to keep engaging? Do they feel that their voices are heard? One thing about disengaging educators is that they'll come to a few focus groups, but if they see your product's not evolving in response to them, they're going to check out, if it's not solving and that feedback is also valuable, right? They will show you with their time what's valuable and what's interesting and so I think you can think about how you can keep measuring that in authentic ways by bringing them close to the product development cycle and including them at as you think about your impact journey in research.

Elana Leoni:

So how do you do that? It's a journey itself to find a representative sample of educators and I always ask myself really critically when I'm doing this type of work, who's missing and fundamentally, there's always somebody missing, but that should evolve too, but let's say you've tackled that challenge and you've got the voices you need and you've gotten the feedback, and potentially you've created a process where you're soliciting consistent feedback. Okay, we got that, but do you work with companies on how to actually take that feedback and create a process so it's not just giving lip service to educators and not wasting their time?

I know that if you're an EdTech company listening, you're like, "Hey, listen, we've got our product roadmap, it's been set for a year. We have budget constraints." We have a lot of constraints that educators may not know about. So how do you bridge that gap of trying to tell them that we're listening and then put it within a product roadmap development cycle?

Katie Boody Adorno:

Yeah. Yeah. So I mean, a couple answers to that. I mean, obviously many product developers, yes, they have their roadmap set, they have their strategic targets set, and meanwhile they're doing AB testing, they're doing other iterative testing to evolve the product so that it's elegant. What unique challenge we have in EdTech that is, I don't think we don't talk about enough, but that's different than consumer driven EdTech or technology products, is that the feedback loop isn't as tight. You think about Netflix or Duolingo or any other consumer grade tool, you're able to functionally AB test or do your product testing 24/7, literally 24 hours a day because there's no barrier to access of that user data.

Whereas when you're dealing with education, you're dealing with in the confines of the academic year, and if it's a enterprise solution, the confines of that school day usage is going to be restricted to when the course is actually happening or as a supplemental oftentimes and you're also dealing with the barrier of accessing that data more meaningfully because of the protections with school systems. So functionally, what does that mean in terms of our value and how we help products source those insights to improve the product? That means we really act as an intermediary and the middleman to help navigate that.

So what that means is yes, we're co-designing the research questions with the schools, but also equally important, the product EdTech companies themselves to understand what insights, what are negotiables and non-negotiables, what's on the table that can be changed and what's not, we just can't get to that or that's not feasible this year. In our research studies too, one thing that's unique about them is they all include action plans. So rather than just being a traditional research study that you imagine on your JSTOR or ERIC website or behind a paywall that has an abstract and it's kind of hard to read and might sit in a website, ours actually have action steps and a plan, and we have a certification called the co-design certification that if a company makes 75% or more of those recommendations within a six month timeline, they qualify for that.

So to certify and signal to the market that this tool has been co-created in earnest and has actionably made steps toward the stakeholder feedback that they sought. So it is pretty thoughtful, I would say, in the intentionality of how we think through how can this really be viable for developers and schools while also iterating to their benefit.

Elana Leoni:

I love that you... Sometimes we used phrases in education and sometimes we overuse phrases in education, but I love that you actually put a certification where the phrase was and say, "Yes, I have practices to prove that this was co-designed by either stakeholders, either in education or students or teachers themselves." So yes, I'm just so excited that that exists and that the biggest loop is once I get that feedback, like I said, and you've kind of navigated with some answers around how to create actions from that, but I would say number one, in education, we're not as transparent.

I feel like for some reason we hold our cards tight when educators just want to know, okay, great, if you can only change 20% of the product with educator feedback, that's great to know I can work within those confines and educators are used to working within a broken system. I don't want to laugh, but they're the most inventive, optimistic at times, people that can say, "Okay, great, we have a problem." A certain percentage of it, you can integrate our voices or be transparent. I don't see too many in terms of EdTech companies do this, but in SaaS companies, they have those product places where you can up vote features.

Katie Boody Adorno:

Yeah, yeah.

Elana Leoni:

Super helpful and it kind of makes me feel like I'm owning the product a little bit too, and I go and check in. So I don't know. You probably have good stories about companies doing that, but I love that now that we know what we know and we're continuously seeking this input, here's an action plan and then here's a certification. I did some things on the social side educators are looking for companies that are saying, "We heard you. Here's our new feature." Don't be afraid to do that. If you're an EdTech company, make sure you're integrating educator voices.

So speaking of that, educator voices, if I'm an educator listening, how do I get involved? If I love a product, how do I tell them things that I love and maybe some areas they can improve? How do I make the most of my feedback or even get started, because a lot of the times educators use lots of products, but they just assume that people don't want to hear their voice.

Katie Boody Adorno:

Yeah, there are a lot of different places you can go. So first off, if you're using a product and you love a product or you're having a negative experience with a product and you have some feedback you want to make known, usually within products there is some sort of feature that you can give feedback. If it's a small startup, I always recommend, and they're a good startup that's hungry for feedback, which almost all of them are. I also recommend just going to their team website and looking who owns product and just emailing them or LinkedIn them directly.

I'm telling you, early stage startups and product people love nothing more than hearing from a user that's excited or has some constructive feedback that can help them solve a problem. So I think I encourage, they're so hungry for that feedback. So I encourage any teacher that's interested to take on the initiative to say, Hey, I have some feedback I'm willing to reach out.

And then as Elana said, and many companies are getting a little more savvy. I haven't seen it as much as EdTech as I'd like to see, but integrating features to make that feedback giving a little bit easier. So I encourage folks to look into any products that they're excited about and see if those features exist. More systemically, there are a couple organizations you can work with too, if this is exciting. So one thing that we do is we compensate both individuals and systems for participating in this kind of research. So you can go to our website at leanlabeducation.org/schools and sign up there either as an individual who wants to give feedback and be compensated for your time to do that on emerging EdTech tools or as a school system.

If you're more interested in the implementation conditions of EdTech, there's another group called EdTech Evidence Exchange as well, who is interested in getting feedback more on the implementation conditions that need to be true within a school, meaning what's the journey that happens once you adopt an EdTech tool, what needs to be true for that adoption to happen, what needs to be true for it to be sustained and useful? So there are more efforts underway, and I think as when people are increasingly curious about how does EdTech make teachers and classrooms lives better or not, I encourage folks interested to look into all of those ways.

Elana Leoni:

Yeah, I think I want to just share a little bit about how if we're in education and specifically in EdTech working with educators, how different it is from an R&D standpoint and I'm sure you've experienced this too, but when I seek out educator feedback, it's almost like I need to twist them a little bit and go, "Okay, what do you really think? What do you really think?" And they're like, "Well, they tried. They get a gold star for trying," and I'm like, no, tell me what they really think because they're so used to kind of just, okay, well, you're asking my opinion so you get a gold star, because people don't generally ask my opinion, but I want to tell the educators listening is that we value your input and we value your honest feedback. I don't even care if it's blunt. If it's blunt, it'll stick with people more too, but do you have experience with that, Katie?

Katie Boody Adorno:

Oh my God, yes and that's honestly why when we do pretty much past concept validation, we often urge companies, we'll record the feedback, but to not be in focus groups because power dynamics are what they are and oftentimes, and I think it's human nature, it's not just educators. When people are confronted with the creator, whether it's an artist or in this case a product developer or a founder, you don't want them to feel bad. It's a human thing. So how might you have a facilitator use a facilitator to try to get to the truth, because typically at first, folks are overwhelmingly positive.

So what we're really looking for is feedback and another way, whether it be good or bad and another way to kind of control for that too, we're experimenting with now is doing some training with educators on product development and research methodology, making sure that they really feel they understand the cycle that they're going through and that constructive adjusting feedback is part of this process, is contributing to this process and we are early in that journey, but finding that that does help. I think when you give the frame of this is where we are, this is what we're doing, this is how this evolves, your feedback is actually a crucial point of the development and co-development of this. We want it to evolve toward your needs, that all is helpful.

Elana Leoni:

Katie Boody Adorno:

Yeah and the product will speak for itself. I mean, that's part of the reason why we also focus on feasibility and implementation studies and compare the data. You'll have verbal feedback and then you'll have your usage data as well and then ultimately impact data we can look at and correlate against and all of those things. There should be a through line between all of those data points and you should be able to see some trends.

Elana Leoni:

Yeah. So in education, we've talked about how hard it is for R&D and in general, R&D is hard. It takes time. It can be expensive, but there is more momentum around it, and it may be attributed to the ESSE funding requirements or the technology bloat. We're seeing post pandemic and stakeholders are now asking, "Hey, does this improve student outcomes? Does this do what you say your product actually does and can you provide research to do it?" So I feel this momentum, you obviously feel this momentum, but you kind of don't see a huge increase in it. We see very little of it and also very little of it done well and I read a great blog post by you on Substack, we can put it in the show notes, but you cite a study from the LEARN platform that showed of the hundred most used EdTech tools, only 26% had published studies aligned to the ESSA Tiers of Evidence, and then roughly 39% of them had published any studies at all.

Why do you have to be a unicorn to have a study to say your product actually works? Why do you think this is all happening? I know there's so many reasons. It's a big topic.

Katie Boody Adorno:

Yeah. Yeah and it's one of my favorite topics to discuss because I think it's fascinating because you look at this, we're talking about products that facilitate learning for children and a market that's been pretty much completely deregulated has had very minimal regulation historically. And the way regulation has come into this space has been through ESSER funds. So a quick primer on that, the ESSA or ESSA Tiers of Evidence are federal guidelines that there four different levels that connote how rigorous of research has an education solution engaged in from minimal research or research that demonstrates a rationale all the way up to a random control trial.

And then there were federal funds through ESSER that were given through schools to try to incentivize the adoption of evidence-based tools. There was never a moment where we're saying, "Hey, all EdTech has to be evidence-based." It was just a moment of saying, "Hey, there are federal funds that are available and we recommend you apply them toward evidence-based tools." So that was actually important and we've seen, I think because of the ESSER funds post COVID and people concerned about learning loss and the need for tutoring, we've seen an uptick in interest in companies wanting to get evidence, but at the core of this, what we've seen is that because it's still kind of up to the company, it begs a bigger question of what are the incentives for companies to engage in R&D and what we see is that because there's been below, as you said, Elana, of EdTech after pandemic, we saw record high investing happening in EdTech.

We saw these federal funds become available for evidence-based technology tools. We saw schools close down and go remote and over-rely on technology. So for all of these reasons, we saw probably too much EdTech come into the market and now we're at this moment of culling, federal funds are going to run out, schools have gotten savvier, and they're looking at cutting down. They want to know what actually works. They want to know what's actually useful. They want to know what actually is getting used and so for all of these reasons, we're seeing EdTech companies either do one of two things or both.

Either say, okay, we're going to double down on R&D and we're going to make this our differentiator or it's not required, and we're going to double down on sales and making sure that we stay in business or we're going to double down on R&D as a means to sales to stay in business. So I think we're seeing those kind of things play out right now and I think in large part it's because it hasn't been a requirement and while there's federal funding that has increased for R&D, I think it's worth noting that the Federal R&D budget for an education is less than 1% of what the United States Government allocates to R&D and the military, for instance, and any other industry. So it pales in comparison to what other industries are investing in R&D.

Elana Leoni:

Wow. I sometimes get stuck in the EdTech bubble, and I assume that... I don't assume, but I just take it for granted and you're right, compared to other industries, R&D is a huge part and I think it's the thing of the longer product feedback cycle, like you talked about with academic outcomes with the school year, with all of the things that, and then we've also seen shoddy research and stuff that were like, "Gosh, they spent so much money and it wasn't really definitive at all," or they had a confirmation bias in it, or it wasn't a representative sample, or they don't have enough data. All of those things I think of when I look at studies and there's, you name it, there's a high dosage tutoring study out there, and I kind of have to squint and go, is this real?

Katie Boody Adorno:

Well, I think you're totally right. There's a lot of shoddy research out there and the high dosage tutoring thing is a perfect example where you had rigorous foundational research about the impact of tutoring, but under the context of in-person learning, that rigorous research that everyone was citing was tutors in real life pre pandemic. That's where that study came from and then people were making the assumption that that would translate to a digital context. So we're seeing a huge challenge here in the sense that not only is R&D underfunded in EdTech and not regulated from a federal perspective, but also it's just really hard to do well with the feedback loop challenges with technology products that are constantly evolving.

And we actually have a charge as a research community that we have to evolve the way we do research so that it's more efficient, so that it's more affordable, so that it's more relevant to these technologies, especially now within the context of AI, where we have more emergent technologies than ever before that are moving way faster than any of our traditional rigorous R&D models have and that poses a large question where I understand why companies are going, this is a lot, we're interested in understanding how our product impacts learning, however, how do we do that while also keeping up with the pace of product development, evolution, staying afloat, we have all these competing priorities. So it is just a really interesting, I think, sector-wide issue we're grappling with.

Elana Leoni:

Do you feel like R&D is, when I look at it potentially could be, and I might be botching this, because you're the expert here, but I almost see it as a gradient. So if you want to incorporate educator voice and maybe you don't have a budget to, but you have some way to consistently get educator feedback, you can at least start surfacing up testimonials, surfacing up feedback, showing qualitatively how it's impacting students and educators. Maybe that's the beginning of the gradient, right, because you're not really seeing if it does, it's just their opinion and it's not a representative sample and doesn't have a lot of data around it, but then you eventually could do a full out study and say, "Yes, in all environments, in all school settings, this works."

I got to see the George Lucas Educational Foundation create the Lucas Education Research, and they did that with project-based learning. So they wanted irrefutable evidence that PBL works in every single grade level environment. They were going for the holy grail. So there's this gradient in between, and I think sometimes we might conflate like R&D has to be at that, it works and it's irrefutable. I don't think you need that, right? Maybe that's part of it of why it's so hard and people don't do it. There's this in-between stuff.

Katie Boody Adorno:

Totally. Totally. Like I said earlier, I mean it really is a journey and I think we often overlook, we capitalize the R in R&D and we forget about the D in education pretty much and so we forget that development is a crucial part of that. You have to go through that developmental process that's creative and iterative to get to something that is used with fidelity, that's remotely static that you can actually measure against other variables and we forget about that, but it's a crucial part.

So I would urge education companies that are at an earlier stage to think through, what can I do now that's oftentimes foundational research. You can build a logic model. There are many people that can support you with that. Digital Promise has some free certifications you can do for logic model development and then from there, really honoring that if you're doing human-centered design in earnest at the developmental stages of the product, that is still part of the R&D journey, and that's setting you up for success, because if you design a great product that has been co-designed with educators across a variety of contexts, you'll see an increase in usage fidelity, which will then unlock insights that can tell you more about how this product is facilitating learning.

Elana Leoni:

Yes, and anyone listening, I would rewind those last 15 seconds of what Katie said and replay it because she's basically saying, if you do the things and you ensure that you're integrating educator voice into your product, all of the things will come eventually too. You certainly want to measure that. All right, so if I'm an educator listening, I think that you provided some really great resources around how you can give feedback and educators listening, I want you to know that your feedback is so welcome and you are the front lines and I don't know, an educator founder, a founder of an EdTech company that wouldn't love to hear your feedback.

So don't ever feel that you're bothering people and Katie said, even just reach out to them on LinkedIn if you want, or Twitter or whatever. Give your feedback that way because you love the products, you use it, it is so welcome and then there's ways that there's organizations like Katie's organization that will actually pay you for your feedback. So we'll put all of those resources in the show notes as well, but if you're an EdTech company, there's a lot we've said, Katie, but you want some parting thoughts on, we have demonstrated it's widely needed.

We've talked about the reasons why it hasn't occurred, and we've given you some practical ways to start integrating educator voice and making sure you're being transparent of putting in your product roadmap, but is there anything else you want to say around maybe this is your chance to get them to the home plate of actually doing the research?

Katie Boody Adorno:

Yeah, yeah, sure. I think just doubling down on what I said before, I mean the development is part of the research itself. Qualitative insights and getting to a product that is used with fidelity should be the North Star for early-stage companies and happy to help and think through that. Also, just kind of a quick plug around the way educator roles are changing too. I really think there couldn't be a more important time to think about how co-development of education, technology products is happening. We're looking at teacher shortages, we're looking at teacher burnout and exhaustion, all the educators that are listening, not only is your role important to help improve technologies, but to help shift the sector to what extent does your role need to change and should these tools be following how educator roles need to change in this kind of new frontier of education that we're entering into.

So I think it's a kind of a call to action on all sides of how do we lean in and I think this is a once in a lifetime opportunity to co-create a new future, a new way forward that is evidence-based that honors educator's brilliance and capacity and might unlock kind of a new paradigm of how we do things.

Elana Leoni:

Gosh. So well said. I don't even want to add anything to that, because that was so beautiful. Why don't we shift gears. We could talk about this for a very long time. I think you have even changed my mindset around this so much. It's a beautiful thing to be able to approach something so complicated, but break it down into small actionable steps too and like you said, you don't just do the research, you want people to do things and have action items. So I hope the listeners, when you do listen to this around, okay, it's not so complicated sometimes we put things in our mind around this is such this big thing, this big endeavor, but beautiful things can happen in small incremental steps.

So why don't we switch gears in the world of education and you as a human being in this challenging but rewarding space of EdTech, you're also a founder of a company, I know what it's like in that roller coaster of a journey how do you, after challenging days, reward yourself by trying to figure out, okay, I am fundamentally drained, I'm going to look at a wall today, but I need to be on tomorrow. So how do you recharge your batteries during those tough challenging times?

Katie Boody Adorno:

Yeah, I love this question. I mean, I think most people from the pandemic I went through, it felt like two and a half years of staring at a wall alone and coming out of that recently, I've joined a hip hop dance class and I'm really into... Recently my fiance and I got a farm property. So we've been fixing that up and spending a lot more time in nature, and I've been really pushing myself to kind of get out of my comfort zone. As a founder I think I tend to be a bit of a workaholic, but I forget that I draw such great inspiration from things outside of my sector. So being in nature, watching how the seasons change and how agricultural cycles work, being in a dance class and analyzing new movement, all of those things feed me and help me think about problems in different ways that oftentimes gets me to answers faster and more unexpected ways in my work than otherwise. So constantly looking for inspiration. Otherwise, right now, those two are my most fun past times. Yeah.

Elana Leoni:

That's beautiful and those are those moments where like you say, you get out of your day-to-day and just naturally your brain thinks about the problem differently and you approach it differently and my team always comes back from vacation going, I've been thinking about this, but not intentionally, but it's totally different and it's shifted. So I love that you brought that in from two different, really different activities of dance and activity and different intellectual stimulation versus farming and getting out in nature.

Katie Boody Adorno:

Yeah.

Elana Leoni:

So Katie, thank you so much for your time. Gosh, I've learned so much in this 45 minutes, but if people want to learn more from you, how can they get in touch with you and your organization?

Katie Boody Adorno:

Sure. You can always email me. I'm Katie@leanlabeducation.org. You can find me on LinkedIn or Twitter and always DM me. It's @KatieBoody as well. So that's it.

Elana Leoni:

Awesome. So we will put all of these resources in the show notes. There were a lot of different things she put on the evidence exchange and the other places. We'll put all of that in the show notes and the show notes can be found at leoniconsultinggroup.com/52. So that's the number 52. You're episode 52 by the way, this is exciting. We're going past the 50 mark.

Katie Boody Adorno:

That's great.

Elana Leoni:

Which is a big milestone for us. Just me as a podcast host, not really knowing what I'm doing, but loving learning and learning from experts like you. So everyone, thank you so much for joining. We will see you next time on All Things Marketing and Education. Take care.

Thanks so much for listening to this week's episode. If you liked what you heard and want to dive deeper, you can visit leoniconsultinggroup.com/podcast for all show notes, links and freebies mentioned in each episode. And we always love friends, so please connect with us on Twitter @LeoniGroup. If you enjoyed today's show, go ahead and click the subscribe button to be the first one notified when our next episode is released. We'll see you next week on All Things Marketing and Education.