This episode features RISE Research Fellow Yue-Yi Hwa in conversation with Jennifer Opare-Kumi, a final-year doctoral researcher at the Blavatnik School of Government, University of Oxford.  They cover a breadth of issues including the potential for targeted instructional programs to contribute towards improved child mental health outcomes, why mainstreaming children’s mental health during early learning might improve their educational and other life outcomes, and the need to adopt an expanded view of the ‘learning crisis’ currently affecting countries in the global south.  

Links

• Jenn Opare-Kumi (webpage)
• Foundational Learning and Mental Health: Empirical Evidence from Botswana (working paper) by Jennifer Opare-Kumi
• Cognitive and Socioemotional Skills in Low-Income Countries: Measurement and Associations with Schooling and Earnings (journal article) by Alice Danon, Jishnu Das, Andreas de Barros, and Deon Filmer (RISE Pakistan)
• Socioemotional and Academic Learning Before and After COVID-19 School Closures (working paper) by Stephen Bayley, Darge Wole Meshesha, Paul Ramchandani, Pauline Rose, Tassew Woldehanna, and Louise Yorke (RISE Ethiopia)
• Researching Socio-Emotional Learning, Mental Health and Wellbeing: Methodological Issues in Low-Income Contexts (working paper) by Stephen Bayley, Darge Wole Meshesha, Louise Yorke, Paul Ramchandani, and Pauline Rose (RISE Ethiopia)
• Measuring the impacts of teachers in Vietnam: teacher value-added and student cognitive and non-cognitive skills (paper in progress) by Pedro Carneiro, Paul Glewwe, Anusha Guha, and Sonya Krutikova
• Youth Impact’s page on Teaching at the Right Level in Botswana (webpage)
• Teaching at the Right Level Africa (webpage)
• Pratham’s Combined Activities for Maximized Learning (CAMaL) teaching-learning approach (webpage)

Guest biography

Jennifer Opare-Kumi

Jennifer Opare-Kumi is a Doctoral Researcher at the Blavatnik School of Government. Driven by a passion for efficient, evidence-based policy making, she researches ways to improve education and mental health outcomes for young people in the Global South through government and non-governmental interventions and policies.

Yue-Yi Hwa

Yue-Yi Hwa is a Senior Education Specialist on the evidence translation and synthesis team at the What Works Hub for Global Education. Previously, Yue-Yi was a research fellow and research manager for Research on Improving Systems of Education (RISE), where she focused on synthesising research on teachers and management. She has also been a research fellow for the Penang Institute in Kuala Lumpur and a secondary school English teacher in Selangor, Malaysia. She holds an MPhil in comparative government from the University of Oxford and a PhD in education from the University of Cambridge, where her thesis looked at interactions between teacher motivation, accountability policy and sociocultural context.

Attribution

The continuation of the RISE Podcast has been made possible through funding from the UK’s Foreign, Commonwealth and Development Office. The Blavatnik School of Government at the University of Oxford supports the production of the RISE Podcast.

Producers: Julius Atuhurra and Katie Cooper

Audio Editing: James Morris

Welcome to this episode of the RISE podcast. My name is Yue-Yi Hwa. I'm a Research Fellow with the RISE Programme and today I'm speaking with Jenn Opare-Kumi who is a final year doctoral student at the Blavatnik School of Government at the University of Oxford. Jen and I talk about a study that she conducted on how teaching at the right level in Botswana affects children's mental health outcomes. And along the way, we get into many exciting questions like tricky issues around measuring squishy things like mental health, what happens when you talk to parents about their children's mental health, and classroom teaching that is so exciting that people actually ask for more math lessons. Welcome to the RISE Podcast, Jenn.

Thanks Yue-Yi.

I'm so delighted to have you here. And I'm looking forward to this conversation for many reasons. And one of those reasons is that, you know, we've had research from RISE Ethiopia, looking at social emotional learning alongside academic learning. We have work in progress from RISE Vietnam researchers looking not only at academic outcomes, but also non-academic ones, like executive function, self perception, independence. And we also have your RISE working paper about foundational learning and children's mental health in Botswana. But we haven't had the chance to discuss any of these on the RISE podcast. And I realize, I've just thrown around a lot of terminology about various aspects of children's well-being and development there so to help us set the scene what exactly do you mean when you talk in your paper about mental health?

Thank you. Yeah, that's a great question. And thank you for inviting me today on the podcast. I think sometimes we use social emotional learning non-cognitive outcomes, which you've mentioned a few, mental health and well-being quite interchangeably. So when I speak about mental health in this research study, I'm using WHO (the World Health Organization's) broad definition of mental health, which looks at a state of well-being in which individuals realize their own potential, their capacity, can cope with the stresses of life can work productively, and are able to make contributions to their communities. And similarly, the Lancet Commission of Global Health looks at mental health as an asset or resource that enables positive states of wellbeing. So we see how mental health can really be encapsulating of the potential, the human potential, our ability to access that. And so I think when we think about mental health, what has been helpful is to classify it a little bit into two different buckets or classes, thinking about externalizing mental health symptoms, which would be your disruptive behaviors and attention, hyperactivity, and internalizing mental health symptoms, such as anxiety, depression, and trauma. And often we hear more about the internalizing mental health symptoms or problems, rather than the externalizing ones. When we talk about mental health, they all include all of these symptoms.

Great, that's really useful background, Jenn. And I think the the concepts you've started to flag for us already suggests answers to this next question I'm going to ask. So I just wanted to get into the fact that, you know, we have so many efforts now to improve children's foundational learning, and why alongside these efforts, is it really important to shine an additional spotlight on mental health? Or another way of putting that is what's the connection between children's mental health and their academic or their cognitive learning?

Yeah, so in this research study, I think it's straddling a line between research and advocacy for more attention to children's mental health and mental well-being, especially after our worldwide collective experience of COVID-19. We can all, in general terms, experientially talk about our experiences of mental health or being in good mental health or well being, we can understand why that's important. And academically, I think we need to catch up in terms of the global public goods that we have as research outputs. And in terms of what public policymakers I would say, look at when they think about investing in in different areas in development, and global children's mental health is an area that is under resourced and researched, underfunded. So we know currently, the evidence shows that 10 to 20% of all children and adolescents have some type of mental health challenge. And even more devastating in that is that 90% of the global population of children and adolescents actually live in low income and middle income countries. So in the Global South, the majority of our children will have some kind of mental health challenge as they grow up and as they develop, as a recent study in Sub Saharan Africa shows that 1 in 7 children had a significant mental health difficulty and this covered depression, anxiety, disruptive and reactive behaviors and disorders Post Traumatic Stress Disorder. I think this area highlights the intensity of resource investment that is required to ensure that all children grow up healthy. And with a sense of overall well-being and life satisfaction.

I'm going to ask a follow up question to that, that might go in a couple of different directions. So the background to this follow up question is, as you know, I come out of the RISE thematic synthesis team, where we talk a lot about how it's really important to prioritize foundational literacy and numeracy as the reason that, a fundamental shared reason across education systems, regardless of social cultural context, and also because when you have complex systems, complex social policy delivery, prioritization really matters. But as you're seeing very clearly, you know, as I know, as well, from my only two years of classroom experience, factors like mental health can have such extensive knock on effect. And it's really hard to tell what might be a binding constraint. And in some settings, this may well be a binding constraint. So I'd actually like to, this might be a leading question because of my biases, I hadn't actually thought before you mentioned it about externalizing symptoms as a part of mental health, because I would just think of those I'm used to thinking of those as like disruptive behavior. Yeah. And classroom management is something I struggled with a lot as a teacher. So could you help connect the dots a little bit, now that my long preamble is over, about two years of classroom learning, and specifically externalizing mental health symptoms in the open system, that is the classroom setting.

It's so challenging, right? In classrooms that I've been in and observed, and teachers that I've spoken to, typically what you would maybe remember also having experiences that these quote unquote, difficult children are classed as such, and almost ignored in the classroom setting. So the rest of the children can get on with the learning of the day, teachers are under a lot of pressure, of course, to get through their syllabus and good time, and make sure everything is done. And if it's one or two children, those children, maybe there is a learning difficulties teacher or school lead, that the teacher then thinks, you know, I'm doing my best I'm gonna assign so and so and so and so to, you know, to, to this learning difficulties school lead, because I just don't have the time, I just need to get on with it. But what we see is that these externalized mental health problems link back to academic success or the lack thereof. And that links back to more internalized mental health difficulties such as trauma, anxiety, depression, which knocks back on to academic indicators of success and learning. So you can see the cycle is so connected and linked, that I think it leads us to think about how do we use the ongoing CPD (continuous professional development) that is already underway. Is there a way to incorporate an awareness, skills tools to support teachers to better identify and support these learners to ensure that some of these, these issues and difficulties can be reduced or stopped before they progress. And some of it is not overly complicated. There are multiple interventions that happen in schools, because we are all invested in children learning. And the argument I'll make today is that these interventions with children in schools can be leveraged for other outcomes that supports children's holistic development.

And that leads nicely into the next thing I wanted to ask you about, actually, which is so the fact that one thing that's been discussed quite a lot in the intersection of education and international development now are targeted instructional programs, which have been found to be quite effective in improving foundational literacy and numeracy across quite a wide range of contexts. And in the paper that we're here to talk about today, you look at one such program, so teaching at the right level and Botswana, and you find that a positive impacts on children's mental health. So can you tell us what this program did? Your findings about children's mental health and also out of my personal curiosity, why you wanted to do the study in the first place?

Okay, so this is a multifaceted question. So I'll try take it piece by piece and feel free to interrupt me at any point to ask for more detail or maybe even less. So in terms of this research study. What I wanted to do was leverage an ongoing intervention in Botswana, teaching at the right level, as you just mentioned, and I believe that listeners of the podcast would be familiar with this intervention, but for maybe for newness I can make a very brief introduction, but teaching at the right level is a targeted foundational numeracy and literacy intervention, which has been backed by multiple evaluations and proven to improve learning outcomes, improved learning outcomes in multiple contexts, as well as multiple delivery models. And by that we mean whether it is the teacher, volunteer facilitator, each time the intervention improves learning outcomes. And the innovation really here is a simple one in that when we think about how children are typically learning in the in basic classrooms, is that they're grouped by their grade level and not their learning level. So teaching at the right level says, you know, why don't we group children guided by their level of knowledge, rather than the assigned grade, and targets instruction in a fun and interactive way, for a set period of time. And, you know, through this regrouping, and assessments and targeted instruction, children learn foundational skills are and are numerous and literate. So in Botswana, this intervention is being rolled out, in collaboration with Young Love now called Youth Impact, and the Ministry of Basic Education, who have all committed to adapting and scaling teaching at the right level in all primary schools in Bhutan. And previously, I was the Research Lead and Program Manager for Youth Impact, where I was supporting to scale teaching at the right level in all primary schools in Botswana. And in that role, I had the opportunity to to chat and observe multiple students, teachers, parents, Ministry of Basic Education officers, and one thing was becoming very clear is that one, teaching at the right level works, and you can see it, and it's such an inspirational thing to see, children that have over and over again, parents have maybe lost a little bit of hope, because they're really struggling. Teachers also feel slightly exasperated with not knowing how to support this child with their current set of tools that they have to explain certain concepts, and incomes, pedagogy that really, I think, expands the options, expands the options and our communication style, our teaching style, the way that we bring students into the learning process. And we could all see this happening. And it's such a nice thing when what you see is confirmed with rigorous evaluation and, you know, measurements as well. And I think we've spoken about this previously before it is it's so satisfying that what you intuitively know is also confirmed but with evidence.

And what you hope to be true and possible in the world.

Exactly. It's so hopeful. So as that was happening, what I was also observing is a certain level of confidence and comfort with the students. Students were eagerly and easily raising up their hands to respond to questions. And this is no small feat in a classroom. I'm sure you can attest to this. But usually we have all the kids that are super confident and know the right answer that raise up their their hands. But in a TARL classroom, even kids that don't know the answer and are probably wrong, still raise up their hands because they want to participate. This is so incredible. This is actually this is game changing for learning. And that the facilitators are not just kind and they atone, but also inviting of children to try even when they are wrong. And their other peers are supportive, when the student is wrong in also offering their responses, but not in a way that brings down the students that was wrong. So there were many things that were happening in that classroom that we were observing, that hadn't been systematically, evaluated, documented. And then speaking to teachers as well. They were saying, you know, children seem happier to learn, they want to learn, they're more confident. They just want more of TARL, which actually means they want to more of learning. They want more classes, which I mean, I don't know the last time you spoke to a primary school student, but no one says I want another math class, but they're saying I want another TARL class. So this really inspired me to want to understand more systematically, what was happening in these TARL classrooms. Was there anything that we could more rigorously evaluate outside of the learning outcomes, we already know, teaching at the right level improves learning, fantastic. But is there something else that's happening, which would be so beneficial to understand? Because it would help children beyond the academic success, but actually life success in terms of the developmental indicators and outcomes? So I really set out to understand, you know, are there any other other other mental health outcomes that maybe we are, we are overlooking, and it's such an opportunity to understand this in the school setting, because if we can replicate this, then we can, we can do this and other places as well. So I was really excited by that notion that something else could be going on. And I think the the basic hats of a researcher is that curiosity, you know, we want to ask these questions, we, we observe something, we think about what we've read in the literature. And we're like, Okay, I think this is a gap. Let me go investigate that. And that's what happened in that moment in a classroom, looking at a very eager young child answering questions, happily interacting with their peers. And I just thought to myself, you know, what else is going on? I would love to find out.

That's super exciting. And what did you find out?

Oh, great, thank you for bringing me back to what did I find out? So what I did is I worked closely with one of the education regions in Botswana called North East region, specifically the city of Francistown, where I worked with 15 schools that were implementing teaching at the right level. And the way that they were doing that was by classes. So for example, class four, A is doing TARL, and four B is not doing TARL in term one. But in term two, for B, we'll be doing TARL. So in each school, I had treated "treated" schools, the schools that were implementing teaching at the right level, and control schools, schools that were yet to implement teaching at the right level. So I was able to compare these at baseline at endline, I used what's called a difference in difference methodology. And what I do find is that there are meaningful differences between the children that are in teaching at the right level, and those that are yet to be exposed to teaching at the right level, the kids that are and teaching at the right level, have significantly less emotional and behavioral difficulties compared to the kids that are not yet in teaching at the right level. And I found that so hopeful, and really the tip of the iceberg for multiple studies to similarly want to use a battery of instruments to test this.

Great. That's, that is, as you see, so incredibly encouraging and promising. And you just mentioned a battery of instruments. So that points to something I want to get into next, which is measurement. So I think a lot of what you've said just now pointing so strongly to the fact that in this complex world that we live in, things are bundled right learning outcomes and mental health are bundled together. interventions like TARL, which might be called targeted instruction, involve a lot of other things. It's not just the targeting and the differentiation, but there's bundled together with tools and routines and educator peer networks and coaching that facilitate joyful pedagogy. But because of, you know, limited human working memory, etc. Sometimes we do need to parse out specific elements of these bundles. And measurement is one area where that comes up, right? So. And I guess I'm interested in two areas of that passing up from the bundle and measurement. So one is that in your study, you use two psychometrically validated screening tools for Children's Mental Health. And your enumerators collected both student self reports and teacher observations, what students? So from the sort of evaluation point of view, I'm curious to just hear your thoughts about how difficult it was to do all this measurement of mental health and how satisfied you are with these tools for measuring mental health in the field in the real classroom in the moment, the mess of the everyday. And the second part of that it's that I think another domain where we often need to kind of pick things out of bundles and try to standardize them is in global monitoring and advocacy. Right? So besides the first area of evaluation second areas, I'm just curious, could you foresee a convergence to some kind of universal baseline benchmark for Children's Mental Health Systems the way you know, we have infant and child mortality rates as baselines for public health systems and there seems to be a growing consensus about certain measures of foundational literacy and numeracy as a baseline for the learning part of education systems. So any and all thoughts on this messier are welcome.

Yeah. So before I get into the instruments, it just occurred to me when you were talking about Teaching at the Right Level again that in India where it originates, Pratham also call it KAMAL, which is an acronym. But in Hindi also means magic and wonder. So it's it's such a wonderful precursor to what we're finding as well, with all the things that TARL can be. But thinking about measurement, I think it would be wonderful to have some sort of repository of measurement tools that are tailored for lower middle income countries for global south countries for low resource settings. And then the study, as a doctoral researcher, I wish I had, you know, more resources, more manpower, a team to spend a very long time doing this. But I did the best that I could with what I had and spent a decent amount of time looking out for various tools that had good psychometric properties. So thinking about validity and reliability measures that were relevant for my population use. So making sure that it wasn't a tool that was used for an adult population, but was relevant for young people, and had a feasible scoring protocols. So you know, these, these were basic guiding principles, but more. So what was really important was a very systematic back translation, and loads of piloting. So speaking to lots of children testing out speaking to lots of parents testing it out. And what is interesting, as you're piloting tools is a little bit of I think an unintended outcome that I found in the study is that the parents were telling me that, oh, Jenn, I didn't actually realize that I could speak to my children about some of these things. I didn't realize that I need to ask my child some of these questions. I didn't realize that I don't know the answer to some of these questions regarding my child. And I think even in of itself, that is a valuable outcome for the communities that I was working in. Because, again, with expanding the options, which I think is so valuable, in any context, is giving whoever you're working with more tools to engage, whether it is with the students or with their child.

Yeah, having having the language to articulate and understand our realities matters so much, right. But what about uncertainty, the global competitive level, is this a domain where having a narrow measure will be useful because it can focus attention? Or how does that balance against the trade off of, you know, the whole Goodhart law thing of, if you make a measure of target, it becomes a bad measure?

Right, but we need to know what we're measuring in the first place. And if it's something that's important, so it is a catch 22. But I will say that I think a lot of this will be driven by big funders, from what I know, about some of the big funders are driving the use of common mental health metrics. So the Wellcome Trust in the UK, the National Institutes of Mental Health and the US are really pushing to standardize mental health measurements, just for the for the general use of being able to compare to being being able to understand where increased investment is needed. But I think this then becomes valuable at a local regional policy level, where we can use those same tools to also understand where we invest local resources, where we, we realize that maybe there is a capacity gap and where we can reach out for more support. So I do understand the trade offs and the limitations, but we have to start somewhere.

That makes a lot of sense. And I mean, it'll be fascinating to see along the way, what sorts of social cultural differences and conceptualizing will help emerge as these discussions continue to happen. Well, Jenn, the final question that we ask all guests on the RISE podcast is, what is one thing you wish other people knew about education systems in Sub Saharan Africa?

Thanks, Yue-Yi. So I interpret your question a little bit as a call to action. And I think the "crisis of learning" that we know to be affecting millions of children worldwide, not only limits the quality of education, but also what I've argued for today is their holistic development potential. And an Africa this crisis is, of course, compounded by the lack of context, policy relevance data on young people's mental health and well being as well as their education outcomes. So in this context, I think more research is definitely needed in reclaiming this learning crisis, highlighting the need for policy reforms. Prioritising pedagogic approaches which facilitates holistic learning and development, particularly in Africa. And I think overall, we really need an increased and genuine South South collaboration and research. Thanks, Yue-Yi.

Wonderful. Thanks so much, Jenn. And here's hoping that many people RISE up and respond to your call to action. Well, it's been a pleasure, Jenn, thank you for being on the podcast today.

Thank you. This was fun.