(Episode 160) Meta-Research Uncovered: The Role of Research Enablers

Meta-Research Uncovered: The Role of Research Enablers

So, um, welcome to our podcast session on meta-research and the role of professional staff. My name's Alix Brodie-Wray. I am a faculty impact development manager at the University of Leeds, and I am a fairly new podcast host for Research Culture Uncovered. Um, joined here today by, um, my good ARMA co-chair colleagues, Dr.

Chris Daley, head of research and innovation evidence and analysis at LSE, um, and Dr. Amy Milligan, who's head of research support and administration at the Saïd Business School at the University of Oxford. We wanna talk to you a bit today about what meta-research is and what research managers and administrators, or more generally research enablers, can, uh, input into the meta-research landscape.

So if you're new to the concept of meta-research, what are we talking about? Well, meta-research in a nutshell is research on research, research about research, the processes, practices of getting it done, getting it published, getting it funded, and uncovering any sorts of biases that will improve the research environment more generally.

I'll share some more definitions of that in the show notes. Uh, but without further ado, I thought I might give a little overview about why I'm interested in meta-research. So I don't have a PhD. Um, I did try to have a PhD at one point, but I quit. I was part-time. I was working on cultural studies, doing stuff around, uh, psycho geography, nostalgia, and vintage fashion.

But after about seven years of being part-time, I just kind of got tired of it. Various things went down and I think the fire just went out for me. However, concurrent to all that, I've been working in the research support space for quite a long time and, um, couldn't help but notice some, you know, reflect on our practices and some of the research culture complexities around how we do research.

And I got involved in a really cool project at Leeds about the experiences of staff in our rather large faculty of arts, humanities, and cultures working in the research support space, and their experiences of career progression, their experiences at work. And that really was such an insightful project.

It made me realize that even if I don't have a proper certified research background or, uh, anything too rigorous behind me, that some of those skills still came into play around Testimony and oral history even, if that doesn't sound too highfalutin, and just noticing the sorts of power dynamics and I guess more broadly the culture around research.

Um, so when the opportunity came up through the Association of Research Managers and Administrators for special interest group, uh, co-chairs, I thought, "Why not?" You know, maybe as a seasoned research professional, I had a fair bit of insight into research and wouldn't mind learning more myself and also encouraging other people to, to kind of recognise the skillsets they have.

I'd like to add as well, you know, not at all exclusively from my PhD, um, as much as it wasn't completed, and that work, but just from being ‑ environment and, and seeing how it worked and thinking about what goes on behind the scenes. So I, I'll turn to Chris and Amy and ask them a few questions about why they got into meta-research.

So, so should I start with you, Amy? Why did you get into meta-research?

:: 03:58

That's a good question, Alix. I think, um, it was a gradual coming to meta-research. I trained as a historian and studied the history of books and publishing, and did some research in that area for a while, including working on the history of Oxford University Press, um, which obviously publishes quite a lot of research, and I think became interested in the publishing of research, and gradually when the postdocs dried up in history, moved into a role in academic publishing.

And so looking at things like research integrity and the way research is, is reviewed, published, shared, disseminated from a publishing angle became quite interesting to me, particularly having come from a research background myself. Um, and then I saw this job come up at the University of Oxford for somebody to really manage all those processes, how research happens, how it gets published, um, from the, the, uh, academic's point of view I thought might be interesting, um, and, and applied, and it, and it's true.

There is a, there is a lot more than I had, that I had, than I had even thought about, I think, um, in research on research. Um, and it's been interesting bringing all of the strands of my own history, um, to inform how I do my job as a research administrator, I think.

:: 05:31

How about you, Chris? What kind of, what point in your career did you sort of think, "I could really call, I'd, you know, link into the meta-research landscape"?

:: 05:42

f in the kind of early to mid-: 2010

So I kind of first got involved in that. Then I actually got a job after I finished my PhD in a university library and discovered that there was actually a, a whole world of kind of practice-based research within the library community. So there were library journals, there were a lot of my colleagues who worked in kind of the more the metric side of things, were interested in scientometrics, research evaluation, and they published on this, and they, they published in journals.

And so I thought, "Oh, actually, you know, you don't have to necessarily be an academic to still be involved in some kind of research." And that kind of research you could be involved in if you were in a professional services kind of role was more of that kind of meta research, research on research kind of world.

So I kind of stumbled across it through working in libraries. But I think then when I moved into, um, more kind of research office roles, I was a- and I became slightly more senior, I was asked to do a lot of work, a lot of reporting work, um, a lot of work that was around leading programmes or initiatives.

And I kind of realized that the best way to get buy-in for these from both senior research managers, but also from academics who are try-trying to often sell programmes to, like, this is what we should do in terms of a research strategy and things like this, um, was to have strong underlying evidence. Um, and so I started- Mm-hmm

to kind of realize that the kind of approach I should take would be not just to come up with ideas and put it to academics, but also undertake some research and develop a kind of evidence base to support anything I did. So I kind of realized I was doing a kind of meta practice-based meta research through, through my day job as well.

So I, that's how I kind of stumbled into it, one, through libraries, and then through just having a bit more responsibility in my roles in research management.

:: 07:56

That's really interesting, you said about libraries, because here at Leeds, we have the wonderful Nick Shepherd. When I mentioned we were doing this podcast, he was like, "Oh, I know all this..."

He had this wealth of knowledge about meta research, you know, and it was, it was really interesting 'cause I think I've come at it fr-from a different angle to him. And I think for me it was about impact actually, to some degree. It was almost like the confluence of that impact agenda, as they called it then.

It gave us opportunities, I think, and I could see examples then, people in roles like ours contributing to that research. So that set a sort of precedent. And then I guess some of the research development work I did at the University of Brighton with the Concordat, um, maybe someone made a little quip that, um, you could tell they'd asked a researcher to do- To do this Concordat rollout scheme.

And I was like, "Oh, I think I'm actually... Yeah, there are, there are various skills going on here that I've got through just various different threads." Amy, did you have another thought on that?

:: 08:53

Yeah, I was gonna say it's, it's interesting that you mentioned people publishing in the field of meta-research, because that's another angle- Mm

that brought me to the subject. One of the journals I looked after was actually a research on research journal, and so speaking to the editors of that and coming up with ideas for, for the journal got me really interested in, in how the researchers look at research and how it's done, um, and how it might occasionally be done a little bit better.

Um- Mm-hmm ... and I think the evidence base is definitely part of it, but also applying that kind of scrutiny to research, um, and looking at what, what's fair, what's inclusive, what's successful, what's maybe less successful or less inclusive, um, of analysis is, is just as valuable for our ecosystem as it is for the other things that researchers study.

:: 09:47

Yeah. That idea of fairness and equity actually reminds me of work I've done on Athena SWAN. Do you know what I mean? So, uh, I was lucky enough to head up a working group, um, for faculty operations, uh, no, organization and culture, and, um, yeah, that was, that was fun 'cause I was leading academics for the first time, or herding cats as I think they sometimes call it.

But, um, it worked quite well and in a way that, that kind of EDI focus can be another way in, can't it? Where you, you might, you might come to a point in your career where you think, "I've got something to say about EDI at a university." Where the... You f- may feel more comfortable even starting out in that space, which is like, you know, uh, explicitly inclusive in terms of figuring out, you know, how your skills fit in potentially to what is effectively better research, even if people don't call it that.

So Chris, I'll start with you this time. So what do you think that research managers and administrators or research enablers more broadly can contribute to meta-research that might be distinct from academics?

:: 11:03

Yeah. Um, so I think the, the primary area of expertise that we have is around kind of understanding what actually functions on the ground in terms of- Mm

the research innovation ecosystem. So for example, you just think of someone who's in a Or think of the range of activities that there are within a research office. It covers a kind of broad breadth of things from pre- to post-award. So people who work in sort of research professional type roles typically have kinda intricate understanding of things like funder policies and strategies, detailed knowledge of like grant applications and the actual processes across mul- Mm

multiple different funding bodies. But they have knowledge about kind of, uh, things like research integrity, research security, regularly scrutinize government R&D policies, uh, so picking out what this might mean for different university types or different disciplines, et cetera. Um, and I think also the other key role is around kind of this translational work.

So the meta- meta-science, meta-research, research on research is, is quite popular with academics. There's a kind of broad- Mm ... range of academics across a range of disciplines who are involved in this. But so there is a kind of a gap there for people who have these kind of translational skills, who can kind of take the academic evidence and find ways of translating that into kinda concrete policies or kinda process proposals.

And I think that that's where kinda people who are in more professional roles have a kind of really important role. And I should also add that I looked at the UK Meta-Science Unit. I looked back at their first round of research grants and what they put in their core guidance, being a kind of good research manager, um, and they, they, they defined, really helpfully, they defined meta-science in two ways.

They said on the one hand that it's researchers using rigorous social scientific methods. But then they said that it was also, and I'm quoting here, "a community of practice united by an interest in designing, implementing, and evaluating innovative modes of science funding and delivery." So they've recognized that it is both about rigorous social scientific methodologies and approaches, which would probably be more dominated by academics, but then it's also about then using that evidence to actually kinda transform the research innovation system to make it more equitable, to make it more efficient, more effective.

And that's where really people in more professional roles are really gonna be crucial. Um, so yeah, I'd say that's where, where we kinda come in.

:: 13:47

Yeah. Amy, any additional thoughts there?

:: 13:50

Yeah. I think, I think I agree with all of that. I think meta-research can enable research managers and administrators to contribute to research by providing evidence about the things they do.

They can gather data on their own processes and activities and advise on policy, as Chris says. But I think the other- Part that interests me is that we can put those research findings back into practice in a way that researchers can do sometimes, um, but in their own, own activities. I think research m-managers and administrators can affect perhaps a larger group, um, with a, a positive change.

So if we introduce a new way of supporting people with grant applications, that will affect a number of people and perhaps make a, a positive change across, um, a larger group. And I think that that can be really valuable. I think our input in policy can do that too. So yeah, I think it's, it's both contributing information, expertise, and data about how we do what we do, but also then making sure that the, the feedback loop is complete and putting a kind of new policies, recommendations, and, and evidence into our own decisions about how research is managed and supported.

:: 15:18

Yeah. That's brilliant. And I think that, you know, reflecting what you said as well, Chris, around the meta-science unit, um, and that two-tier kind of understanding, it's like bringing in an equal weight to those aspects of, um, of meta-research or meta-science. Um, 'cause I think it's, it's a, it's a sort of perennial problem perhaps in the research space.

I think it's reflected in the way that we... You know, when we're having conversations around the academic and non-academic divide or around certain types of research at a research intensive institution, there's a little bit of lopsidedness there, isn't there, in terms of understanding the theory versus the practice.

So I think that's what's quite nice about research managers, administrators, and professionals, and enablers being involved in this in that way. It's that reflection of equal respect or equal recognition of both the theory and the practice and, and our role in the theory as well. Does that make any sense?

Almost see it like two shapes, you know, where one's normally more highly esteemed and visible than the other. Practice in so many different contexts in the university, and as an arts and humanities person, you know, we talk about practice research, but, you know, practice more generally and, and applying research, y-you know, is a big part of what we can bring into it, but not, but not the only part.

So instead of seeing like one big chunk of monolith of recognition and esteem, and then this sort of us holding it up, you know, like Atlas or something with the, with the world, it's like that interplay between them and that like parity of esteem. That's my utopia anyway, which is nice. So You know, I did touch on a little bit at the beginning of my intro about, about my non-PhD- Mm

the PhD that never was, and whether or not that had anything to do with my interest in meta-research because I'd been a re- I'd been an administrator for much longer than I'd been a researcher. So, you know, I started off 20 years ago at the University of Brighton. I was lucky enough to do my master's for free at Sussex, uh, because we shared a med school, and then I was doing this part-time PhD moving into the research space.

I already had this massive background of dealing with spreadsheets. I'm just gonna put it very bluntly. I mean, it's more complicated than that really. Um, but dealing with information and data and negotiating the higher education landscape even before I started off. So for me, I'm not sure- Yeah ... how much it was to do with my PhD at all or my non-PhD, and maybe more to do with the practice I even had generally in terms of basic kind of information handling skills, um, that sort of drew me meta-research as someone who might feel outside of that.

What do you think? Do you need a PhD to do research? Who wants to go first? Anu, you look poised to go first.

:: 18:19

I'll go first. No is the simple answer. I, I do have a PhD, but- Mm ... I did research in order to get my PhD, so clearly you don't need it to get started. Yes.

:: 18:30

Of course.

:: 18:31

Yeah.

:: 18:32

That's so funny, isn't it? We think of it as like the thing, and it actually what's led into it- is the thing.

:: 18:38

So, I mean, I, I won't say that it hasn't helped. Um, certainly having a solid grounding in particular research methods which you learn in a PhD can be really helpful when conducting meta-research, but it's not a, a, an absolute requirement. I think certainly solid information handling skills, I think is what you said, that can be learned in a variety of ways, and the academic route is one way to learn that.

But on the job, um, is certainly another and a just as valuable way, um, to, to learn that skill. I think I would say that when looking at the field of research, it is helpful to have an understanding of how universities work and researchers work, and I think we can gather that as administrators, but it also helps to talk to academic colleagues if you're thinking about things that particularly affect academic colleagues, so, um, career paths- Mm

for example. So having the, the access, I think, to a, a lab or in our case a university or research institute, that's valuable too and, and you can kind of fill in the gaps in your own expertise perhaps by talking to other people or bringing on, um, colleagues like Chris who have- Really strong data skills if that's not your cup of tea, or perhaps you're better at, at talking to people and doing the kind of qualitative approach, and your, your colleague can rely on you for that.

So I think you don't have to do it all by yourself, so you can definitely fill in some of the, the gaps in your own knowledge by working with really helpful and intelligent, knowledgeable colleagues. Mm-hmm. But no, you don't need your own PhD. You just need a solid plan, a lot of support, and the willingness to learn a new skill.

Hmm. That's great. Chris, anything to add on that?

:: 20:33

Um, yeah, I'd agree with Amy that you don't- Yeah ... have to have a PhD. Um, it can be, can be helpful, um, just because you've had experience of delivering a research project in that regard. Mm. But it's, in terms of the kind of specifics, uh, specific knowledge might not necessarily be helpful.

I mean, I did a PhD in, uh, a humanities discipline. I was looking at British science fiction and the Cold War for my PhD, so... And I've moved now to a role where I'm looking at large data sets, and I've had to sort of re-skill in some ways and do kind of courses in data analysis and things like this that, um, you know, they didn't require a PhD.

These were things that I learnt on the job in my research management job. And so it, it can be helpful, but it's not absolutely essential. What I would say is important though is that you do need to have a kind of intellectual curiosity about the research and innovation ecosystem. So the ability to kind of think just beyond your day job, as it were, to think about how your role contributes to kind of the broader system that you're in, I think that's really important for any kind of good meta-scientific research, is that you have that kind of...

You think, "Right, I do this role, but how does this relate to the kind of broader national system, the broader international ecosystem, et cetera?" Be just interested in those kind of things. I think that is important. But again, it doesn't need a PhD to do that. You can just read around the subject and become interested in it.

etrics this, uh, this year in: 2026

And in order to kind of build up a project team during that sandpit, I needed to kind of work with not just people who are in the same sort of role as me, but I had to reach out to academics as well. So- I think that ability to kind of be able to bring people together who are from diverse kind of roles, so be able to kind of speak to academics and talk to them about what you would bring to a project or how you might want them to help you with a project, I think is really important.

So that kind of ability to navigate a very kind of interdisciplinary environment is really important as well. But again, it doesn't ne- mean, mean you need to have a PhD. You can, you can-- There's lots of people who could do that with just being great networkers, basically. Um, so yeah, they're the kind of tips I would give for being involved in this.

:: 23:14

Oh, that's interesting. Yeah. Yeah, that's really interesting. I've lost my train of thought now. I had a little follow-up there. I'll cut this bit out. At the ARMA conference last year, we did our survey, didn't we, into how research managers, administrators coming to the conference felt about meta-research, being involved in that.

And aside from time as a barrier, there was a sense from some people that like, "Why should we?" So it was really interesting what you were both saying there about, you know, finding those connections or, you know, not everybody has to do it. Not everybody has to do meta-research. So we have this sometimes, you know, especially with the really well-intentioned work that we do through the technicians' Concordat and encouraging technicians to be recognized for their input into research, of accidentally placing the same pressures on that staff group.

And it's came through a little bit, didn't it, in the REF, you know, planning exercises that were going, going into the, the latest iteration of the REF and people were talking about why. And I've been in rooms where academics have said, "Why would we want to include research professionals and technicians?

Surely they're not expected to do research." Um, so I think I always want to sort of say, you know, if you find yourself drawn to this, it's so useful the advice we've had so far about doing that. But, you know, we often have to be careful of some of the unintended consequences in terms of pressure that people feel when we talk about this.

So it's almost an option, isn't it? It's a direction you could go in. It-- people shouldn't feel like they have to go in. Yeah. No.

:: 24:58

Yeah. Um, yeah. So just to jump in. No, I don't- Mm-hmm ... I don't think that it is. And I think actually it's important to kind of make a caveat around for anyone who wants to get involved in meta-research who's in a professional services role, that it is a time-consuming thing to do.

Mm-hmm. And that there is a kind of barrier around what our roles are designed to do and the kind of additional pressure that you would add on top of it by being involved in a kind of meta-research project. Because, uh, so I, I work with RLUK for their-- they, with the AHRC, have a programme for librarians to kind of build up their confidence for Applying for research grants.

Um, and one of the kind of major barriers and areas that we have to navigate through that, that scheme is the way in which people say, "Well, my, my contract is for me to deliver particular services for our academics or students, et cetera. So how am I meant to find the time to sort of like move into researcher mode and do, do kind of a meta-science project?"

Um, and so I think that that's something that we need to be very frank about, that, that, that is a challenge, um, because I mean, I've, I've have this meta-science unit grant, and I find that it's quite tricky 'cause I'll be going from my normal week where I'm having one-to-ones with people in my team, having team meetings, having regular meetings, having, you know, delivering to deadlines that are set like week by week and stuff.

That's just part of my day job. And then I kind of find I've also got to do this grant as well, and I have to switch- Hmm ... into a kind of different mode of working, which is very much around kind of a... I would describe it as kind of long form work where I'm having to kind of really interrogate this material in quite a lot of depth, have to kind of think about how I'm gonna disseminate these findings in terms of academic papers, in terms of policy outputs, things like this, which is a kind of completely different way of working to my day job, which is much more, you know, I work in a kind of role that is a bit more kind of research-y foc-focused in a way, but it's still a professional services role where I have kind of deadlines to meet and regular meetings to attend and things like that.

So I think that it's really important that we are frank with people, that is, it is challenging, but it's not impossible and it is actually very rewarding if you can do it because of the thing I spoke about earlier, about that ability to collaborate with really interesting people across the ecosystem.

So it's... I'm working at the moment with computer scientists at the University of Edinburgh and going up there next week, and that's like really, really exciting to be doing that and to be working with these people who are brilliant. Uh, but it, but it obviously is a bit different to what my day job is, and I think we have to just be honest with people about that.

:: 27:47

Yeah. I think that's true, and I think that some institutions will be more open to professional services staff taking on research projects in a, in a significant way. And I think part of the reason they would be open to it is because those research projects are usually designed to help research work better, more efficiently, more inclusively to improve things.

And I think that's one reason why professional staff are interested in participating. They, they also want to make things work better, and I think that drive to share your insight into how things might be improved is one good reason to get involved in, in meta-research. But I think that's a reason to do it yourself.

I think we should also Remember that there are other ways to engage with meta-research which don't require you to apply for a major grant and take on a whole new job outside your regular job. There are often calls for participants to share their expertise in ongoing research on research. So if you see a survey about how professional services staff feel about this policy or that process change, feel free to lend your expertise because that research might then inform a meta-research sci- study which goes on to improve a policy or a practice.

Similarly, if you are interested in research findings and integrating those into your own thoughts about how you do your job or perhaps improving a process locally, I would encourage you to explore some of the meta-research that's already been published so that you can add some of that, um, knowledge to your own, um, kind of ways of working.

:: 29:27

Yeah. That's so useful. And I was thinking as well about changes in the higher education landscape and the way we do research, and I know a lot of institutions it's felt a bit like... I'm not sure how universal the experience is, but I've, I've kind of gathered from speaking to other people with this sort of shrinkage of the sector a little bit, p- maybe people feel a bit inhibited either because of the structures, they're sort of clamping down on this is academic, this is not academic.

Maybe there's a bit of a sort of rollback on what a few years ago people were calling the third space, just in terms of capacity. But I really liked what you were both saying about the, the benefits as well of that, while recognizing the limitations or the, the moment that we're in, which seems a little bit precarious.

Um, so that kind of leads on to my next question. What next for meta-research? I know we had a lunchtime drop-in which focused a little bit on, um, the Meta-Science Unit and the opportunities there, but what next more generally for the involvement of research enablers and managers and administrators in the meta-research landscape?

Chris?

:: 30:48

So there's obviously a recognition within government that this is a really important Area that it wants to fund, um, because it feels that the, as Amy was talking about, that the kind of benefits, uh, are gonna be really substantial if we kind of apply kind of evidence-based findings to the research system.

This will kind of produce great efficiencies, potentially kind of make it fairer, make it more transparent, make it more creative, and things like this. So there's obviously an appetite within government to fund this. And really interestingly, I've seen just in the last few weeks, a number of adverts from organisations like RAND Europe and the Tony Blair Institute have been advertising for roles that are kind of for people working in their R&I policy and metascience teams, um, in kind of officer type roles, and that's really interesting.

So they've obviously picked up on the fact there's been greater investment from government. They're also seeing this as a big thing within research and innovation funding and research innovation policy, so are invested in that in terms of staffing as well. Um, and, and I wonder whether th- this might translate into universities as well, um, that there starts to be more of a kind of interest in having these kind of roles that kind of use data and evidence to kind of help with decision-making at a local level.

And, but I think it's really important that research managers and administrators really kind of speak up about our expertise and our role in this area and don't feel kind of afraid to be part of this because I think metascience generally... So I was at the Metascience conference at UCL last year, and it-- while there were quite a few research managers there or people from funders and things like this, it was very academic dominated, you know, largely social scientists, I would say, but it was very, very, very researcher dominated.

But as we've seen from the metascience definition of what the landscape is, there is a really big part for us to play in that. So I think we need to be confident and speak up for our role because I think it is, you know, funders and government do wanna hear from us, so I think we should be confident in that.

:: 33:25

I think that feeds on really nicely to what my answer to that question was, what's the future for, for meta-research? And I think much of the meta-research that's already happened has been really focused on the publications, so kind of looking at the bibliometrics and at open science publications and at academic research culture and career paths and so on.

But I think there is a growing understanding that the ecosystem of research includes a whole variety of career pathways of different people with different skills doing different jobs. And so I think really the future for meta-research is to start thinking about the ecosystem as a whole and how the other parts of that landscape, we keep using these big words, but how those kind of are impacted or impact research.

And I think very much That research managers are going to be a part of that conversation, both as research subjects, but also as knowledgeable experts on this world in which we are all operating. So I think that expansion of meta-research to include a broader vision of what supports, enables, and constitutes research is the next step, really.

:: 34:38

:: 35:00

Publishers.

:: 35:01

And partners. Yeah, publishers, of course. And then, you know, I think about, you know, I work in the impact space, so you know, sometimes you see fascinating looking roles working for charities or other organisations, um, around evaluation. It's, it's almost like-- And I can kind of see we might feel like the sector's contracting, but maybe there's actually a broadening of like our skillset in, in different settings, which I think is a nice optimistic note to end on.

So, um- Mm-hmm ... thanks so much Chris and Amy for talking to me about meta-research. It's endlessly fascinating.

:: 35:41

:: 35:42

Share Episode

Shownotes

Transcripts

Follow

Links

Chapters

Video

More from YouTube