How do you respond to peer reviews for journal articles? Do you respond to every single reviewer comment? How do you respond to this strategically? On this #AskAnAcademicAuntie we have Dr. Heather Millar (@hlmillar) and Dr. Carmen Ho (@carmenjho_) dispensing a little auntie wisdom.

Thanks for listening! Get more information, support the show, and read all the transcripts at academicaunties.com. Get in touch with Academic Aunties on Twitter at @AcademicAuntie or by e-mail at podcast@academicaunties.com.

Hi there, this is Ask an Academic Auntie, our bonus episodes of Academic Aunties where we take your questions and try to impart a little auntie wisdom. Do you have a question? Send us a message on Twitter at @AcademicAuntie or email us at podcast@academicaunties.com. If you send us a voice memo, you'll even get your question played on the air.

We'd love to hear from you. 

We are here today with two amazing people who I actually went to graduate school with. And I'm so jazzed that they're here to join us today to answer a listener question. With us today is Heather Millar, who is an Assistant Professor in Political Science at the University of New Brunswick, and we also have Dr. Carmen Ho, who is an Assistant Professor in Political Science at the University of Guelph. 

I am so happy that both of you are here today to answer one of our listener questions. So the question is

Yeah. So this is such a great question. The first thing that I do is I pull out the substantive comments for each reviewer and I copy and I paste them into Word document verbatim.

Um, sometimes I'll shorten the comments if their citations or parts that are repetitive. But what I'm trying to do is capture the substantive parts. And what I'll do is, uh, label each comment. So like reviewer one, comment one, comment two, comment three, and italicize the comments. Then what I'll do is start by writing out my response to each comment and what specifically I'm going to do to address their suggestions. And I try to do this really, really quickly, right. At the very beginning. So I have, um, kind of a laundry list for all of the comments and what I'm going to do. And this helps me process the comments in my own mind. And it also creates a draft response memo.

Now, the second thing I do is I make a calendar based on this laundry list. I start with the end in mind the deadline for when I have to submit everything, then I just kind of work my way backwards. And I have a week by week timeline with sub goals.

And then the third thing that I do is I build in external accountability. So this meant telling five different people, five different kind and generous people, including Heather, um, that I'm going to be sending them my work in progress at different dates. 

And the reason why I do this is that I find that teaching and service, they both have accountability built into them. So you have to give a lecture, you have to respond to emails, you have to show up for that service meeting. So it's easy for these things to take priority over research and writing. So having external accountability, um, or having these mechanisms for external accountability, having these people who I know I have to send my work in progress to, lets me kind of have a reason to continue to work on my work and my research and to have these you know, specific deadlines and I match up the deadlines with my sub goals. 

I really, really, really like what you said at the last part with respect to putting in accountability mechanisms. Our worlds, our lives get so busy, so you need kind of that external push to make sure that this gets done as well.

Um, so thank you so much for that. Auntie Heather?

I really appreciate what Carmen set out. It's excellent practice. I like spreadsheets and I like lists, so I'm very comfortable. I definitely do that first kind of breaking the comments down. I also tend to give myself about a week to have a grump. 

Mm. 

I've had great comments from some journals and I've had comments where I really wasn't completely sure that the reviewer read the article or read with the eye of well, you're not writing the article I would have written. I think there's an emotionality often to the first critique that's important just to let yourself have that moment of just, okay, before you go back and then, and then try to really understand what it is that they're really trying to tell you. Or what's the piece of the argument that the reviewer is really sticking on or is wanting you to develop further.

And I know, I think that that would just be the other thing that I would add.

Yeah. So I'm so glad that Heather brought that up because so in the first week, so, so this is also really interesting because, uh, for me, I'm comparing these two pieces, the APRS piece was my dissertation, it was very personal. It took a long time to get this to where it was. And so it was actually a lot harder to process the reviewer comments. But for the Lancet piece, it was a coauthored piece. Wasn't a project that I'd worked on for a very long time. And so there wasn't this emotional piece. I just read the comments and I was just like, yeah, this makes sense. Okay. Let's see what we can do to address these things, but certainly for APRS, that first week I really existed in my own head and I thought, oh my goodness, I got eight pages of single space comments from four reviewers from four of them. was so much. Single-spaced and it was, it was just, it was a lot. And the only way I knew how to take that was, I must've really bombed this. I did, I did something wrong, I don't know what to do. And so I existed kind of in my own head. I kind of, I knew what I had to do next, but there was still like this piece of me that was really anxious and really upset.

And then I had a really nice chat with Heather and, and, you know, Heather and Heather actually don't know if we talked about this afterwards, but for me after that conversation, that for me was the turning point. 

So what you had made me feel was that it wasn't me, this wasn't, these comments weren't specific to me, this was just part of the process.

And you had kind of shared that you had received comments like these before, and to me, like hearing it come from you. So I think you're incredibly smart. I think you're very thorough with your research. I think you're very clear. And if you are getting comments like these, then it didn't feel like it was specific to me, like something I had done wrong.

And so then it was kind of, for me, very much a turning point for, okay. I can like, it was that grump, I think. Like that period that I had to grump about it. And then I could kind of move forward from there with a little bit less emotion, I think. But I found it so helpful to talk to you after getting the comments.

I think there's like ideological gatekeeping in that, you know, you're publishing something or you're writing on something that doesn't align with previous research I, as the reviewer have written about. So I'm going to either really, really critique it or reject it, but then also like the methodological gatekeeping of like, you know, oftentimes we talk about it in the, the context of quantitative researchers viewing qualitative research as less rigorous.

As just stories, um, or just opinion. And then that is also kind of met with a lot of critique, uh, or rejection, but I've also seen it done the reverse way where qualitative scholars will dismiss quantitative work. So I just think that there's a lot of luck involved in terms of who your reviewers are and for the two pieces that I've gotten through I think I've also been really lucky to have had reviewers who, you know, gave a lot of feedback. Like the APSR piece like eight pages single space is like a lot of feedback but it was in good faith, I think. And not all reviewers are like that.

That's a super important reminder. And I think this also alludes to a good answer to the second part of the question. Do you respond to every single reviewer comment? I mean, I think you can tell when you read the reviews, when the comments are done in good faith or when the comments are not being very generous or are written from the standpoint of someone who doesn't really get your argument and is actually annoyed that you're not making your argument the way that they would make it, which isn't the purpose of peer review, right?

The purpose of peer review is to be rigorous, but also to take to heart what the author themselves tried to accomplish in that piece. It should never be about what you would have written. And I think that's a flaw that a lot of peer reviewers make as well. 

You know, it's so hard to put your work out there. Right? It's so hard and it's always, so it's so stressful when you see the journal has emailed you back and I actually don't even really want to open it when I see that the decision has been made. Right? But any advice to our listeners who are still, you know, new to this whole peer review thing? 

Don't give up because a piece that isn't fitting at one journal really may fit at another journal. Just know that it's a very slow moving but iterative process, but hopefully, you know, the kernel of the idea will, will land eventually. 

I actually wanted to kind of follow up on a part of the question, which was, do you respond to every single reviewer comment and I'd love to hear what both of you think. I personally respond to every comment. I try to respond to every comment just because I know how long the process takes like myself having been a reviewer. It takes a long time to read a piece, to write like thoughtful and constructive comments. So somebody has, you know, taken the time to do this. I really do want to engage with what it is they're saying. It's not to say that I'll make all of the changes that they recommend.

But for the things that I don't actually change, I'll explain why. And so I don't know if this is common. I don't know how you both approach this. But I try to respond to all of the comments.

Yeah, I do too. I've had a bad experience where I tried not to respond and then I got the second round of revision. So I think, uh, yeah, maybe the fact that I'm getting multiple rounds actually just means I'm not responding well enough to the comments. I also think you can respond, but respectfully disagree.

It's not that you have to agree with the reviewer. You just have to have a conversation with them, and so I agree with Carmen. I do, I do try to respond to all of the critiques. Sometimes it's hard when it's, uh, you know, triple barreled critique in one sentence where, depending on the reviewer.

So that can be challenging, I think, where you're like, I don't know what this, you know, I don't actually know what they're saying. And so it's hard to respond. That's why I think breaking out the comments into individual little nuggets is much easier.

I respond every single comment too for journal articles, right. In a situation where I find, I mean, I do that anyway. Right. Like I actually, you know, I do everything that both of you do. Put down all of the comments. And then I write in my letter, my responses to each comment. Right? And there's also the editor's letter that writers can provide where you can provide further contextualization to your responses as well.

Um, cause I think it's important for reviewers to see that you have attempted in good faith to respond to their comments as well. 

I've heard of people who say like, you know, I don't respond to all of the comments. You don't need to respond to all of the comments. It's not necessary to do that. But my feeling is just, you know, if someone has taken the time to, to write this up, I would like to engage with them just because, you know, like, um, Dr. Joyce Green was talking about

But I also got this really useful piece of advice. Um, and someone had told me that when I'm actually writing up my responses, I shouldn't just be like, yeah, I I've addressed your comment. I should also think about selling the revisions. Um, just to make it clear that, you know, I've engaged with the comments, I've addressed, the concerns, I've made the necessary changes, here they are.

But I think responses can be framed in a way so that the reviewer knows that, you know, you did the work and you've attempted to like really engage with their thoughts.

Yeah, I think, I mean, I've had, I've had the weird experience the other way of reviewing where I feel like I caught, I said a comment and then got mansplained back by the, by the writer. Um, which was, which was a bit like, you know, sorry. I'm like tweaking my face. Um, it definitely rubbed me the wrong way.

I think the piece that I reviewed was actually by a more experienced scholar. And so they didn't like what I was questioning. But, uh, I was like, well, you might know that information, but it's not coming through in the text. Right? Again, I think it's just thinking about it, it's a conversation with your peer reviewer, which is very difficult, because it's a conversation with a blank screen, I mean it's a bit like being on a Zoom call and not seeing anything.

Right. That's what is so ridiculous about peer review on some level is that is a conversation when we don't know who the other person is.

This is great. Thank you so much. Auntie Heather, Auntie Carmen for these fantastic words of wisdom. If our listeners want to follow you on social media, are you on Twitter?

Do you want to share your handles? 

Absolutely. Uh, you can reach me at @hlmillar. That's M I L L A R.

And thanks so much for having me Auntie Ethel. Um, so I am also on Twitter. I'm under Carmen, Jacqueline Ho and my Twitter handle is @carmenjho_ underscore. 

Awesome. Thank you so much.