Artwork for podcast The Language Scientists
S1: E9: Mouse-tracking: How to Study Language Online
Episode 98th November 2022 • The Language Scientists • De Montfort University
00:00:00 00:24:44

Share Episode

Shownotes

In this episode, Dr Zavaleta speaks with Dr Anue Kukona about psycholinguistic research. In particular, they discuss conduction research over the internet and the advantages that come with that in terms of a more diverse sample of participants, mouse-tracking as an alternative to eye-tracking, as well as predictions in language that goes at three times the normal speed! Join us for an interesting episode! 

Can you help us by completing our survey? We would like to know what you liked about this episode, what you would like to hear about next and also whether you have encountered any barriers in your language learning journey. Click here for the link to this 5-minute survey. 

Follow Dr Zavaleta on twitter:  @dr_klzavaleta  

Or get in touch with her via email: kaitlyn.zavaleta@dmu.ac.uk   

 

The Language Scientists Podcast website: languagescientists.our.dmu.ac.uk 

 

Follow Dr Kukona on Twitter @AnueBK 

Or get in touch via email: a.p.bakerkukona@greenwich.ac.uk or Dr Kukona's webpage 

 

Link to an article on mouse-tracking as mentioned in the episode:  

Kukona, A., Jordan, A. Online mouse cursor trajectories distinguish phonological activation by linguistic and nonlinguistic sounds. Psychon Bull Rev (2022). https://doi.org/10.3758/s13423-022-02153-6 

Transcripts

If you have a connection to languages, this is the podcast for you. Whether you're a language learner, a language teacher, a language researcher, or someone who's just interested in languages. I'm Dr. Kaitlyn Zavaleta and alongside Dr. Marie-Josee Bisson. We are the language scientists and this is our podcast. We are senior lecturers in psychology at De Montfort University, and we conduct research into the area of language learning. Throughout this series, we hope to translate the science behind language learning into informative and useful practical advice. So whether you're a language learner, teacher or researcher, sit back and enjoy. So today we have Dr. Anue Kukona on this podcast. Welcome Anue.

Dr Anue Kukona

Hello. I'm delighted to be here.

Dr Kaitlyn Zavaleta

We're glad to have you. Now, both Marie and I have had the pleasure of working with you at De Montfort, and now you are a senior lecturer at University of Greenwich in Psychology. Is that right?

Dr Anue Kukona

Yes. Yeah. I moved slightly south.

Dr Kaitlyn Zavaleta

Your background, you're obviously also an American, as people can tell by our similar-ish accents here. And you did your Ph.D. in the U.S. at University of Connecticut, is that right?

Dr Anue Kukona

Yes. Yeah.

Dr Kaitlyn Zavaleta

And then you did post-doc at University of Dundee.

Dr Anue Kukona

Yes.

Dr Kaitlyn Zavaleta

So your background is just psychology, though, right? Did you do any other programs or was it like just that's your area, your focus that you live in?

Dr Anue Kukona

Yes, where I did my PhD, I think it was kind of structured as a kind of cognitive science approach, really, where although I guess if you look at my like diploma, it does technically say psychology on it, but it did kind of there were requirements to do classes outside of psychology. So I did feel like I got to do things in linguistics also in kind of discourse approaches that yeah, gave kind of a wider exposure to the issues and questions about language that I think I'm still interested in, but that aren't just necessarily grounded in kind of a psychology perspective.

Dr Kaitlyn Zavaleta

I really like that about programs like that because I mean, we're all psychologists, right? But we we're not just in this little bubble. It's really interdisciplinary. I mean, because I was coming from linguistics into psych, obviously Marie's got a very different background coming from teaching like actual real life teaching of a language. So that's what I just think is really cool about studying language, is that you've got these different perspectives. It's almost like all different lights shining on the same weird thing in the forest, right? To figure out what it is.

Dr Anue Kukona

Yeah. It does seem to me that like human cognition, including language, is this really complicated thing. And, you know, it's going to be. Yeah, I feel like psychologists aren't going to just be able to figure it all out on their own. It does really take all of these different perspectives to really get to the heart of how it works and what's going on. And yeah, and I suppose when you know when your at PhD well at least in the US where I suppose PhDs are a little longer, there is a little more room to kind of explore and figure out what are the things that are interesting. And so yeah, getting all those different perspectives I really enjoyed.

Dr Kaitlyn Zavaleta

Yeah. And you kind of use all these different perspectives in your research. I mean, so I say obviously we're in psychology and that's our focus and everything. But you're pointing out that we're in cognitive science to be completely honest and using linguistics perspectives, but like you're so what if we go back to your language background? What languages do you speak? I mean, how did you get here?

Dr Anue Kukona

I'm one of those really where people are like, Oh, you're a linguist. You must speak a lot of languages. But actually I only speak English. So yeah, the worst case of just speaking a single language, you know, I think language is one of these things that kind of sets humans apart. And it's that kind of feature of language, which I've always thought was so fascinating. But if you really want to, like, understand people, then language seems like a really good thing to focus on because it is this thing that, you know, sets us apart from other species and from that kind of a perspective.

Dr Anue Kukona

You know, I think even though we've actually worked together, I have not heard your viewpoint on that before. So that's really interesting. That's fun actually to hear about, which is like the default assumption, isn't it? I studied all these different languages or I like learning languages, things like that.

Dr Anue Kukona

Yeah, yeah.

Dr Kaitlyn Zavaleta

But yours is actually like, well, this is the difference that we have.

Dr Anue Kukona

Yeah, I do think language is this kind of window onto cognition more broadly. So yeah, I do have that kind of I do see it as a kind of a tool at exploring the mind in a really broad sense.

Dr Kaitlyn Zavaleta

So then obviously I feel a little silly asking this because I know the answer of what you research, but our listeners don't necessarily know the answer of what you do. So when you science, when you do science, when you do your language research, what is it that you're typically studying?

Dr Anue Kukona

A lot of my work has been focussed on language comprehension, so very often psycholinguistics gets split up into either the part where you're producing language so production or you're listening to language and trying to make sense of it. So comprehension. I've always focussed on the latter side of things, in part because I think you can design experiments that are kind of, I don't know, you have a lot more control I think in studying comprehension.

Dr Kaitlyn Zavaleta

You have a lot more control.

Dr Anue Kukona

With production, of course, you know, people say what they want because they want to, not because they're in a study that, you know, you're trying to understand something. Whereas with comprehension, you can control it, you know, the kinds of things people are listening to. So, yeah, I am very practical, I will say. So I've always liked the comprehension side and I've always been really interested in so kind of grammar and syntax and language at a sentence level. So a lot of linguists have focussed on the idea that syntax, if you think about language, is really setting us apart from other species. It's also really syntax which sets us apart because it enables us to kind of describe ideas, talk about things in ways not only that we might not have ever spoken about before, but that actually nobody in the existence of anything has also ever done before. So you can kind of describe things creatively. So I've always been interested in, you know, it's called sentence processing, but it's this way of of, you know, how we can take a sentence that maybe we've never heard before and we can still understand it, we can still comprehend it, and we can do it really rapidly. We can do it even in really like noisy environments, and we can even do it despite the fact that a lot of language is ambiguous and it's not really that clear about, you know, what is being said. So despite all those challenges, we can we can still understand what people are trying to communicate, communicate with us. And I've I've always thought that that's really fascinating and also important. And so, yeah, it seems like an issue really trying to get that. I've always wanted to get to the bottom of.

Dr Kaitlyn Zavaleta

And syntax and you've used the word grammar as well. It just refers to the way the sentence is structured. Right. How else might you describe like syntax versus other parts of language?

Dr Anue Kukona

Yeah. So yeah, I suppose in the broadest sense it's how you put the words together. And so yeah, you know, I always think that anybody who feels like they actually speak a language, it's not sufficient just to know the words of a language. You've also got to know how to put them together in order to, you know, ask to go to the toilet.

Dr Kaitlyn Zavaleta

Coffee, now.

Dr Anue Kukona

Yeah, exactly. I suppose some things you can get.

Dr Kaitlyn Zavaleta

Yeah, but it's limited, right? I mean, you could say like coffee please, please Coffee as opposed to can I please have a.

Dr Anue Kukona

I suppose if you just, you know, you walk down the street like screaming toilet like that, might people might interpret that you need a toilet, but it also might mean that you're calling them a toilet. So, yeah, it does seem like having multiple words and and being able to put them together is, it's an important part of language. Yeah.

Dr Kaitlyn Zavaleta

You study syntax, you study language comprehension. How do you do this? I mean, do you just say, hey, I'd like you to understand some sentences, folks.

Dr Anue Kukona

So one of the the methods I use is eye tracking. So I do a lot of work where we have people listening to sentences that are kind of related to or there about sort of visual stimuli that they're looking at. So this is sometimes called the visual world paradigm for that reason. And the idea is that we try to match up what they're looking at in a scene or in a picture in relation to what they happen to be hearing about. And the kind of, you know, perhaps obvious take home is that there's a there's typically a very close coupling between the kinds of things you're hearing about and the kinds of things that people are then going to be looking at in a picture or in a scene. So you can kind of use it as a tool as to how people are trying to understand what they're hearing, because it then impacts on essentially what they're looking at. If you kind of watch their eye movements.

Dr Kaitlyn Zavaleta

That's pretty cool. And I know I've only done very little eye tracking research, but, you know, it feels complicated, but like at the heart of it it's where you're looking, where you're you're attending to, where you're processing that information, right?

Dr Anue Kukona

Yeah. So it kind of starts to bring in lots of different elements of cognition, right, where you have language, but then you also have a tension there. And then it's also the perception of the scene that you're looking at. So I've always enjoyed it because it, it seems to me that, yeah, I don't know, I've never been that interested in studying language in a vacuum. I always kind of enjoy it as part of this larger cognitive system of perception, attention and all of these things kind of happening together and supporting each other.

Dr Anue Kukona

That's a really cool viewpoint. Now, over the last few years, it's been obviously we've had this pandemic, right? So our research has changed just a little bit. We've gone kind of from that typical lab setup that I think we all were trained-in where we have participants sign up online and they come into the lab and they go into the booth and they see stimuli on the computer and they do their study and then we analyse it later. But things have changed a bit and you've got a kind of a newer way of conducting research and that's something that we were going to discuss today. Could you tell us a little bit about that?

Dr Anue Kukona

of course, in I guess it was:

Dr Kaitlyn Zavaleta

I think that's cool because essentially it's this idea that you're using your mouse to go in a specific direction, right? And that's supposed to be the way we're measuring kind of what would have been our eyes, right?

Dr Anue Kukona

Yeah. So it's basically it's just yeah, it's, it's instead of because in some ways attention isn't necessarily equated with eye movements anyway. Right? You can dissociate what you're attending to relative to what you're looking at. So in the same way that we were using eye movements as a kind of an index of attention, although it was, you know, even there, it was just kind of a proxy for that. Now we're kind of switching to saying, all right, let's use where people's mouse cursors are, as now they're kind of indexed. They're an indication of what it is that they're paying attention to.

Dr Kaitlyn Zavaleta

So when you're talking about doing an online research study, is this the same group of participants as we would get in the lab? I mean, is it just university undergraduates coming in and doing the same thing but like from their computers at home?

Dr Anue Kukona

Yeah, so that's a good question. And in our initial studies, we in some ways we were sceptical as to whether or not this would even work that well. I think when you do online studies you lose a whole lot of control, which is, you know, you now no longer really know where the participants are and what's going on in the background. So a lot of our initial studies were just with the very same participants that we would have used in the lab. And very often these were just our undergraduate psychology students, although I would say that for some time now it's been recognised that an issue with a lot of psychology research is that we've really focused in on a very small specialised group of individuals and we very often kind of extrapolate from them, even though they don't really reflect the, I think diversity of, you know, humans across all of the world and across all of human diversity. So I think although ultimately we were forced into this by the pandemic, it is kind of an interesting solution to an issue that psychology has been facing in, I suppose, recent decades, but also maybe across its entire history as to, you know, who is it that we're studying and how do we make sure that we're we're really recruiting people that, you know, are diverse and that are reflective of the diversity that you see across, of course, human beings. So we actually we have transition now where we are recruiting from much wider kind of populations. And I will say one thing that I was kind of struck by, I was initially worried, although now I think I'm I think it's a good thing, which is that I think across every study I had published up until the pandemic, the average age of participants was something like their early twenties because they really were the kind of prototypical undergraduate student. And our kind of first study that we did completely outside the lab, and then also not just the recruiting from undergraduates. I think the average age had jumped up to like 40 something, maybe 45. And at first I was like, Oh, no, I don't know if this is like, am I studying the same thing? But, you know, I quickly realised that actually the issue was that the kind of samples that I was studying before were the problem. And it was now that, you know, these new samples that were, you know, basically sampling across the Internet was really, you know, something to be, I suppose, celebrating that, you know, a 20 year old isn't necessarily a a representative age of of, yeah human beings. So, yes, there has been this kind of shift where we're now able to recruit much more diverse participants, I think, than certainly we were before when we had to talk everybody into coming into the lab.

Dr Kaitlyn Zavaleta

So then this is actually kind of an ongoing area of concern in in research is that we do tend to have this this population of those people who come into university research labs that we can study and make sense of human cognition. Is that then that's kind of what you're saying here, right?

Dr Anue Kukona

Yeah. So a few years ago, actually, probably a decade ago now, some researchers came up with a really, I think, clever term, an acronym, which is WEIRD. So w e i r d to kind of describe the typical participants in a lot of psychological research. So this was Western, educated, industrialised, rich and democratic participants in terms of, you know, the countries of their origin where they were kind of coming from. And I think they even did even further research on this. And one of the kind of statistics that's always stood out to me is if you look at psychology research, it's something like two thirds of participants are drawn from the USA in psychological research. And the concern, of course, is that, you know, do we really have an understanding of people or do we perhaps have an understanding of Americans?

Dr Kaitlyn Zavalata

That's crazy. I actually did not know that statistic.

Dr Anue Kukona

published work from like the:

Dr Kaitlyn Zavaleta

So then essentially you're able to study someone's kind of cognition through an Internet connection. Would you say that's right, especially doing this mouse tracking research that you're doing?

Dr Anue Kukona

Yeah.

Dr Kaitlyn Zavaleta

Yeah. It's a little bit easier than, say, trying to study cognition in like using Urp or EEG or something where you have the cap and you put it on someone's head and you're able to see how the brain works. It's definitely easier than trying to get someone to come into the lab and and do research in the cube. But yeah, I mean, I think this is a really cool, accessible method of research. Would you say this is something that someone can learn to do or or do their research on fairly easily?

Dr Anue Kukona

Yeah. I mean, I think, you know, it obviously there are limitations and I don't necessarily know that this is a solution to understanding maybe brain functionality. But yeah, I mean, I think, you know, there's certainly a tool that is certainly is like Internet and computer software and things like that have gotten better. I think the tools for being able to do sort of fine grained capturing of behaviour and then also analysis of behaviour has certainly kind of increasing the possibilities that, you know, and the questions that we can address.

Dr Kaitlyn Zavaleta

So is there anything that you in the research that you've done, like the mouse tracking research or any of your eye tracking research that you think is really just like a very cool, interesting piece of information that you were just surprised or it showed something that was just really awesome that you want to share with us.

Dr Anue Kukona

Yes. I suppose one finding that I've just recently submitted so it's fresh in my head is a lot of the work that I've been doing is looking at this idea of prediction. So in language it seems to be the case that we're not just kind of waiting around to hear something and then we kind of comprehend it, but rather we're also doing a lot of prediction of what we're going to hear before we actually do hear it. So one of the recent things that we did was to try and ask, can people really predict when it would be perhaps most useful, which is when potentially you're hearing language that's like really fast so you don't necessarily have a lot of time to try and understand it. And so we did this kind of study in which we took some recordings and we doubled how fast they were. And we found that people could indeed kind of predict what they were going to be hearing next, and they could even do so when we tripled the speed. So we thought its really interesting that it does seem to be that language comprehension isn't just about understanding what you have heard, but also trying to predict maybe what you'll hear next.

Dr Kaitlyn Zavaleta

Oh, that's really cool. And that's something that you've just submitted.

Dr Anue Kukona

Yeah. So, yeah, fingers crossed. It'll be out eventually in the literature.

Dr Kaitlyn Zavaleta

Yeah. So our listeners can be on the lookout for that.

Dr Anue Kukona

It was a mouse tracking study. So, you know, basically people are hearing sentences and you'll hear verbs that are like, the man will ride something and essentially people start moving their mouse cursor over toward write about things like if there's a picture of a bicycle before they actually hear that the man's riding a bicycle. So you get this kind of nice prediction and you can detect it through the Internet with mouse cursor tracking.

Dr Kaitlyn Zavaleta

I think that's awesome. And then it kind of tells us, like the main point of that we were kind of making today is that some really good, high quality research can be done online to understand how people process language, whether it's there, actually production is one way, but their comprehension of language.

Dr Anue Kukona

Yeah, yeah. No, I'm, I'm really excited about the possibilities going forward and yeah, my, my default now is to try and think actually, could I, could I do a study online before maybe I have to figure out some other way of doing it in the lab.

Dr Kaitlyn Zavaleta

Yeah, for sure. I don't even think you would need to be in the lab for that. So that's that's really cool, really accessible research. Well, cool. Is there anything else that you'd like our listeners to be able to walk away and say like, Oh, this is something that I learned today. This is the coolest part. Any last bit of wisdom?

Dr

Yeah, I guess. I suppose following on our discussion, it's when engaging with research, I suppose one thing that I encourage my students more and more and it's something I do more and more is to really try and think about who are the participants that are in the study and what are the implications of that follow from, you know, who it happens to be that's being studied. And yeah, you know, I now, I think I am much more of the kind of feeling that when I see that the average age of the participants are 20, I think, well, okay. Is this a study about human cognition or is it a study about how maybe 20 year olds are, you know, doing the task that's described in the study?

Dr Kaitlyn Zavaleta

That's a really great point, and I love that you made that as well, because I think of you first and foremost as kind of like a methodologies, to be honest, in thinking about research methods and who do we have here? What are we what are we actually making sense of with the with the data and the research and the results and everything?

Dr Anue Kukona

Yeah, definitely. And I think it it does open up quite a lot of like opportunity to, you know, revisit classic old studies and look at who the participants are and then ask, you know, how might this be different in a different group of participants?

Dr Kaitlyn Zavaleta

Well, thank you so much for joining us today Anue. And I will say to our listeners, Anue is actually our last interviewee for the series. So what a very cool note to end on. And I'll just take a really quick minute to remind our listeners that if you haven't already, please take 5 minutes just to go to the show notes and click the link for our survey. It will tell us your thoughts so that Marie and I can plan our upcoming series based on what you would like to hear. So thank you for listening, and thank you to the British Academy for funding our podcast. I'm Dr. Kaitlyn Zavaleta and you've been listening to the Language Scientist podcast.

Links

Chapters

Video

More from YouTube