In this first episode discussing critical AI and digital literacy, Tavinder Mangat (Digital Education Consultant at Birkbeck, University of London) speaks to Amy Aisha Brown (Senior Lecturer in Academic Skills at the University of Portsmouth, London) about Amy's experience of embedding critical AI literacy in her teaching.

They discuss why critical AI literacy is important and how to protect and develop criticality and judgement in the use of generative AI. Amy speaks about how she uses formative student centred assessments to build essential skills. She also reflects on her own experience of the workload of keeping up with rapid changes related to GenAI use.

Mentioned in this episode:

Music by Mykola Sosin from Pixabay

Speaker: Hello and welcome to this series

Speaker: on critical AI and digital

Speaker: literacy, where we'll be

Speaker: exploring what it means to teach

Speaker: critical AI and digital literacy

Speaker: at university.

Speaker: My name is Tavender Mangat.

Speaker: I'm a digital education consultant at Birkbeck,

Speaker: University of London.

Speaker: My name is Amy Aisha Brown.

Speaker: You can call me Amy.

Speaker: I'm a senior lecturer in

Speaker: academic skills at the

Speaker: University of Portsmouth,

Speaker: London.

Speaker: Welcome, Amy.

Speaker: In this episode, we'll be

Speaker: talking about Amy's work on her

Speaker: professional and academic skills

Speaker: for success module at the

Speaker: University of Portsmouth,

Speaker: London, where Amy has been

Speaker: actively working to embed AI

Speaker: literacy skills.

Speaker: Amy, thank you so much for talking to me about the great

Speaker: work that you've been doing.

Speaker: Your module is for level for

Speaker: first year undergraduate

Speaker: students, and it aims to help

Speaker: them develop the academic skills

Speaker: that they need for their

Speaker: undergraduate degree.

Speaker: You've specifically been working

Speaker: with your students to develop

Speaker: their critical AI literacy

Speaker: skills.

Speaker: Can you tell me about why that's so important?

Speaker: AI is becoming a normal part of academic and professional life.

Speaker: And so just like other skills students need, say, working in

Speaker: groups or being effective communicators, it's important

Speaker: that we actually bring the skills into our teaching.

Speaker: If we want all of our students to be able to fully participate.

Speaker: But as you suggested there in your question, it's not just

Speaker: about telling students what they can or can't do.

Speaker: They need to develop the

Speaker: criticality to make decisions

Speaker: around AI.

Speaker: That's deciding when AI tools

Speaker: are appropriate and when it

Speaker: isn't.

Speaker: But it's also about helping them tackle some of those deeper

Speaker: questions, like, how do I judge what I'm looking at?

Speaker: Who or what wins or loses in this situation, and what are the

Speaker: implications of those answers?

Speaker: That's nothing new.

Speaker: It's just critical thinking.

Speaker: But it's so important that they learn to apply that to AI,

Speaker: because they're going to continue to have to navigate

Speaker: change, changes in tech, changes in norms, changes in guidance.

Speaker: And they need to be able to bring that capacity into new

Speaker: situations in their studies, their professional lives.

Speaker: But equally, if I'm allowed to

Speaker: open it out to be fully engaged

Speaker: citizens.

Speaker: You mentioned changes in guidance.

Speaker: Can you give some background on

Speaker: how guidance and generative AI

Speaker: policies in your institution

Speaker: have shaped your approach

Speaker: towards how you discuss

Speaker: generative AI with your

Speaker: students?

Speaker: Some changes in policy have been quite difficult for me in terms

Speaker: of a change in policy, means I've had to shift what I've been

Speaker: doing around AI literacy.

Speaker: For example, there was guidance that students were not to use AI

Speaker: in specific circumstances.

Speaker: And so I did lots of work to

Speaker: help prep students for

Speaker: understanding that.

Speaker: But then new guidance came out that changed that.

Speaker: And when you're trying to embed

Speaker: AI literacy and you're working

Speaker: with new students, students need

Speaker: to be supported with these kinds

Speaker: of changes.

Speaker: And so some of the changes were a little bit difficult for me

Speaker: because I always felt like I was on the back foot.

Speaker: There's always something

Speaker: changing, something to adapt

Speaker: for.

Speaker: There were also changes, for example, around how tech was

Speaker: used in line with policies and guidance, such as a shift in how

Speaker: we acknowledge AI on Moodle, which meant lots of changes to

Speaker: how I designed assessments and the guidance and support for

Speaker: that as well.

Speaker: But actually I kind of agreed with the policy changes.

Speaker: They've made things easier and

Speaker: made things feel a bit more

Speaker: authentic.

Speaker: For example, the current

Speaker: guidance is now, and I'm

Speaker: paraphrasing here, there's not a

Speaker: list of things that they can do

Speaker: with AI or they can't do with

Speaker: AI.

Speaker: There's a general rule that students shouldn't let AI do the

Speaker: work for them.

Speaker: They need to own what they submit.

Speaker: And I found that to be quite

Speaker: freeing, because it enables me

Speaker: to talk to students in ways that

Speaker: it's not just about rules and

Speaker: tools, but about how instead we

Speaker: can think about their decisions,

Speaker: their judgements.

Speaker: It frees up everyone to talk

Speaker: more about ideas rather than

Speaker: worrying about how AI is being

Speaker: policed.

Speaker: And that's not to say that students don't use tools in ways

Speaker: that aren't allowed and are problematic, so that, you know,

Speaker: there's still very much an element of talking about that,

Speaker: but it has shifted the conversation and I've found that

Speaker: change to be positive.

Speaker: And that's great.

Speaker: So you felt freer to talk to your students more openly about

Speaker: the kind of techniques that you think they should be using, or

Speaker: how they can critique their own use of AI?

Speaker: Can you talk about some of the ways that you had those open

Speaker: conversations and discussions with your students?

Speaker: One of the ways I did that was

Speaker: by bringing AI into the room and

Speaker: making it something that we

Speaker: used.

Speaker: But we also then talked about

Speaker: using, for example, we did some

Speaker: simple activities where our

Speaker: students to do things in

Speaker: different ways.

Speaker: So I asked them to do something on their own to start with

Speaker: something, then with group members afterwards, and then to

Speaker: use AI to get some feedback on what they were doing.

Speaker: They then compared those different ways of doing things.

Speaker: Some students use different tools.

Speaker: They compared that different

Speaker: things happen for different

Speaker: groups.

Speaker: So there was an opportunity there to talk about what was

Speaker: working and what was not.

Speaker: But importantly for me, I also got students to map the

Speaker: different ways that they'd used AI and other resources,

Speaker: including me and their peers, and thinking about how useful,

Speaker: quick, trustworthy and so on those different resources were.

Speaker: And that really helped to bring out ideas about what's a good

Speaker: way of doing things.

Speaker: Do we always need to be using AI, even if it's the quickest?

Speaker: Is it the best way of remembering things?

Speaker: What do we get from each other that we're not getting from AI?

Speaker: And conversations like that, that I think are probably the

Speaker: most important thing we can do.

Speaker: I want students to know how to use tools and learn different

Speaker: ways of doing things.

Speaker: But I think what's difficult is to gain, again, that kind of

Speaker: criticality around it.

Speaker: So lots of the tasks and things

Speaker: that we do with AI are do

Speaker: something.

Speaker: Yes.

Speaker: But then let's talk about it.

Speaker: Let's frame it, look at it through different lenses.

Speaker: And in that way you're teaching them how to question everything.

Speaker: And that's what we talk about

Speaker: when we're thinking about

Speaker: critical thinking skills across

Speaker: the board.

Speaker: It's about questioning whatever source material you've got.

Speaker: So you're applying those ideas to gen AI tools.

Speaker: Yeah, absolutely.

Speaker: And it's not going away really that idea that we need judgment,

Speaker: we need critical thinking.

Speaker: We need to make decisions.

Speaker: It's bringing those to the fore and showing how important those

Speaker: are, especially in academia.

Speaker: Yeah.

Speaker: So I think it just brings out the need for that quality.

Speaker: And there are a lot of discussions about authentic

Speaker: assessments and how changing assessments can make them

Speaker: generative AI proof.

Speaker: That's not always guaranteed.

Speaker: You're not going to be able to stop students using gen AI.

Speaker: One of the ways of making assessments more authentic is by

Speaker: making them student centered.

Speaker: Can you tell me about how you've structured your assessments for

Speaker: your module and how you've made them more student centered?

Speaker: So I work with level four

Speaker: students who are new to the

Speaker: university.

Speaker: They come from different places, they've got different skills.

Speaker: They're very much still learning how to be part of the

Speaker: institution, how to navigate it.

Speaker: They might also be in a new country and with all the newness

Speaker: that that brings.

Speaker: And so I want the assessment to

Speaker: work for those different

Speaker: students who may be just

Speaker: starting out in developing their

Speaker: academic skills.

Speaker: So one of the things I try to do

Speaker: is build assessment that's

Speaker: gradual.

Speaker: So they're gradually learning

Speaker: new bits and being assessed on

Speaker: those bits.

Speaker: And I use marking criteria to reflect that.

Speaker: So they might start with being assessed on a simple criterion

Speaker: like are you following the task.

Speaker: But then they will still get

Speaker: feedback on other aspects of the

Speaker: marking criteria.

Speaker: Gradually I'll add another criterion that becomes summative

Speaker: and then another so that students are gradually

Speaker: progressing through the criteria rather than getting graded on

Speaker: everything before they've had a chance to really try it out and

Speaker: get used to it.

Speaker: So how does this relate to AI?

Speaker: Well, in a couple of ways.

Speaker: First, I try and sequence the criteria that are summative in a

Speaker: way that the early ones focus on those things that are really

Speaker: about the student and what we've been doing in the room together.

Speaker: So the aspects that focus on

Speaker: their critical thinking, their

Speaker: reflection, because I want

Speaker: students to recognize the value

Speaker: of those.

Speaker: I want them to see that those human and personal skills are

Speaker: important from the get go.

Speaker: Whereas the aspects that AI can take much more of a lead on the

Speaker: sort of procedural aspects, I guess like academic writing and

Speaker: acknowledging sources correctly.

Speaker: I've pushed those to the other end.

Speaker: So it's a way of saying to

Speaker: students, I first care about

Speaker: their voice and ideas, a kind of

Speaker: value signaling.

Speaker: I guess pushing those procedural

Speaker: type aspects to later on also

Speaker: helps with making sure there's

Speaker: more time for identifying the

Speaker: kinds of issues that can come up

Speaker: with AI use, as well as other

Speaker: kinds of academic practice

Speaker: issues around referencing and so

Speaker: on.

Speaker: I want those to come up before they become critical and require

Speaker: those stressful academic misconduct type discussions.

Speaker: If I do see potential issues in

Speaker: an early piece of work, then

Speaker: I've got time to talk to

Speaker: students.

Speaker: I can also do it in ways that

Speaker: allow me to focus on their

Speaker: ideas.

Speaker: I mean, the impact that those

Speaker: issues have on their ideas

Speaker: rather than jumping in with

Speaker: misconduct accusations.

Speaker: Say, for instance, they've handed in a reflection that

Speaker: sounds great, but seems to be totally different from what

Speaker: happened in the workshop.

Speaker: I can be talking to them about

Speaker: where I'm seeing issues with

Speaker: them meeting the task, or

Speaker: showing their justification

Speaker: skills, and then asking

Speaker: curiously about how those things

Speaker: happened.

Speaker: When AI does come up and it

Speaker: does, it can then be around

Speaker: those judgment questions that we

Speaker: talked about.

Speaker: So I'm trying to recognize that students should have room to

Speaker: mess up a bit and get non-judgmental feedback,

Speaker: especially in a module like this, which has started and

Speaker: finished within about two months of them actually starting uni.

Speaker: That's a very thoughtful and considered approach.

Speaker: Going back to changing

Speaker: university policies and the work

Speaker: that you've been doing to

Speaker: understand them and keeping up

Speaker: to date with the changing AI

Speaker: landscape.

Speaker: Can you talk about how much that impacts your workload?

Speaker: Do you think people disengage with generative AI because

Speaker: they're worried about increasing their own workload?

Speaker: One of the ways that trying to take all this on board increases

Speaker: my workload is I need to keep on top of things.

Speaker: I need to know if that AI guidance has changed for staff,

Speaker: for students, because I'm actively trying to help students

Speaker: with those things.

Speaker: I need to know if a tool has changed.

Speaker: I need to test that tool when it's changed to make sure that

Speaker: the task I'm going to do with students next week is still

Speaker: going to work with that tool.

Speaker: So there's time around kind of keeping up with the zeitgeist.

Speaker: With that, I need to know about AI literacy too.

Speaker: I need to look at what best practices, how we're changing.

Speaker: And those ideas have changed a

Speaker: lot just in the last couple of

Speaker: years.

Speaker: So there's a workload around

Speaker: keeping up with all of that

Speaker: stuff.

Speaker: Then there's the workload of

Speaker: implementing changes or dealing

Speaker: with those changes when they

Speaker: come up.

Speaker: But there's also a lot of creativity needed because we

Speaker: don't have that big bank.

Speaker: You know, we don't have a big bank of ideas and ways of

Speaker: teaching these students about these new things.

Speaker: So there's quite a lot of work

Speaker: that goes on around how do I fit

Speaker: this in?

Speaker: How do I fit AI into these

Speaker: things that we're already trying

Speaker: to do?

Speaker: And yeah, for me, that's, I

Speaker: think the biggest load because

Speaker: it often has to happen quite

Speaker: quickly.

Speaker: As I said before, something changes and quickly changes.

Speaker: I've then got to adapt and I've often got to be quite creative

Speaker: at times when I'm maybe not really giving myself the time to

Speaker: be that creative.

Speaker: So I think those are some of the ways that this takes its toll.

Speaker: And I think why some people might avoid it.

Speaker: Indeed, the module that I was

Speaker: talking about before is not

Speaker: actually going to run from

Speaker: September because we're going

Speaker: through a period of curriculum

Speaker: change.

Speaker: Instead, I'm designing a new

Speaker: module, a new skills module that

Speaker: will be taken by all of our

Speaker: undergraduate students when they

Speaker: first arrive.

Speaker: That new module will be taught by lots of different people.

Speaker: And so I'm rethinking this again.

Speaker: I'm having to again, be quite creative because some of the

Speaker: ideas I had before, some of my ways of working, relied quite a

Speaker: lot on me being comfortable with just going with the flow.

Speaker: I know some of my colleagues would also be comfortable with

Speaker: that, some of them probably a lot more than me, but others

Speaker: might not be.

Speaker: And reasonably so.

Speaker: And I don't know who might need

Speaker: to teach this module down the

Speaker: line.

Speaker: So even though I've already done lots of work on this, I'm again

Speaker: having to do quite a lot of rethinking in ways that I don't

Speaker: have to do around other skills.

Speaker: Things like what a good presentation looks like.

Speaker: Or at least I don't have to be rethinking about those at nearly

Speaker: the same rate as I am around how I'm embedding AI literacy.

Speaker: And what are your plans for the

Speaker: new module that you'll be

Speaker: teaching?

Speaker: What sort of things are you going to bring to it?

Speaker: And is there anything that

Speaker: you'll change based on your

Speaker: experience over the past few

Speaker: years?

Speaker: So I guess some learning that I've had that I'd like to take

Speaker: forward is that anything too concrete may need to be changed.

Speaker: And this is just from a

Speaker: sustainability perspective,

Speaker: something I've reflected on and

Speaker: feel I really need to think more

Speaker: about.

Speaker: If we're talking about the

Speaker: policy at the university or the

Speaker: guidance for students, maybe

Speaker: that's something I do have to be

Speaker: quite.

Speaker: I do have to share with students

Speaker: and make sure that they're

Speaker: focused on that policy or that

Speaker: guidance.

Speaker: But there's other aspects where this can be much more of a

Speaker: student driven activity.

Speaker: Think about something like how they use copilot.

Speaker: Students can be the ones going

Speaker: out, finding out about it and

Speaker: sharing what they found and

Speaker: tried, and judging that

Speaker: together, rather than me trying

Speaker: to give them some step by step

Speaker: guidance.

Speaker: And then we look at the tool and the tools changed.

Speaker: The guidance no longer works.

Speaker: And me, or even worse, one of my

Speaker: colleagues then needs to step in

Speaker: and come with a quick fix on the

Speaker: spot.

Speaker: So yeah, more flexibility and getting students to do things

Speaker: for themselves and working with what they find is some learning

Speaker: that I will be taking forwards.

Speaker: That's a really good tip for

Speaker: keeping activities flexible, for

Speaker: rapidly developing content and

Speaker: basing the activities on student

Speaker: investigation.

Speaker: Thank you so much for joining me

Speaker: today, and thank you to our

Speaker: listeners.

Speaker: You can find more episodes in this series on the By George

Speaker: Channel, and many more episodes will be coming out.

Speaker: So look forward to those.