In this episode of Fibonacci, the Red Olive Data Podcast, we interview James Gardiner, head of data technology for Admiral Group. James has 25 years of experience working with data and has been at Admiral for the past seven years, recently managing its move to the cloud.

We chat about the importance of applying an engineering mindset to data projects, data governance, the art of the possible and more in a wide-ranging chat.

Here are the topics we discuss with their timecodes:

• How James got into data engineering (1m)
• The importance of emphasising an engineering mentality to data teams (6m 9s)
• Giving people the data to make good decisions, including live streams and integrating it with sales processes (7m 17s)
• Understanding customers to give them the best price, while balancing risk (9m)
• Scoping a project and getting both the right people and architecture in place (10m 35s)
• The importance of “the why” of a project (12m 42s)
• Security is one of the cloud’s biggest risks: how to manage that with good governance (14m)
• Training a young team to respond well when things go wrong (17m)
• Applying machine learning to monitoring (19m)
• Mesh architecture and ML Ops (20m 18s)
• Communicating well with your team and re-using code (24m 30s)
• Using data governance as an enabler, rather than a blocker (27m 50s)
• Learning from Red Olive (31m)
• The important skills for people to learn, and data automation (33m 18s)

Hello and welcome to Fibonacci, the Red Olive Data Podcast, all about data

and analytics, where we hear from leading specialists and get their take

on the industry. I'm your host, Nicky Rudd. Today, I'm joined by James

Gardiner, head of Data Technology for Admiral Insurance. James has nearly

25 years of working with data and has been at Admiral for the

past seven years, lately managing its move from on premise to the cloud.

He was a data consultant for 15 years before joining Admiral, working in

the telecoms and finance industries, where he consulted on data solutions

for companies including Lloyds, Barclays, Nokia and WS Atkins. We discuss

data governance, moving to the cloud, and the art of the possible in data

projects. Let's find out more. Let's start off with telling me how you

got into the world of data. What was your journey in?

I finished my degree in digital systems engineering. Way back, oh God, that's

27 years ago I think. My thesis at the time was doing things

with image tracking, tracking objects within images and following them and

things like that. So I started getting into

what was basically a bit of machine learning, so it's what they call

linear regression, and I started getting interested in that. When I came

out of university, I actually landed my first job,

was for a print inspection company. They were doing vision analysis

on print runs. So this could be anything, this could be wallpaper, newspapers,

anything like that. And what they wanted to do was build a system

that identified differences in colour trends while the press was moving,

so they didn't have to stop, saves them a lot of money,

etcetera, etcetera. So I went in there and I started coding in C++

to do some simple machine learning algorithms and things like that, that

they could then apply to the equipment that they sent out that was

bolted to these massive printing presses. Now, I did that for about a

year. And unfortunately it was a very small company, there was only about

12 of us and there was really nowhere for me to go in

there so my options were quite limited. I eventually joined a graduate scheme

for Welsh Water as part of their software division, and they put me

through loads of training with Capgemini and all that, they really invested

in me. I ended up being a junior

DBA SQL Server. So I did that for about two years,

and eventually got out of the junior position, became one of the development

DBAs there and things like that. And then unfortunately I got made redundant,

as did hundreds of other people on this project, you know how it goes,

this typical project just got shut down. And so from there,

I actually went to work for a telecoms company up in Nottingham. I

used to work three weeks at home and one week in the office

there. Basically, I was building and designing databases for them to whole

test data for their mobile devices and things like that, they used to

have big contracts with, at the time, Nokia, Ericsson, and the places that

are testing their handsets, and they always used to do a high level

of reporting on that test data. And I eventually

got into that and I was there for probably another two years

and then I got made redundant again, believe it or not. I got

made redundant again. The bottom dropped out of the mobile industry and

I got made redundant. And so I was looking around then for another

job, and there wasn't many permanent places in South Wales.

So I eventually landed a contract with WS Atkins, where I basically started

looking at the data for the road network, so basically this is the

M4. So in the M4 you have hundreds of sensors underneath the roads that

basically look at traffic flows, look at the speed of the traffic and

things like that, and they actually control the automated signs. So

when you're on the motorway and it says oh, "Slow down to 50,"

and things like that, it's normally because there's something, there's slow

traffic up ahead, like it could be a mile ahead. One of my

jobs was to take that data and actually mine it and start looking

at where all the congestion was on the roads and things like that,

and eventually this would go into road planning. So they would go,

"Right, okay. We need to open up another slip road here,

we need to do another bypass here," and things like that.

So I used to look at all this congestion

and things like that. Again, another two years there as a contractor,

and then I landed another contract at PHS, which then does the washrooms

and things like that. I built a financial data warehouse for them,

it was probably the first proper data warehouse I'd built.

I was there for a couple of years and then I went out

to the finance industry. I had all this data warehousing finance knowledge

and I went to a number of places, it was Lloyds, it was

Barclays, Barclaycard, lots of places like that. I also found myself in

Nokia, I was there for four years with Nokia Music. Again,

as a consultant, building financial data warehouses for them. I was also

with Principality, the building society, I was there for another four years.

Again, building financial data warehouses and things like that, and putting

in teams that could basically deliver these measures that the business wanted.

And then eventually I got a bit sick of contracting and I wanted

to have a bit more power, have a bit more leadership,

responsibility and things like that. So I joined Admiral, and I joined Admiral

about seven and a half years ago now. Since then I've made it

my home. I'm very happy to be there. That's the longest stint.

It is. It is definitely the longest stint. Normally I'd get a bit fed up,

I'd sort of move on. But then again, I was a consultant as

well. So about Admiral, yeah, I've been with Admiral for quite some time

now, and I'm currently the head of Technology for Data for Admiral. And

what that means is that it's all about capabilities. I look after most

of the data people and I construct workable agile teams for data.

As I said, it's all about capabilities, it's all about pushing down that

engineering mentality to the people on the floor to make them think about

data and not just, "Ah just coding away here," that there's a whole

big world and you really gotta understand the bigger picture in these sort

of things. And that's kind of what I do now to this day.

It's got a lot bigger, a lot quicker, 'cause obviously the demand is

just sky high. So now I basically lead and instruct those teams on

how to fulfill Admiral's data requests, and that's where I am now.

And I was gonna say that actually, even in the last

seven years that you've been there, you must have seen some massive changes.

We'll talk a little bit about the big project that's moving from on

premise to cloud, but what things do you think have been drivers over

the last five and a bit years to really push that technology and

that sort of data as a key building block of how the business

move forward? There's two sides to the insurance industry, and every insurance

company, I believe looks at things like that. And first they'll look at

things like value streams and in terms of right pricing, we need to

get the best and fairest prices for our customers. And then the second

is claims. You need to get the best claim service,

but also you have to be aware of things like fraud,

which costs a lot of money, which as the industry has blossomed, so

have bad guys as well, okay? So there's a lot of that out

there as well. But also just essentially giving people the data to make

good decisions, and it's gotta be good quality data. And obviously that's

not always the case, but yeah, I've seen it move from

very static data sets that only get updated once a day to basically

data sets that are essentially getting updated several times a day.

And then you have streaming data and things like that and real time

alerting and things like that. If you can do things like plumber into

your business processes so that your agents on the phone have really good

information about the customer that they are actually talking to,

can actually change or offer new products and things like that while they're

on the phone to them, then I think that's something we've seen in

the last couple of years that's been huge.

Do you think it's been a technology push, or do you think it's

been a leadership push and the sort of a mindset? 'Cause I think

insurance is obviously seen or has been seen as a very traditional setup

and traditional business. Has it been the people that have made the move,

or has it been just the fact that actually there's been advances in how

you can work with the data to give you that reporting and that insight?

I think it's been a bit of everything, to be honest.

I think people have realised what the art of a possible is.

So this is where you get your stream in these embedded processes,

these ML models and things like that that can

help the agent to make decisions on pricing and on a customer.

There's that part of it, and I think leadership have recognised that.

And obviously you can't fail to ignore things like that with the big

players, your Googles and your Amazons and things like that. Obviously there's

an ethics thing that comes into it. There are so many certain things

we're allowed to do. But you've got that side of the coin.

And then you've got the other side of the coin which we now

have these fantastic systems that can handle large amounts of data.

And if you're a data scientist or a pricing analyst or anything like

that, you're continually looking for other things to basically give that

customer the best price. Or you may be looking at things and actually

go, "Well, this guy is a bad driver here. He may not have

made any claims, but other things have happened. There's only so much we

could do, obviously, but other things," they are a higher risk,

so therefore they deserve a higher premium. And it's about protecting ourselves

as well as our customers and things like that. So it's a bit

of everything, really. And you can't ignore this change in architectures

and in technology as we've gone through time like this. When you're thinking

about all these changes, how do you go about thinking, "This is what

we're gonna do first," and actually the next steps?

First off, the experience will tell you the pitfalls, and it depends what

type of data project you're really starting. Because what you generally

start with is some sort of POC to prove out the technology,

prove out an idea and make sure it's quite ring fenced and your

acceptance criteria are quite well understood and measurable. So I think

it starts with that. And then you have to look at the personas, who's gonna

be accessing this data, whether or not it's a machine that's accessing the

data or a person, an agent and things like that. And you kind

of have to list those out, and you have to go and actually

talk to people. A lot of what my guys do is that they're

very heavily welded in with the business. A lot of business people actually

sit within our delivery teams to help and advise and go,

"That's right, that's not right." "These are the questions we need to be

asking." Exactly. And I think that's really important. That's kind of how

we start. And obviously you gotta get your architecture right. There are

internal processes for that and things like that. It can be a bit

frustrating sometimes and things like that, but that's the way of the world.

But essentially, yeah, I think that's what you start with. You go,

"Right, what's the scope of this? Who are the people accessing this data?

All those personas and things like that. What technology platforms are we

looking at to build this? What capabilities do we need to run some

POCs and things like that?" And also cost is a massive one as

well. Everyone's tightening their belts at the moment, so what is the cheapest

solution but the best solution overall that we can basically grow and we

can maintain properly and support properly? They are all things that need

thinking about. You generally drag in a lot of people into that process,

but I think that's where it starts. And then you start thinking about

also people from a delivery perspective, "Have I got the right capabilities?

Do I need to get some contractors in? What does that do to

the bottom line?" All these sort of type stuff. So

even though I'm quite technical, it does start a level above,

and you really have to get those things right first and a good

outset, and everyone needs to understand why we're doing what we're doing.

I think, yeah, they'll start with the why. I think that's really important

for us. The amount of times you've had a coder or something,

just, "Alright. Okay, so do you understand what you're doing here?"

"Because what I know what, I've gotta get this data from here to here."

Yeah, but why are you doing that?" "Oh, don't really know."

And that doesn't work, and I think... I think it's good to have

those meetings and explain the fundamental aims of what you're trying to

do and in what context, and then everyone can really get on board

and feel like a part of it. Those are the things we really

start with, but then obviously the next phase then is, "Let's do it.

Okay?" And that's when you have to organise teams,

you have to start... We're very agile, so you break everything into stories

and things like that. Everything get sized up.

Again, you're still looking constantly at the capabilities, have we got

enough people from networks to plumb this in? It's not just a data,

I just need data engineers, it's all sorts of disciplines across the board.

And I think that's where you start, basically. You at the moment obviously

have moved from on premise to cloud. I suppose the biggest project in

more recent times, I'm guessing. What sort of challenges has that thrown

up? Have there been things that you've had to consider differently than

perhaps having a non premise sort of set up? We're still halfway through

that at the moment, there's a lot of work, there's always something to

do, but security is the big one, I think.

With our on premise networks, there's a little bit of a safety blanket.

It's not all safe. Do you know what I mean? If someone wants

to get in, they can probably get in. But it's a lot easier

to deploy things knowing that you have things ring fenced and people won't

be able to exploit them. So security is a massive one,

because when you push it up into the cloud, it's so easy to

just make something public by mistake and things like that. So we do

spend a lot of time on security and making sure everything's encrypted in

transit and encrypted at rest and all this.

You do find though, a lot of the cloud services and all this

kind of happens by default, but you still gotta check it out and

make sure it's within your standards, if you like. So security is a

massive one. The other things to consider is governance, which changes drastically

as well when you're going to cloud because it's so easy for things

to get out of control. Because of the power of cloud,

essentially is something that I've led that I haven't learned before,

is that you can push date up into cloud and give everyone access,

but God, it becomes a massive mine field and it turns into the

Wild West if you're not careful. So you really have to basically keep

an eye on that governance, making sure that people have only got access

to the data that they need to do their day to day job.

And I think that's very important. It's always important on premise as well,

but with the power of the cloud and more people can do,

it can really get out of control so quickly.

There are a number of things that I consider, but one thing else

I'd like to say is that if you've got good governance and good

lineage of your current on premise systems, it becomes a lot easier to

go to cloud. Because you understand how the data is getting used.

It goes back to those personas again, who's using it, it's all documented

and things like that. In that case, it's a lot easier to move

and replicate and put those same controls in. So it's important to get

it right wherever you are. So yes, cloud does make a difference,

but you really should be putting these rules in place throughout,

no matter where you are. I think it's interesting you say that because

obviously there's great opportunities with cloud, the speed and

the openness, as you say, but yeah, a lesson learnt from something in

the cloud is probably a harder lesson than if it's actually still just

an internal system, a bit more closed down. Have you had to do

a lot of training or sessions with your team to get them thinking

differently with those points and those challenges? So there's always technical

training, that's a given, but like I said, it's having that engineering

mindset. Admiral's a very young company, we've got a lot of young people

working there, come straight out of university and we can train them up

and things like that. They need to understand the repercussions of things

going wrong. We have processes and things like that to control that,

but we'll generally give that insight into failing systems, a lot of the

time. Because you have to code to have resilience and make things robust.

So quite often we'll have reviews of code go, "Right, okay,

what happens if that thing goes wrong?" And it goes, "Oh, well,

it re tries." "Okay, alright, that's great. It re tries. But does it

clean up the message made when it went wrong?" It's things like that.

So it's having an engineering mindset. A lot of what I try to

put into context is that, look, would you put an aircraft in the

sky if you built it like that? What if something failed on the... Is

there a backup system? How does it get maintained, how does it get

checked? There's warning lights and things like that that you get?

Have you got the same warning lights and things in that in the

code and the data pipelines that you've put into this system?

That's why it's important to monitor, because everyone's brought up with

a CICD mentality, and a lot of CICD doesn't work when you have

a data project going on. And this is why you have data ops

instead of dev ops, because it's not only the code that has to

go through that process, it's the actual data itself. And you can get

bad data, so your code may be the best code written in the

world, but you get some bad data into it and everything can fall

down. And so you have to have these checks, you have to have

this live monitoring and things like that. And that's where I find machine

learning is actually coming into play, because you can actually apply machine

learning to that monitoring, to actually go, "Oh, this premium looks low

today," or on this percentage of measures or things like that.

You can actually get machine learning to actually invoke that alert and

then somebody can react to it and things like that. So although we

look at machine learning as, "Oh yes, gonna help this, gonna make... Better

profile my customers," and things like that, there's so many more applications

throughout the whole data pipeline, if you like, and that data flow,

that I think sometimes people ignore that. It's like turn the skills on

its head and use the machine learning to help make your systems as robust

and get your data as clean and as quality as possible.

I think it's interesting you say that, 'cause I was just about to

ask you about machine learning and ethics. 'Cause you'd mentioned that.

And I think there is that, yes, you can think about it from

a technology point of view and stuff moving very, very quickly,

but that role of a person in there just checking and putting in

the perimeters of kind of, "This is gonna be safe," or, "This is

ethical," or, "This is actually what we mean by good data,"

and actually the checks and the processes, is still very important.

Are there any particular trends or ways of working that you can really

see how that would jump start or that would really, really push a

different way of thinking? Well, there's a number of new ways of rolling

out data teams and rolling out data architectures and things like that,

one of them being mesh. There's this concept of a mesh architecture and

God, you really gotta be mature for some of this stuff,

and you really have to have those engineering principles embedded.

Because the whole concept of mesh is that you decentralise basically,

okay, and you farm that out to value streams and things like that.

Now, traditionally it's been like a data warehouse team and they've done

all your data and they've ingested it and things like that, but they

become a blocker, they become a blocker because they can only prioritise

so much, and someone over here in, I don't know, household insurance wants

to do some stuff and they can't. So what you try and do

is you try and farm out those capabilities to other value streams,

but that can be difficult unless you've got some blueprints for the central

governance because you want everything to work the same way

so you can maintain it. There's definitely some things that in the delivery

space that are being suggested. Everyone talks about mesh. I went to Big

Data London this year, and everyone was talking about mesh, but no one's

really nailed it. And it's the same thing with machine learning,

you get our pricing analyst thinking, "Right, okay, we've got a good machine

learning model here, we've trained it well. Right. We've got to get it

live." But because it's hooked into the whole quoting process and things

like that, you really gotta be careful that that is correct.

And so it can take a long time sometimes for these things to

get live, and this is where you have things like ML Ops springing

up. Okay? So it's not about the model itself and the machine learning

model itself, it's more about how quickly can you get that value into

the mainstream. And this is what ML Ops is giving us,

and essentially it's a set of governance rules really,

that I think data scientists aren't used to

dealing with that. Data scientists, they're in their own little world,

they're in a bubble and things like that. Now they have to think

in terms of engineering and what matters, "What if this model goes wrong?"

That sort of thing, they have to think about that. So everything has

to be thoroughly tested. And that takes time. So although you've got this

whole, "Oh yeah, ML Ops, AI," and things like that, you have to

determine the risk of certain things, and if you start

relying too much on it without having that whole framework around it that

gives you that robustness, you could be in a lot of trouble.

I was gonna say the kind of reporting and that, like you say,

having different people with different roles and responsibilities across

that sort of data pipeline, if you like, how do you go about

making that reporting and that kind of auditing and the reporting seamless

for everybody at the their worst? 'Cause it's such a big thing,

isn't it? That actually, you could break it down and make it so you're working

in pockets, but that actually would completely blow the overall name of

having this kind of thing. Have you got particularly with Admiral, ways

of working that you think you've nailed that? Or have there been questions

that you've along the way thought, "Actually, if we ask this with these

certain people, we are gonna have a better outcome at the end."?

I think, again, its mindset is a lot of it. So yes,

you can have these different teams and they're all working on different

things, but again, we go back to why.

Okay? Why you're doing this? You're an essential part of the cog...

An essential cog, sorry, not essential part. That is basically gonna deliver

this value at the end, and you are an essential part of it.

I think if people understand that, they generally

start to go out and communicate. And I think that's what it is,

it's all about good solid lines of communication. And obviously that's what

Agile gives you with the stand ups, your retrospectives and things like

that. This is what I'm heavily involved in, is that something is not

working, you change it. That's the the whole point, nothing's etched in

stone. If something's not performing or something went wrong at the end

of the last delivery or whatever, you change it, and communication is a

big factor in that. So when you have teams and you have them

in those silos and they're not communicating, you'll find that they're not

gonna line up at the end of your sprint or whatever your delivery

mechanism is, you have to change it. So

we get people to work together. That's the key. And communications is hard

when we're all working remote and things like that, but

it's still achievable, very much so. But it is about comms.

No one wants to come in and write something bad that doesn't work

at the end of it. But they didn't know, they were just sort

of carrying on. So it's important to get strong leaders to bring people

together and actually go, "This what we're doing. This is how we're doing

it. Everyone clear what they're doing?" "Yes." In the morning and stuff

like that. "If you're not, come and see me, we'll go and speak

to someone else," or whatever like that. That's kind of how it needs

to work. Within the data teams at Admiral, I think we've cracked that.

I think we have. We try to modularise as much as we can,

so somebody doesn't write a code for a specific things, they write it

for many things that can basically be used by another team.

So we try and write these modules and frameworks

for people to work in, rather than, "I'm just writing code." And I

think that's what helps. But there's also a lot of unit test checking

and things like that, there's a lot of automation that you can basically

put in, and we do this at our retros and things are like that. We went,

"Right, that went wrong, didn't it last time? Okay, what we're gonna do

about it?" "Right, we put an automated check in for that."

So as soon as someone deploys something into a test environment,

runs a host of unit test scripts or things are that and goes,

"No, that ain't gonna work with this," so we can build that in.

And we quite often do that. Don't capture everything. 'Cause it is hard.

Data is hard, I'll tell you that now. But if you have that

mindset and that ways of working, and that monitoring mindset and that robustness,

that engineering mindset, you can overcome it. Do you think that the governance

is kind of more stable or more solid because of that understanding of

data engineering mindset? With the governance? Yes, I think with governance

we've been in a better place than Admiral has ever been.

We bake things in, and again, our engineers think about it because they

want the best kudos for their bit of code.

So the only way you're gonna do that is get people actually using

it and using what the output of that code has produced.

Now, if you've got good governance that has put that into a data

catalog 'cause you've coded that up or is automatic or whatever,

then people will find that data and they'll use it. Okay? And this

is another thing about, probably dovetails into the working in silos.

You develop something for a specific thing, I know it has to happen

sometimes, but if you can write it so that it's open ended and

basically other people from outside the intended recipient can see it in

the data catalog, and within reason, they're allowed to see it and all

that. There's security and stuff like that. They can discover through a

data catalog and basically go, "Oh, I didn't realise someone had done that,

calculated that over there. I'll go and use that." And then they attribute

to the data catalog themselves by saying, "I am using this for this

purpose." That kind of mentality then grows, because everyone used to see

data governance is like a blocker, but essentially there's ways and means

of doing it where you can bake it into that whole delivery process

and actually make it an enabler and not a blocker. I think it

took a lot of companies a long time to understand that,

because I think they just thought of it is a regulatory thing and,

"Oh we've got to do this, we're ticking boxes here and there,"

and stuff like that, but I think you do it properly and it

is so valuable and it can aid operational efficiency. People have gotten

better decision making because the quality of the data has increased.

I used to go into meetings and some guy used to go,

"Well, this is what we sold this product for yesterday," and another guy

would go, "Well that don't match my figures." And you're like,

"Well, who's right?" And now with this data governance you've got a gold

standard, and now you got the guy going, "I got these figures."

"Oh, well I got these figures and they're different, but mine

have a tick on them, okay? They've been checked, so they've been gold

standard, this is actually what happened." The other guy's got no response

against that. And this is the type of things that it can enable,

but I think it takes a while for people to see that because

they do see it still as a blocker. But if you know data an you

know how it's used, you can really see the advantages of having that

strong governance. And I think what you're saying there as well,

is that actually just the quality of the reporting across an organisation

and also out to customers, will be better because you know that you're

using correct data and you're using it in the right way,

and it is standardised. It's not just the reporting, you look at the

AI and the machine learning and the data scientists and things like that,

it used to be they'd have this, "I just want all data. All

the time, as quick as you can, all data, all format, and I

want as much data as you can give me," and things like that. I

think the emphasis is slowly drifting away from that now to actually go,

"No, I want quality data." Because if you're training your models on bad

data, you're gonna get bad decisions, okay? So I think the emphasis has

now moved from, "Give me as much data as you can,"

to, "Just give me clean data." And I think that's an important step

because I don't think I could say that to you two years ago,

but I think now there's a machine learning model for everything,

I don't think anyone's really pushing the boundaries of machine learning.

Yeah, don't get me wrong, at Admiral we have custom ones and things

like that, and probably data scientists if they heard this in Admiral they'd

probably have a few choice words to me,

but to be honest, someone else has done it. Someone else has done

something very similar somewhere. You can pretty much buy a model off the

shelf, okay? And everyone is doing it, it's all like self service.

I can go on to Google's marketplace and I can buy

load of models and get the result I want.

What the difficult thing is the data engineering to get the data quality

up to a level when you can train that model, and I think

that's where the emphasis has changed there as well. So it's not just

the reporting, it's definitely for other applications as well.

So with the teams that you've got there, it sounds like obviously there's

a sort of sharing of knowledge and sharing of code, and you have

obviously done a project and read all of... Been involved with it. What's

it been like working with external consultants? Has that challenged the

thinking? Has it kind of steered the project?

You definitely learn from them. Because at the end of the day we're still

quite immature in some ways, Admiral's cloud journey.

Things take a long time and things like that. And you need that

external support, you need good people who have basically done this before

and know what the pitfalls are. You're still in charge, but you're in

charge of the outcomes, "These are the outcomes I have. Okay?

It needs to work like this." People like Red Olive as well, they come

in, they're classic engineers, they're trained in first principles.

And they will go, "Well James, if you do it like that,

then that could break there." So that's great advice. And so they're thinking

the same way, which is fantastic. You don't have to educate them in

robustness and maintenance and things like that. You just learn a lot from

them. You go, "Oh, okay. Well, we did some similar for the couple

of guys two years ago that did this. Would that work?"

And it's just bringing that experience to it. Because cloud is an experience

and you really have to sort of think differently. And especially when you're

on prem, you design for performance and when you get to cloud,

you're actually designing for cost. Because it's so easy to just throw another

100 processes at it and a shed load of memory, but there's a

cost implicated with that. So you still gotta be super efficient,

because otherwise those costs can run out of control and things like that.

And it's good that our external consultants and things like that, they've

all fed into that. They've gone, "Well, you can do it this way,

it's robust, it's really good, but it will cost you this and you

have to figure that out," and that can be very difficult.

But having people that have done it and things are like that,

that can really guide you essentially on that design from a cost perspective.

And also from a robustness perspective. It's a balance, isn't it?

So yeah, they've been really good. Well, though we've been talking data,

we've actually been talking about communications and people throughout this.

For people who are now coming into the data industry, what

skills would you recommend that they have? And what kind of things would

you be looking for for somebody joining industry now?

To be honest, we take on a lot of grads, a lot of

young people that may want to get into this sort of thing.

The key skill or two key skills I'll put down from a technical

perspective, SQL and Python. Because pretty much everything runs off those

two things. Or is interfaced with them in some means.

But what we really look for is... You can teach those skills.

If you've got a bit of a mathematics or science background or things

like, even if you haven't. But you know how an Excel spreadsheet works,

you know how to look through it, you know to organise data and

things like that, those are the key sort of technical skills we look

for. But we look for behaviours, and we do a lot of competency

based type interviews where we look at, "Give us an example of a

time when something went wrong?" And that may not be in data,

or the business could be something completely external, but we look at the

mindset about how people handle it. So there's a little bit of psychology

in there and things like that, but what we're looking for is people

who are conscientious, who care about what they're actually doing, and that's

important. So all the technical stuff, you can learn, you could go on

100 YouTubes there, right? And you can know concepts better than us.

But when it actually comes to implementing it and your day to day

job, I think your psychology and how you're made up has a lot

to do with it. So we do look for a lot of people

who are very driven, as I said conscientious, really care about what they're

doing, and you can make a really good start to mould them into

a classic data engineer. So that's what we look for.

My last question for you, what is the most exciting thing that you're

most passionate about for the data, pushing forward into the future?

Data automation. Because I know we talked about machine learning and AI

and things like that, and those things are great and very impressive and

things like that, but I think unless you're one of the top players,

they can be very difficult to implement on scale.

But what I'm interested in at the moment is a lot of metadata

sort of data driven pipelines, whereby, rather than deploying code you're

making more configuration changes and things like that. So it's the automation

of it all from a delivery point of view,

I like the idea of. Some people have had a go,

WhereScape being one of them, they're really good, but there needs to be

a next level. So when we're talking about... We've spoken about things like

governance, data cataloging and things like that, that type of thing needs

to happen automatically. So we talk about automation in terms of automated

testing or automated deployments and things like that, but what about actually

automating the actual development? Because a couple of years ago, a lot

of people just hand coding things, and there's a lot of now low

code solutions where you still need to have the technical ability and know

what's going on underneath, but you can template a lot of it.

So I can automatically template that it adds, I'm bringing new data in,

it adds to this data catalog. Or I publish a report, it automatically

gets added to this data catalog and things like that. So that's what I'm

really interested in, because everyone goes after the really exciting stuff

and things like that, but nobody thinks about the implementation so much.

Because I live in that space and all my problems have been in

that space, that's what I kind of get excited about now,

can I have a team of three people that can deliver something in

say a month, rather than a team of 10 deliver it in six

months? And that's kind of where I'm going at the moment,

because we're up to our eyeballs at work and it never stops.

And you can only grow so much before it becomes hard to manage.

But if we can deliver quicker and automate more and maintain our quality

standards and our governance standards and things like that, that's what

excites me. Now, that may be a bit of a boring answer,

but that's the honest truth. A fascinating take from James on the importance

of people, teamwork and communications in data projects. Join us for the

next episode of Fibonacci, the Red Olive Data Podcast, where we'll be joined

by another data expert sharing their thoughts on latest trends in AI and

big data, along with some great hints and tips.

Make sure you subscribe to Fibonacci, the Red Olive Data Podcast from wherever

you get your podcast, to make sure you don't miss it.

That's all for today. Thanks for listening. I've been your host,