Welcome to Fibonacci, the Red Olive data podcast. This new, regular podcast is another way for us at Red Olive to help empower businesses to extract the value from the data they hold. We will be talking to different figures within the industry and asking them about the latest trends in AI and big data, getting their take on the issues that matter to them and amassing some hints and tips along the way. 

In our first episode, we are joined by Chief Data Officer Robin Hayden. He has worked for over 20 years in the field of data and has a wealth of experience in machine learning, AI, large scale engineering, analytics and product development across various chief data officer roles. He has delivered over 100 machine learning models into production, been responsible for tens of millions in additional revenue and hundreds of millions in spend.

Our wide-ranging chat covers lots of different areas, including:

• How edge computing could lead to several benefits, such as greener computing. 
• What the future might look like as products become connected and smarter.
• The crypto space and its relationship to a more democratic sharing of resources.
• Data ethics and how the bias in some algorithms could be an opportunity for civil society to have a conversation about years of discrimination.
• The opportunity offered by the cloud for engineering teams to focus on activities that differentiate their offering from their competition.
• CI/CD as a provider of early, fast, cheap feedback in the development process.

Have a listen to the podcast and let us know your views by emailing hello@red-olive.co.uk. If you like what you hear, please subscribe to the podcast on whichever service you get them from.

- [Nicky] Hello, and welcome to

the Red Olive Fibonacci Podcast.

The podcast all about the brilliant world of data

covering future trends and topical tech.

We'll be joined by experts in the data sphere

to share their opinions and advice.

I'm your host, Nicky Rudd.

Today's episode is the first of a two-part interview

recorded during lockdown with Robin Hayden.

Robin has worked for over 20 years in data

and has a wealth of experience in machine learning,

AI, large-scale engineering, analytics,

and product development.

In various chief data officer roles,

he's delivered over 100 machine learning models

into production,

been responsible for tens of millions in additional revenue

and hundreds of millions in spend.

The first part of the conversation covers a lot of ground

from crypto and AI,

data ethics and Skynet,

to the best use of the cloud

for businesses looking to update,

and how to run a data migration project.

So let's go.

(introductory rock music)

- [Nicky] I thought we could start off

if you could just tell me a little bit about

how you got into the world of data.

- [Robin] It was a bit of a long journey.

I actually, I was quite into this sort of AI space

from very early on,

but you couldn't really get a job in that

unless you're a researcher and or something in that space.

And you know, that wasn't an option that was open to me

at the time.

So I went off and did

a lot of sort of early internet type work.

And it started to become apparent

that we were able to do really smart things.

It was basically all based off of big data

and it was starting to become apparent

that we now had enough data

that you could go beyond doing

a sort of ad hoc analysis.

You could really start to extract decisions.

And it was obvious that at some point

it was going to change the way we did all of computing.

I sort of moved my career to focus primarily on data

from that point it's 80% data engineering

and 20% intelligence.

But it is the 20% that's really fascinating.

- [Nicky] It's a really exciting industry

to be involved with,

even in the work that I've been doing

over the last couple of years.

The step changes really just have jumped.

What do you think of the most exciting changes of foot?

How do you see things developing over the next

two to three years.

- [Robin] In the space, I would say definitely.

I think the proliferation of edge computing

and the ability to push intelligence

all the way out to the edge

is something that is, some people know about it,

but it isn't as widely understood in this space.

People think of big data as big and expensive

and this sort of planet killer in many cases

because these big models,

and they are some big models

and they are expensive to build like GPT-3,

or Google's competitor to that recently,

which was,

I think GPT-3 was several billion,

a few hundred billion parameters or something.

And Google's equivalent, not that long ago,

was over a trillion parameters.

And those things really do take

enormous amounts of power and computing and stuff to build.

But the flip side is that the computing,

it takes orders of magnitude more energy

to process something,

to push something all the way to the cloud and back.

It actually, even just pushing data

from the processor into RAM on your machine

versus processor registers,

is it very expensive?

Like that distance,

that extra distance costs a lot in computing terms.

The idea that things like neural networks,

that basically they're doing matrix operations

over and over again,

that's basically taking the same data

and recycling it,

and reprocessing it over and over again in the processor.

So in many ways it's actually very power efficient

and this idea that once you have these in these models

and they can do inference

out at the edge on some very low power devices.

I mean, there's this real, you know,

as they're starting to

become embedded in all sorts of things,

cars obviously being the most prominent of those,

but in almost anything that's out in our environment

and all sorts of sensors and things,

you're getting really low power chips and things

that are optimised for this.

And I think our whole world

may well become much, much more intelligent.

And you combine that with things that are emerging,

like the crypto space,

which if you go beyond the currencies

is really just people exploring

how far can we take this distributed computing,

this sort of collaborative computing,

and perhaps that we've had a large amount of centralization,

which I suppose is a bit counter to

what was the original sort of spirit of the internet

when it was very open for everyone.

And now we've got this consolidation

around a few big powers.

And there's a lot of political backlash to that.

And all the rest at the same time,

you've got things like crypto space developing

and despite the sort of hype,

and I'm sure a lot of people may still get hurt

in the investment world there.

But I think the idea of being able to encrypt

and process things very widely

and developing those sorts of,

they're almost like democratic networks

that allow people to share processing of data and stuff,

and the file storage and stuff

in ways that it couldn't do before.

And you leverage that

with the sort of ubiquitous intelligence

and the ability to run intelligent algorithms on everything.

It's hard to actually rarely,

to even fully imagine what that might mean.

Honestly, if you think of an entire world

where everything is more intelligent,

where, you know,

also where there's not necessarily

one or two big controllers of that world.

That could be a very radically different world.

Then, of course, if you zoom forward even further

and you look at things like quantum computing,

which doesn't solve everything,

that particular class of problems,

but certainly then there are some very big problems

and things that were much, much harder to do.

And so if you zoom forward another decade or so,

maybe two,

I don't know however long it takes to do that.

You know, then we just have this

another sort of order of magnitude

for a certain class of problems

and other sort of order of magnitude improvement

in sophistication that we're able to achieve.

So it just feels like this idea that,

I don't really obsess much about generalised intelligence,

and those people who think about

Skynet's going to take over the world.

I think it's impossible to say how far we are from something

when we don't know, we don't have a map to getting there,

if you know what I mean.

We could be a decade away.

It could be a million years away.

Nobody really actually knows.

So I don't really spend much time thinking about that.

But I do think that it's obvious

that over the next decade even,

all of our products are going to be much more intelligent.

And not just the products we're used to using

on the internet.

All of our products.

The things that are in our house,

and in our cars,

and in our gardens,

and that sort of thing.

And that's really exciting.

- [Nicky] What would you say

when it comes to that kind of intelligence,

and what's your take on

the kind of the ethics around it,

and that kind of ethical data question?

- [Robin] Technology is neutral,

but nuclear efficient processes is just nuclear efficient.

When you put it into bombs,

it's obviously a bad thing.

And I think the same could be true of a lot of technology.

So there will be people who abuse them.

This is not dismissing that.

I think it's very important that as a society,

we think about ethics,

and we think about things like bias and all the rest.

I don't think we're actually worse off as a society.

I think it's that swings back and forth.

We get more sophisticated,

we keep progressing.

And I think there will be moments of disruption.

Maybe it's things like

the crypto space wrestle some of their power back

from the centralised control.

Some of the big cloud providers and things have now,

or maybe it's something else.

And in that moment, opportunities will be created.

And of course you will have

people who take advantage of that,

and there will be negative side effects,

but I've also no doubt that I was doing,

for example, when I did my master's,

I did quite a bit of stuff that was medically focused.

I'm not in that space now at all.

The things I was looking at like epilepsy,

and heart conditions,

and things like that.

And looking at trying to detect certain conditions,

or forecast them and that sort of thing.

So the one or two things I did,

what really drove this kind of home to me,

the power of this stuff,

is that today,

if you took like this particular sort of epilepsy condition

that we're looking at,

they would have five or six experts sit around,

a sneeze of people who takes well over a decade before

they can even at the level where they're qualified

to sit in front of a screen.

And then you've got like maybe five of these,

sometimes looking at someone with this condition.

That's a problem right now.

As humanity has gotten where there's seven billion of us,

and maybe we get to ten billion or more, who knows,

and we just have a critical shortage

of those kinds of skills.

And so that sort of analysis and diagnosis and stuff,

we're not displacing doctors and things entirely,

but we certainly augmenting them

and making it possible for a lot more people

to scale up things like healthcare.

And so you can't possibly see that as an all bad thing.

That is if it's seven billion people now,

probably one billion of us have okay health care.

Some people would argue how okay our healthcare is,

but it's certainly much better than the other,

than the vast majority of the population.

So I think those kinds of things,

automating intelligence,

scales up a lot of things which make it easier

to sustain the people that we have

with a lot better healthcare,

probably better food,

probably better legal representation.

That's another field where

legal representation is extremely expensive,

but you're getting all of this stuff emerging,

which is automating some of the analysis in the legal space,

which is probably going to democratise legal representation.

There's all those kinds of things

that I just can't help seeing the opportunity for good.

For sure, there are some issues,

there are things like bias

and other things in our data sets.

And I think on that particular thing, bias,

it's probably worth talking about.

Bias isn't,

there's a bad bias and there's a good bias, right?

When you hire somebody for a job,

in a sense, they've got a certain amount of bias

built into them,

and bias, in the sense that

they have come to know a certain domain

and they've come to get familiar with things.

So they lean towards, when something happens,

they lean towards saying,

oh, my experience tells me that this is probably that.

And that in the machine learning world,

there's this notion of bias-variance trade off.

And in fact, in that sense,

you don't want models that are completely unbiased

because every time they get a new data point,

they would just leap to a new conclusion

and they wouldn't actually be learning anything.

So they learn a bit of bias,

which is actually useful.

The kind of bias that we talk about

and we think is bad,

is when we learn biases that are not useful.

We learn things that are incorrect.

Like the fact that let's say the colour of someone's skin

determines whether they're good at a particular job,

which is latently incorrect.

And so that's not useful bias.

But all that's doing,

it's not that the technology has introduced the bias.

What it's doing,

is it's forcing us to confront the fact

that sort of bias has been implicit in all of our data,

and in all of our examples and things.

So we're training the stuff off of that

and it's learning the bias

and it's forcing us to confront that and say,

hold on, perhaps we need to correct that.

But it's not the algorithm that's causing it,

it's decades and millennia and years and years

of bias that we've developed as a society.

You could even say,

it's a positive that this is triggering that conversation.

And now that it's scaling that up

and people are saying,

hold on, you're scaling up algorithms

and things that are now re-entrenching

this bias we want to get rid of.

So it's amplifying the conversation,

which, I think it's a good thing.

- [Nicky] Let's talk a little bit about scale

because the work that you've done with Red Olive

on a couple of projects,

most recently,

it gave me a sense part of it was moving to the cloud,

and I'm guessing that's really where scale comes in.

But what would you say is the big deal with cloud?

What are the big business drivers?

Where does that enable you to go next?

- [Robin] Cloud, for me, it's about,

you could slide other things in here, not just cloud.

What you're trying to do all the time

is you're continually trying to stand

on the shoulders of giants.

So how we progress as a society,

as we build on the things

that other people have done before us,

this idea of don't reinvent the wheel.

We make progress by basically,

we do more than the generation before us

by using everything that they've done

and adding to it.

And when you build things, say build software,

or build algorithms,

and that sort of thing.

What you do find,

sometimes people at the earliest stages in their careers,

they're very interested in learning

the underlying mechanics of things.

So they go off and they want to build everything

from the ground up.

And sometimes what happens is you get people more senior.

Very often, they start to change their attitude,

or they start to become, has someone else built this?

And then what they look for

is the stuff that someone else hasn't built yet,

and then they build that.

And usually it's building on all of the things

people have built before.

Cloud aside, even things like a normal software development,

almost everyone now uses frameworks

and all sorts of things that other people have built,

and they build on top of them

and they wire them together.

And cloud is one of those things.

It's saying that you don't differentiate yourself by say,

installing a database.

So that's just something

that sort of has to be done to facilitate,

let's say, the analysis or the machine learning,

or whatever work that we then do on top of that.

And so what cloud is giving you,

it's giving you the ability to offload

all that non-differentiating stuff.

And economic terms,

one of the things that certainly served us well

is this certain amount of specialisation.

And what it's allowing you to do,

is it allowing you to take the experts,

the top specialists in those areas,

in setting up that infrastructure

and making it available to services

and that sort of thing,

concentrating them in a few places,

and then reusing their efforts

and everyone else reusing their efforts.

So that for me is that the point with cloud,

is you're not trying to save money,

you're trying to increase productivity.

You're trying to not build things again,

spend all your effort building

the stuff that's going to improve your business

and your domain in some way.

Rather than installing a database,

or installing, I don't know, it could be

a streaming platform,

or whatever the case may be.

And it really does increase your productivity.

If anyone has worked with,

let's say, in data space, like a big data warehouse,

it's really easy these days with something

like say, big query.

And you just use it, right?

It's obviously still things you have to do,

but there was a time when you had to spend a lot more time.

You have to have a lot more DBAs

and things optimising your system.

And that's, if you're a business,

that's got a team of, I don't know,

you might have a team of two people.

If one of them has to do all of the maintaining the database

and optimise that all the time, that's half the capacity.

But even bigger teams,

you've got a team of a hundred.

I can bet you that if you're doing it on-premise,

the 20% of those is permanently looking after

your environment.

And that's 20% you're not spending

on some sort of business differentiator.

- [Nicky] With that, you're talking about the teams,

how do you get it so that team and culture

are tied into that.

Because I'm guessing there is a sort of,

a bit of an education

that might come along with that kind of

moving to the cloud.

And how differently you might have to work

than in a traditional sort of data and IT projects.

How important is it to have the vision

and be able to communicate that internally

to the rest of the organisation?

- [Robin] It's not really a hard sell anymore

to tell everyone that everyone's moving to the cloud

and we got to do it too.

Now, people may not always recognise why.

I think that the people who, if you've got say,

technologists and people who are working on the platforms,

for example,

they're always very willing to do it

because they know it's where everyone else is going

and they want their careers to progress.

I think executives use those increasingly,

because everyone's doing it.

It just seems that's the kind of common wisdom, right?

So that's what you do now.

I think the important thing now,

it's not so hard to sell people on the idea of doing cloud,

I think it's just probably more important

just to make everyone understand,

what is it do you want from cloud?

Why are you doing cloud?

Why is everyone doing it?

You're getting those kinds of philosophies out there

and saying, look, all we're trying to get is,

we're trying to offload all that non-differentiating effort

because that's where you will get people

trying to build things themselves,

or you'll get,

it's a hard thing sometimes to move something

when you've got a complicated on-premise platform,

for example.

And it could be data,

it could be any other software.

And it's actually quite hard to move from one to the other,

so you had a lot of this sort of

lift and shift type mentality.

And the reason that doesn't work is because

it doesn't leverage this idea that you are floating,

all the non-differentiating work.

If you're fundamentally still doing most of the instal

and maintaining all the applications.

So a lot of startups,

but even bigger organisations

are getting there with sort of saying,

okay, actually, why don't we, you know,

shift all of this,

well a lot of stuff, to something that's more

server-less.

Not the idea that it's as a service,

I just use it to just write some code

and deploy that,

rather than have to manage all the intermediate steps.

That in itself is a lot of work to get there.

So it is actually sometimes easier for people

just to think in terms of, okay,

what if I just do what I was doing,

but to do it on the cloud.

And I think that's probably where people fail

and when they end up increasing their bowl,

because this is like renting or buying a house

in some respects.

If you're just going to do exactly what you did before,

then you're better off buying than renting

because they've got a margin on top

and they're doing a little bit of work underneath the hood.

But if you don't change your attitude,

you will just spend more money

and you won't get the benefits.

So I think that's the important part

is understanding what you're trying to get

is additional productivity,

and other things as well.

There's some philosophies there that I think people,

I don't know,

maybe it's because I've been doing it so long,

it doesn't seem new to me anymore,

but I'm not sensing that I'm getting a lot of resistance

to the ideas anymore.

But I think other things people don't always fully grasp,

if they're not in that world,

emphasis changes from things like

managing availability of service

to managing costs.

And what I mean by that is,

if you use kind of properly

and use things like auto scaling and all the rest,

and you use like chaos monkeys

and these things where you're actually

deliberately trying to take out services to see,

to make sure things are resilient.

You try to take out services,

say in the middle of the day,

and then because if they are going to break,

it's better for them to break while everyone's around,

basically is a philosophy with something like that.

But that's all facilitated by this idea

that as a consumer,

you can use the cloud as if it's an infinite resource.

And that's great,

because you can build services that never go down.

So the first thing,

you've got to philosophically get into that mindset.

We live in a different world

where things can be incredibly robust.

Where they can just scale up and down

as our market scales up and down.

And that is actually a great attribute

for any organisation to have.

To be able to adapt like that to circumstances.

The flip side, of course,

is that before your costs were then finite,

in the sense you bought some stuff

and your service might go down because it's overloaded,

but your costs were still fixed.

So that's where I think

you need to pay a bit more attention to cost management

and that sort of thing.

And that can be tricky.

Like I've been in a company where, you know,

early stages of the projects,

I was like, how much is going to cost?

Well we've just started doing it,

like we've just launched.

I have to spend a few months

figuring how greedy the service is.

And you kind of work through it for a bit in the beginning.

But you do have to spend some time,

work through that,

and then spend a lot of time putting together

those mechanisms where you actively trade off

that sort of cost versus availability and performance

for your customers, and that sort of thing.

So it's things like that,

it's those nuances I think,

is understanding you're trying to be more productive.

You can build a sort of bulletproof service,

very easy to port internationally

and all sorts of things if you do it right.

But understanding all those things

and specifically designing for that,

so that you actually get the value from cloud.

I think that's the bit which

you still have to spend time banging the drum,

making sure everybody understands that

it's not so hard to convince people to go to cloud anymore,

if they're not on cloud already.

- [Nicky] Do you think that

within the development side of things,

people are braver knowing that

if you're using a cloud platform,

that it gives you more scope to try things out

and if it doesn't work.

So you're kind of doing a bit more of that CI/CD

as you're moving along, that's how you develop them.

Do you think they sort of come together almost,

they fit together a little bit like hand in a glove.

- [Robin] They do,

in theory kind of CI/CD is, I don't know,

orthogonal to the cloud,

they compliment each other.

But you could do a lot of that stuff on-premise.

It makes it easier to do it on cloud

because you can just spin up and down instances on demand

and that sort of thing,

because you can treat this thing

as almost an infinite resource.

It makes it very easy

to build these very flexible pipelines and stuff.

But at its heart,

the idea of agile is about continuous feedback.

And it seems obvious that if you wait until,

if we look at traditional, much earlier philosophies

where you had waterfall type philosophies

in development and stuff,

where people would design whole big programmes

and projects and stuff,

and then you phase them.

You had design in the front,

and then you had this development,

and then you had testing,

and you had some sort of release.

And the problem with that stuff is all the UAT,

and all of that sort of thing was very late in the day.

And the projects took so long,

that lots changed in the meantime.

And you got to the end

and then you figured out that there were all sorts of things

that needed to change.

The world had changed,

Or there's a few, or things have been misinterpreted.

And there's a lot of people in the middle.

There's a lot of messages that have to be understood.

And there's a lot of misinterpretation that happens there.

So there's nothing quite like

putting actual product in front of people saying,

did I get it wrong,

or is it still in the context of my environment and stuff?

And so that's rarely,

agile,

a lot of it basically centres around that.

It's can we get feedback very early in the process?

Can we have short cycles?

Can we have lots of sort of show and tell type things?

Can we, and it depends on the product you're building,

but can we continuously,

can we encourage that continuous feedback

and then see those problems early?

Because it's a lot cheaper to fix something

when you've written three lines of code,

than at the end of the project

when you've written a whole complex thing,

several things all depending on each other,

and you have to unwind many things.

So if you say that's the heart of agile,

then the CI/CD thing just isn't natural fit to that.

I'm saying, so talking about to the cloud,

in the sense that that is about,

CI/CD is about really just putting things out often,

and quickly,

and efficiently, right?

Automating the stuff that you can automate.

The process from checking in your code

through to automating the testing.

And of course,

there's a discipline there about writing a lot of tests

and test first development, and that sort of thing,

which makes that work, right?

It gives you the confidence to look at

these large organisations that have

tens of thousands of releases a day and stuff.

They can only do that because

the process is designed so that every time

something is checked in and merged to main,

it's automatically tested and all the rest.

It's not entirely bulletproof.

You're much more confident in an environment

where something won't go all the way into production,

unless it gets thoroughly tested,

and this whole process is really automated.

That sort of gives you a confidence.

It means you can deploy very often.

And if you can deploy the full things very often like that,

then that fits that sort of feedback cycle.

You've got lots of little changes happening all the time,

and occasionally something will go wrong,

but then it's going to be one small thing

that you can roll back and you can learn from.

And you become a very adaptable.

And I'm not using the term agile with all the baggage

of what we think agile means in software.

I just mean you've truly become an agile organisation

in the sense that

you become very sensitive to little changes

in your environment.

And you change really quickly,

and you evolve fast.

And of course,

if you can evolve quickly,

then you've got an inherent advantage over something

that doesn't evolve quickly.

So that's where I think CI/CD fits in.

Again, it enables you to evolve really quickly.

How it relates to cloud,

is cloud has enabled us to, because

cloud has also allowed us,

so in the past, if you wanted to do something new,

you would have to go and get quite a large CapEx budget.

Let's say to buy some licence,

to instal some equipment,

to do all of those things upfront.

And so obviously in cloud, now,

if you want to do something small,

you can do something small.

And that's a key point, right?

You can go and try.

I talked about Google,

let's say Amazon,

I've got something like Kinesis or something.

There are products that, I actually prefer,

some of them it's the underlying product,

so let's say like Kafka or something.

That underlying product,

I think it was a really good product.

There's other things that have come along

in the streaming space that kind of improved on it a bit.

Historically, you've had to go and buy,

you have to go to a case to spend like

$100K+ with someone to get started.

And of course you can go into Amazon,

in something like Kinesis,

and you can just start.

And you can pay like a few quid just to do a POC, basically.

So that fits into the cycle of just being able to

iteratively

do the next thing.

See if it works.

It fits this whole picture of continuous evolution

in very gradual steps.

- [Nicky] Excellent advice from Robin there.

But we're going to have to leave our conversation

and pick up on it in the next episode

where he'll be talking about the sorts of skills

people need to develop to get ahead in the industry.

It's full of interesting and useful info,

so make sure you subscribe

to the Red Olive Fibonacci podcast,

from wherever you get your podcasts

to make sure you don't miss out.

That's all for today.

Thanks for listening.

And catch up next time.