In this episode of FutureNow, we speak with David Biron, Head of Networks at The Francis Crick Institute, one of Europe's leading biomedical research centres.
David shares the realities of leading a five-year data centre network transformation programme, supporting thousands of researchers, and maintaining resilience in an environment where disruption can have real consequences.
We also dive into how to maintain stakeholder trust throughout a major transformation project. This episode offers valuable lessons for network and infrastructure leaders across healthcare, education and research.
*This discussion reflects The Crick's experience of their Data Centre Network Transformation and related projects at The Crick and does not constitute a general endorsement of Block as a supplier.
Yeah, so we had sort of a 5-year road map and that actually started with the firewalls. So that was the first
Speaker:first piece of equipment to be replaced, and then we moved on to probably one of the biggest pieces of
Speaker:the puzzle, which is the Wi-Fi. So to get to every corner of the building that large sort of 93,000 square meters
Speaker:of building to actually change the change the Wi-Fi um that's a that's not a small not a small task. So
Speaker:yeah that and that is where you've got equipment spread out across the building and to actually get someone to
Speaker:that area and we have areas that are sort of it's high containment as well.
Speaker:So, you can't just go in there. You have to get special permission. So, yeah, it's been a long been a long process.
Speaker:Um, but that's worked. Yeah, we've got there. We've got our new Wi-Fi in there, and then we moved on to our other probably biggest piece of
Speaker:work, which is changing the edge switches. So, the building split into sort of four quadrants. Um and
Speaker:we've got sort of over 50 like switch hub rooms that all need the equipment replacing. So we're sort of logistically
Speaker:sort of arranging the downtime as I mentioned before sort of and actually ripping and replacing and they're not small devices. They're chassis. So
Speaker:it's like a forklift job. You got to lift that in and put the new put the new stuff in. And you have to be confident that it will work because once you
Speaker:forklifted it out, it's going to be a lot of work to put it back in again if something doesn't work. And I imagine with the amount of research that's
Speaker:going on there downtime is just it can't happen. You've got to have solutions around that. How maybe did you expand on that throughout the project?
Speaker:Yeah. So this is the biggest challenge is actually dealing with downtime.
Speaker:And I think the ideal is always to reduce bandwidth to services than completely take them down. But, one
Speaker:of the key things sort of taking down services is actually the communication out to people because at least if people
Speaker:understand why it's happening and what's happening, they're more perceptive to you making changes for
Speaker:suddenly someone turns up one day with a ladder and they're like, well, who's this person? What are they doing?
Speaker:So, the work that happens at Francis Crick Institute is of such high value and high importance. How critical is the uptime of that network infrastructure?
Speaker:And I guess can you maybe expand on uh the real world impact that has if any if there were to be any downtime?
Speaker:There's sort of various impacts, but effectively if you stopped that experiment from happening, they could lose time. If they actually lost that
Speaker:data for that experiment to happen, that could be catastrophic. It could be months' worth of work that actually gets
Speaker:lost. So the design of the network infrastructure, how does that support scalability for the massive high
Speaker:performance compute environment that you have?
Speaker:So I think the key elements in the design that we're really happy to see is actually we've got the bandwidth that's
Speaker:needed. Um the existing network was struggling in regards to the actually interfaces to plug the devices in. We were running a 40 gig. We
Speaker:needed 100 minimum for the for the new stuff coming in. And we were always at a yeah we effectively what we were having
Speaker:to do with the infrastructure is we're having to bolt on new Cisco stuff to support the old or to support the new equipment going in. Um and actually it's
Speaker:worked out really well for us because we've been able to use existing investment with the new network that Block have put in. So actually it's not wasted money,
Speaker:it's not wasted effort. We've actually been able to bring that new stuff in.
Speaker:Um, and particularly for high performance computing, you've got to get the data to the end user as fast as possible and they've got to get the data to the platform as fast as possible.
Speaker:So, yeah, that actual connectivity and bandwidth is key and that's been built into the solution and we're happy
Speaker:now that there isn't sort of a single point of failure or a sort of a drawback in that overall design.
Speaker:Yeah. Yeah. kind of and I guess when you look at kind of what's running across that infrastructure I guess that world-class research that really relies on
Speaker:world-class data and keeping that protected as well how do you find sort of balancing the sort of openness but also the security in such high stakes
Speaker:environments so I think the main thing is and the main goal is not to block the science from happening
Speaker:so um, and I think the key thing with a lot of the work that we do is the early engagement particularly with new projects If you've got new science
Speaker:projects happening, you need to know beforehand because if you don't know about it beforehand, then it comes in really late, you've got to suddenly come
Speaker:up with a solution. So, one of the key things we've got in place within The Crick is a network access control solution. So, it
Speaker:actually knows what's going on to the network and we can control what joins the network as part of the solution. And just looking at the building, it's such
Speaker:an iconic building, the Francis Crick, and it's adopted quite a lot of smart technology already across the organisation.
Speaker:How are you finding managing that as a network team and how have you had to adapt there?
Speaker:So we were lucky that we were sort of a smart building from day one which has its benefits but then the criticality of
Speaker:the network is higher because the last thing you want to happen is someone holds their pass up to the door and you can't and you can't get in. So yeah
Speaker:everything is controlled. So the lighting, access control, security it is all part of that part of that smart
Speaker:building. So it is a challenge and that's why we always looked for an always on network because yeah you can't afford that downtime.
Speaker:Yeah it just adds so much more criticality to getting that network right and the transformation project that you've been on. So you've been leading this 5-year transformation
Speaker:project and also just going back to your point around how many sort of organisations and stakeholders that must be involved in the Francis Creek. How do
Speaker:you kind of like keep the teams and the scientists confident in what you're doing while you evolve this network?
Speaker:So, I think that goes back to the communication because yeah, you need to communicate out to people actually what the benefits are. Yeah.
Speaker:And what it's going to mean to them because often as technical people and we're all guilty of it is sort of saying
Speaker:all the technical advantages but people don't care about they just want it to work. So that's the most important thing is to describe it in their terminology.
Speaker:So we're a trusted sort of, trusted pair of hands to sort of rip and replace this sort of stuff because you are disconnecting and disrupting things.
Speaker:And I think sometimes it's having the key stakeholders because if you speak to the right stakeholders, you can actually get some key bits of
Speaker:information to say actually if you did that at 3:00 a.m. you could do it without any problem. Yeah. Yeah.
Speaker:Um, and that's and that's a really key really key thing is that stakeholder engagement.
Speaker:Yeah, I can imagine. And you must have quite a strong team behind you as well there within your network team and the wider infrastructure team to be able to manage such change.
Speaker:Yeah. So relatively small team. So it's seven people including myself. But, we really gel as a as a team sort of
Speaker:lots of different personalities lots of different skills but yeah we sort of come together and we're really strong in in lot in
Speaker:lots of areas. So yeah people we often have people ask us questions in the office and we can generally answer those and support
Speaker:but the most important thing is to support users in whatever they need to do.
Speaker:Yeah. So just going back to the topic around trying to keep scientists and the teams across the organisation happy. How do you balance the challenges
Speaker:posed by shadow IT but also providing that sort of lab autonomy across the organisation? So I think sort of shadow
Speaker:IT is a problem throughout sort of institutions, I came from the university world and that was that was really common, it's no different at
Speaker:The Crick because people want to do their experiment get it set up as quick as possible and yeah sometimes
Speaker:they buy whatever they need and then expect it to work and that's always a challenge and
Speaker:the way we've tried to work against that is just we've tried to enable them to do what they need to do but the early
Speaker:engagement. So a lot of our procurement teams are like asking those questions from day one and not yeah saying to people well actually yeah you need to
Speaker:engage us first because you could end up with any sort of piece of equipment whether it's like a consumer or enterprise grade device. And there are
Speaker:mechanisms like you can have something that is could be an internal only device. And there's mechanisms within the network to enable
Speaker:that to happen. You've got some stuff you just want it to talk to the internet and nothing else. So yeah, we've got a way of dealing with that
Speaker:within the network, but the most important thing is the early engagement because you don't want to be trying to reverse engineer something that somebody's bought to get it to work.
Speaker:Yeah, that makes sense.
Speaker:And I think when we were speaking a while ago, you said that the future of the data centre is going to look very different.
Speaker:what do you think's coming next and how are you preparing for that within your team? So I think the challenge is
Speaker:always in regards to the data centre. It's sort of having enough power in London if you want to run your own AI, having enough cooling.
Speaker:I've come from Cisco Live a couple of weeks ago, everything is moving to direct liquid cooling. So the infrastructure from your traditional air data centre is very very different.
Speaker:So if people want to do that it is a challenge and us as a building we don't have a car park you can
Speaker:wheel a container in and actually run a data centre from there. So we're looking at various opportunities but particularly in the world of networking
Speaker:there's a technology called ultra ethernet which is something that's coming into play and
Speaker:the AI workloads can't cope with your standard Ethernet offering. You have to have something different. So that's something as a team we're sort
Speaker:of learning about and that's one of the key takeaways at Cisco Live was looking into that technology.
Speaker:Yeah. And in terms of the innovation that's going on there, how do you balance that innovation versus risk when you're working at such high stakes?
Speaker:I think you never want to be the first person to do it, but you don't want to be the last either. So yeah, I think institutes and universities always watch
Speaker:each other on what they're doing in the world. So there's announcements from some of the people we partner
Speaker:with stuff that they're doing. So it's really interesting to hear about what they're doing and their experiences.
Speaker:Yeah it must be very helpful in terms of learning about those insights. And in terms of your transformation project. So, what's the next steps now you've
Speaker:gone through the transformation are you looking at anything around more automation and observability? So the most important thing was not to disrupt
Speaker:services. So some of it was a like for like replacement because then that gives us what the baseline to build on. So what we're
Speaker:hoping to do now is where we completed the replacement program is actually enjoying what we what we bought and building on that. So there's lots of
Speaker:bits where we're going to need to cause disruption, but actually it's for it's for the greater good overall. To make sure the devices are connected correctly
Speaker:and we're providing the service that people need.
Speaker:So obviously across the organisation there's a lot of use of AI across research and the technology that's coming in there. How are you and your
Speaker:team looking at AI at the moment around whether that's automation or reduction of workloads?
Speaker:Yeah. So that's an interesting question because within the team I think we're look sort looking to reduce
Speaker:workload of like mundane tasks. So yeah, as you mentioned, automation is something we're looking into. One of the critical things which people
Speaker:find really tricky is sort of if you have a technical support issue and Cisco always say, oh, get a technical support
Speaker:file. If you can have an AI bot that goes out and gets all the information beforehand and just pass it on to Cisco, then it saves my team hours of work just
Speaker:trying to get all the information that that you actually need to diagnose and solve a problem.
Speaker:Yeah, I can imagine. Yeah and what's next for you then? What's your ambitions and where do you want to take this next?
Speaker:Um, so I think some of the key things that we've that we've got is sort of um the network access control I said before
Speaker:was a key thing. So it's augmenting that with the with extra data from other systems and allowing us, if we needed to
Speaker:block something from the network you do it in one place or one place only and then it propagates out to all of those other other systems. Because we do have a
Speaker:multi- vendor network. So it's not just we obviously have Cisco and a data centre provided by Block but we've got an HPE network for the rest of that and
Speaker:it's actually bringing all of that stuff together. And my brief was always to get the best fit of technology for The Crick and that has built us into a multi
Speaker:vendor network. Which has been a good it's been a good thing. We're confident that we've got all the right solutions
Speaker:and technology in place. Um, but of course all vendors will tell you, well, if you buy all of our stuff and only our stuff, it'll work really really well
Speaker:together. So, it it's always weighing that against the multi vendor versus single vendor.
Speaker:Yeah, it makes sense. And in terms of The Crick itself, is there I can imagine, you know, things are evolving and changing all the time there in terms of what they're asking of you. Is there
Speaker:anything coming down the line in terms of what they're looking to do as an organisation? Any big changes that are going to have an impact on you and your team?
Speaker:So I think the way they built the building was it can move, like
Speaker:even the walls can move. So we have a constant stream of changes finding more space within the building to
Speaker:support the science. One of the big projects that we did over the last sort of 18 months was they actually built on the roof. So you ended up with
Speaker:cranes actually building on the roof space to which is the Skylab. So that was sort of code name Skylab but that
Speaker:was really interesting because people thought well you can't find any more space in the building and actually they did it and we had to provide a network to support that.
Speaker:Wow. Enough to keep you busy then I imagine.
Speaker:And if you look at maybe if we talk about your time at The Crick what's been your biggest achievement to date? I think the biggest achievement to date
Speaker:for me personally was obviously becoming head of networks but yeah particularly for the team completing this 5-year replacement
Speaker:program and all the effort and that's gone into it. Um yeah it's
Speaker:not a small task. So yeah thanks to all the team for their efforts in getting everything over the line. Yeah really proud of what we've achieved.
Speaker:Yeah I bet. Thank you. Yeah and the partnership with Block how's that worked and kind of what are the key ingredients to making that successful?
Speaker:Yeah, so I see Block as a key partner. They've got some yeah some really good skills inhouse working with Cisco to deliver the
Speaker:deliver the project for us. And yeah, a safe pair of hands like an extension to our team because we've got a lot of knowledge
Speaker:on site within the team. But yeah, we don't have all the latest and greatest technologies and the knowledge of
Speaker:those. So Block have been a really key partner to help us deliver that.
Speaker:I know the team at Block have really enjoyed working with you on this project and really keen to continue that partnership as well. It's really exciting to see what you've got coming
Speaker:down the line next as well and thank you for joining us today. I'm sure the audience as well will find this really fascinating. Thank you.
Speaker:Thank you. It's great to be here.