Please Rate and Review us on your podcast app of choice!
Get involved with Data Mesh Understanding's free community roundtables and introductions: https://landing.datameshunderstanding.com/
If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here
Episode list and links to all available episode transcripts here.
Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here and their great data mesh resource center here. You can download their Data Mesh for Dummies e-book (info gated) here.
Sid's LinkedIn: https://www.linkedin.com/in/siddharthin/
In this episode, Scott interviewed Sid Shah, Head of Product Data and Analytics at Airtel, a large India telecom operator. To be clear, he was only representing his own views on the episode.
Before we jump in, it's important to note that Airtel are doing data mesh on 'hard mode'. Because of regulatory requirements/restrictions, they are all on-prem. That means extra challenges when it comes to securing compute resources.
Some key takeaways/thoughts from Sid's point of view:
Sid started by talking about some of the challenges of data and analytics in a very large organization that has grown through acquisitions - data is siloed across the organization and you can't get sight of the customer journey across lines of business. To address this, a few years ago they started landing data from across the organization into a central data lake. You can probably see where this is going - a central team getting more and more overloaded while there is an ever increasing demand for data. Add to that a data platform where the only teams sophisticated enough to use it being in the central data team and you have major bottlenecks, poor quality governance, the business teams weren't sure what to trust or what data really meant, all combining for a bad analytical experience. So that was the stage for why they started to look at data mesh.
Even before coming to data mesh as a potential solution, Sid and team thought the biggest problem was poor data ownership. The teams who knew the data couldn't own the data even if they wanted to - they didn’t have a platform they could actually use or the data fluency to own their data work. The other issues could really be traced back to that - poor quality, bottlenecks, low trust, etc., it all came back to poor data ownership.
According to Sid, coming across data mesh and all the other organizations that were implementing - including the stories from this podcast and the community - validated that they weren't the only ones experiencing these data and analytics issues. And it gave them not just validation but hope and a set of talking points and proof points to leverage internally for driving understanding and buy-in.
Sid and team understood early on that changing data ownership is not a "trivial exercise". It's not just changing the responsibility, you have to enable domains to actually be able to properly own their data or it's just the same problem of bad data and bottlenecks but just new people to point the finger at. They decided to not try to solve challenges at the small scale but come up with an approach that was going to really target the pain at the enterprise level, so that's why they arrived on data mesh.
Two things that helped drive data mesh forward as a concept at Airtel were a general understanding of data mesh and its aims at the senior leadership level in the organization and again feeling that pain for long enough that people were ready for a fix. A third aspect was creating a way for data mesh to actually happen at Airtel specifically - how could data mesh work in their organization without trying to change everything about the organization and how people/processes worked? Essentially, how can you explain how this would all work to people at their level of involvement - senior leadership doesn't worry much about what tech your platform uses - and that your mesh implementation won't disrupt the entire organization in a bad way while addressing a lot of the causes of data pain.
As a number of guests have pointed to, when speaking to the business, Sid recommends to speak to their business pains. Data quality issues, not being able to nimbly react to the market based on data, reliability issues of things breaking, etc. The business wants to deliver on their objectives, not talk about how you are building the platform. Think about time to experimentation as well - if you can take the time to deploy an experiment in the market from weeks to days or even hours, what can that do for them? The product and engineering orgs love the sausage factory tour so keep it to just them :)
When looking to drive buy-in and understanding of data mesh with the technical teams, the engineering and product teams/leaders, Sid brought in a number of people from other organizations doing data mesh. People got used to the idea of data mesh and also, if they are seeing many internal presentations on data mesh, they understood the momentum was building so they'd have to get on the bandwagon. Scott note: this is part of why Data Mesh Understanding exists - not everyone can easily bring in those outside speakers.
When they were just starting out their journey, a number of domains gave the quite fair pushback of they didn't have the skill sets to do data mesh, to own their own data. They also didn't have the resourcing to take on the additional work, both people and compute - recall, Airtel is all on-prem. So Sid and team had central resources - including people - to loan to those domains who wanted to own their data, who were bought in, but needed that help. Another aspect was to find champions and develop hero stories, where someone not that advanced around data was able to successfully build a dashboard or similar to inspire others and lower the perceived bar to doing the data work themselves.
Sid and team knew that if the domains couldn't successfully own their data, the work would fall back on his team. So they had a bit of an extra incentive to make it work 😅 So they made sure to only take on the amount of work they could handle instead of onboarding every domain they could/that was interested right at the start. Just make sure you communicate effectively/transparently. Scott note: It's absolutely normal and reasonable to push domains out in time if you don't have the capacity to work with them, whether that is resources or platform capabilities around their use cases.
At Airtel, Sid and team didn't succeed on their first attempt at data mesh - in fact they "failed miserably". They weren't fully prepared and didn't have a lot of the issues figured out. Luckily their company didn't throw in the towel but the failure helped them understand what really mattered. It was about making data mesh possible, making it a part of the strategy so teams could feel like it was okay to lean in, and building out the resourcing - platform, compute, and people sides - to make this an actual possibility.
Sid didn't see his early data mesh role as trying to convince every domain, win over everyone upfront. It was about getting out the first few use cases and working with a few domains. It was more about finding collaborators that could help drive to a scalable platform. It was about finding the friction points as domains look to own their data and then addressing those friction points. Really focus on what you need to accomplish and don't try to take on the world :)
It's going to be important - and somewhat challenging - to keep all your stakeholders engaged at the start of your journey according to Sid, especially those not participating early. It's important to keep engineering and product leaders engaged but also the people doing the actual work in the domains and the data team. And of course, you need to keep the business stakeholders engaged as they are the ones who will do a lot of the work but also get the biggest benefit. But it's also crucial to understand each group will require different approaches to keep them engaged. Focus on that incremental progress and find leverage points instead of trying to tell them about everything. For Airtel, the product and engineering teams were excited about faster product delivery and quicker delivery of insights.
For Sid, it was important to communicate to leadership that the data mesh journey would take a few years and to get their buy-in that this was a long-term approach, not a quick quarterly fix. But most teams really only care about what is going to be coming, what value will be delivered, in the next quarter or two. Again, focus on delivering updates about what the audience cares about and understand you will have multiple personas to cater to.
At Airtel, budgeting, especially around data, has been done centrally and not in each domain according to Sid. So, for the first year of their data mesh journey, the data team isn't even showing people exact budgets/costs for their domains around their data work but they intend to build a perspective on overall expected costs. This is a big change in thought for the domains from when the costs were all on a central shared data team. The data team didn't want to switch from essentially an afterthought to billing the domains too quickly but they do want the domains to consider costs.
The return on investment conversation has been somewhat difficult to nail down in many use cases for Sid. What value do you attribute to faster time to market? If something now takes 1 day instead of 4 weeks, there is a very clear business value BUT what is the measurement of that business value? Time savings is nice but ends up being pretty low in the grand scheme at Airtel. There are also new use cases that could not have been done before. But overall, it's hard to point to an exact return figure. Scott note: this is where you can flip this around on the lines of business and ask them to tell you how much value it generates for them to do a use case or to get to market quicker/easier and do smaller scaled, fine-tuned experiments.
Sid and team are still figuring out things like the domain onboarding model and many governance aspects in the platform and out. Should they have an embedded expert in things like data modeling go into domains to help them get up to speed? Now that sharing data is far easier, how do you make sure data is well governed across a number of dimensions like quality, security, privacy, etc.? It's not all figured out and that's okay :)
Learn more about Data Mesh Understanding: https://datameshunderstanding.com/about
Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/
If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/
If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here