Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/
Please Rate and Review us on your podcast app of choice!
If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here
Episode list and links to all available episode transcripts here.
Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.
Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here and their great data mesh resource center here. You can download their Data Mesh for Dummies e-book (info gated) here.
Ryan's LinkedIn: https://www.linkedin.com/in/ryandolley/
In this episode, Scott interviewed Ryan Dolley, an Independent Business Intelligence (BI) Consultant.
Before we jump in, lots of things in the key takeaways are marked as potentially controversial. Because much of what Ryan covered hasn't really been stated by a lot of people. So until we have more consensus, of course things _could_ be controversial.
Some key takeaways/thoughts from Ryan's point of view:
Ryan started with a bit of inspiration and/or a call to action to fellow business intelligence people: data mesh and a lot of other movements in data are creating a massive wave of innovation further up the data stack / data and analytics process. So BI people can either get with the wave and ride the changes to greater value or get washed over by it. Ride that wave :) Upskilling can be painful but it can mean a major upgrade in capabilities and value. And being highly tech-capable in BI is a great career move right now.
According to Ryan, the last mile of analytics where BI really fits has been somewhat ignored in most modern data approaches and technical innovations. Tableau defined an era of BI and while Tableau isn't going away, that era is coming to a close. So he believes things like data mesh need to focus more on the last mile of analytics - or at least specifically call things out more often as to what changes and how in BI. Whether we call that self-serve or anything else. What does the upstream changes from data mesh mean for BI? How can we take advantage of these improvements in data processing, ownership, quality, etc. to deliver actual insights to those that matter?
Ryan asked how should we actually do corporate BI in a data mesh type setup? Right now pre data mesh, most organizations have centralized BI teams. And while domains will probably need some BI capabilities, a recurrent theme in the podcast is that most exec questions are not single domain questions. So we build this huge amount of interoperable data, who is in charge of bringing that together at the corporate level? Who actually owns creating and maintaining C-suite and board-level dashboards and reports? Are we expecting execs to build those themselves? 🤣 And historically, central BI / Data warehouse teams haven't really loved embedded BI teams, often calling them "shadow IT". Can they play nicely together? If yes, how?
There is often too much of a 'we'll cross that bridge when we come to it' relative to BI as part of most modern data strategies according to Ryan. While BI may not actually be part of the exact data mesh implementation at some organizations - there's all kinds of definitions and each org is extending the self-serve platform to encompass different aspects of analytics - BI definitely should be part of your data strategy. Otherwise, what is the point of doing data work if not to, you know, generate actionable insights? This is especially an issue when trying to embed insights and analytics in general into other applications.
An important aspect of a data mesh implementation for Ryan is what is left to the BI tool and what is pushed down into the data product*. There isn't a lot of guidance or discussion happening exactly on what is a best practice or how people are implementing embedded insights, just that it's the future. There needs to be far more conversation about who owns data throughout the data and analytics process lifecycle. Is it all on domains?
*Scott note: episode #40 w/ Xavier Gumara Rigol covers this well
For Ryan, there are a few reasons why the general data warehouse has not been as successful as it could be. To actually interact with the data warehouse, people need a decent level of technical and BI skill. So when it comes to data mesh, we don't want there to be an even higher skill level required. So domains likely need to own some of their own BI, sharing insights from the domain in a consumable way to the rest of the organization. We can't only have domains sharing data, we need to be explicit about sharing information to a purpose. Is that to get the data, the insight, or the action that should be taken based on the insight? Scott note: there will be people in your organization that are only information consumers - taking info from dashboards and reports made for them. It's up to the organization to figure out if that is acceptable or to try to upskill those people against their will.
While product oriented thinking is picking up in more areas of data, it's still not very common in the BI world according to Ryan. Most BI teams have no real idea of how useful any of what they produce is to the organization. Sometimes that is they don't have the systems set up to even monitor it but if they do, they typically don't understand how usage translates to value or honestly even look at the usage metrics. That means the BI teams lack the insights to make their BI assets as valuable as possible - what are the SLAs people need to make the most use of this data? How timely or accurate or complete does this need to be to drive the most value and be trustable? If BI people develop those relationships and that information flow, they can evolve what they produce to continue to meet needs. We need to stop producing work into the void; who is using this BI asset and why and how can we make it better and/or less costly to produce?
A really important aspect of designing usable BI is thinking about the actual user in Ryan's view. It's often overlooked too and is part of product thinking: what is the product market fit for the BI asset / data product you are developing? How do people want to interact with it? How far along the pathway from raw data to insight to "so what" - the action that should be taken - is it the producer's responsibility to own and do the actual work? Does the consumer understand and agree? How can we generate things in data to create understanding and trust with the minimal amount of effort? Dashboards are great if you're monitoring a known question but we also need to get far better at answering how do we persuade someone with data.
Ryan believes BI people need to move past the idea that the output is the end of the process. That dashboard or report or whatever is a communication mechanism about what is currently known. But it's not as if there is a single answer that stays the same. How many customers do we have is not a one-time answer question. There needs to be a lot of work to bring more iteration into BI, especially collaborating with consumers as both sides learn more about questions they are trying to answer. So BI teams need to move past being order takers and move to being part of the productization process around data and insights. Iterating in BI is not common practice but needs to be!
BI is probably the most important aspect of your data culture according to Ryan. That's simply because it's the place where the vast majority of your organization will interact with your data, at the BI level. Whether that is in generating insights or merely consuming them, BI should not be left until the end of your data strategy. But it often is left to the end specifically because it is where there is the most human interaction and change management and that's the hardest part of change management, dealing with those pesky humans. Don't try to put it off until the end as it will just create more pain than is necessary.
Ryan said "begin with the end in mind." It's quite easy in data work to get lost in the how instead of the why. The how is the fun/interesting technical challenges part. What are you trying to achieve? BI at the end of the day is not about generating insights, it's about supporting actions that help the business. So how do you build your BI strategy into your data mesh implementation strategy so you are not just making data available and trustable? How can you move past just generating insights? How do you think about the end - making smarter business decisions driven by trustable data - and work backwards? Don't lose sight of what you are trying to achieve.
Ryan wrapped up the conversation by returning to the crucial points of integrating in product thinking to BI and upskilling. We can go further than we ever have before with data and BI but we have to embrace new ways of working, both on the strategy/data culture, and the technology. But just trying to advance your BI practice with technology is going to miss a whole lot of the value.
Quick tidbits:
We have to figure out in the domain if BI is a primary role or is it part of the necessary skillsets involved but not a separate role. It will probably be domain dependent, not even organization dependent, based on the complexity, size, and data value of the domain.
Right now, most organizations want to embed insights into their general flow applications - e.g. the CRM for the sales team - but they aren't doing the back-end work to make that possible.
Putting a business analyst in the boardroom simply as a vehicle to live answer questions usually causes more issues than it solves. It leads to a high pressure environment where you can't be sure of quality. Scott note: fully agree. There's a long rant I have about this…
Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/
If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/
If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here
All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf