While companies large and small have made considerable gains in building scalable and sustainable architecture, we’re left with the uncomfortable questions:

is what we’re doing genuinely providing value?

Do we really know who our users are and understand their needs? If so, can they generate insights in a fast and reliable way?

As long as users don’t complain and pipelines don’t fail, does that mean all is well? For all our investment in data, are we seeing the return?

These questions that touch on the concept of value are challenging for an internal platform that doesn’t sell to or make money from our users. Moreover, the added complexity of quantifying the value that data brings forces us to consider alternative definitions of value:

• Top-down: How does the data platform align with and contribute to company goals?
• Bottom-up: Would users choose to use the data platform, and is it easy to use?

When the data platform is viewed purely from a technical lens, we put ourselves at risk of forgetting our users and their needs, building solutions before we’ve understood the problem and further increasing the data/value gap.

Accessing and working with data is a user experience.

Principles

All of our data efforts are a user experience. But users are not one and the same; they are diverse and evolving, fast. For this reason, we should start by considering our different user personas (“who are our users?”).

We also need to consider the motives and reasons behind needing a data platform in the first place. We should at least give consideration to user intentions (“what do our users want to do with data?”), broadly similar to use cases but at a much higher level.

For the multifaceted data platform we need to make life easy for both our users and ourselves by breaking it down into easily identifiable product components (“how do we present our data platform to our users?”). These components are accessed by users through interfaces, typically representing third-party or internal tools.

Together with other data leads and interviews with key business stakeholders, we investigated each of these areas one by one. To ensure it didn’t become a one-off exercise we made them as visible as possible internally, e.g as part of onboarding a new Data or BI Engineer. Doing this also incentivizes us to keep them relevant and up to date.

User Personas

User Intents

Why would anyone make use of a data platform in the first place? What are the key actions our combined user personas should expect to be able to perform autonomously? These were the questions we asked during this step.

Product Components

Joining the dots

By taking a step back and linking the use cases, user personas, components this enables us to:

• Look at our data platform as a manageable set of product components we can measure more easily and build OKRs around.
• Identify gaps in our offering, where important user intents are not linked to a live product component. This is particularly useful as a starting point for building a longer-term strategy.
• Decide how we best organise our teams around common KPIs.
• How we structure our roadmap and communications to the business on what we’re delivering, to who and why.

Main reading sources:

• Continuous Discovery Habits, Teresa Torres
• Inspired: How to Create Tech Products Customers Love, Marty Cagan
• How to Move Beyond a Monolithic Data Lake to a Distributed Data Mesh, Zhamak Dehghani
• How to Build your Data Platform like a Product, Barr Moses

https://medium.com/stuart-engineering/how-were-building-our-data-platform-as-a-product-f89142b6547f