Let’s be clear from the beginning. For a product manager, there is no real argument against data-driven decision making. Thinking otherwise would be a shameless attempt at a #hottake.
I’m not saying this because I work for a company obsessed with data (the mat outside the elevator literally says, “We love data.”). Data should be an essential part of a product manager’s working day.
As product managers, one of our main tasks is making decisions, but we’re not omniscient figures. Basing your decisions on KPIs (rooted in data) seems the perfect solution. This way, decisions are transparent and comprehensible for the stakeholders and developers. There is no need for endless discussions based on opinions.
Data is one of the most effective tools a product manager has, and if we are not using it, we are doing it wrong.
We won't become "data-driven" just by looking at numbers, so let’s go over the best practices to implement into your working day.
1. Understand the purpose of KPIs
As a product manager you have many data sources at your fingertips, but not all of them are helpful. Worst case scenario? Different sets of data pointing in opposing directions. You must understand where the data comes from, and which aspects of your problem are represented.
Customer feedback, either through individual customer interviews or consolidated customer survey results, is a valuable guide. Especially for B2B products, due to the slim quantity of data created through interactions with your tool. However, if one customer represents huge revenue potential, analyzing their dedicated feedback can be worth it.
As the famous Henry Ford may or may not have said: “If I had asked people what they wanted, they would have said faster horses.” While feedback alone should not predict your next steps, it can be the right place to start.
Then there is "product analytics”, which should tell you: how your product is used, the user path, and the most and least often used functionalities. It is usually based on tracking, implemented in the user interface. However, don't underestimate data generated from everything happening in the background, and don’t only track page views or clicks.
Also, strongly encourage your IT to log system performances and errors – it can help you to better identify bugs and performance improvements.
2. Choose The Right Metrics
This seems obvious, but it can be tricky. At the start of every user story, there tends to be a problem; perhaps an unaddressed use case, or a user experience in need of improvement.
As soon as the problem is identified and properly solved, define the KPIs needing changed. Set up a hypothesis and don't be afraid to bring stakeholders or developers in on it.
Choosing KPIs early makes sure the number is properly tracked so you can establish a baseline measuring the effect of the change. If it is not yet tracked properly, the tracking can be part of the first iteration of the implementation. Making the KPI part of early concept drafts means you can use it early to challenge the feature purpose.
Don't make the mistake of looking at unactionable metrics. The rise or fall of a KPI needs to tell you something. Sometimes it can be clear – even before the first numbers come in – in what direction your actions should take depending on the results.
3. Being Data-Driven vs Being Data-Informed
It hurts a little to write but, data is not everything.
The above examples show you the past, leading to the status quo. The best case? Discovering patterns and making predictions based on them.
Data shows you where a funnel leak might be located, or what impact a change could have. If you change something, success or failure is visible in your reports. We can use data to challenge plans and ideas.
However, data won’t show us how to create an outstanding experience from scratch. Data driven decisions help you achieve a local maxima of performance, but only innovation will lead to higher improvements.
Team creativity and experience is the best way to identify solutions on a bigger scale. It takes a lot of work (just as much as relying on data).
Analyzing data helps you focus energy on the important things. Potential solution must withstand the reality of data. This much will be as true in five years as it is today.
That said, feedback from your stakeholders and customers can show you why something is not working, and it provides context for data and your decisions. What customer expectations and motivations are incompatible with which product features? That’s something that data – love it as we do – can’t tell us.
too long; didn't read
a) Gather data from multiple sources, but acknowledge the data source and its potential biases.
b) Bringing data into the decision making process means choosing actionable KPIs with clear messages.
c) Communicate data with your team to make the status quo transparent for everybody involved.
d) Use qualitative feedback combined with data to get the real story.
e) Being data informed is the ultimate goal, and being data driven will lead you there.
This blog post looks at some actual GDPR text to see exactly what is (and isn't) about to change. Read it now.