How can your product team leverage declarative data to run better experiments?
The Paradox of Declarative Data
Imagine having access to constantly updated, accurate insights that you see in column B for every incoming customer. Although, in my experience, most product teams have & rely on insights from column A to make serious decisions or answer very complicated product or strategic questions.
Why is that a problem? Relying on data from column A will give you only half of the picture, but with data from column B that’s a different thing, you start seeing the full picture.
Sounds cool, right? Yes! And to unlock this possibility, you need to start collecting declarative data from your customers constantly and start using it in combination with other data types — behavioral and enrichment data.
And here is the paradox, declarative data is the key to a full picture of your customers, but that is also a type of data that is typically ignored the most or not taken seriously.
Why? Because there is a thought (or should I say stereotype) in the air that sourcing data directly from your customers is not reliable, they will lie, be not precise, etc. But your customers are not that stupid and they are staying at the door of your product because they have a problem they want to solve. And guess what, they are happy to give you clues as to how you can help them.
How your product team could use declarative data?
First of all, I strongly believe that the biggest part of any product manager’s daily routine should be Product, Business, and Customer Discovery. Not operations, not execution, not creating countless PRDs, not sitting on countless meetings and aligning on god’s know-what, but DISCOVERY.
How otherwise you will understand where to navigate your product to solve your customer’s problems in the simplest and the best possible way? Surely not on your today's 5th meeting 🤦. You will only get there by focusing on DISCOVERY.
Do you want proof? Here’s a very great example of a very successful company … Atlassian. A PLG pioneer. A simple graph showing the amount of investment in R&D vs other companies.
And, to me, experimentation is one of the core pillars of having a great discovery process in place. Experimentation is a very complicated topic, and it always starts with culture. But, today we will talk about how declarative data could fuel your experimentation and take it to the next level.
Declarative data could lead you to a much better understanding of where & for whom to run experiments
It is the same as with your own body. The more meaningful data you have about your own body the better you know where to push to make it function better.
Same with your digital product.
Imagine you start getting insights like this on a daily basis for all of your customers:
Customers from Segment C who report coming from a blog "How to skyrocket your product" have a 5.6% activation rate. The average activation rate for customers from the same Segment C (but with different attributes) is a 3.2%. Also, you start getting the insight that customers from the “How to skyrocket your product” blog don’t understand how to set up the profile. They also report that the main reason to start using your product is better design & fast-reacting customer support.
Suddenly you start seeing opportunities laying in plain sight. You start seeing what could be fixed quickly, which segment is underperforming, where they need help, or even what they are missing compared to your competitors.
Imagine having this, in much more detail, for all your customer segments. You will get straightforward clues as to where (at which point in the journey) and for whom (what is the most struggling segment, or what is the segment, that has low-hanging fruit opportunity) to run your experiments.
Declarative data could help target your experiments with much higher accuracy
Why? Because different customers have different problems they came to solve with your product, and what work for one group, will not work for another.
Trying to fix all problems with a one-size-fits-all approach will result in fixing zero problems bringing more churn. Same if you will apply the solution to Problem A for customers experiencing Problem B. 👇
To avoid this, you need to deliver the right solution to a specific group of customers with a specific problem.
And guess what? Declarative data-based segments, paired with behavioral and enrichment data and a decent targeting system, like Optimizely will help you to target your customers on a very granular level based on their intents, expectations, and backgrounds.
Declarative data could help you to perform a much better analysis of your experiments
Think of an experiment that you have launched for all of your new customers. 3 weeks after, you got to statistical significance and you are ready to analyze the results to see the impact.
The analysis is completed, and you see that there were no impact (0). You make the conclusion that the experiment does not make a difference for your new customers and you ditch it.
But what if for Segment A (business customers with a particular problem), this experiment showed an uplift across all metrics, and for Segment B (consumers with a different problem) this experiment lead to a certain decrease? But overall it looked like close to 0 effect.
Breaking down experimentation data by declarative data-based segmentation will serve as the magnifying glass in this case, helping you to spot the granular effect on different user groups allowing you to make better-informed decisions having a huge impact on your whole experimentation cycle.
Why? Because a poorly-informed decision to kill an experiment could also kill a huge long-term opportunity, and vice-versa a poorly-informed decision to continue an experiment cycle could put you on the wrong path and lead to resources being spent in the wrong direction = missed success.
The bottom line
Declarative data done right is a very powerful weapon. Though getting to the point described in this article will take time, there are a few things you could start doing now:
- 🤝 Start trusting your customers
- ❓ Start asking them meaningful questions trying to clarify what problems they have, what are their expectations and what are their backgrounds
- 🧠 Start using this data in planning and decision making
This will already do a huge impact. Good luck 🚀.
