Risk-driven analysis (1/3): how I balance quality and agility

This is part 1 of the 3-part series on the risk-driven requirement analysis I cultivate at Elements.cloud. This article introduces the concept and explains what I mean by ‘risk’.

What is risk-driven analysis?

As product managers or business analysts, we have to deal with feature requests raised by customers, internal stakeholders (sales team, customer success team, marketing team), engineers and finally, our ideas about which problems to solve for our target market.

In an ideal world, we would take each idea through a rigorous validation process. For product managers, that would mean extensive market research (both qualitative and quantitative) that would lead to designing an ideal Minimal Viable Product to test our assumptions. For business analysts, that would mean running a lot of in-depth discovery sessions with the relevant stakeholders.The trouble is that our backlogs  (the list of all requests and ideas) can often be extremely long and our deadlines often extremely short. For example, at the time of writing this article, our product management backlog has nearly 400 ideas and requests, and it keeps growing with a steady pace of around 5 per week. And our small team of product managers simply does not have the time or the capacity to run it through the ideal validation process (not without decreasing our agility and potentially angering both business stakeholders and customers alike).

Hence why I designed a risk-driven approach to our idea / requirement analysis to quickly identify which need a thorough investigation, and which can be sped up through the process. I can best summarise that approach with the following visualisation:

In risk-driven idea analysis, you qualify the risk of an idea based on how well evidenced and understood is the customer problem, job to be done and user journey, and then use that risk to determine how much further validation is needed before you feel confident in building a solution.

How to understand risk?

First of all, let’s imagine a widespread cost vs value matrix like the one below (I prefer to put value on the X-axis):

For an idea, we try to assess how valuable it may be (for the customer in a way that can generate value to the business) and how costly will it be to build (time, effort, financial cost, opportunity cost). This allows us to understand the value vs the cost placement of an idea.

If you do it for many ideas, you can end up with a mental or physical picture like the one below. It seems simple, right? We can quickly identify ideas worth implementing and those not worth the effort.

However, the picture above is misleading. What is missing is the visualisation of risk. By risk, I mean uncertainty about the value and cost hypotheses. Firstly, any solution is almost always more complex to implement than we think (for one, we don’t always know all the dependencies or anticipate technical debt we may discover).

Secondly, and more importantly, in product management we tend to fall in love with our ideas or be misled that just because we would want a solution like that, everyone else will as well. As a result, the actual value of our idea to the target customers and the business may turn out much more minor. If we say that ‘feature X will bring lots of value’, what evidence do we have to back this up? More often than not, it is our gut feeling, imagination or wishful thinking. 

In business analysis it might be less so, since a lot of ideas come in the form of business requirements or projects from the business stakeholders. Nonetheless, ideas almost always come in the form of elaborate feature requests, often championed by one stakeholder. But stakeholders more often than not lack the real understanding of what the technology allows us to do or how difficult it is to accomplish. And there is also a question of buy-in from other people in the business.

Hence why the accurate picture for any new idea, accounted for the levels of risk, is more like this:

You can easily see that even when an idea is classified as a ‘big bet’, if we are mistaken about its actual cost (usually higher) or value (usually lower), then the idea may end up being classified as absolutely anything else.

And if we compare it to a different idea, one that may at first seem like a loser, with varying bars of error around it, we can see that both of them have the potential to be both a fantastic success or an utter failure.

Because any idea, with its pretty extensive error bars, cannot be appropriately classified, and more importantly, cannot be that effectively compared with other ideas, the entire exercise of assessing value vs cost is completely pointless…

…unless you work on decreasing those error bars.

The more desirability, feasibility and viability testing you do, the smaller and smaller the uncertainty becomes, and therefore more and more accurate your estimation becomes.

Comparing risk as well as value and cost

With risk-driven product management, neither an apparent ‘loser’ is dismissed out of hand, nor an apparent ‘quick win’ is automatically scheduled for development unless the error bars have been minimised.
The size of those error bars dictates how much time and effort we have to put in de-risking those ideas.

In future articles, I will dive deeper into what distinguishes different levels of risk (at least to us) and what methods or frameworks of analysis we utilise to de-risk ideas. 

What's next?

You can continue reading on risk-driven analysis in part 2.

A methodology I use to de-risk ideas is called Total Story Visualization. It is a powerful and syncretic technique that combines several other methods into single, logical approach. First, you need to become familiar with UPN diagramming notation. Within TSV there is also a very specific approach to understanding and capturing jobs to be done.

You can learn the entire framework with practical examples to hone your skill and understanding in my masterclass in Total Story Visualization: