Jurassic Park Moment: What shouldn't be AI'ed

Whenever a new technological revolution comes about, in these days 'the AI revolution', entrepreneurs and developers worldwide try to apply it to every conceivable use-case to see what will resonate with the market. And from time to time, we get our own Jurassic Park moment.

In the 1993 movie, the character played by Jeff Goldblum utters one of my favourite movie quotes and product management cautionary lessons:

This cautionary tale isn't just applicable to the ethics of cloning dinosaurs (in case you haven't seen the movie - bad idea!). I would argue that it applies chiefly to the realm of product and technology development. While we have learned in the past year that generative AI presents trully transformative capabilities, there are certain tasks it should never automate—chief among them is the automatic conversion of customer feedback and feature requests into user stories. But that is what we are increasingly being served.

What are 'stories'?

I am a big believer in using user stories and job stories for software development. And that topic alone deserves a proper deep-dive and explanation which I promise to cover in future articles. For the time being however, let's just say that 'stories' are a specific way of formatting software requirements that is meant to illustrate the context and the customer value we are trying to accomplish.

• The user stories, most known, should be captured in the 'As a [persona / user type], I want to [perform certain activity], so that I can [achieve desired outcome]'. 
• The job stories, less known but arguably more powerful, should be captured in the 'When [context when activity happens], I want to [perform said activity], so that I can [achieve desired outcome]' .

The stories are a powerful analysis tool and I often argue that the first audience of the story are not the developers, but the product managers and business analysts. The reason is that the stories, if they are to represent the desired user experience, must come from rigorous and in-depth analysis and understanding of the customer need. When they are, they become a powerful mechanism for innovation. I have experienced this many times that when stories articulately and logically communicate the desired outcome, it becomes easy for the development team to creatively brainstorm the best means of achieving it.

However, many people don't understand the real purpose of stories in software development. They prioritize the form of a story and ignore its purpose, which results in the pointless 'story theatre'. The stories created for the sake of creating stories end up being circular, surface-level time wasters. Like the one below (you can enjoy more examples on the official 'Shit user story' X channel):

Ai-ing ourselves into feature factories

Which brings me to my grief. There are more and more tools out there that offer to bring the power of AI to turn user feedback into user stories or generate stories automatically from a provided text. I want to call out one tool in particular, Kraftful, and call out many users who use Taskade. I don't have a specific problem with the latter tool, I find it very useful. But I've seen a ton of posts on linkedin and twitter of people boasting how they copy-paste customer feedback into it to generate stories. I have a much bigger issue with Kraftful that embedded feedback -> user story automation into their product. 

AI algorithms that churn out user stories based on customer feedback focus on structure: 'As a [User Role], I want to [Action], so that [Benefit]'. While this format is commonly accepted, as I said above, it's merely a framework. The real value lies in understanding the 'why' and 'how' of the user's need, not just the 'what'.

User feedback, 99.999% of the time, is just a surface-level signal. In nearly a decade of product management experience I have rarely seen feedback from the customers or stakeholders being in-depth and articulating both the root problem, urgency and the context in which the solution they are asking for would be used. That is why a good product manager or business analyst would not act directly on a piece of feedback, not even when it has been voted on or submitted 100 times by other users. Not without the additional insight of why, how and when.

The mindless automation of customer feedback / feature requests into stories, and by extension, shipped features, reminds me of the old Simpsons episode when Homer was tasked with designing a new car. The Homer Simpson car was a colossal flop that bankrupt the company and the image of said car has become ubiquitous in product management circles with 'feature creep'. That is what I imagine our products looking like if product managers and business analysts abandon their responsibilities to the AI.

AI can generate stories, if given proper input

That being said, I do think that AI can and should be used to generate stories, design documents and other development artifacts. After all, at Elements.cloud we have developed our own automatic story generator. There is a difference though. 

With the AI, and any automation actually, the general rule is 'garbage in, garbage out'. The AI is not the issue with story generation, it is the input we are feeding it to do it. If you feed the AI engine the full context of what the customer is trying to accomplish, why, when the need for it arises, and who else might be involved, that comes from rigorous user research, then it would be a complete waste of time NOT to use AI to speed up the documentation and capture of our findings. If you however look at AI as a way of simply accelerating mundane tasks and you do not even recognize the need to sanitize and improve the data you're feeding it, then AI will be your doom.

Arguably, I might owe Kraftful an apology, since they could argue that it is up to the users to feed their tool well captured, structured data. And if users end up analyzing and turning surface-level feature requests to story generator, that's on them. And while not without its merit, as product developers we need to be extremely wary and take responsibility for what our users might do with our products if they are misused. We cannot abdicate all responsibility. Let's use AI to augment our abilities, not to diminish the value of the human touch in critical processes like business analysis. 

What's next?

I am an advocate for process-led business analysis and after nearly a decade of product management experience I have come up with a powerful, syncretic technique called Total Story Visualization. TSV combines jobs to be done framework, Universal Process Notation, stories, user experience design and system automation design to help product managers see the complete picture of the solutions they are building.

You can learn the entire framework with practical examples to hone your skill and understanding in my masterclass in Total Story Visualization: