Summary
Challenge: Data Quality Team from an international insurance company faced a high volume of tickets and lacked capacity to consistently perform RCA. Separate tracking in Airtable led to manual effort, errors, and time-consuming validation, resulting in an inconsistent RCA process.
Solution: A custom Rovo Agent was implemented to automate RCA by analysing Jira ticket data alongside synced Airtable records. With the help of AI, it suggests root cause themes and specific causes, incorporates user feedback, and includes a human approval step to ensure accuracy.
Outcome: The solution achieved 70% accuracy in identifying root cause themes, reducing manual work and improving consistency. It enhanced data quality as well as freeing up the team to focus on more complex analysis.
Background
In our work with Convex, an international speciality insurance firm, we are continually exploring how we can automate internal processes for its business teams. Convex has embraced Rovo across its Atlassian toolset – particularly in Jira Service Management (JSM).
We have created a range of custom Rovo Agents to unlock greater time efficiencies, tighter quality control, and happier users for our client.
In this case study, we’re focusing on a custom Atlassian Rovo Agent to enhance Root Cause Analysis (RCA)
Challenge
Occasionally, an underwriter or other business user of Convex’s various IT systems will discover an error or inconsistency in data stored in one or more of those systems. They may then raise a request to have the data corrected. The requests are logged in Slack and, via Atlassian Assist, automatically raised as tickets, i.e. work items, in Jira Service Management (JSM). The tickets are picked up and resolved by the company’s Data Quality Team. The Data Quality Team will not only correct the data error but also try to discover the root cause of the error and fix that, to prevent similar data quality errors from happening again. The Convex Data Quality team were receiving such a high volume of tickets that they did not have the capacity to analyse the root cause of each of them, nor to establish a consistent process for doing so. As a result, team members could only focus on one or two subsets of tickets, and root cause analysis on the remaining large number thus had to be omitted.
All root causes were documented in a large Airtable, complementing the data correction requests tracked in Jira. This led to some problems, including human error, because of the very large datasets and the need to comb through so much information. Senior team members therefore often had to check root causes for accuracy – which took time and often resulted in additional fixing and reworking.
Solution
AC automated the entire RCA process. As part of this work, we created a custom Rovo Agent to gather context, analyse data, and suggest the likely cause of a data issue – all before human team members even began investigating.
We also connected the datasets in Airtable to Jira for greater cohesion and accuracy using the Atlassian Marketplace app External Data for Jira Fields. Now, whenever a new entry is added to Airtable, an automated trigger fetches the relevant Airtable data/column and syncs it with Jira. This ensures that the Rovo Agent has all the information it needs to conduct RCA.
How does the custom Rovo Agent work?
The Rovo Agent collects ticket data from Jira and pulls relevant data from Airtable (we used automation rules and API integrations to pull in the data we needed). The Agent then suggests the most likely root cause ‘theme’ (this could be anything from human error to configuration issue). With over 4,000 historical root cause entries in Airtable organised by themes, identifying a single theme to focus on is an essential first step.
The Rovo Agent then runs through all root causes associated with that suggested theme. Each theme has anywhere from 1-10 associated root causes, and the Agent uses AI-driven analysis to match patterns in the data and suggest the most relevant root cause. We designed the Agent to provide a rationale as to how it made this root cause recommendation, so human users could understand how it came to a decision. Crucially, users could add comments if it was the wrong call, on which the Agent could draw from for an improved result next time.
Finally, we added in an approval step. Convex requested this, to ensure that a human Root Cause Analyst could review the Agent’s suggestion. We built this using Jira Service Management forms.
Benefits
Convex has been pleased with the accuracy of the custom Rovo Agent so far, and the automated process as a whole.
Operational Excellence Consultant at Convex remarked, “I expected far fewer accurate responses… Rovo has exceeded my expectations.”
During a dedicated testing period, the Rovo RCA Agent could correctly identify the root cause theme 70% of the time. Over time, Convex expects the Rovo RCA Agent to guide the team towards improved data quality and accuracy – alongside freeing up human time to focus on the most complex RCA cases.
