Each month, MetricNet highlights a Key Performance Indicator (KPI) for service and support. We define the KPI, provide recent benchmarking data for the metric, and discuss key correlations and cause-and-effect relationships for the metric. The purpose of the column is to familiarize you with the Key Performance Indicators that really matter to your organization, and to provide you with actionable insight on how to leverage these KPIs to improve your performance This month, we deviate from our standard format. Instead of focusing on a single metric, we will discuss the measurements related to artificial intelligence (AI), as well as how you can gauge your AI technology’s effectiveness. What Do We Mean by Artificial Intelligence? Before delving into AI metrics, it’s crucial first to establish what we understand as AI. Much of what is touted as AI today is simply a dressed-up form of human intelligence masquerading as artificial intelligence. Some of the industry’s well-known AI tools do nothing more than scan knowledge articles and regurgitate what they’ve read. That isn’t AI, and it’s certainly not an example of machine learning. I’ve had the opportunity to see over a hundred AI tool demonstrations in the past few years. Only recently have these demonstrations convinced me that we’re approaching a critical turning point with AI. We are now witnessing genuine demonstrations of AI tools that surpass the ability of human experts and continually learn and improve over time. The ultimate test of any AI tool for service and support is this: Does the tool, without any human intervention, lessen ticket volumes, resolve issues more rapidly, decrease total cost of ownership (TCO), and enhance the customer experience? If it ticks all these boxes – and improves over time – then it’s genuine AI, driven by machine learning. Assessing AI Our definition provides considerable insight into AI metrics as we can measure ticket volumes, resolution times, TCO, and the customer experience. An efficient AI tool will positively influence all these measurements – without requiring human intervention. TICKET VOLUME An effective AI tool will specifically decrease tickets per user per month. Back in 2018, I wrote an Article of the Month on Tickets per User per Month. Presently, the average for tickets per user per month stands at roughly 1.1 tickets at level 1 and 0.5 tickets for desktop support. The most efficient AI tools will lessen these figures, often quite significantly. An effective AI tool is not uncommon to reduce tickets per user per month by 50% or more. RESOLUTION TIMES Mean Time to Resolve (MTTR) is a standard metric for resolution time. You can view my article on MTTR here. Primarily, MTTR is a level 2+ metric. It is usually tracked for desktop support, level 3 IT, field support, and vendor support, but not at level 1. At level 1 (the service desk), the First Contact Resolution Rate (FCR) is the common metric used to measure resolution time. Regardless of whether it’s for level 1 support or level 2+ support, effective AI will enhance both metrics: MTTR will decrease, and FCR will increase. Depending on your starting point, the reduction in MTTR and the increase in FCR can be quite dramatic. It’s not uncommon to see reductions of 50% or more in MTTR and increases of 20+ percentage points in FCR. TOTAL COST OF OWNERSHIP AI curtails TCO in two ways: by lessening ticket volumes and by initiating a shift to the left. We’re all aware that the ideal ticket is the one that never occurs. Effective AI makes this a reality by proactively identifying and fixing issues before they become tickets. With AI data mining algorithms, it’s now possible to operate at Level -2 on the shift left spectrum (demonstrated in Figure 1 below). This “search and destroy” capability, which was not feasible before AI, reduces TCO by decreasing ticket volumes and saves enterprise customers time and money by averting many tickets in the first place. CUSTOMER EXPERIENCE Customers are extremely sensitive to FCR and MTTR. Improving these metrics with AI – reducing MTTR and increasing FCR – leads to a superior quality customer experience. You can read more about the customer experience here. Empirical benchmarking data has unequivocally established the cause-and-effect relationship between MTTR, FCR, and the customer experience. When MTTR decreases and FCR increases, the customer experience improves. A comprehensive list of metrics impacted by AI is shown in Figure 2 below. Industry Case Study Since I mentioned Shift Left above, I will share an industry case study from a large insurance company that leveraged AI to shift left and reduce their TCO. Their Shift Left metrics before and after the deployment of AI are summarized in Figure 3 below. As you can see, their AI-fueled performance considerably improved their shift left metrics. Although this company’s cost per ticket increased at each level, their total cost of ownership decreased dramatically due to the considerable reduction in ticket volume – from 1.91 to 1.02 tickets per user per month at level 1, and from 0.52 to 0.31 tickets per user per month at desktop support. Figure 4 below summarizes the insurance company’s TCO before and after the Shift Left initiative. This support organization reduced their TCO from $116 million annually to $79 million annually, a staggering savings of $37 million in just one year! Key Takeaways We are at the dawn of AI, and the technology will continue to evolve for many years to come. AI has, and will continue to be, a disruptor in the industry. Initially, it will eliminate the need for agent-based commodity support – consider Microsoft Office, Windows, password resets, and other easily resolved issues. Then, as machine learning makes each deployment of AI progressively smarter, even the most complex support provided by today’s customer-facing agents will be replaced by more intelligent bots. Yet the majority of agents see AI and automation as a positive development because it will enhance careers for the better. Much like today’s auto industry assembly line workers are engineers monitoring computer screens while robots actually build the cars, the support technician of the future will become a support engineer who monitors, coordinates, and directs the efforts of the AI bots. For the most skilled and talented in the industry, the future of IT service and support has never been brighter! Please join us for next month’s Article of the Month: The Metrics of Problem Management, a strategic set of metrics that focuses on ticket prevention rather than ticket resolution.
