This story is featured in the WorkLab newsletter. Sign up for it here.
Every company needs to be able to assess where and how mistakes get made. Let’s say a banking customer experiences delays when they’re applying for a new loan. That dissatisfaction means that customer service employees, in turn, spend valuable time dealing with complaints and inquiries. Figuring out why those delays happen—and fixing the issue—can directly affect the bottom line.
For an institution like Australia’s Bank of Queensland, with a few thousand employees serving 1.4 million customers, identifying overlooked risks is critical to reducing costly errors in the future. To do this, it uses a common problem-solving method: root cause analysis. “The process is essential for maintaining high standards of customer satisfaction and operational excellence,” says Bank of Queensland’s Head of Partner Programs for Group Tech, Bernadette Demasi. But it’s also resource-intensive. Our researchers at Microsoft teamed up with the bank to explore how AI could improve speed and efficiency.
Their findings: AI access—along with custom, targeted prompt development—can help diagnose those bottom line–busting issues more quickly and accurately.
What we did: Our researchers gave 14 people access to Copilot, while a control group of 21 did not have it. Participants were asked to analyze a simulated “risk event”—loan approval delays—and tasked with identifying and cataloging the potential reasons for the delay.
Because we know there’s a learning curve with AI, we also wanted to see whether giving targeted guidance could help Copilot users hit the ground running. The treatment group received tips on task-specific optimization, including sample prompts that encouraged asking for narrative responses (“imagine telling the story…”) and “think-aloud” prompts (“imagine you’re thinking it through with a colleague…”).
After both groups were finished, the research team compared the quality of the analyses and how long each group took to write them.
What we found: Analysts using Copilot were able to determine the root cause 51.8% faster, a remarkable result. In fact, more than half of the analysts with AI access completed the task more quickly than the fastest analyst without access to AI. Despite the relatively small sample size, the performance differences between the treatment and control groups were so pronounced and uniform that the results are statistically significant.
Bankers Work Twice as Fast with Copilot
In a Microsoft study at Bank of Queensland, analysts using Copilot were able to finish a difficult analysis 51.8% faster than those without Copilot.
We also saw that analysts with access to Copilot had consistently high-quality outcomes compared to the more variable quality from the non-Copilot users. And using AI significantly improved the effectiveness and clarity of the analyses.
Survey results also suggest that AI users had a much better experience. More than a third of analysts with Copilot found the task less draining than did those without, which suggests that Copilot significantly reduced their cognitive load. Other positive results were even more uniform: 93% of the treatment group agreed that Copilot improved the quality of their root cause analysis and reduced the effort involved in completing it. Every participant with access to Copilot agreed that it helped them answer questions about contributing factors for the risk event, and that going forward they wouldn’t want to do this type of analysis without it.
What it means: Completing root cause analyses improved Bank of Queensland’s ability to identify and manage risks, and using Copilot has reduced the company’s analyst time significantly. They estimate that equipping 1,000 employees with Copilot could enhance productivity so much that it is equivalent to adding 120 new employees. “We have more work to do than people to do it,” Demasi says. “Adding capacity through AI allows us to work through resource constraints and supports our teams to gain the capacity to focus on higher-value work.”
Just as important: The results indicate that it’s not sufficient to simply give your people access to AI with no guidance. To get great results, you need to work together with your teams to give targeted direction on how to adopt the technology, supporting them to push the boundaries on how to use it to their best advantage.
Microsoft is a technology company, a small local company, with few employees, no offices, and almost making no profit… >>
Please visit the firm link to site
