First part of a new series called "Exploring Analytically." Got ideas for other topics? Let us know!
One frequent challenge I hear from stakeholders is that their data is too noisy.
For example, Mean Time to Resolve or Mean Time To Repair can sometimes jump around based on a small handful of poorly resolved/repaired issues.
The key problem is that averages are very responsive to outliers. That is, a small number of extreme results can really mess up your averages.
Some teams try to account for that by filtering out outliers. That's not a great practice, because it can be a slippery slope of manipulating data to try and reach something that "looks reasonable."
Instead, I recommend you transition these reports from average to median reports. Here's an example of a Resolution report showing (for the same data) averages and medians:
You'll notice that the average has some incredible peaks (varying by 8 times) that makes it hop above and below our "goal" (the dotted horizontal line). Meanwhile, the median (or Top Percentile "TP" 50) is still impacted, but stays closer together.
The median is saying that the "typical" incident is still resolved between 6 to 12 days. Half of incidents are resolved quicker; half are resolved slower.
The advantage of percentile-based reporting like medians is that you can show the "typical" experience while also representing the outliers. Here's a TP 50/90/99 version of the same report:
The TP 50 time shows the median (again, half of incidents were resolved quicker, half slower). The TP 90 shows the 90th percentile: 90% of incidents were resolved quicker than this line. The TP 99 shows the 99th percentile: 99% of incidents were resolved quicker than this line.
Looking at these together gives you more information. The median tells you the typical experience callers are having with support. The TP90/99 are telling you how far your worst incidents are from that typical experience.
When those lines are close together (as in March to April above), not only are you giving a good quality of service, but you're giving a consistent quality of service. When those lines diverge, you know your support organization is being impacted by some whopping outliers.
Want to learn how to create these kinds of reports? We have a video on that topic here.
