Apportioning Knowledge Article deflection success by category

[Chris Armstron1]

Hi All,

We're using the Knowledge article deflection metric.

Formula: customer has viewed a knowledge article and not raised an incident within the next 24hrs = 1 deflection

We can see that there appears to be quite a lot of successful self-service taking place. It's recently sharply increased because we've added AI search to our Virtual Agent so that it provides knowledge articles in the conversation, leading to more customers seeing the articles.

However, we would expect to see a reduction in calls to the IT Helpdesk as a byproduct of increased self-service success - But we cannot see an overall reducing in call volumes. Our management is keen to see this as a tangible benefit of self-service - reducing calls. We would like to be able to view knowledge article deflection on a more granular level to see if there are smaller correlations between increases in deflection for individual topics, such as iPhone, printing etc. 

Is anyone aware of a way to find this information without additional customisation or configuration of the product?

Kindest regards,

Chris

[David Kay]

Michael - If I were rewriting that paper today, I would dial back them emphasis on phone surveys.  Among other things, the phone-answering landscape is pretty different now! 

I know Oracle had good luck with opt-in exit surveys--opt-in meaning they asked if you'd be willing to take a survey at the beginning of the web session, and then popped it up at the end.  Reasonable response rates are the goal so you can minimize non-response bias.

[Dr Evil]

I think one thing that is not being clarified is where or not these are internal or external customers? This is important context for determining how to help. For example for my company are self-service knowledge is only for internal usage and we developing the reporting now. So for us a defection would be tied to the location and the individual using the self-service KBA. If the user views a KBA/Virtual Chat session and if that person doesn't create an incident with the same/similar keywords within 3 days we count that as a successful deflection. We are doing this through performance analytics currently. 

[David Kay]

Once again, not in general a deflection.   And internal vs. external doesn't matter much in this case.

[Dr Evil]

How is it not a deflection? If you have a customer that utilizes a self-service article and doesn’t generate a phone call to the helpdesk, why would we not call that a deflection? If the goal is to use self-service to prevent generating an incident, and we are using articles to provide information to deflect calls coming into the helpdesk, then it should be considered a deflection.

I understand your logic with contact rate, and you mentioned that most people who attempt self-service were not going to open a case. I believe clarifying internal vs. external is relevant. Internally, you most likely understand your call/incident volumes and reasons for creating a case/ticket. However, if we are talking about an external customer, your explanation of contact rate makes sense.

Internally, teams, especially a helpdesk, may want to show monetary savings tied to using self-service knowledge over a phone call to the helpdesk. Call volumes fluctuate, and many complex factors influence call volumes on an unpredictable sliding scale. For example, the CrowdStrike fiasco is one of the most notable recent examples, but there are similar impacts every day that aren’t as significant but still influence call volumes. Everyone’s environment, reporting capabilities, and impactful developments affect call volume, sometimes without a predictable pattern. Deflections are a simple concept that can be used to show value, but they may not necessarily impact the calls coming into the helpdesk due to sometimes complex variables.

[Mike Van Vooren]

I think your scenario is the exact reason why this is a complicated calculation. I was discussing this topic internally and sharing the details in David's white paper.  His chart shows where users can tried to self-solve their own issue but were unsuccessful.  That bucket of users also did not create a ticket. 

That is the exact situation that colleague had experienced himself.  He went to our knowledge base looking for information about a specific topic.  He clicked the only relevant article that appeared.  After reading it, he found it did not contain the information he was looking for.  Then he left.  He never planned to open a ticket.  He tried self-service and it didn't work and it did not lead to an esclation.  That can not be considered a deflection in that scenario.