How to Measure Case Deflection When AI Search Suggests KB Articles Before Case Creation?

Mohit42 · 8 hours ago

Hi Experts,

We are looking to implement and measure Case Deflection within our ServiceNow CSM portal and would appreciate guidance from anyone who has implemented a similar solution.

Our use case is as follows:

A user navigates to the Get Help page.
The user enters a Short Description of their issue.
AI Search automatically recommends relevant Knowledge Articles based on the entered text.
The user opens and reviews one of the suggested Knowledge Articles.
If the user finds the answer and does not create a case, we would like to count this as a successful Case Deflection.

We are trying to understand:

Is this scenario supported out of the box through Self-Service Analytics and Deflection Configuration? [github.com]
What is the recommended way to track the user journey from:
- Get Help page access
- Short description entered
- KB article viewed
- No case created within a defined time window
How are others measuring the "no case created" condition (e.g., 7 days, 14 days, etc.)? [github.com]
Are there any best practices for configuring AI Search and Knowledge Management to accurately capture these deflections? [servicenow.com]
Has anyone implemented custom reporting or Performance Analytics dashboards to report on these metrics?

Our primary KPI is:

A user views a KB article suggested by AI Search from the Get Help page and does not create a case within a defined period, which should be counted as a Case Deflection.

We would appreciate any guidance, implementation examples, or lessons learned from similar CSM deployments.

Thanks in advance for your help!

Best Regards,
Mohit Kumar Jain

vaishali231 · 6 hours ago

Hey @Mohit42

Yes, this scenario is achievable in ServiceNow, and the use case described is very close to what Self-Service Analytics (SSA) and Deflection Configuration are designed to support.

However, I would make one important distinction in the KPI definition:

KB article viewed + no case created within a defined period should generally be treated as a Potential Deflection, rather than automatically as a Confirmed Deflection.

A Confirmed Deflection should ideally have an additional success signal, such as positive feedback/helpful interaction, depending on the configuration and available activity pattern.

1. Is this supported OOB?

Yes. Self-Service Analytics provides a framework to capture self-service activity and determine whether the interaction resulted in a deflection.

For your use case, the main OOB components would be:

CSM Portal / Get Help

AI Search

Knowledge Management

Self-Service Analytics

Deflection Configuration

Self-Service Analytics Dashboard

Performance Analytics, if additional business-specific reporting is required

The important point is that SSA provides the deflection framework, while the exact attribution of "this article was recommended by AI Search from this specific Get Help interaction" should be validated in the target implementation.

2. How should the "no case created" condition be measured?

I would avoid implementing a custom Scheduled Job simply to check whether a case was created after a Knowledge article was viewed.

SSA already provides the framework for evaluating subsequent activity within the configured deflection window.

For example, if the configured window is 7 days:

Customer searches for an issue.

AI Search recommends a Knowledge article.

Customer opens the article.

If the customer creates a case during the configured window, the interaction is treated as No Deflection.

If no case is created within the configured window, it can qualify as a Potential Deflection.

If there is also an appropriate positive/helpful signal, it can qualify as a Confirmed Deflection.

The exact activity pattern and configuration should be validated against the ServiceNow release and CSM portal implementation being used.

3. What about a 7-day vs 14-day window?

I would not consider 14 days automatically better.

The window should represent the normal period in which a customer would raise a case if the Knowledge article did not solve the issue.

For example:

Window	Consideration
1 day	Strong immediate self-service signal
3 days	Useful for short-lived issues
7 days	Good starting point for many CSM use cases
14 days	More conservative attribution
30 days	Attribution becomes much weaker

I would start with 7 days, measure the results, and then adjust based on actual case-creation behavior.

The longer the window becomes, the more difficult it becomes to confidently attribute "no case" to the article.

4. One important point about the proposed KPI

The proposed KPI is:

User views a KB article suggested by AI Search and does not create a case within a defined period.

I would actually report two metrics.

Potential Case Deflection

AI Search recommendation → KB Article Viewed → No Case Created within the configured window.

Confirmed Case Deflection

AI Search recommendation → KB Article Viewed → Positive/Helpful Feedback → No Case Created within the configured window.

This distinction is important because simply not creating a case does not necessarily prove that the article solved the customer's problem.

The customer may have:

Left the portal.

Solved the problem somewhere else.

Contacted support through another channel.

Decided not to pursue the issue.

Created a case after the selected window.

Therefore, I would use Potential Deflection as the broader KPI and Confirmed Deflection as the stronger success metric.

5. How would I track the complete user journey?

I would model the journey as a reporting funnel:

Get Help page access

Short Description/Search submitted

AI Search results

KB article clicked

KB article viewed

Potential Deflection

Confirmed Deflection

Case Created / No Case Created

For the OOB portion, I would rely on Self-Service Analytics activity tracking and Deflection Configuration.

The area I would specifically validate is the AI Search attribution.

There is an important difference between:

Customer viewed a Knowledge article.

and:

Customer entered a Short Description, AI Search recommended a specific article, the customer opened that article, and the customer did not create a case.

If the business requires the second level of attribution, verify that the current AI Search/Get Help implementation exposes the required activity information to SSA.

If it does not, I would add lightweight custom instrumentation rather than replacing the complete SSA framework.

6. Would I create a custom table?

Not initially.

My first approach would be to use:

OOB AI Search

OOB Knowledge Management

OOB Self-Service Analytics

Deflection Configuration

OOB Self-Service Analytics Dashboard

Only if the required AI Search attribution is not available OOB would I introduce custom event/activity tracking.

If custom tracking is required, the minimum information could include:

User/session

Search submitted

Search timestamp

AI Search result

Knowledge article

Article viewed

Article view timestamp

Case created

Deflection window

Deflection outcome

I would avoid storing unnecessary customer or search information if it is not required for the reporting use case.

7. Reporting / Performance Analytics

For reporting, I would create a funnel rather than only showing article views.

Recommended KPIs:

KPI	Purpose
Get Help Visits	Self-service traffic
Search Submissions	Search demand
Searches with Results	AI Search effectiveness
KB Click-through Rate	Recommendation effectiveness
KB Article Views	Knowledge engagement
Potential Deflections	Possible cases avoided
Confirmed Deflections	Stronger deflection signal
No Deflection	Case created after self-service
Deflection Rate	Overall effectiveness
Top Deflecting Articles	Best-performing Knowledge
Failed Searches	Knowledge gaps
Deflection by Product	Product/service analysis
Deflection by Category	Issue analysis

For example:

Potential Deflection Rate

Potential Deflections / Eligible Self-Service Engagements × 100

Confirmed Deflection Rate

Confirmed Deflections / Eligible Self-Service Engagements × 100

I would avoid using:

KB Views - Cases Created

as the deflection rate because that can significantly overstate the actual deflection impact.

8. AI Search and Knowledge best practices

AI Search quality will directly affect the quality of the deflection metric.

I would focus on the following areas.

Knowledge quality

Clear and problem-oriented article titles
Good article descriptions
Correct Knowledge Base
Correct user criteria
Appropriate metadata
Product/service information
Relevant categories
Synonyms
Regular content review
Avoiding duplicate or obsolete articles

AI Search

Validate search sources
Validate Knowledge visibility
Review result ranking
Monitor zero-result searches
Review unsuccessful searches
Analyze frequently clicked articles
Analyze articles that are recommended but still lead to cases

One particularly useful metric is the number of cases created after a customer interacted with a recommended Knowledge article.

Those cases can indicate that the recommended article did not actually solve the customer's problem.

9. Knowledge Gap analysis

I would also use unsuccessful searches as an input for Knowledge Management improvement.

For example, regularly analyze:

Top failed searches
Searches with low-quality results
Articles with high views but high case creation
Frequently searched topics without relevant Knowledge
Outdated or duplicate Knowledge articles

This creates a continuous improvement cycle for Knowledge Management and AI Search instead of treating Case Deflection only as a reporting metric.

10. Recommended implementation approach

I would implement this in three phases.

Phase 1 – OOB validation

Enable Self-Service Analytics.
Configure Deflection Configuration.
Configure the deflection window.
Validate Knowledge activity patterns.
Validate case creation correlation.
Review the OOB Self-Service Analytics dashboards.

Phase 2 – AI Search attribution

Validate the complete Get Help interaction:

Get Help page
Short Description
AI Search
Recommended Knowledge Article
Article View
Subsequent Case Creation

Confirm that this specific journey is captured by the SSA activity framework.

If the exact attribution is not available OOB, introduce only the required custom activity/event tracking.

Phase 3 – Business reporting

Create a Platform Analytics / Performance Analytics dashboard containing:

Get Help Visits

Search Submissions

AI Search Results

KB Views

Potential Deflections

Confirmed Deflections

Cases Created

Deflection Rate

11. Final recommendation

For this requirement, I would not build a completely custom Case Deflection solution.

I would start with:

AI Search + Knowledge Management + Self-Service Analytics + Deflection Configuration

and use the OOB SSA deflection framework as the foundation.

The main area I would investigate carefully is the AI Search-to-article attribution from the specific Get Help page, because your KPI is more specific than simply "Knowledge article viewed."

Most importantly, I would define the business KPI as:

Potential Case Deflection: A customer engages with a Knowledge article recommended through self-service and does not create a case within the configured deflection window.

And:

Confirmed Case Deflection: A customer engages with the Knowledge article, provides an appropriate positive/helpful signal, and does not create a case within the configured deflection window.

This approach gives you a more defensible KPI and avoids overstating the number of cases actually prevented.

For reference, ServiceNow's Self-Service Analytics documentation and Customer Service Self-Service Analytics dashboard provide the relevant OOB framework for deflection measurement.

*************************************************************************************************************************************

If this response helps, please mark it as Accept as Solution and Helpful.

Doing so helps others in the community and encourages me to keep contributing.

Regards

Vaishali Singh

Servicenow Developer
Linkedin - https://www.linkedin.com/in/vaishali-singh-2273361bb