Understanding Scheduled Sentiment Analysis Frequency and Multiple Executions in CSM

MahekMM
Tera Contributor

9 hours ago

Hi everyone,

We are investigating the behavior of Sentiment Analysis for CSM Cases and would appreciate clarification on the expected scheduling and execution behavior.

Our current configuration is:

  • Table: CSM Case
  • Eligible assignment groups: Groups starting with CS
  • Eligible states: New, In Progress, or On Hold
  • Scheduled trigger frequency: Every 7 days

We have the following questions:

  1. If the scheduled Sentiment Analysis runs on Monday at 10:00 AM, and a new eligible case is created on Wednesday at 12:30 PM, should Sentiment Analysis run for that case immediately, or will it wait until the next scheduled run on Monday?
  2. If an eligible case is updated multiple times between two scheduled runs, how is Sentiment Analysis handled? Does the next scheduled run evaluate the case once using the latest information, or can multiple Sentiment Analysis executions be triggered?
  3. How is Now Assist usage/token consumption calculated for Sentiment Analysis? Does each actual Sentiment Analysis/LLM execution consume a separate Now Assist usage?
  4. We are seeing multiple system-generated processing jobs associated with Sentiment Analysis. Is this expected behavior, and how do these processing jobs relate to the configured 7-day Sentiment Analysis schedule?
  5. If multiple Sentiment Analysis executions are observed for the same case, what is the recommended way to determine whether these are expected executions or duplicate processing?
  6. Is there a supported way to trace a specific case through the flow of scheduled Sentiment Analysis → event processing → actual GenAI/LLM execution → Now Assist usage?

We would particularly appreciate clarification on how newly created cases and cases updated between scheduled runs are handled, and how actual LLM executions should be distinguished from background/event processing.

Thanks in advance for any guidance.

0 Helpfuls
  • 110 Views
1 REPLY 1

Abhishek Pal
Giga Guru

5 hours ago

Hi @MahekMM ,

Your questions are valid because there are three different layers involved here:

1. Sentiment scheduler
2. Actual Sentiment Analysis / LLM execution
3. Now Assist usage/assist consumption

They should not be treated as the same thing.

For the OOB CSM Sentiment Analysis Case skill in Australia, the trigger is schedule-based.

The OOB scheduled job is:

Sentiment analysis scheduled job (case)

and its frequency is controlled from:

Now Assist Admin
-> Skills
-> Customer
-> CSM
-> Sentiment Analysis Case
-> Define Trigger

Let me answer your scenarios individually.

1. New case created between scheduled runs

If your frequency is:

Every 7 days

and the scheduled execution runs:

Monday 10:00 AM

then an eligible Case created:

Wednesday 12:30 PM

does NOT normally trigger an immediate Sentiment Analysis execution just because the Case was created.

With the OOB scheduled configuration, it becomes eligible for processing when the next scheduled sentiment run executes.

So conceptually:

Monday 10:00
-> Scheduled run

Wednesday 12:30
-> New eligible Case created
-> No immediate scheduled sentiment execution

Following Monday 10:00
-> Scheduler runs
-> Case is evaluated if it still satisfies the configured availability conditions

This is important because a 7-day frequency means sentiment can potentially remain uncalculated/stale for several days.

If the business expects near-current sentiment, I would not configure a 7-day interval.

ServiceNow allows the trigger interval to be configured in:

Days
Hours
Minutes
Seconds

but reducing the interval increases the number of Now Assist actions and therefore potentially increases assist consumption.

2. What happens if the Case is updated many times?

Assuming you are using only the OOB scheduled trigger and have not added another custom/manual trigger:

Case updated Monday
Case updated Tuesday
Case updated Wednesday
Case updated Thursday

does NOT mean four separate Sentiment Analysis executions are automatically created because of those four updates.

When the scheduled sentiment process reaches the Case, Sentiment Analysis evaluates the current Case information available at that time.

The OOB skill uses Case context including items such as:

- Short description
- Description
- Priority
- State
- Activities
- Task SLA

The sentiment calculation also considers recent/requester comments, historical sentiment/trend, SLA information, response timing and tone.

Therefore the next analysis is based on the latest available Case context.

However, I would not rely on assumptions about duplicate execution purely from the schedule.

The platform provides execution and usage logs, which are the correct way to confirm how many actual skill/LLM executions occurred.

3. Does every LLM execution equal one Now Assist usage?

There is an important distinction here.

Do not calculate consumption using:

Number of tokens
or
Number of background jobs

Now Assist usage is tracked as:

Skill Action
x
Assist Ratio
=
Assists consumed

ServiceNow records this in:

Gen AI Usage Log
[sys_gen_ai_usage_log]

Useful fields include:

Assists
Document
Document Table
Feature
Skill Config Id
Status
Created
User

Therefore, an actual Sentiment Analysis skill execution can generate Now Assist usage, but you should check the configured skill's Assist Ratio instead of assuming:

1 LLM call = exactly 1 billable assist

Token counts are a separate technical metric.

They can be reviewed in:

Generative AI Log
[sys_generative_ai_log]

which contains:

Prompt Token Count
Response Token Count
Model
Started At
Completed At
Status
Prompt
Response

Token count is useful for technical/model analysis; Assist consumption is the entitlement metric.

For contractual consumption, use Subscription Management / Now Assist usage rather than calculating from token counts.

4. Why are there multiple system-generated processing jobs?

This does not automatically indicate duplicate Sentiment Analysis.

Conceptually the processing can look like:

Sentiment scheduler
-> Find eligible Cases
-> Queue/process work
-> Invoke Now Assist skill
-> Invoke Generative AI
-> Persist sentiment result
-> Update usage/log records

The scheduler/background/event infrastructure can therefore create multiple processing activities while there may be fewer actual LLM calls.

Do NOT count:

sys_trigger records
Scheduled Job executions
Event records
Async jobs

as Now Assist executions.

The important evidence is:

sys_gen_ai_usage_log
and
sys_generative_ai_log

5. How to determine whether Sentiment Analysis actually executed twice for one Case

This is the troubleshooting path I would recommend.

First open:

sys_gen_ai_usage_log.list

Filter:

Document Table = sn_customerservice_case

Document = <Case sys_id>

Skill Config Id = <Sentiment Analysis skill config>

Sort:

Created descending

This shows the Now Assist capability usage associated with that Case.

Then open:

sys_generative_ai_log.list

Filter using:

Skill Config Id = Sentiment Analysis skill

and the relevant execution/time window.

The Generative AI Log also contains a reference to:

Gen AI Usage Log

This lets you correlate the actual GenAI execution with the usage record.

So your trace becomes:

CSM Case
|
v
sys_gen_ai_usage_log
|
v
sys_generative_ai_log
|
v
Actual LLM execution

If you see:

2 scheduler/background jobs
but
1 sys_generative_ai_log execution

that is not two LLM executions.

If you see:

2 distinct successful sys_generative_ai_log records
for the same skill/Case/time window

then investigate why the skill was invoked twice.

Check whether you have:

- More than one Sentiment Analysis trigger
- Custom Flow/Business Rule invoking the skill
- Manual testing
- Duplicate scheduled configuration
- Cloned/custom sentiment skill
- Historical sentiment processing enabled

6. Historical processing is separate

There is another OOB job called:

Update sentiment historical records

ServiceNow documents this separately from the normal sentiment scheduler.

Its default state is:

False

When enabled, it calculates sentiment and trends for historical records.

Therefore, if you observe unexpected additional processing, verify whether this job has been activated.

The two jobs have different purposes:

Sentiment analysis scheduled job (case)
-> Ongoing sentiment refresh according to configured trigger frequency

Update sentiment historical records
-> Historical/backfill processing

Make sure you are not interpreting historical processing as duplicate normal scheduled processing.

7. Recommended end-to-end trace

For your scenario I would trace one Case like this:

Case
sn_customerservice_case
|
v
Check availability:
Assignment Group starts with CS
State = New / In Progress / On Hold
|
v
Sentiment analysis scheduled job (case)
|
v
Sentiment Analysis Case skill
|
v
Gen AI Usage Log
sys_gen_ai_usage_log
|
v
Generative AI Log
sys_generative_ai_log
|
v
LLM processing
|
v
Outputs:
Sentiment
Sentiment Trend
Sentiment Reasoning

Then separately:

sys_gen_ai_usage_log.Assists
|
v
Now Assist usage / Subscription Management

This gives you a much cleaner way of differentiating:

Background processing
vs.
Actual GenAI execution
vs.
Assist consumption.

8. How to validate your 7-day configuration

I would create one test Case:

Assignment Group:
CS Test

State:
New

Then record:

T0:
Case created

T1:
Add customer comment

T2:
Add another customer comment

T3:
Scheduled sentiment run

After T3 verify:

Case:
Sentiment / Trend / Reasoning

Then:

sys_gen_ai_usage_log
-> Case sys_id

and:

sys_generative_ai_log
-> Sentiment skill

You should then be able to prove exactly how many actual GenAI executions occurred.

9. Recommended schedule

I would reconsider the 7-day frequency if this information is meant to help agents proactively identify customer frustration.

A sentiment result that is refreshed only once every seven days can become operationally stale.

A better pattern is normally:

More frequent schedule
+
Narrow availability/filter criteria

rather than:

Very broad eligible population
+
7-day schedule

For example, concentrate the skill on active Cases where sentiment provides operational value.

This also helps control assist consumption.

ServiceNow specifically notes that reducing the scheduler interval increases the number of actions and assist usage, so schedule and availability conditions should be designed together.

10. Monitoring usage

In Australia you can also monitor the skill from:

Now Assist / AI Analytics
-> Skills performance

and, on supported patch levels:

Now Assist Skill Details
-> Usage

You can review:

Total executions
Active users
Executions by channel
Assist consumption

For entitlement-level consumption use:

Subscription Management
-> Subscriptions
-> Now Assist usage

ServiceNow updates the skill-level Subscription Management usage data nightly.

So the short answers are:

Q1:
New Case between runs?

-> Waits for the next scheduled execution when using the OOB scheduled trigger.

Q2:
Several Case updates between runs?

-> They do not inherently create one scheduled execution per update. The next scheduled processing evaluates the latest Case context.

Q3:
Usage calculation?

-> Use sys_gen_ai_usage_log / Assist Ratio. Do not use token count or number of scheduler jobs to calculate assists.

Q4:
Multiple system processing jobs?

-> Can be expected. Background jobs/events are not equivalent to LLM executions.

Q5:
How to identify duplicate execution?

-> Correlate Case + Skill Config through sys_gen_ai_usage_log and sys_generative_ai_log.

Q6:
Supported end-to-end trace?

-> Scheduler -> eligible Case -> Sentiment skill -> Gen AI Usage Log -> Generative AI Log -> sentiment result -> assist reporting.

I would avoid modifying the OOB scheduled-job script just to troubleshoot this behavior.

First use the OOB GenAI/Usage logs to establish exactly where the duplicate activity is occurring.

Official ServiceNow references:

Sentiment Analysis Case configuration:
https://www.servicenow.com/docs/r/customer-service-management/now-assist-for-csm/configure-sentiment...

CSM Sentiment Analysis skill inputs and triggers:
https://www.servicenow.com/docs/r/customer-service-management/now-assist-for-csm/additional-skill-in...

Generative AI Controller tables:
https://www.servicenow.com/docs/r/intelligent-experiences/generative-ai-controller/generative-ai-con...

Monitoring Now Assist usage:
https://www.servicenow.com/docs/r/platform-administration/monitoring-now-assist-usage.html

Now Assist Skill Details:
https://www.servicenow.com/docs/r/intelligent-experiences/now-assist-skill-details.html

ServiceNow Employee article - CSM Sentiment Analysis:
https://www.servicenow.com/community/csm-articles/servicenow-otto-for-csm-sentiment-analysis/ta-p/33...

Hope this helps!

If this response helped, please mark it as Helpful.
If it resolves your questions, please Accept it as Solution.

Kind Regards,
Abhishek Pal

0 Helpfuls