Scheduler performance metrics
Summarize
Summary of Scheduler performance metrics
Scheduler performance metrics offer a snapshot of how scheduled jobs are performing within the ServiceNow AI Platform®. These metrics help customers monitor and manage the execution of various scheduled jobs, including reports, script executions, timers, and asynchronous business rules that support background processing for better user experience.
Show less
Key Metrics
- Completed Scheduled Jobs: Tracks the number of scheduled jobs completed per application node. This includes traditional scheduled jobs and special cases like asynchronous business rules triggered by UI actions or integrations. A sudden spike in completed jobs may indicate potential processing overload and should be evaluated with other metrics.
- Scheduler Mean Queue Age: Measures the average age of unprocessed jobs in the node’s in-memory scheduler queue. A high mean queue age suggests that the node may be struggling to process jobs promptly. Customers should investigate long-running jobs and consider optimizations such as reducing frequency or staggering executions. From the New York release onward, nodes can return unprocessed jobs back to the systrigger table for reallocation, which should be verified for proper operation.
- Scheduler Queue Length: Represents the count of claimed but not yet executing scheduled jobs in a node’s in-memory queue. Sustained high queue length (over 60 minutes) accompanied by increasing queue age may indicate processing bottlenecks at a node. If multiple nodes exhibit this, it could suggest instance-wide overload or performance issues.
Practical Guidance
- Monitor these metrics together to identify processing delays or overloads affecting scheduled jobs.
- Investigate the types and sources of executing scheduled jobs to pinpoint potential performance issues.
- Address long-running jobs by optimizing frequency or staggering execution to improve throughput and reduce queue buildup.
- Verify that job reallocation functionality is working correctly to prevent job processing backlog.
Additional References
These scheduler metrics complement other ServiceNow performance metrics such as instance observer reports, anomaly insights, database performance, event queue metrics, and node health. Together, they provide a comprehensive view to maintain optimal job scheduling and platform performance.
The metrics provide the Schedulers performance snapshot within the ServiceNow AI Platform®.
Completed Scheduled Jobs
The number of scheduled jobs completed per node.
There are many types of scheduled jobs. They could be scheduled reports, scheduled script executions, one-time jobs like SLA or Workflow Timers, or repeating jobs like Email Sender or Event Processor. In addition to what you might think of as traditional scheduled "jobs" there is also a couple special case of operations that run on the scheduled job queue called, Asynchronous business rules. These can be work that was triggered from the UI or an integration but runs as a scheduled job to achieve asynchronous behavior. Typical examples of Asynchronous business rules are Discovery Sensors or perhaps a job that iterates over a large number of child records, making so many updates that it would not be a good user experience to have to wait for them all to complete.
- When scheduled job counts suddenly shoot up this can be concerning as it may overwhelm processing and cause lag.
- This should be considered together with other job metrics like the scheduler age, scheduler queue length, and pending jobs counts.
Scheduler mean queue age
The average age of all unprocessed jobs in an application node in memory scheduler queue (i.e., the delta between the job being claimed by the node or placed in the queue and the point at which metrics were collected). Note that this does not include time the job spent pending in the sys_trigger table between its 'Next Action' time (that is, when it should have run) and when it was claimed.
A large scheduler mean queue age can indicate that an application node is struggling to process claimed jobs (that is, those in its in memory queue) in a timely manner.
Scheduler queue length
The number of claimed scheduled jobs in each application nodes in memory scheduler queue. Note that this does not include scheduled jobs which are currently executing on a worker thread on the application node.
It is common for application nodes to claim multiple jobs at once and therefore have a significant scheduler queue length at any point in time. If a single node has significant queue length for a sustained period (that is, more than 60 minutes), where queue length does not drop down to 0 and scheduler age is increasing, this may indicate the node has an issue processing scheduled jobs.
If multiple nodes see similar issues it may indicate either all nodes are having issues processing scheduled jobs or the instance is simply overwhelmed by scheduled jobs.
- Check scheduled jobs executing on application nodes to determine type or source.
- Address performance or scheduling of jobs as necessary to prevent them overloading worker threads.