In the em_event table, can anyone explain how the Process Duration is calculated?

[sivkr]

My assumption was that Process Duration represents the time difference between the Created and Processed fields, but that doesn’t seem to be correct.

I also checked the sa_performance_statistics table. It appears that events are grouped into buckets, and based on those bucket queries, they are processed to create actionable alerts. However, in our instance, the count_per_sec and avg_millisec columns in sa_performance_statistics are empty and not populated with any values.

My requirement is to create a dashboard showing:

• Total events created
• Total duration taken to process the events

We are displaying this report in 1-hour intervals, but the Process Duration we are assuming is not accurate.

Can anyone clarify:

1. How is Process Duration actually calculated?
2. How can we get values for count_per_sec and avg_millisec in the sa_performance_statistics table?
   
   Please find the attached screenshots.
   
   

[sivasankaris]

In Event Management, Process Duration represents the actual time the Event Management engine spends actively processing an event, and it is measured in milliseconds. It’s more of an internal execution time than an end-to-end timestamp difference.

What this duration includes is the work done by the EM engine itself, such as:

• Event rule evaluation

• Correlation logic

• Alert creation or update

• Deduplication checks

• RCA and aggregation processing

Internally, ServiceNow calculates this as the difference between the processing start time and processing end time. These timestamps are runtime values used by the engine and are not exposed as fields on the em_event table.

That’s why you’ll notice:

• Processed – Created does not match Process Duration

• Two events created at the exact same time can still have very different process durations

In short, Process Duration reflects CPU execution time inside the EM engine, not wall-clock or end-to-end processing time,

Hope this clarifies how the field works internally, if this is helpful please mark it as helpful and accept the solution.

[sivkr]

Hi @sivasankaris 

Thanks for the detailed clarification on how Process Duration works internally. That really helps!

I have a couple of follow-up questions:

Do you have any insights on how the avg_millisec and count_per_sec columns in the sa_performance_statistics table are populated?

Are these values calculated automatically by the Event Management engine, or do we need to enable/configure something on our end to start seeing data in these fields?
My requirement is to prepare a data visualization report that shows:

Total number of events received in the em_event table per day
Display this in 1-hour intervals for the entire day
Include the total time taken to process these events
Is there any recommended approach or best practice to achieve this? For example, can we leverage any existing tables or metrics in Event Management to calculate the processing time accurately for these hourly buckets?

Any inputs or guidance on these points would be greatly appreciated!

[sayalishinde]

Based on the screenshots, it looks like the Processing duration in the em_event table is calculated as the difference between the Created and Processed timestamps.

For example, in the first event record:

• Created: 13:13:53
• Processed: 13:14:02
• Processing duration: 2 milliseconds

However, the Processing duration does not appear to represent the complete difference between the Created and Processed timestamps. The timestamp difference is around 9 seconds, while the Processing duration is only 2 milliseconds.

This suggests that the event may spend some time waiting in the event-processing queue or bucket before processing starts. The Processing duration may represent only the actual time taken by the Event Management engine to process the event, rather than the total end-to-end time from event creation until processing is completed.

The sa_performance_statistics records also appear to support this. The Event Processing records show the processing count and duration for individual processing buckets, while the Total Of Event Processing records show the aggregated values. For example:

• Count: 46,590
• Duration: 2,851,968

The average processing time can be calculated as:

Duration ÷ Count

2,851,968 ÷ 46,590 ≈ 61 milliseconds

This also matches the eventProcessingAverage value of 61 shown in the Extra Data field.

Therefore, for the dashboard:

• Total events created: Use the count of em_event records grouped by the Created time.
• Total event-processing time: Use the Duration value from the Total Of Event Processing records in sa_performance_statistics.
• Average processing time: Calculate Duration ÷ Count, or use the eventProcessingAverage value from the Extra Data field if required.

The Count Per Sec and Average (milli) columns being empty may be expected for these Event Processing records, since the relevant values appear to be stored in the Extra Data JSON instead. It may be worth checking whether those fields are intended to be populated for this specific performance-statistics type.