"[PA AppSec] Daily Data Collection completed with warning" daily email to ServiceNow admin account since upgrade to Quebec

[sbh]

Every day since we upgraded our prod instance to Quebec, our ServiceNow admin account gets this email. It's only a warning, but I don't know if I'm being warned I'm logging too much and that this could impact performance. Or does this just mean I should change settings?

Subject: [PA AppSec] Daily Data Collection completed with warning

Body:

Job Name: [PA AppSec] Daily Data Collection
Run time: 8 Seconds
Inserts: 96
Deletes: 0
Warnings: 1
Errors: 0

Message from job log: Not saving all records for Indicator:<sys_id> Result has 25,764 records. Only allowed subset of 5,000 records are stored

[Saurabh Gupta · Accepted solution]

Change the property "com.snc.pa.dc.max_row_count_indicator_source" as per your need. By default it is 5000.

Available system properties | ServiceNow Docs

If my answer replied your question please mark appropriate response as correct so that the question will appear as resolved for other users who may have a similar question in the future.

Regards,
Saurabh

[sbh]

Thank you! I'm not sure what that data collection is used for. Do you know if there's a way I can find out which source(es) is/are sending all that data to be collected? I don't know if it's crowding out something that would be more useful. Is there anything that explains where this data comes from?

[JP-ODU]

If I may piggyback on this discussion... I found this info immensely helpful, thank you! But when I go to my version of com.snc.pa.dc.max_row_count_indicator_source, it says it's already set to 50,000 but my PAs are saying "only allowed subset of 5,000 records are stored." Shouldn't 50,000 be allowed? What am I missing?

[Pablo1]

Hi JP-ODU,

there are actually two separate properties, one that helps remove the warnings stated in the original post and the other which determines the threshold for indicator sources from collecting data from tables.

1.  com.snc.pa.dc.max_row_count_indicator_source - Maximum number of rows that are allowed to be fetched from an Indicator Source

which in the last number of releases has been replaced by the optimized version for data collection: 

com.snc.pa.dc.hsql.max_row_count_indicator_source - Maximum number of rows that are allowed to be fetched from an Indicator Source in optimized data collection

2. Message from job log: Not saving all records for Indicator XXXXX is due to the property: com.snc.pa.dc.max_records - Maximum number of records that are stored during a data collection i.e. the number of snapshots allowed. 

So the value 50,000 default doesn't affect the total number of records stored as snapshots during data collection. Two different properties.