Getting "[PA AppSec] Daily Data Collection completed with warning" emails ever since updating from Quebec to San Diego

Jeremy Dean · ‎09-14-2022

We are getting "[PA AppSec] Daily Data Collection completed with warning" emails hourly ever since updating from Quebec to San Diego.

Reviewing the job logs, I see a warning:
Not saving all records for Indicator:f83563cab7721010aed8e6e5de11a96b Result has 7,832 records. Only allowed subset of 5,000 records are stored

Searching around on the issue I found this article "[PA AppSec] Daily Data Collection completed with warning" daily email to ServiceNow admin account s... which indicates you can adjust the system property "com.snc.pa.dc.max_row_count_indicator_source" to include as many records as you need. I updated the number to 50,000 and that seems to have stopped the error in our Pre-Prod instance, but the same change was not successful in Dev. And evn if it were, I would still like to understand:

What does this message mean? (The error Not saving all records for Indicator...)
What does this job do? ([PA AppSec] Daily Data Collection completed with warning)
Why did this start when we upgraded to San Diego
What is the proper resolution (I don't like the idea of randomly increasing numbers on system variables.)

Any help is most appreciated - thank you!

Adam Stout · ‎10-17-2022

The log is showing you the sys_id of the indicator that has the issue. If you search for that in pa_indicators, you will see the indicator. I would reactivate this indicator but disable the "Collect Records" option. This warning is related to collecting (storing the sys_ids) more records than allowed based on the system property. Storing the sys_ids of every user isn't particularly valuable, but the count of active users is important to have as a reference for other indicators.

View solution in original post

jfrancione · ‎02-22-2023

thank you for your responses.. I ended up opening up a case with HI.

Bert_c1 · ‎10-03-2022

That indicator source if defined for the sys_user table, and has no conditions. So all records are returned. The indicators that use that indicator source for the filter:

pa_indicators_list.do and use Indicator source "System Users"

check those to see how many records are returned.

Bert_c1 · ‎10-03-2022

As far as setting the Active to false on those three indicators, you will no longer see any data for those going forward. that does not cause any other negative consequences. The data shown on any related dashboards/widgets will only show values before the date you de-activated those indicators.