Consequence of (re-)running historical data collection

LenaLindel_f
Kilo Expert

Hej,

   

We have been running daily data
collections for a few months and have new break-downs we want to apply
historically. Can we run a historical job for, for example, incident indicators
that we have daily collections for other break-downs without "destroying" the data
we have already collected?
 

Can you please describe which
type of data that is not collected in historical collections? I recall from our
implementation that there was some
difference but cannot find any documentation around that.

Thanks,

Lena

4 REPLIES 4

D van Heusden
ServiceNow Employee
ServiceNow Employee

Hi Lena,



in the job you can change the indicators you are collecting for to only collect for a specific breakdown. I suggest you create or reuse the [PA Incident] Historic Data Collection job. Add the indicators for which you want to collect the data historically (I'll dive into what indicators and breakdowns you can or cannot collect historically) and specify the breakdown (Eureka) for which you want to collect or list the breakdowns you would like to exclude (Fuji) from the collection. You can make the changes in the 'Collect' column for each of the indicators. Also make sure you set the other indicators to 'false' that you want to exclude altogether. You will also have to think about whether or not you want to re-collect the indicator score itself, do this by unchecking the 'Collect indicator' column. Typically I would also exclude this because you are only interested in collecting the new breakdown scores.



Differences in Eureka and Fuji are because you more often than not want to collect for a set of new breakdowns that you've added to an indicator. In Eureka you'll have to change the breakdown for which you want to do a collection and run the job again, whereas in Fuji you can easily exclude any breakdowns that you've already collected for and you can easily collect for the newly added breakdowns.



As to when you can or cannot collect historically, this depends if the breakdown element changes over time. If that's the case, for example an incidents assignment group, you will probably not be able to accurately collect historically because the incident will only retain the latest assignment group that the incident was assigned to. Collecting historically would then show that latest assignment group and not any previous assignment groups that the incident could be on. If you want that information, you should look at building indicators on the metrics table, but that is something outside of this discussion that we'll cover in a later post, blog or KB article. Another example is the incident state, as all incidents should ultimately be in the closed state so you won't be able to collect this historically.



Typically, breakdowns that rarely or infrequently change you can easily collect historically like category, subcategory, priority, caller, caller organization, configuration item, business service, etc... And sometimes you are just looking for trends and you do not need complete accurate scores so that basically additionally determines whether you want and can collect historically.



Let me know if you succeed. Kind regards



Fuji example for indicators added in a job with 1 indicator active and multiple breakdowns and the indicator excluded from collection:


Historically collect for single breakdown Fuji.jpg


Thanks for your prompt response David!



We are running Fuji and we have noted and used the ability to exclude break-downs from historical data collections (we tend to create new each time to keep traceability of when we ran historical jobs).



BUT …the omitting indicator scores from being collected was a new and useful piece of information… let me just see if I understood that correctly — we can, for a particular indicator, select only the break-downs we would like to collect data for relating to that indicator leaving the daily scores intact?



And I think you have answered my question on what we lose by re-running historical jobs — if I already have values for a Break-down from daily collections I will lose any changes to break downs as PA captures only the latest recorded BD value so for example if an incident was re-prioritized that will get lost during the historical collection.


If your investigations find something more out I ´d be all ears!



I will be back if I fail the historical BD collection.. but with your explanation I think it will work fine.



Have a nice day!



Regards,


Lena


"BUT …the omitting indicator scores from being collected was a new and useful piece of information… let me just see if I understood that correctly — we can, for a particular indicator, select only the break-downs we would like to collect data for relating to that indicator leaving the daily scores intact?" - That is correct.


Thanks David — we will try that!



Regards,


Lena