Servicenow provides various options to get the data from different discovery sources. The data flows into Servicenow via different applications like 3rd Party Applications, Excel imports and places into CMDB Tables.
While doing these activities, it is always better to use IRE Engine to insert/update records into CMDB Tables because it has all the features to determine which discovery source can update via Reconciliation Rules (or) determine whether a record can be inserted / updated via Identification Rules and so on.
The process to import this kind of data starts with creating Data Source, Field Mapping & Transform Scripts. Finally some integrations requires a Scheduled Data Import. Regular way of scripting the transform is cumbersome and lot of maintenance work is required even for a small changes in IRE Rules. To address this, we have an OOTB class called "CMDBTransformUtil" that helps in determining inserting/updating the records properly and ensure to minimize the creation of duplicate entries.
Example
Table - Company Resources
Identification Rules are like this
Reconciliation Rules are like this
The below data from an Excel sheet needs to be imported
For the above table, the Data Source, Transform Map & Transform Script is configured like this
Data Source#
Transform Map#
Transform Script#
Now, Load all records from Data Source => Go to the Import Set ==> Transform. You can observe the records in the Table like this
When it comes to updating of these records, Since we are using CMDBTransformUtils, it update records using IRE Engine and we don't need to write any unnecessary logics within the Transform Scripts.
Note: The Low Priority is ACC-Visibility & the High Priority taken as "ABC Application"
| ABC Application | ACC-Visibility |
| Can update Name, Serial Number, Description, Location, Manufacturer | Can only updates Description field |
Now, lets see how a row will update based on the DataSource/Discovery Source. Here is the code for your reference.
OnBefore Script# using ABC Application as Data Source
(function runTransformScript(source, map, log, target /*undefined onStart*/ ) {
// Add your code here
var ctu = new CMDBTransformUtil();
ctu.setDataSource("ABC Application");
//#commented#ctu.setDataSource("ACC-Visibility");
ctu.identifyAndReconcileEnhanced(source, map, log);
ignore = true;
if (ctu.hasError()) {
var errorMessage = ctu.getError();
log.error(errorMessage);
} else {
log.info('IE Output Payload: ' + ctu.getOutputPayload());
log.info('Imported CI: ' + ctu.getOutputRecordSysId());
}
})(source, map, log, target);
Existing Record from Company Resources Table
I have changed the name & description as below from the Import set Rows
After Executing the Transform again, the row updates like this. Since it was using "ABC Application" as Data Source, it updated both the fields after matching rules.
If you change the Datasource to "ACC-Visibility" in transform script and execute the transform, since it is supposed to update only Description field, the remaining fields update will skip like this.
Important:
- It is always better not to toggle "Apply to all attributes" when you create a Reconciliation Rule. Rather select the fields independently.
- "Update with null" needs to be selected in case if would like to update with a null value when the field value is not empty.
- If you would like to update a value when it is empty with Low Priority Reconciliation Rule,
- Make sure the "CMDB Properties==>Identification/Reconciliation Properties ==> Allow the update of an empty field by a lower priority data source" to true.
- Use the "Identification Simulation" when things are not working to see how the IRE Engine behaves of a specific record.
- Use "SNC.IdentificationEngineScriptableApi" in case if you would like to completely skip the Data source, Transform and go with your own way of importing the data. This can be useful when you are dealing with REST Outbound to get the data via API.
- You may also think of creating a "Data Stream" via Flow Designer when you deal with REST Outbound to get the data via API.
Please feel free to add your comment/any other guide lines.
Thank you,
Narsing
