CI Reconciliation between different import sets

[Beth11]

We want to use the discovery reconciliation engine to co-ordinate between multiple unique data sources. We have multiple data sources writing into the same CI. Some of these sources are more trusted than others. Unfortunately, it doesn't look like there's any way to configure the reconciliation rules or data precedence rules to treat import sets differently; they're all handled as the same 'source' in the CI class manager.

Is there any way to get around this? I'd love to be able to build custom data sources that the reconciliation engine will recognize (corresponding to different import sets) for ease of maintenance, but how else can I tackle this issue? Potentially this will all end up pretty complex as we move forward with many different sources of truth, and I'd like to get things set up to handle that while it's still early days.

Thanks in advance for any advice.

[Beth11 · Accepted solution]

For the curious, or anyone hitting this issue themselves, here's the solution I ended up with. 

First, I added the source I wanted to use to the choices available for the field cmdb_ci.discovery_source. 

Then, I extended the script include CMDBTransformUtil to add a second initialization function, one that would allow me to pass the data source to the function instead of it being set statically.

var CMDBTransformUtilSourced = Class.create();
CMDBTransformUtilSourced.prototype = Object.extendsObject(CMDBTransformUtil, {
    initialize: function(customDataSource) {
		gs.log('custom initalize');
		this.json = new JSON();
		customDataSource = typeof customDataSource !== 'undefined' ? customDataSource : 'ImportSet';
		this.dataSource = customDataSource;
		this.transformEntryTable = 'sys_transform_entry';
		this.cmdbTransformStatsTable = 'cmdb_import_set_run';
		this.CmdbAPI = SNC.IdentificationEngineScriptableApi;
		this.CmdbUtil = SNC.CMDBUtil;
		this.outputPayload = '';
		this.outputRecordSysId = '';
		this.error = '';
		this.transformResult = 'e';		
    },
    type: 'CMDBTransformUtilSourced'
});

Then, I followed the steps here: https://docs.servicenow.com/bundle/london-servicenow-platform/page/product/configuration-management/... but replaced their function with mine.

(function runTransformScript(source, map, log, target) {

	// Call CMDB API to do Identification and Reconciliation of current row
	var cmdbUtil = new CMDBTransformUtilSourced('MySource');
	cmdbUtil.identifyAndReconcile(source, map, log);
	ignore = true;

	if (cmdbUtil.hasError()) {
		var errorMessage = cmdbUtil.getError();
		log.error(errorMessage);
	} else {
		log.info('IE Output Payload: ' + cmdbUtil.getOutputPayload());
		log.info('Imported CI: ' + cmdbUtil.getOutputRecordSysId());
	}

})(source, map, log, target);

It's important to note the caveats in the article I linked to, because there are a lot of ways this can go wrong if it isn't quite configured properly, but so far this is working well for me. I hope it helps someone in the future.

[Marek10]

hi Beth,

Thanks for inspiring topic and solution! However, we found out that (probably) the same could be achieved with ootb CMDBTransformUtil class with setDataSource method as per: https://developer.servicenow.com/app.do#!/api_doc?v=madrid&id=CMDBTU-setDataSource_S

Best regards,

Marek

[jleon]

If you need to know how to add a custom Data Source in an IRE Data Source rule, you can check this

https://community.servicenow.com/community?id=community_question&sys_id=80313f201b018990cdd555fa234bcb20