Duplicate ci cleanup in cmdb

[Scott Howes]

Hello,

We have some duplicate ci’s in our cmdb due to manual entries. I would like to use the de-dup process to resolve them into the master ci and then delete them but I do not know of a way to manually enter them into the de-dup wizard. I have tried to get the system to recognize them as duplicates and create de-dup tasks but so far unsuccessful. They also do not show up as duplicates on the cmdb dashboard.

The duplicate ci’s are all in the same class which is extended from hardware and the identification rules are all derived from the hardware class. The manually entered ci’s only have the name filled out on their record and they aren't being discovered or updated. The issue is that we have change requests and incident tickets related to them. We want to move all of the requests and tickets to the master ci and delete the manually created ci’s.

In the ci table we changed the name of one of the duplicates to be the same as the master ci and then ran discovery against it thinking this might create a de-dup task but it did not. Discovery brought over all of the information that is on the master ci, added those attribute values to the manually created ci and set the discovery source to ServiceNow. Now we have 2 ci’s that are loaded with the same attribute values but aren't recognized as duplicates. Also, some of the incident tickets were dropped from the ci’s (both manual and discovered) but the ci’s are still showing on the incident tickets and still pointing to the ci they were originally assigned to.

I would like to know the best way to deal with this as part of our duplication clean up process.

TIA!

[Sandeep90]

Can you check if the property "glide.identification_engine.skip_duplicates" is set to true? if it is, then it could explain why it didn't create the duplicate task.

You can just run the correctness health job from CMDB view - > CMDB Health dashboard jobs, this will run the duplicate processor and create the duplicate task for all the CIs. if there are duplicates of more than 5 then a duplicate task is not created and it can be changed from property "glide.identification_engine.skip_duplicates.threshold"

[CMDB Whisperer]

Try this as a starter.  It starts with the assumption that the duplicate_of property has been set on one CI to point to the other, which you can set directly on your own via the list view.  Then, for any CI where the duplicate_of value is set, create a deduplication task to list both the CIs.  If you wanted to support more than two CIs you'd have to adjust the logic accordingly.  I have used a similar script for my customers and it worked great!  Of course, always test on a non-Production system before running on Production.

	createDeduplicationTask: function(grCI) {
		var dupTask = new GlideRecord('reconcile_duplicate_task');
		dupTask.initialize();
		dupTask.setValue('short_description', 'Manually created deduplication task.');
		dupTask.insert();
		
		var dup1 = new GlideRecord('duplicate_audit_result');
		dup1.initialize();
		dup1.setValue('duplicate_ci', grCI.getUniqueValue());
		dup1.setValue('duplicate_id', grCI.getUniqueValue());
		dup1.setValue('table', grCI.sys_class_name);
		dup1.setValue('follow_on_task', dupTask.getUniqueValue());
		dup1.setValue('discovery_source_duplicate_ci', 'Manual Entry');
		dup1.insert();
		
		var dup2 = new GlideRecord('duplicate_audit_result');
		dup2.initialize();
		dup2.setValue('duplicate_ci', grCI.getValue('duplicate_of'));
		dup2.setValue('duplicate_id', grCI.getValue('duplicate_of'));
		dup2.setValue('table', grCI.sys_class_name);
		dup2.setValue('follow_on_task', dupTask.getUniqueValue());
		dup2.setValue('discovery_source_duplicate_ci', 'Manual Entry');
		dup2.insert();
		
		return dupTask;
	},

[Carolyn Rast]

You can manually create a de-duplication task:

// where <sys-id1> and <sys-id2> are sys_IDs of CIs in the cmdb_ci table
var sysIDs = '<sys-id1>,<sys-id2>';
var dupTaskUtil = new CMDBDuplicateTaskUtils();
var deDupTaskID = dupTaskUtil.createDuplicateTask(sysIDs);
gs.info(deDupTaskID);

SNow Reference: https://docs.servicenow.com/bundle/sandiego-application-development/page/app-store/dev_portal/API_reference/CMDBDuplicateTaskUtilsAPI/concept/CMDBDuplicateTaskUtilsAPI.html