Can I recreate Duplicate Task for duplicate CIs?

[Henrik Jutterst]

I have some ~20 CIs in the cmdb_ci_pc_hardware class, that are marked as duplicate and have a reference to master CI, using the duplicate_of field that's pointing to the correct master.

However, the Duplicate task to handle these CIs seems to be missing and I'm not sure how I can recreate it.

glide.identification_engine.skip_duplicates = true
glide.identification_engine.skip_duplicates.threshold = 5

Default settings for both, but the Scheduled CMDB Health Jobs does not create a new Duplicate Task, and I can't find an old one where the name of a duplicate CI is referenced.

Question: What is the default way to move forward in this case? 
I want the user to have a Duplicate Task to work on and remediate the Duplicate CIs

I've been reading up on these articles:
Detecting duplicate CIs

How does the Identification and Reconciliation Engine Work?

Duplicate CIs

Also, what happens to the ~20 CIs that I have since threshold value is 5? Is it correct that I need to edit the threshold to ~30 in order to get a Duplicate task for these...?

Script to find duplicates by fqdn:

//Find duplicate fqdn
var grCI = new GlideAggregate('cmdb_ci')
var dupRecords = []
	grCI.addQuery('active','true');
	grCI.addAggregate('COUNT', 'fqdn');
	grCI.groupBy('fqdn');
	grCI.addHaving('COUNT', '>', 1);
	grCI.query();
	
	while (grCI.next()) {
		dupRecords.push(grCI.fqdn.toString());
	}
	gs.info(dupRecords);

I also created a script to manually set correct main ci:

////// Mark CIs as duplicate of another CI
try{
	var queryString = ''; //<--- set encoded query to the CIs to be marked as duplicate of
	var grCI = new GlideRecord('cmdb_ci');
	grCI.addEncodedQuery(queryString);
	grCI.query();

	while(grCI.next()){
		grCI.duplicate_of = ''; //sys_id of master CI
		grCI.update();
	}
}

catch(ex){
	gs.info('Exception from background script: ' + ex);	
}

[Anshu_Anand_ · Accepted solution]

You can create a duplicate task of your own

use this script

// where <sys-id1> and <sys-id2> are sys_IDs of CIs in the cmdb_ci table

var sysIDs = '<sys-id1>, <sys-id2>';
var dupTaskUtil = new CMDBDuplicateTaskUtils();
var deDupTaskID = dupTaskUtil.createDuplicateTask(sysIDs);
gs.info(deDupTaskID);

It will create the de-duplicate task, there you can select the master ci and OOTB workflow will take care of rest

https://docs.servicenow.com/bundle/rome-application-development/page/app-store/dev_portal/API_refere...

Mark answer as correct if it helps

[Henrik Jutterst]

Thank you so much @Anshu  That was exactly what I was looking for.

The link you sent had a very good script to mass create Duplicate tasks. Here's my version of it. Basically I just modified the filter and what CI type to look for.

/**
 * CMDBDeduplicateTaskUtil().createDeduplicateTask() example
 * 
 * OVERVIEW
 * 
 * This script will allow you to search the CMDB
 * for records that may be duplicates based on the field
 * and filter definitions of your chosing
 * 
 */

try {

    //SET INITIAL VARIABLES

    //Set the table that you would like to check for duplicates - note this MUST be a CMDB table
    var cmdbTable = 'cmdb_ci';

    //Add a filter as an encoded query if desired - only objects within this filter will be matched
    // this query filters PC Hardware CIs are not Retired 
    var filterQuery = 'sys_class_name=cmdb_ci_pc_hardware^install_status!=7';

    //Which field shall we use to check for duplicates?
    var matchField = 'name';

    //Do matches need to have the same cmdb_ci class? - This prevents child device types from matching if enabled i.e. Computer, Server
    var sameClass = true;

    //Should empty values be excluded from match?
    var excludeEmpty = true;

    //Minimum number of duplicates required to create a de-duplication task - must be at least 2
    var numberOfMatches = 2;


    //USE GLIDE AGGREGATE TO IDENTIFY POSSIBLE DUPLICATES

    var dupeSetAgg = new GlideAggregate(cmdbTable);
    if (filterQuery) dupeSetAgg.addEncodedQuery(filterQuery);
    dupeSetAgg.addAggregate('COUNT', matchField);
    dupeSetAgg.orderByAggregate('COUNT', matchField);
    dupeSetAgg.groupBy(matchField);
    if (sameClass) dupeSetAgg.groupBy('sys_class_name');
    if (excludeEmpty) dupeSetAgg.addNotNullQuery(matchField);

    //Group by domain if domain separation is enabled (prevent matches across domains)
    if (gs.getProperty('glide.sys.domain.partitioning') == 'true') dupeSetAgg.groupBy('sys_domain');

    //Must have at least 2 CI records that match
    if (numberOfMatches < 2) numberOfMatches = 2;
    dupeSetAgg.addHaving('COUNT', matchField, '>=', parseInt(numberOfMatches));

    dupeSetAgg.query();


    //LOOP THROUGH EACH SET OF MATCHES

    while (dupeSetAgg.next()) {
        var dupeRecordCount = dupeSetAgg.getAggregate('COUNT', matchField);
        var dupeRecordsList = [];
        var dupeAggQuery = dupeSetAgg.getAggregateEncodedQuery();
        var dedupeShortDesc = 'Found ' + dupeRecordCount + ' records where ' + matchField + ' = ' + dupeSetAgg.getValue(matchField) + '\n';
        var dedupeDescription = '';

        gs.log(dedupeShortDesc, 'Dedupe Task Script');

        //Prepare notes - Header row
        dedupeDescription += 'NAME | CREATED ON | UPDATED ON'


        //GATHER ADDITIONAL DEVICE DETAILS (NEED SYS_IDs)

        var dupeDevices = new GlideRecord(cmdbTable);
        dupeDevices.addEncodedQuery(dupeAggQuery); //Use the exact query that the aggregate used for this group
        dupeDevices.query();

        while (dupeDevices.next()) {
            var devName = dupeDevices.getValue('name');
            var devCreated = dupeDevices.getDisplayValue('sys_created_on');
            var devUpdated = dupeDevices.getDisplayValue('sys_updated_on');
            var devId = dupeDevices.getUniqueValue();

            //Add CI to notes
            dedupeDescription += '\n' + devName + ' | ' + devCreated + ' | ' + devUpdated;

            //Add sys_id to list of matched records for this group
            dupeRecordsList.push(devId);
        }
        gs.log(dedupeDescription, 'Dedupe Task Script');


        //CREATE DEDUPLICATION TASK

        //Convert array to string - .createDuplicateTask() expects a comma separated string
        var deviceIdString = dupeRecordsList.join();

        var dedupeUtil = new CMDBDuplicateTaskUtils();
        var dedupeTaskId = dedupeUtil.createDuplicateTask(deviceIdString);


        //UPDATE DEDUPLICATION TASK - ADD DETAILS

        if (dedupeTaskId) {

            var dedupeTask = new GlideRecord('reconcile_duplicate_task');
            dedupeTask.get(dedupeTaskId);
            dedupeTask.setValue('short_description', dedupeShortDesc);
            dedupeTask.work_notes = dedupeDescription;
            dedupeTask.update();

            gs.log('>>>Created de-duplication task ' + dedupeTask.getValue('number') + ' for ' + devName, 'Dedupe Task Script');

        }

    }

} catch (er) {
    gs.log(er, 'Dedupe Task Script');

}

[Anshu_Anand_]

awesome !