How are duplicate CI's identified?

[jaredscharf]

Are duplicate CI's identified as data is processed through the IRE? Or is there a job that runs daily that looks at the cmdb data and compares it to identification rules to identify duplicates? I'm asking because we have recently modified our identification rules, but there currently are many duplicate CI's/duplicate CI tasks in the CMDB. Ideally what I would like to do is delete the duplicate CI tasks table and have duplicate CI's identified under the new identification rules. What is the best way to do this?

[jaredscharf]

In the example above, if a deduplication task was created because the name matched, then why would Discovery not update the CI with the matching name instead of creating a new one? This is the exact point I am confused on. 

[jaredscharf]

What confuses me is why a duplicate CI would get created in the first place. If the purpose of identification rules is to prevent duplicate CI's, then how would a duplicate CI get created and then identified as a duplicate according to the same identification rules that allowed creation of that CI?

[Bhuvan]

@jaredscharf 

As long you are using Identification and Reconciliation Engine during data load or integrations or discovery, IRE engine will make sure duplicate CIs are not created in the system. Sometimes manual load and import sets are created without using IRE which would result in duplicate CIs and that would need to be remediated via De-duplication tasks.

You can refer below documentation link for more details on how IRE works,

https://www.servicenow.com/docs/bundle/zurich-servicenow-platform/page/product/configuration-managem...

De-duplication Tasks - CMDB Workspace

You can refer below 'get well playbook' on how to handle duplicate CIs in CMDB

https://noderegister.service-now.com/kb?id=kb_article_view&sysparm_article=KB0829103

I hope you appreciate the efforts to provide you with detailed information. As per community guidelines, you can accept more than one answer as accepted solution. If my response helped to guide you or answer your query, please mark it helpful & accept the solution.

Thanks,

Bhuvan

[jaredscharf]

Thanks Bhuvan- your response makes sense, but I still am confused on what generates the deduplication tasks? I get that duplicate CI's can get created through import sets not going through IRE or manually created CI's, but what is the trigger for a deduplication task to get created if CI's then?

[Bhuvan · Accepted solution]

@jaredscharf 

De-duplication tasks are generated via Scheduled Job that runs daily. See below for more details,

You can go to script includes 'DuplicateTaskDataPopulator' and check for more details on how de-duplication tasks are created and related tables used for logic and data population

I hope you appreciate the efforts to provide you with detailed information. If my response helped to guide you or answer your query, please mark it helpful & accept the solution.

Thanks,

Bhuvan