Duplicate Records in CMDB

[mburnham]

We have what appears to be multiple duplicate records in our CMDB in a particular class.  There exists a Standard Identification rule in place with the criterion of "Name."  

Why would we have duplicate records with the same name with the Identification rule in place?

Duplicate remediation tasks are not being generated and Discovery is not recognizing duplicate records exist.  How would these existing records be merged or deleted etc.?

[mburnham]

Currently we only have one MID server on a discovery schedule for this class so as near as I can tell only one source.  I'm told every CI should be going through the IRE.

[mburnham]

Currently we only have one MID server on a discovery schedule for this class so as near as I can tell only one source.  I'm told every CI should be going through the IRE.

[Harish Bainsla]

Hi @mburnham how many data source is there to populate ci  and check weather there are IRE proper defined because It means your data does not satisfy any of the identification criteria.

To avoid duplicates, please review your incoming data and try to identify unique parameter(s). Create/modify your rules accordingly to uniquely identify existing records.

for more check below link

https://support.servicenow.com/kb?id=kb_article_view&sysparm_article=KB0869861

if my answer helps you mark helpful and accept solution

[mburnham]

sorry for the multiple replies - I didn't think they actually submitted.

[AJ-TechTrek]

HI @mburnham ,

Lets discuss 1st Why duplicates can still appear despite Identification Rule

Even when you have an Identification Rule (e.g., using Name):

* ServiceNow Discovery / Integration uses that rule only if:
* The incoming payload includes the identification attributes (here: Name)
* Possible causes:


1.The duplicate CIs came in from different data sources (e.g., manual, import set, integration, discovery).


2. The Name field value in the payload was missing, different, or had leading/trailing spaces, special characters, or casing differences (web01 vs. Web01).


3. The Identification Rule might have:
* Conditions that only apply when certain other fields are present.
* Not covering some edge cases (e.g., when Name is null, it creates new CI).


4. Manual creation: someone created CIs manually instead of via discovery.

Steps to fix & prevent duplicates:

Step 1: Validate Identification Rule
* Go to: Identification Rules → open the rule for that class.
* Check:
* Name is in the "Mandatory" section.
* No conflicting rules.
* Confirm that the rule covers all relevant scenarios.

Step 2: Analyze the duplicate data
* Compare duplicates:
* Are the Name fields identical?
* Same casing, no spaces?
* Check:
* Discovery source
* sys_created_by / sys_updated_by
* Timestamps
To identify which integration or discovery created them.

Step 3: Enable / tune Duplicate Detection
* Activate and configure:
* Duplicate CI Detection plugin (com.snc.ci.duplicate).
* Configure duplicate detection rules for that class.
* This lets the system generate duplicate remediation tasks.
If tasks aren’t being generated:
* Maybe the plugin isn’t active.
* Or the rule isn’t configured properly.

Step 4: Remediation - merge or delete
Once you identify true duplicates:
* Use Duplicate CI Remediator to:
* Compare field-level differences.
* Merge records safely.
* Or manually merge/delete:
* Keep the oldest or most accurate CI.
* Re-link related child CIs and relationships.
Don’t just delete — always merge to keep relationship integrity.

Step 5: Strengthen prevention
* Make Name a required field on data entry forms.
* Review discovery patterns or probes:
* Ensure Name is always populated and correct.
* Add additional identifiers (Serial Number, IP, etc.) to the Identification Rule if applicable.

Step 6: Ongoing governance
* Monitor CMDB Health dashboard → Duplicate Compliance score.
* Set up scheduled duplicate detection jobs.
* Educate data entry users not to manually create CIs for discovered classes.

Please appreciate the efforts of community contributors by marking appropriate response as Mark my Answer Helpful or Accept Solution this may help other community users to follow correct solution in future.
 
Thank You
AJ - TechTrek with AJ
LinkedIn:- https://www.linkedin.com/in/ajay-kumar-66a91385/
YouTube:- https://www.youtube.com/@learnitomwithaj
ServiceNow Community MVP 2025