CI Identifier only looks at first Priority attribute to create a de-duplication task?

EikeK · ‎08-11-2023

Hi all,

I have a question regarding the prioritization of CI Identifier entries for creating de-duplication tasks. I am not using discovery, but I want to use the CMDB View Dashboard to show me CMDB data quality.

I am running into an issue with using multiple identifier entries on one class.

On the Hardware table I have following 3 attributes:
Prio 1: Name

Prio 2: Serial Number

Prio 3: MAC Address

I created two hardware CIs with different name, serial number but with a matching MAC address. When I run the correctness job no de-duplication task is created. However if I switch the priority of Name & MAC address (so that MAC is now Prio 1) and re-run the Job it will create a de-duplication task.

This is happening on my PDI as well as on customers instance.

I already tried setting the same priority on all attributes, that didnt help.

Is that working as expected or is there something I missed?

Thanks & Best Regards

Fabian Kunzke · ‎08-11-2023

Hey,

This is indeed intended and the way identifiers would work.

If you have multiple Identifiers, the whole process will look at first identifier and determine, if it finds a match (in your first example that would be the name). If it finds one match, great. We are done.

If no match is found, well, onto the next identifier rule (the serial number). Found one match? Great, we are done.
If not, onto the next one.

The de-duplication tasks are created in the same way. Yes, you do have duplicate MAC addresses, but it does not matter to the degree that the identification rule would deem it a duplicate (as it stops after the first match). So for the first rule set, the duplicate isn't "there" as the name is unique.

Changing the priority to have the mac address first will lead to "oh, i found 2 CIs here!". Therefore, a de-duplication task is created.

This is best described here: https://support.servicenow.com/kb?id=kb_article_view&sysparm_article=KB0726425

The duplicate detection uses the IRE engine which in return utilizes the rulesset as described above. Whenever it finds exaclty one match, there is no need to run through the other identifier rules.

NOTE (this is from my own experience, i don't know if this is a general "issue"): For some reason imports using the IRE (e.g. the discovery) WILL run through all of the rules. I don't know why, but it makes the duplication detection a bit weird. The discovery will create a de-duplication task whenever ANY rule matches more than one CI even if the identification rules result in exactly one match.

I personally prefer the duplicate detection from the CMDB Health and therefore deactivate the option that the discovery generates duplicates...

If this answers your question make sure to mark it as correct/helpful. This helps other to find solutions more quickly when searching the community.

Regards

Fabian

View solution in original post

Fabian Kunzke · ‎08-11-2023

Hey,

This is indeed intended and the way identifiers would work.

If you have multiple Identifiers, the whole process will look at first identifier and determine, if it finds a match (in your first example that would be the name). If it finds one match, great. We are done.

If no match is found, well, onto the next identifier rule (the serial number). Found one match? Great, we are done.
If not, onto the next one.

The de-duplication tasks are created in the same way. Yes, you do have duplicate MAC addresses, but it does not matter to the degree that the identification rule would deem it a duplicate (as it stops after the first match). So for the first rule set, the duplicate isn't "there" as the name is unique.

Changing the priority to have the mac address first will lead to "oh, i found 2 CIs here!". Therefore, a de-duplication task is created.

This is best described here: https://support.servicenow.com/kb?id=kb_article_view&sysparm_article=KB0726425

The duplicate detection uses the IRE engine which in return utilizes the rulesset as described above. Whenever it finds exaclty one match, there is no need to run through the other identifier rules.

NOTE (this is from my own experience, i don't know if this is a general "issue"): For some reason imports using the IRE (e.g. the discovery) WILL run through all of the rules. I don't know why, but it makes the duplication detection a bit weird. The discovery will create a de-duplication task whenever ANY rule matches more than one CI even if the identification rules result in exactly one match.

I personally prefer the duplicate detection from the CMDB Health and therefore deactivate the option that the discovery generates duplicates...

If this answers your question make sure to mark it as correct/helpful. This helps other to find solutions more quickly when searching the community.

Regards

Fabian

EikeK · ‎08-11-2023

Hi Fabian,

Thank you for the extensive explanation, much appreciated.

Do you know from your experience if there is somehow a way to still check for multiple attributes? So even if the Name of the CI is unique, I would still need to search all entries if there is another entry with the same MAC address for example?

Thanks again & Best Regards

Fabian Kunzke · ‎08-14-2023

Hello,

As far as i know, no. There is no property that would change this behaivor. If you find it, please let me know, i have been looking for it for a while now.

Regards

Fabian