IRE reconciliation rules on child tables override rules on parent tables - how do you overcome this?

[mbourla]

I’m looking for feedback from other people who use the IRE (Identification & Reconciliation Engine) to manage data feeds into their CMDB.  Specifically to overcome an issue we’ve been having with as-designed IRE behaviour, around reconciliation rules on child tables always overriding rules on parent tables.

One of the “features” of reconciliation rules is that a rule on a child class/table overrides a rule on a parent class/table, regardless of the relative priorities, or and regardless of whether the CI in question is even coming in on the child rule’s feed.  It’s described here. 

That has been causing issues for us, where feeds were being blocked from updating fields that we expected them to be able to update.  Below is an actual example.  At the time I raised a HI case with ServiceNow, and they clarified why we were seeing that behaviour.

We’ve been thinking about how to avoid this problem.  The solution we have arrived at is to have rules on the tables where fields are actually defined.  For example, instead of one rule on cmdb_ci_computer for Attributes of Name, Serial number, Operating system, and CPU count, there would be two rules:

• On cmdb_ci_computer : for Operating System, CPU count
• On cmdb_ci : for Name, Serial number

This way, the same attribute is never listed on reconciliation rules on different tables in the class hierarchy, since the same attribute would always be on the table where it is defined (we have used cmdb_ci instead of cmdb).

But this isn’t how the Docs pages describe the use of Reconciliation rules, or how you’d naturally want to have them set up.  And I can’t believe all ServiceNow users have had to do this.

So if you use IRE and have defined reconciliation rules, I’d be really interested to know if you’ve had this same problem, and how you’ve got around it.

Many thanks!

Michael

Example situation

Here is an actual example situation we had earlier this year, involving 2 feeds from data sources called “SnowSoftware” and “Network Inventory”:

• Feed #1 : from source “SnowSoftware”, primarily bringing in computers and servers.  With a Reconciliation Rule on cmdb_ci_server at Priority 100.  Attributes list includes Name.
  
  
• Feed #2 : from source “Network Inventory”, bringing in various types of network device.  With a Reconciliation Rule on cmdb_ci_hardware at Priority 200.  Attributes list also includes Name.

Feed #2 tried to update Name on a CI that was not coming in from SnowSoftware.  The CI was of class Network Appliance Hardware.  But since Network Appliance Hardware extends from Server, and there is a Reconciliation Rule on Server for SnowSoftware, that takes precedence over the “Network Inventory” reconciliation rule, because rules on child tables override rules on parent tables.  That is despite that particular CI only coming in from “Network Inventory” and it will never come in from “SnowSoftware”.

Network Appliance Hardware	<== Target CI class, for CI from Network Inventory feed
Server Hardware
Server	<== SnowSoftware rule here, overrides Network Inventory rule
Computer
Hardware	<== Network Inventory rule here
Configuration Item

[Anshu_Anand_]

Some queries

1. Why not using network gear table for network devices? Its sibling of server table and you can create two separate reconciliation rules 1 on server and other on network Gear and no one will interfere each other.

2. If you want to use Network Appliance Hardware you had to create reconciliation rule on only server and Network Appliance Hardware. Remove reconciliation rule from hardware table .

3. I believe some CIs are common in both data sources other wise, the child rule always override parent rule.

4. Have you defined Data refresh rules? This is important when you define multiple data sources for same CI.

The overall IRE reconciliation is controlled from cmdb_datasource_last_update table.

You need to check this table .

Clear the table in your DEV environment. fresh run the data sources and observe as per the rule goes. You will get the clarity how the precedence is taking over . 

Last update and Data source dictate which CI field will be updated as per the priority and data refresh rule(if defined)

Hope it helps

[mbourla]

Hi Anshu, and thanks for coming back so quickly.

That was just an example. That "Network Inventory" feed can populate CIs on quite a number of different tables, both under Server and under Network Gear. So at the time we built it, the obvious place to put the Reconciliation rule was on Hardware, rather than on all the different possible target CI classes.

Yes, if we'd known that it might clash with rules for other discovery sources, in hindsight we could have put the rules on other tables.

But that's the whole point... it's not ideal to have to define the reconciliation rules for each discovery source, taking into account the requirements of all the other sources.

We currently have perhaps 10 feeds into the CMDB, and in time that will almost certainly keep growing. So each time we build a new feed, we would have to go back and possibly adjust or refactor the rules for the all the other existing feeds too. Which seems a great deal of work to keep them all under control otherwise, and not an ideal situation.

You mentioned the cmdb_datasource_last_update table. Yes, I always thought that was how reconciliation would decide whether to allow a CI field to be updated, just based on the priority of the current source vs the priority of the source that last updated that field. But as ServiceNow explained to me in response to my earlier support case, if there is a reconciliation rule on a child table, that *immediately* stops the current source from updating the fields in common with the parent rule - it never even gets as far as looking at cmdb_datasource_last_update.

Which is why we've decided to always put Attributes on reconciliation rules on the tables where each attribute is defined in the CMDB schema. Then this parent table vs child table issue won't arise, and it will indeed always go by cmdb_datasource_last_update (plus the Data Refresh rules, as you say).

So that was why I wanted to find out how other customers handle this situation. Knowing that some customers will have many more feeds into their CMDB than we do currently. In case we're doing something that isn't necessary, or there's a better way to do it.

If you can share how you do this yourself, that would be useful.

Regards
Michael

[Anshu_Anand_]

For my case, i had 4 data sources, 3 for computers, 1 for network gear table

So separate reconciliation rule for each source.

Its best to define Each data source in reconciliation rule if the CI has incoming data from multiple sources.

For example , i wanted discovery as priority 1 and altiris as 2 to update the fields for that CI .

It only make sense when , for same CI there are multiple sources of data bases. 

Why you not create all reconciliation rule on hardware table only and in condition filter by class

when defining priority?

[mbourla]

Yes that's basically what we've done.  In your case it worked to have them all on hardware.  Our feeds update a large number of different classes.  So the equivalent is to have all the reconciliation rules on the tables where the fields are defined.  So rules on cmdb_ci for fields  like Name and Serial Number that are defined on cmdb_ci.  And rules on cmdb_ci_computer for fields like Operating System and CPU Count that are defined on cmdb_ci_computer.  And so on.  Which works, but it's not what the documentation indicates should be needed.  That's why I was keen to find out how other people had set up their reconciliation rules, to overcome this parent-overriding-child behaviour.  

Many thanks for your input - much appreciated.