Is there a way to use the IRE for Reconciliation without using it for Identification?

[Greg Hoschar]

We are trying to update an existing JDBC Import from SolarWinds to leverage the IRE so that we can include and prioritize multiple data sources to build our CMDB. The problem is some incoming nodes share the name names or serial numbers, leading to overloaded CIs.

• My company does not have any ServiceNow Discovery licensing and will not procure it for the foreseeable future. No solution requiring a Discovery license (such as Service Graph Connector for SolarWinds) can be used at this time.
• We do not leverage IntegrationHub ETL because of concerns about the additional costs associated with IntegrationHub.
• We want separate CIs for each node in the data source. In several cases the overloaded serial number is used for multiple virtual nodes that the infrastructure teams leverage explicitly for Change Records.
• The serial numbers are populated in the data source via SNMP query and will not be altered manually.
• The previous coalesce field is Node ID. A sequential value that only has meaning in this data source.

We are having trouble finding a solution that leverages the IRE yet will create a record when an incoming node from this data source has an existing name or serial number.

We basically want to use the IRE for reconciliation of data across multiple data sources, yet use the old method of coalescing on Node ID for identification of the record.

Does anyone have any recommendations?

[Antoine Bonhomm]

Hi Greg,

Have you ever try to add a new Identification Rule in CI Class Manager for the dedicated class ?

This new Identification rule will be based on your Node ID attribute with an higher priority than Name and Serial Number rules.

After this operation, I think you could identify the right CI with IRE.

Have a nice day !

[Greg Hoschar]

Thank you, Antoine. I have considered that and a higher priority identification rule based on Node ID does allow the IRE to identify the correct CI if it exists.

Unfortunately it does not allow for new nodes with the same serial number if they do not already exist.

For instance:
Let's say CI#5 has name "X" with serial number 1234 and Node ID 5678. We also have an identification rule at priority 150 (before the serial # rule at 200) that matches on Node ID. All the other Identification Rules are default OOB Hardware class Identification Rules.

Now we introduce a new virtual device Y with serial number 1234 and Node ID 5679. When it comes into the IRE:
Priority 100, it looks for serial number & serial type. No match. Next rule.
Priority 150, it looks for NodeID = 5679. No match. Next rule.
Priority 200, it looks for serial number 1234. Found. IRE says this device matches CI#5.
CI#5 is updated. Name becomes "Y" and Node ID becomes 5679.

When device X is next processed, there is no Node ID 5678, so it matches CI#5 on the priority 200 serial number rule. The values for CI#5 will flip flop between the import records and whichever is processed last wins.

This is why I seek a way to ignore the other identification rules for this particular import and match only on Node ID. If the Node ID does not exist in the CMDB, I want to create a new CI. I know this is not best practice and it will produce duplicates, but we knowingly make this compromise in order to have those devices available for ITSM tasks.

[Antoine Bonhomm]

Thanks Greg for your additional informations, I've a better view of the situation now.

I understand all CIs, physical or virtual, are on the same CI class (Hardware), right ?

Could you add a child class ?

You say " ignore the other identification rules for this particular import " but maybe it's "use a specific rule for a particular class"

[Greg Hoschar]

Most are Network Gear or children thereof, but some belong to other CI classes. In any case, we don't want to remove the serial number identification rule completely.

The whole reason we are leveraging the IRE is to use other data sources and be able to prioritize them. I would prefer to use serial number for identification wherever possible, though this will likely lead to more issues if other data sources cannot find the correct CI.

Since originally posting this thread, I have discovered the Source [sys_object_source] table and believe it may be the key to conditionally bypassing the IRE. If we search the Source table by Discovery Source and Source Native Key and no value turns up, it means that this combination has not been seen by the IRE. In that case, I will bypass the IRE, insert the CI per the old pre-IRE transform map, and then manually insert a Source table record for the Discovery Source, Source Native Key, and newly inserted CI. That way we can leverage the IRE for prioritized updates, but new devices from this data source will create a fresh CI.

The problem is that all references I have found leverage IntegrationHub-ETL to set Source Native Key. A  simple Transform Map based IRE implementation does not supply Source Native Key, so the entries show the Discovery source in both Name and ID columns of the Source record and prevents us from using the Source table as described above. I think we may be able to use CMDBTransformUtil.getPayload() and setPayload() to manually insert the value, but there are concerns about the impact of juggling large JSON payloads for thousands of records per run.

Does anyone know of a method to specify Source Native Key in a Transform Map onBefore script without manipulating the payload or using IH-ETL?

We are currently working with the owners of the SolarWinds instance to see if there are other device based identifiers available that we can leverage to help distinctly identify the CI. If this effort doesn't lead anywhere, we may have to explore the Source table based solution.