Unexpected Outcomes when Updating Relationships with IH-ETL

[DavidG796156893]

We recently encountered a challenging situation while trying to automate the maintenance of CMDB relationships.

Relationships are core to a CMDB, providing significant value by indicating, for example, which IT infrastructure enables which Business Application Service Instances.

Relationships can be populated through several mechanisms. Although manual entry is possible, automated options are preferable wherever possible. Relationship data can be collected from requesters at the time of provisioning, for example, “Please select which Service Instance this new server is for”; from feeds, such as external provisioning systems; or from ServiceNow Discovery

The issue we encountered arose when ingesting relationships through IH-ETL. We initially thought this would be an easy, low-code, out-of-the-box mechanism.

However, a relationship is not a CI! It has no identification rules and is not in a cmdb_ci subtable. As a result, adding or removing relationships is only possible in the context of a CI. The challenge is that the parent and child CIs must be submitted as records to IH-ETL. IH-ETL does not support a lookup-only type of transaction. This means that even when the goal is only to insert a relationship, the parent and child CIs are also updated.

At a minimum, this is a problem because ServiceNow shows that the parent and child CIs have been updated; for example, the “last discovered” and “discovery source” fields are updated. At worst, we encountered an issue where a parent’s third-party unique ID had changed. We expected the submitted record to create a relationship to the new parent CI, but instead the relationship stayed with the existing parent and the parent’s unique ID was updated. This occurs because IH-ETL maintains a source_native_key mapping.

To help prevent this in the future, we used reconciliation rules to make our feeds the lowest possible priority for updating the parent. However, this does not prevent the parent’s “last discovered by” and “last updated by” fields from being updated.

An alternative workaround would be to remove the parent or child CIs from the IH-ETL data and write a custom post-processing business rule that triggers after IH-ETL processing to directly insert or delete relationships using Glide. This is an unfortunate outcome when the goal is to remain out of the box and low code.

We are now working on a more general API for external systems to submit CMDB relationships. For this effort, we are bypassing the IRE and using traditional Glide APIs, along with the Suggested Relationships table, as a check for which valid combinations can be submitted.

I’d love to hear of any other interesting experiences others have automating and putting governance around relationships in the CMDB.

[Kieran Anson]

The challenge is that the parent and child CIs must be submitted as records to IH-ETL. IH-ETL does not support a lookup-only type of transaction

The API provides a query endpoint / function that allows you to query to understand what operation will occur (insert Vs update) along with extensive metadata. Is this not sufficient? 

At a minimum, this is a problem because ServiceNow shows that the parent and child CIs have been updated; for example, the “last discovered” and “discovery source” fields are updated

When passing a payload, you can define settings on what changes can be made. One of these is whether the last discovered timestamp is updated. 

Also the following is useful on why the last discovered field has lost some of its value / important / purpose 

https://support.servicenow.com/kb?id=kb_article_view&sysparm_article=KB1893055