Deduplication process run on software installation table doesn't update sys_updated_on field

[ho_jessjia_]

Hi, our teams perform scheduled data ingestion from ServiceNOW to Databricks based on the sys_updated_on field. However, for records in the software installation table, when deduplication process is run, the sys_updated_on field is not updated.

As a result, the latest deduplicated version of the record can't be ingested into Databricks.

To resolve this issue, as a mediator between ServiceNOW and Databricks side, I need more context to understand how this deduplication process works, and provide suggested solutions to both sides.

Can anyone provide me relevant information/knowledge/potential workaround regarding this? Thanks!

[fknell]

Hi @ho_jessjia_,

It depends on the action performed in the de-dub task. The default is to keep the older record and migrate information. In case you don't update attributes or migrate only relationships, you might end up with an unchanged older record and the new record would be deleted.

Hope this helps!

[ho_jessjia_]

Thanks for clarification, in such case, how you track whether a record undergoes deduplication process?

[fknell]

Hi @ho_jessjia_,

It should be an automated process either by Discovery or IRE. 

Here you find the feature explained Duplicate CIs remediation 

Hope this helps!