Managing CMDB data deletion
The Integration Commons for CMDB (sn_cmdb_int_util) store app uses post-script and transform map methods to remove imported Configuration Management Database (CMDB) records that were deleted later from the source third-party application.
Post-script method
The post-script method checks the Source [sys_object_source] table for configuration items (CIs) not updated in the last run for a specific source feed. It then removes any non-updated CIs from the associated target CMDB tables synchronously, handling the removal of target records one at a time.
Most Service Graph Connectors use the post-script method to manage the removal of any CMDB records.
The post-script method uses the RecordRemoval script to handle various record deletion strategies. Hard deletion involves permanently and completely removing records from the CMDB, making them unrecoverable and suitable for data that is no longer needed. Soft deletion marks records as deleted without actually removing them from the CMDB, allowing for potential recovery and preserving historical data. Optimized soft deletion, managed by the Identification and Reconciliation Engine (IRE), enhances this soft deletion approach by using a specialized table, such as the extension attributes table, and applying reconciliation rules to manage deleted records more efficiently. By default, the table can hold up to 100 records at a time, but as a user with the admin role, you can adjust this limit by setting the sn_cmdb_int_util.ire_payload_items_size system property. This property is not available by default and you must create this property of type integer within the Integration Commons for CMDB (sn_cmdb_int_util) store app. For more information, see Add a system property.
Transform map-based method
The transform map-based method checks the Source [sys_object_source] table for configuration items (CIs) not updated in the last run for a specific source feed. It then utilizes the multi-threaded deletion approach to remove any non-updated CIs from the associated target CMDB tables concurrently, handling the removal of all target records within an import set at once. Multiple import sets are created to store target record data within the Integration Commons Remove Record [sn_cmdb_int_util_remove_record] staging table enabling multi-threaded deletion of records. Therefore, reducing the data processing time.
The Integration Commons Remove Record [sn_cmdb_int_util_remove_record] staging table tracks non-updated CIs by storing target record details including the discovery source, source feed, target table sys_id, and target table name of these CIs. The on-start script defined within the Remove Record Transform Map then deletes the records from the associated target tables.
The transform map-based method is used by a Service Graph Connector to improve the data processing time. For example, the SG-Tanium Remove Software data source within the Service Graph Connector for Tanium removes software data using the transform map-based method. See Service Graph Connector for Tanium.