Managing CMDB data deletion
Summarize
Summary of Managing CMDB Data Deletion
The Integration Commons for CMDB (sncmdbintutil) provides methods for efficiently managing the deletion of Configuration Management Database (CMDB) records that have been removed from third-party applications. This process is essential for maintaining accurate and up-to-date CMDB data.
Show less
Key Features
- Post-Script Method: This method identifies configuration items (CIs) that have not been updated since the last run and removes them from the CMDB synchronously. It supports hard deletion (permanent removal) and soft deletion (marking records as deleted) strategies.
- Optimized Soft Deletion: Managed by the Identification and Reconciliation Engine (IRE), this approach improves soft deletion efficiency by utilizing a specialized table and reconciliation rules. Admins can configure the maximum record limit using the sncmdbintutil.irepayloaditemssize property.
- Transform Map-Based Method: This method checks for non-updated CIs and employs a multi-threaded deletion approach to remove records concurrently, thereby reducing processing time. The Remove Record staging table tracks these records for efficient deletion.
Key Outcomes
By utilizing these methods, ServiceNow customers can ensure that their CMDB remains current and accurate, streamline the data deletion process, and improve overall data management efficiency. The ability to choose between hard and soft deletion strategies allows for flexibility based on data retention needs.
The Integration Commons for CMDB (sn_cmdb_int_util) store app uses post-script and transform map methods to remove imported Configuration Management Database (CMDB) records that were deleted later from the source third-party application.
Post-script method
The post-script method checks the Source [sys_object_source] table for configuration items (CIs) not updated in the last run for a specific source feed. It then removes any non-updated CIs from the associated target CMDB tables synchronously, handling the removal of target records one at a time.
Most Service Graph Connectors use the post-script method to manage the removal of any CMDB records.
The post-script method uses the RecordRemoval script to handle various record deletion strategies. Hard deletion involves permanently and completely removing records from the CMDB, making them unrecoverable and suitable for data that is no longer needed. Soft deletion marks records as deleted without actually removing them from the CMDB, allowing for potential recovery and preserving historical data. Optimized soft deletion, managed by the Identification and Reconciliation Engine (IRE), enhances this soft deletion approach by using a specialized table, such as the extension attributes table, and applying reconciliation rules to manage deleted records more efficiently. By default, the table can hold up to 100 records at a time, but as a user with the admin role, you can adjust this limit by setting the sn_cmdb_int_util.ire_payload_items_size system property. This property is not available by default and you must create this property of type integer within the Integration Commons for CMDB (sn_cmdb_int_util) store app. For more information, see Add a system property.
Transform map-based method
The transform map-based method checks the Source [sys_object_source] table for configuration items (CIs) not updated in the last run for a specific source feed. It then utilizes the multi-threaded deletion approach to remove any non-updated CIs from the associated target CMDB tables concurrently, handling the removal of all target records within an import set at once. Multiple import sets are created to store target record data within the Integration Commons Remove Record [sn_cmdb_int_util_remove_record] staging table enabling multi-threaded deletion of records. Therefore, reducing the data processing time.
The Integration Commons Remove Record [sn_cmdb_int_util_remove_record] staging table tracks non-updated CIs by storing target record details including the discovery source, source feed, target table sys_id, and target table name of these CIs. The on-start script defined within the Remove Record Transform Map then deletes the records from the associated target tables.
The transform map-based method is used by a Service Graph Connector to improve the data processing time. For example, the SG-Tanium Remove Software data source within the Service Graph Connector for Tanium removes software data using the transform map-based method. See Service Graph Connector for Tanium.