Components and process of Identification and Reconciliation
Summarize
Summary of Components and process of Identification and Reconciliation
The CMDB Identification and Reconciliation functionality in ServiceNow Yokohama release is powered by the Identification and Reconciliation Engine (IRE), along with various rules and tasks. This functionality ensures that Configuration Items (CIs) are uniquely identified and reconciled to maintain data integrity in the CMDB by preventing duplicates and controlling authoritative data updates.
Show less
Key Components and Processes
- Identification: Determines whether a CI already exists in the CMDB or if it is new, using identification rules or unique IDs from data sources.
- Reconciliation: Controls updates to the CMDB by allowing only designated authoritative data sources to write CI and attribute data. This process applies only during updates, not for new inserts, and uses reconciliation and IRE data source rules.
- De-duplication Tasks: When duplicates are detected during identification and reconciliation, they are grouped into tasks for review to understand duplication causes.
- Reclassification Tasks: If a CI matches but requires a class change (upgrade, downgrade, or switch), and automatic reclassification is disabled, a task is created for manual review and decision.
Identification and Reconciliation Engine (IRE)
IRE is the core rule-based engine that processes incoming CI data using identification, reconciliation, and data source rules. It centralizes and enforces consistent CI identification and reconciliation across diverse data sources, preventing duplicates and ensuring only authoritative sources update the CMDB.
APIs for Identification and Reconciliation
ServiceNow provides several scriptable APIs for integration and enforcement of identification and reconciliation before data is stored in the CMDB:
- createOrUpdateCI(): Creates or updates a CI based on IRE rules, committing changes to the CMDB.
- identifyCI(): Simulates create or update actions without committing data, useful for validation.
- createOrUpdateCIEnhanced(): Supports enhanced IRE features such as partial payloads, partial commits, and deduplication, with automatic enabling of deduplication and partial commit when partial payloads are used.
- identifyCIEnhanced(): Similar to createOrUpdateCIEnhanced but does not commit changes, used for pre-commit validation.
- CMDBTransformUtil: Specifically designed for Import Sets to apply identification and reconciliation processes to imported data.
These APIs are accessible in scoped applications via the Configuration Management For Scoped Apps (CMDB) plugin, enabling centralized and consistent data management.
Customization and Rule Inheritance
Predefined identification and reconciliation rules exist for many base system tables and can be customized to fit organizational needs. New CMDB tables inherit rules from their parent tables unless rules are explicitly created at the child level or updated at the parent level.
Practical Benefits for ServiceNow Customers
- Ensures unique identification of CIs, preventing data duplication in the CMDB.
- Maintains authoritative control over CI updates, ensuring data accuracy and consistency.
- Enables automated and manual handling of CI reclassification and duplication scenarios.
- Provides APIs that integrate with various data sources and workflows, supporting flexible and controlled data ingestion.
- Allows customization of identification and reconciliation rules to align with organizational requirements.
The CMDB Identification and Reconciliation functionality is supported by the Identification and Reconciliation engine (IRE), rules, and tasks. Identification rules, reconciliation rules, IRE data source rules, de-duplication tasks, and reclassification tasks determine how IRE identifies and reconciles CI.
Concepts and Components of Identification and Reconciliation
- Identification
- Process of uniquely identifying CIs, to determine if a CI already exists in the CMDB or if it is a newly discovered CI that must be added to the CMDB. Identification processes rely on identification rules, or on unique IDs for CIs that data sources can provide.
- Reconciliation
Process of reconciling CIs and CI attributes by allowing only designated authoritative data sources to write to the CMDB at the CI table and attribute level. The CMDB is updated in real time as records are being processed. There is no staging area to verify the reconciliation activities before they are committed. Reconciliation processes rely on reconciliation rules and IRE data source rules.
Reconciliation is required only for update operations, when the identification process identifies a CI in the CMDB that matches an incoming CI in the payload. When IRE inserts a new CI, reconciliation is not applied.
- De-duplication tasks
- If the instance encounters duplicate CIs during the Identification and Reconciliation process, it groups each set of duplicate CIs into a de-duplication task. Review the information in these tasks to see how it was determined that these CIs are duplicates.
- Reclassification tasks
- During the CI identification process, a matched CI might need to be upgraded, downgraded, or switched to another CI class. If automatic reclassification is disabled, then the system generates a reclassification task. Review the information in these tasks, and decide whether a manual reclassification of the CI is appropriate.
- APIs
The Identification and Reconciliation APIs are a centralized set of APIs that can be used with different sources of data such as Discovery, Monitoring, or Import Sets. You can use it to enforce Identification and Reconciliation before data is stored in the CMDB. Data sources do not directly write to the CMDB. Instead, they call APIs first to ensure that the data being written does not introduce inconsistencies.
Identification engine APIs are accessible in scoped apps. The Configuration Management For Scoped Apps (CMDB) plugin (com.snc.cmdb.scoped) allows a scoped app in scripts to use the prefix 'sn_cmdb.IdentificationEngine.<method>' to access identification engine APIs. The Configuration Management For Scoped Apps (CMDB) plugin is activated in base systems.
- createOrUpdateCI(): A scriptable API that creates or updates a CI based on identification and reconciliation rules.
- identifyCI(): Similar to the createOrUpdateCI API, but does not commit the result to the database. Use this API with a given payload to find out if the identification engine will perform insert or update operations, without committing the operation.
- CreateOrUpdateCIEnhanced(): A scriptable API that provides the functionality of enhanced IRE features such as partial payload, partial commit, incomplete payload, and deduplication of payload items. You can select the enhanced features to use. However, if you enable partial payloads, then deduplication of payload items and partial commit are automatically enabled.
- identifyCIEnhanced: Similar to the createOrUpdateCIEnahnced API, but does not commit the result to the database. Use this API with a given payload to find out if the identification engine will perform insert or update operations, without committing the operation.
- CMDBTransformUtil: An API to be used exclusively with Import Sets to apply Identification and Reconciliation processes to data imported by Import Sets.
Predefined identification are included for many of the tables in the base system. You can customize these rules for your organization. When a new table is created in the CMDB, it derives identification and reconciliation rules from its parent table if these rules exist. To apply identification and reconciliation rules to a new table, create the rules either at the child level or at its parent level.
Identification and Reconciliation Engine (IRE)
- IRE prevents duplicate CIs by uniquely identifying CIs.
- IRE reconciles CI attributes by allowing only authoritative data sources to write to CMDB.