A Deep Dive into Data Ingestion and Management in CMDB
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2 hours ago
Introduction
The central archive that contains information relating to an organization's IT setup is the ServiceNow Configuration Management Database (CMDB), which contains a complete and precise picture of all Configuration Items (CIs) in the organization and their relationships. For an organization to carry out its business processes effectively in the areas of IT Service Management (ITSM), IT Operations Management (ITOM), etc., it is of critical importance to ensure that its CMDB is accurate and dependable. This article talks about data in-uptake tools, data transformation tools, and best practices in configuring a dependable CMDB.
1. Service Graph Connectors: Smooth Integrations
Service Graph Connectors are pre-built, service-certified APIs provided by ServiceNow that can be used to make it simple to retrieve CI details from different external tools like AWS, Qualys, SCCM, Intune, Dynatrace, etc. It has been developed in a way that minimizes initial setup as well as maintenance time. However, it should be noted that while these are extremely efficient in use, most of these connectors are usually an extra cost and are not usually included in a standard Personal Developer Instance (PDI). It must also be understood that most of the external tools will require their own respective connectors.
2. REST API as the Transport Layer: The Communication Backbone
Essentially, the core of the integration process revolves around the REST API, which acts as a backbone in facilitating communications between ServiceNow and other systems. Service Graph Connectors use REST as the medium of exchange, which acts as the core transfer mechanism. The payload can be large JSON structures that contain the rich attributes of each CI. This exchange creates room for interoperability and adaptability within diverse environments.
3. IntegrationHub ETL: Data Transformation Engine
IntegrationHub ETL: This is the robust data wrangling engine of the ServiceNow platform and is meant for shaping and refining external data before it is imported into the CMDB. ETL excels when the feeds come in from different sources and assorted data formats. It excels when the feeds come in different forms, such as CSV files, JSON, FTP, and other feeds that demand significant adjustments in the fields. ETL enables users to get the job done from the initial intake of the data through nuanced cleansing and loading of the data.
Major steps in the ETL process include:
Data Source: specify where your raw data is located, e.g., CSV, FTP, JSON, API, etc.
Import Set: This is a stage table that holds the raw data temporarily.
Transform Map: these are used to map the import set to the target CMDB tables.
Class Mapping: routes the input data to the suitable CMDB class.
Field Mapping: this maps a source field against a CMDB attribute.
The ETL Map Assistant offers a guided setup, which covers essentials, data source setup, source-to-target mapping, field as well as class mapping, and also testing, making your experience smooth throughout.
4. Identification and Reconciliation Engine (IRE): Keeping CMDB Accurate
Identification and Reconciliation Engine (IRE) plays a vital role in maintaining the precision and uniqueness of CI data in the CMDB, particularly if there are different data sources, for example, AD, SCCM, AWS, Intune, Qualys, where there can be overlaps of data. The key function of IRE is to ensure that there are no duplicate CIs, and only trusted sources can update CI attributes. This acts as a central resource for handling data integrity, particularly when managing multiple sources, which makes it a central framework. Here is a very simple explanation of how the IRE logic works:
5. Manual Import Set + Transform Map: A Simple, Quick Path
If you're using the application of loading data in bursts, using smaller sheets of data, or you just want an uncomplicated method of performing the mapping without intricate data manipulation, the application of the Manual Import Sets and then using the Transform Maps on their own provides an uncomplicated and direct approach. This works well in circumstances when the amount of data isn’t voluminous and the rules of transformation are uncomplicated.
6. CMDB Integration Dashboard: Clarity and Oversight
CMDB Integration Dashboard
The CMDB Integration Dashboard helps view all data flow in one place. It has the advantage of presenting clear visions of run status, errors, and overall integration health. This dashboard usually comes bundled with the Integration Hub plugin. This powerful tool is a must-have feature for any large enterprises with complex CMDB infrastructures, e.g., BT, Orange, Vodafone, JPMorgan, etc.
7. Kicking Off a CMDB Project: Practical Implementation Steps
A successful rollout of a CMDB implementation typically occurs through a staged approach known as Crawl-Walk
- Crawl: Begin small with foundational CI classes like Computer, Network, Server, Switch, and so forth. Governance must be decided on with regards to these key CI classes.
- Walk: More CI classes, bring in automated discovery, and start with basic service relationships.
- Run: Service mapping, lifecycle management of CI, and advanced CMDB health monitoring and reporting.
This incremental approach provides a foundation that allows organizations to gradually improve the features and functionality of their CMDBs, while still ensuring the integrity and trustworthiness of the data they represent.
If you found this article useful, please mark it as Helpful. It helps others find the content more easily 👍🙂
