**** As of September 2023, this blog is obsolete as the data model became GA ****
**** INNOVATION LAB RELEASE ****
This article applies to an innovation lab release, the details of the data model described in this article are subject to change prior to going general availability.
A new application named "Service Graph Connector for Kong" has been introduced on the innovation lab. The initial release of this app only contains a new CMDB data model which will facilitate future service graph connectors for populating the model. In the release there are no integration components included which populate the model. Once we have vetted the data model with customers, we will move the classes to the standard "CMDB CI Class Model" application on the store to be used in production instances. In v0.2 of the SGC for Kong we will introduce the actual integration components for populating the data model from Kong Gateways.
Let's first understand the data model.
Core Data Model
To begin, look at the two new classes below.
Core of the new API data model
If you are familiar with the OpenAPI specification, then some of the terminology used in this article will be familiar. Otherwise, I will elaborate where possible. The API (cmdb_ci_api) is designed to store information about the top level API like name, version, and base URL. the API Component (cmdb_ci_api_component) is designed to represent the various resources of the API. These will include unique method and path combinations, varying authentication requirements, ports used, and more.
An example of how this might look can come directly from ServiceNow if we look at the REST API Explorer.
ServiceNow Example
In this example, we are looking at the CMDB Instance API which would presumably be stored in the base API class (cmdb_ci_api). The various resources below that like "GET Query records for a CMDB class" would represent the different API components (cmdb_ci_api_component) related to the API.
Expanding the Data Model to API Gateways
In addition to these two new classes, there are a number of classes surrounding these two which will be used when populating from API Gateway sources. Expanding on the core model, we continue to build out the data model as follows:
Expanded data model for API Gateway data
A couple of initial points to notice are that the API class now has a new child class named "Managed API" (cmdb_ci_managed_api). This class has a dependency on the API Gateway class which the core API class does not have. Likewise, the API Component class has two new child classes, API Frontend (cmdb_ci_api_frontend) and API Backend (cmdb_ci_api_backend). These two new child classes are used to store common components found in API Gateways which represent the client facing frontend (request) endpoint and the request fulfilling backend service which provides the response.
We can look at an example of this using the Azure API Management service.
Azure API Management Service example
While the table "cmdb_ci_azure_gateway" does not currently exist, hopefully you can extrapolate how this will look in the future with that being a child class of the API Gateway class introduced in this new data model. In the picture of the Azure API Management service, you can see a number of APIs. When choosing the "Echo API" we see what Azure calls the various operations of the API. These are what we will normalize into the data model as API Components. However, we will not store them in that parent class, but use the additional data of this gateway to populate the API Frontend class with the "frontend" values like "POST /resource" and the API Backend class with the "backend" values like "http://echoapi.cloudapp.net/api" and related data about that resource.
We do not plan to replicate everything in the gateway into the ServiceNow CMDB, but rather, our intent is to capture the relevant data for achieving outcomes in the following use cases.
Use cases
Security Operations and Vulnerability Management: Being able to associate security incident or vulnerabilities with APIs can offer new insights and faster responses to one of the fastest growing attack surfaces in organizations today.
AIOps / ITOM Health: APIs are now part of monitoring and observability solutions. Being able to understand the health and/or outages of APIs is critical to keeping applications and services running.
Change Management: The availability of APIs within the CMDB provide a way of tracking major and minor changes to APIs in the same way as other organization technology assets. This can help IT operators identify root causes faster when outages occur.
APIOps: The emerging practice of APIOps brings together DevOps with GitOps to facilitate the full lifecycle of APIs. Having this data within the CMDB gives multiple stakeholders the visibility of such assets in their workflows.
Example Data to Test
Using the ServiceNow example API above, this is how the data could be entered into the API (cmdb_ci_api) and API Component (cmdb_ci_api_component) classes, which are the two main classes to test during this innovation lab release period.
API Example Data
Once the API is defined, the components can be added and related to the API. Here is an example of one component that is related to the API.
Example API Component Related to API
Once you have the components of the API defined, you should be able to see a dependency view similar to the following.
Example dependency view
From this point, alerts can be bound to these CI's, dynamic CI groups could be created for service views, incidents and changes can be created against the CIs, vulnerable items can be filed, and any other workflow where you use CIs.
Placement of the API Class
Some customers may be asking why are we creating the new API class directly from the base cmdb_ci class. The answer is that the conclusion to do so was made after multiple alternatives were considered. First, the endpoint hierarchy was considered and more specifically the http endpoint class. This was abandoned due to the hierarchies tight connection to service mapping and impact it would have on that capability. Similarly, endpoints do not show on dependency views or service maps and we want customers to be able to visualize the APIs in the environment.
Secondly, the application service hierarchy was considered. Again, the hierarchy for app services was initially introduced by the service mapping team and there is considerable logic behind all records in that hierarchy including the continual evaluation of impact rules for each and every application service. When loading 100's, 1000's, or event 10's of 1000's of APIs into that hierarchy, there was adverse behavior and visualization in the system.
Finally, a new service type was considered directly under the Service class (basically being a sibling to the application service hierarchy). This approach met challenges with other product lines which would have to be modified to account for a new service type as they currently only perform against the existing three service types.
For these reasons we have extended directly from the cmdb_ci class to offer the most flexibility without impacting other product lines existing functionality.
How to Access and Provide Feedback
The innovation lab application is currently "hidden" and only available upon approval. If your organization is in a position to load the new API data model detailed above in a sub production instance and populate the data given the concepts above, and can provide feedback on how it is working in certain process flows, please let me know.
Some initial issues to note are that we already plan to expand the width of the "ID" attributes on the classes as well as the "path" attributes as they are currently limited to 40 characters.
15 Comments
