Real-time CMDB: Identification and Reconciliation API

[ben_yukich]

The identification and Reconciliation API is a relatively new capability in the ServiceNow platform that often goes (tragically) unnoticed, and under appreciated. This must end. It's one of the most important capabilities to be aware of if you're driving any part of your CMDB from an external source. Have you ever wondered how Discovery and Service Mapping are so good at NOT creating duplicate CIs in the CMDB? Have you ever noticed on your Discovery Status records the Devices tab has a status of "Created CI" or "Updated CI" for each device scanned? This capability has existed for quite some time behind the scenes, but now you can use the exact same framework from any piece of script and any external REST call!

So, let's break this down and figure out how to make it useful. There are two ways you can use the I&R API: direct from SN script; from a REST web services call. Either way, the way you communicate with the API is going to be the same: you'll be using a JSON payload to describe configuration items and relationships. So, let's start with a simple example payload and script runnable in Scripts - Background:

var payload = {
       items: [{
               className: 'cmdb_ci_win_server',
               values: {
                       name: 'Win Server 100',
                       ip_address: '10.20.30.40',
                       mac_address: 'ABCD1234',
                       ram: '2048'
               }
       }]
};
var input = new JSON().encode(payload);
var output = SNC.IdentificationEngineScriptableApi.createOrUpdateCI('ServiceNow', input);
gs.print(output);

I get a response about what action was taken:

*** Script: {"items":[{"className":"cmdb_ci_win_server","operation":"INSERT","sysId":"1d848b174fc12a40fd2bfc5f0310c775","identifierEntrySysId":"Unknown","identificationAttempts":[{"identifierName":"Hardware Rule","attemptResult":"SKIPPED","attributes":["serial_number"],"searchOnTable":"cmdb_ci_hardware"},{"identifierName":"Hardware Rule","attemptResult":"NO_MATCH","attributes":["name"],"searchOnTable":"cmdb_ci_hardware"}]}],"relations":[]}

So, that's a long line, but you get a lot of great detail in there about precisely which identifier matched, and what record was inserted (an easier to read example will be included below).

Now let's say we want to update this same server (perhaps we doubled the memory), but this time we wan to do it via the REST API... easy! Just navigate to the REST API Explorer and select the "Identification and Reconciliation API" in the API Name field (within the "now" namespace). You'll see that there is only ONE action you can take: a Create or Update CI POST operation. To call this successfully, you must provide a parameter that I glossed over in the script example, the data source, to indicate where the data is coming from (this must be a valid entry in the discovery_source field of the cmdb_ci table, you can extend this of course - it is used for important capabilities like datasource precedence). You must also provide a JSON payload of the same form shown earlier. Let's try with one that has an update to the ram value:

{"items":[{
       "className":"cmdb_ci_win_server",
       "values": {
               "name":"Win Server 100",
               "ip_address":"10.20.30.40",
               "mac_address":"ABCD1234",
               "ram":"4096"
       }
}]}

Once this is executed we should receive a 200 response with a slightly different response payload than we saw before, indicating an UPDATE rather than an INSERT:

{
   "result": {
       "items": [
           {
               "className": "cmdb_ci_win_server",
               "operation": "UPDATE",
               "sysId": "1d848b174fc12a40fd2bfc5f0310c775",
               "identifierEntrySysId": "556eb250c3400200d8d4bea192d3ae92",
               "identificationAttempts": [
                   {
                       "identifierName": "Hardware Rule",
                       "attemptResult": "SKIPPED",
                       "attributes": [
                           "serial_number"
                       ],
                       "searchOnTable": "cmdb_ci_hardware"
                   },
                   {
                       "identifierName": "Hardware Rule",
                       "attemptResult": "MATCHED",
                       "attributes": [
                           "name"
                       ],
                       "searchOnTable": "cmdb_ci_hardware"
                   }
               ]
           }
       ],
       "relations": []
   }
}

So, I get excited about odd things... but there is no denying that this is SWEET! In the past, I've stepped through how to trigger a full discovery via web services (Real-time CMDB: Disco as a Service). Now, you can add to your bag of tricks - imagine calling the I&R API as the first step in populating your CMDB, or perhaps to augment the CMDB with devices that would not ordinarily be discoverable (hello, IoT).

Worth noting that this can be used for an array of items, not just one at a time, and also relationships. For example, here's how you would include an application and the fact that it runs on a specific server:

{"items":[{
       "className":"cmdb_ci_app_server_tomcat",
       "values":{
               "name":"tomcat",
               "running_process_command":"xyz",
               "running_process_key_parameters":"abc",
               "tcp_port":"8087"
       }
},{
       "className":"cmdb_ci_win_server",
       "values":{
               "name":"Win Server 100",
               "ip_address":"10.20.30.40",
               "mac_address":"ABCD1234",
               "ram":"4096"
       }
}],"relations":[{
       "child":1,
       "parent":0,
       "type":"Runs on::Runs"
}]}

There are a ton of details glossed over here (like configuring identification rules). To go deeper, take advantage of some of the great content out there by manishgupta richardbrounstein and mikebuckner (NOTE: these are K16 resources which are only available to full conference attendees who are logged in):

Identification & Reconciliation (repeat 1 of 2)

Use CMDB APIs for Identification & Reconcilation

Here are several publicly accessible resources that are related for those without access to K16 content:

Product Documentation

createOrUpdateCI Scriptable API

IdentifyReconcile REST API

CMDBTransformUtil — identifyAndReconcile() API