Interested in a ServiceNow event built for developers? Registration for now[dev]26 is officially open!

ben_yukich
ServiceNow Employee

The identification and Reconciliation API is a relatively new capability in the ServiceNow platform that often goes (tragically) unnoticed, and under appreciated. This must end. It's one of the most important capabilities to be aware of if you're driving any part of your CMDB from an external source. Have you ever wondered how Discovery and Service Mapping are so good at NOT creating duplicate CIs in the CMDB? Have you ever noticed on your Discovery Status records the Devices tab has a status of "Created CI" or "Updated CI" for each device scanned? This capability has existed for quite some time behind the scenes, but now you can use the exact same framework from any piece of script and any external REST call!

So, let's break this down and figure out how to make it useful. There are two ways you can use the I&R API: direct from SN script; from a REST web services call. Either way, the way you communicate with the API is going to be the same: you'll be using a JSON payload to describe configuration items and relationships. So, let's start with a simple example payload and script runnable in Scripts - Background:

var payload = {

      items: [{

              className: 'cmdb_ci_win_server',

              values: {

                      name: 'Win Server 100',

                      ip_address: '10.20.30.40',

                      mac_address: 'ABCD1234',

                      ram: '2048'

              }

      }]

};

var input = new JSON().encode(payload);

var output = SNC.IdentificationEngineScriptableApi.createOrUpdateCI('ServiceNow', input);

gs.print(output);

I get a response about what action was taken:

*** Script: {"items":[{"className":"cmdb_ci_win_server","operation":"INSERT","sysId":"1d848b174fc12a40fd2bfc5f0310c775","identifierEntrySysId":"Unknown","identificationAttempts":[{"identifierName":"Hardware Rule","attemptResult":"SKIPPED","attributes":["serial_number"],"searchOnTable":"cmdb_ci_hardware"},{"identifierName":"Hardware Rule","attemptResult":"NO_MATCH","attributes":["name"],"searchOnTable":"cmdb_ci_hardware"}]}],"relations":[]}

So, that's a long line, but you get a lot of great detail in there about precisely which identifier matched, and what record was inserted (an easier to read example will be included below).

Now let's say we want to update this same server (perhaps we doubled the memory), but this time we wan to do it via the REST API... easy! Just navigate to the REST API Explorer and select the "Identification and Reconciliation API" in the API Name field (within the "now" namespace). You'll see that there is only ONE action you can take: a Create or Update CI POST operation. To call this successfully, you must provide a parameter that I glossed over in the script example, the data source, to indicate where the data is coming from (this must be a valid entry in the discovery_source field of the cmdb_ci table, you can extend this of course - it is used for important capabilities like datasource precedence). You must also provide a JSON payload of the same form shown earlier. Let's try with one that has an update to the ram value:

{"items":[{

      "className":"cmdb_ci_win_server",

      "values": {

              "name":"Win Server 100",

              "ip_address":"10.20.30.40",

              "mac_address":"ABCD1234",

              "ram":"4096"

      }

}]}

Once this is executed we should receive a 200 response with a slightly different response payload than we saw before, indicating an UPDATE rather than an INSERT:

{

  "result": {

      "items": [

          {

              "className": "cmdb_ci_win_server",

              "operation": "UPDATE",

              "sysId": "1d848b174fc12a40fd2bfc5f0310c775",

              "identifierEntrySysId": "556eb250c3400200d8d4bea192d3ae92",

              "identificationAttempts": [

                  {

                      "identifierName": "Hardware Rule",

                      "attemptResult": "SKIPPED",

                      "attributes": [

                          "serial_number"

                      ],

                      "searchOnTable": "cmdb_ci_hardware"

                  },

                  {

                      "identifierName": "Hardware Rule",

                      "attemptResult": "MATCHED",

                      "attributes": [

                          "name"

                      ],

                      "searchOnTable": "cmdb_ci_hardware"

                  }

              ]

          }

      ],

      "relations": []

  }

}

So, I get excited about odd things... but there is no denying that this is SWEET! In the past, I've stepped through how to trigger a full discovery via web services (Real-time CMDB: Disco as a Service). Now, you can add to your bag of tricks - imagine calling the I&R API as the first step in populating your CMDB, or perhaps to augment the CMDB with devices that would not ordinarily be discoverable (hello, IoT).

Worth noting that this can be used for an array of items, not just one at a time, and also relationships. For example, here's how you would include an application and the fact that it runs on a specific server:

{"items":[{

      "className":"cmdb_ci_app_server_tomcat",

      "values":{

              "name":"tomcat",

              "running_process_command":"xyz",

              "running_process_key_parameters":"abc",

              "tcp_port":"8087"

      }

},{

      "className":"cmdb_ci_win_server",

      "values":{

              "name":"Win Server 100",

              "ip_address":"10.20.30.40",

              "mac_address":"ABCD1234",

              "ram":"4096"

      }

}],"relations":[{

      "child":1,

      "parent":0,

      "type":"Runs on::Runs"

}]}

There are a ton of details glossed over here (like configuring identification rules). To go deeper, take advantage of some of the great content out there by manishgupta richardbrounstein and mikebuckner (NOTE: these are K16 resources which are only available to full conference attendees who are logged in):

Identification & Reconciliation (repeat 1 of 2)

Use CMDB APIs for Identification & Reconcilation

Here are several publicly accessible resources that are related for those without access to K16 content:

Product Documentation

createOrUpdateCI Scriptable API

IdentifyReconcile REST API

CMDBTransformUtil — identifyAndReconcile() API

16 Comments
James Fricker
Mega Sage
ben_yukich
ServiceNow Employee

Shoot, sorry James those resources were from K16 and clearly have some restrictions on access. Did you attend Knowledge 16? I will update the post to include some public resources as well. Sorry about that!



Edit: I have confirmed with our community team that the K16 resources are only available to full conference attendees. If you feel you should have access but are having trouble, contact community@servicenow.com


joeteasdale
ServiceNow Employee

with the api call identifyAndReconcile is it possible to pass along a discovery source value? I have data coming into various import sets and need a way to differentiate for precedence rules.


ben_yukich
ServiceNow Employee

Absolutely possible (and encouraged) - my bad for not making a clearer point of it in the above post.


  • If you're using the API from script, as in the first example, you just need to change an argument:
    • SNC.IdentificationEngineScriptableApi.createOrUpdateCI('My Data Source', input);
  • If you're using the API via REST, it just takes adding an additional query parameter "sysparm_data_source" to your request
ranjanireddy
Tera Contributor

Can you please explain how we can call I & R API in Tranform Map. I went through SN community and noticed that we can call the below method in On before transform map script.



var cmdbUtil = new CMDBTransformUtil();
cmdbUtil.identifyAndReconcile(source, map, log);
ignore = true;



Will the API automatically fetches the mapped fields or do we need to send it as a payload in onbefore script.


joeteasdale
ServiceNow Employee

First you need to set up the field mappings. No need for coalesce since the reconciliation handles how to find a matching entry through the CI Identifier rules. Next, add your on_before script that you have below.



The one shortcoming however is if you need to map a field that is type reference and the identifier is something other than the display value or sys_id. If that is the case you will need to write some logic to search on the related table to find the record to associate. Here is a good example of that:


joeteasdale
ServiceNow Employee

Sorry — Here is the example:



var resourcId = source.u__resourceguid;


var gr = new GlideRecord('cmdb_ci_computer');


gr.addQuery('correlation_id', resourceId);


gr.query();


if(gr.next()) {


source.u__resourceguid = gr.getValue('sys_id');


}



var cmdbUtil = new CMDBTransformUtil();


cmdbUtil.identifyAndReconcile(source, map, log);


ignore = true;


tkalpa005
Tera Expert

Hi All,


After Jakarta upgrade,API isn't updating the record, every time it inserts a new reccord, and it happens for all className, Identifier and indetifier entry are looks good, any idea   why it is happening ?


Identifier behaves wired for SNC.IdentificationEngineScriptableApi.createOrUpdateCI() API, It insert new record instead of updating the same .


request:


var payload = {


      items: [{


              className: 'cmdb_ci_msd',


              values: {


                      "company": "c94993314a362312005cfc20c1aa15d2",


                              "correlation_id": "9091",


                              "device_type": "SAN",


                              "ip_address": "99899999",


                              "model_id": "HM800S",


                              "name": "hgdfdgdfdg",


                                "serial_number": "480981",


                              "u_firmware_version": "vvvvvv",


                              "u_management_url": "URLLLLL",


                              "u_subclass": "e0300aa14fe676c0e6b642818110c76b"


              }


      }]


};



var input = new JSON().encode(payload);


var output = SNC.IdentificationEngineScriptableApi.createOrUpdateCI('ServiceNow', input);


gs.print(output);



Response :



identification_engine : createOrUpdateCI: Matched 1 records and 0 relations in 3msec


identification_engine : Waited 1msec for mutex named IDENTIFICATION_ENGINE_INSERT


identification_engine : Encountered an insert during delay locking, restarting processing under lock


identification_engine : createOrUpdateCI: Matched 1 records and 0 relations in 2msec


*** Script: Calling processCIParents for EXP0171044


*** Script [ReferenceCI]: [DEBUG] Starting to create/remove relationships for hgdfdgdfdg


*** Script [Symfoni:SymCMDBReferenceCIRelationships]: [DEBUG] 1507732189016: Managing relationships : child: , child_name: , child_class:


*** Script [Symfoni:SymCMDBReferenceCIRelationships]: [DEBUG] 1507732189017: Managing relationships: parent: e4fd70fcdb39434062f27a0bbf9619a4, parent_name: hgdfdgdfdg, parent_class: cmdb_ci_msd


*** Script [Symfoni:SymCMDBReferenceCIRelationships]: [DEBUG] 1507732189017: Managing relationships: previous: e4fd70fcdb39434062f27a0bbf9619a4, previous_name: hgdfdgdfdg, previous_class: cmdb_ci_msd


*** Script [Symfoni:SymCMDBReferenceCIRelationships]: [DEBUG] 1507732189017: Managing relationships: field: provided_by


*** Script [Symfoni:SymCMDBReferenceCIRelationships]: [DEBUG] 1507732189018: Previous parent or child was null, nothing to remove


*** Script [Symfoni:SymCMDBReferenceCIRelationships]: [DEBUG] 1507732189018: Parent was null, nothing to create


*** Script [ReferenceCI]: [DEBUG] Starting to create/remove relationships for hgdfdgdfdg


*** Script [Symfoni:SymCMDBReferenceCIRelationships]: [DEBUG] 1507732189019: Managing relationships : child: , child_name: , child_class:


*** Script [Symfoni:SymCMDBReferenceCIRelationships]: [DEBUG] 1507732189019: Managing relationships: parent: e4fd70fcdb39434062f27a0bbf9619a4, parent_name: hgdfdgdfdg, parent_class: cmdb_ci_msd


*** Script [Symfoni:SymCMDBReferenceCIRelationships]: [DEBUG] 1507732189019: Managing relationships: previous: e4fd70fcdb39434062f27a0bbf9619a4, previous_name: hgdfdgdfdg, previous_class: cmdb_ci_msd


*** Script [Symfoni:SymCMDBReferenceCIRelationships]: [DEBUG] 1507732189020: Managing relationships: field: u_rack


*** Script [Symfoni:SymCMDBReferenceCIRelationships]: [DEBUG] 1507732189020: Previous parent or child was null, nothing to remove


*** Script [Symfoni:SymCMDBReferenceCIRelationships]: [DEBUG] 1507732189020: Parent was null, nothing to create


identification_engine : 1 Main CIs insert commit time is 87msec


identification_engine : Processed 1 records and 0 relations in 2msec + 87msec (waited 1msec for mutex)


*** Script: {"items":[{"className":"cmdb_ci_msd","operation":"INSERT","sysId":"e4fd70fcdb39434062f27a0bbf9619a4","identifierEntrySysId":"Unknown","identificationAttempts":[{"identifierName":"OCI Storage Identifier New","attemptResult":"NO_MATCH","attributes":["name"],"searchOnTable":"cmdb_ci_msd"}]}],"relations":[]}




Identifier entry :



find_real_file.png



Do you have any idea ?



Thanks


Kalpa


ben_yukich
ServiceNow Employee

Hi Kalpa, I can't say for sure with the example you gave. I replicated your identifier, ran the same script in my instance, and it behaved as expected (the first run, a new record was created; the second run, the record was updated). The output you've provided above is consistent with an initial run of the script where there is no matching cmdb_ci_msd record named "hgdfdgdfdg" - if you run the same script a second time and the identifier engine again finds no match, I encourage you to reach out to our support team.



Best,


Ben


Kasper XYZ
Tera Expert

Hi


Any Idea on how to ensure that the related assets and models are created correctly when using the Identification and Reconciliation API?


Regards Kasper