SCCM Import Data Source, Reconciliation, and Data Precedence

[tenagra]

Hey everyone.

Having an issue with the SCCM Import process setting discovery sources so that the reconciliation rules and data precedence work as expected.   Let me explain (sorry for it being long).

We've been using SN Discovery for the better part of the last year for our all of our servers.   We now have a new project to bring in the workstations.   Rather than using Discovery, we are going to do it via SCCM 2012 V2.   The benefit being 3 fold.   1) Capture who the workstation belongs to, 2) incorporate Client Software Distribution, and 3) incorporate Software Asset Management.   The various teams would also like SCCM to populate any server information that isn't current populated and of course, create records that don't already exists.

During our testing, I've found that we need to use reconciliation and data presence rules to remove conflict around who can update.   These are based upon the 'Data Source' field on the CI.   So I setup the following rules, which I assume to be very basic in the belief that this is say, "ServiceNow Discovery is king of the world, with the one exception of setting the 'assigned_to' user, where SCCM Rules"

A) Reconciliation Rule:

        Data Source: ServiceNow

        Applies To: CMDB_CI

        Active: True

        Attribute: Left empty (which I understand to mean, allow updating of all fields)

B) Reconciliation Rule:

        Data Source: SCCM Import (custom choice added for this, rather than import set)

        Applies to: CMDB_CI

        Active: True

        Attribute: assigned_to

C) Data Precedence

        Applies to: cmdb_ci

        Data Source: ServiceNow

        Order: 100

        Active: True

D) Data Precedence

        Applies to: cmdb_ci

        Data Source: SCCM Import

        Order: 200

        Active: True

I've created a CI record for my personal work station and set the discovery source = ServiceNow.   I set a random Serial Number and random assigned_to user.   My expectation is that the assigned_to will get updated via the SCCM import, but the SerialNumber, which doesn't match what is in SCCM, should remain the same

Here is where I need some help or I missing something.   As original stated, the SCCM Import doesn't populate the Data Source field.   It is NULL for any new records, and any records that are updated don't have the Data Source field updated from 'ServiceNow' to 'SCCM Import' as it should after the update.   So, my first thought is that I need to resolve that so that reconciliation will work as expected.   As such, I tried to directly set the discovery_source in the transformation script (as shown in #1 below)

Tried to set the Discovery Source directly in the transformation script.  

target.discovery_source = 'SCCM Import';

I then run the SCCM Import.   The CI does update the Discovery Source and the assigned_to as expected.   However, it also updates the serialnumber field, which it shouldn't have done.

Thinking the issue was that something was out of order (like the field was populated after the data had been updated, I followed the advice from this set of videos on reconcilation and data precedence   https://servicenow.app.box.com/v/sourceprecedence.   Basically, it says that to get the Data Source to be populated during an Import, you need to create a 'OnBefore' script in the transformation and add the following code:

(function runTransformScript(source, map, log, target /*undefined onStart*/ ) {
       var cmdbUtil = new CMDBTransformUtil();
       cmdbUtil.identifyAndReconcile (source, map, log);
       ignore=true;
})(source, map, log, target);

Easy enough.   After adding, I start failing during the transformation portions.   See the following in my transform history logs:

Identification and Reconciliation Result: {"items":[{"sysId":"Unknown","identifierEntrySysId":"Unknown","errors":[{"error":"MISSING_MATCHING_ATTRIBUTES","message":"In payload missing minimum set of input values for criterion (matching) attributes from identify rule for table [cmdb_ci_hardware]. Add these input values in payload item '{\"className\":\"cmdb_ci_computer\",\"values\":{\"chassis_type\":\"undefined\",\"discovery_source\":\"SCCM Import\",\"name\":\"\",\"os_domain\":\"\",\"correlation_id\":\"\",\"serial_number\":\"\",\"model_number\":\"\",\"last_discovered\":\"\",\"assigned_to\":\"\",\"cpu_count\":\"\",\"manufacturer\":\"\"}}'"},{"error":"ABANDONED","message":"Too many other errors"}],"identificationAttempts":[{"identifierName":"Hardware Rule","attemptResult":"SKIPPED","attributes":["correlation_id"],"searchOnTable":"cmdb_ci_hardware"},{"identifierName":"Hardware Rule","attemptResult":"SKIPPED","attributes":["serial_number","serial_number_type"],"searchOnTable":"cmdb_serial_number"},{"identifierName":"Hardware Rule","attemptResult":"SKIPPED","attributes":["serial_number"],"searchOnTable":"cmdb_ci_hardware"},{"identifierName":"Hardware Rule","attemptResult":"SKIPPED","attributes":["name"],"searchOnTable":"cmdb_ci_hardware"},{"identifierName":"Hardware Rule","attemptResult":"SKIPPED","attributes":["ip_address","mac_address"],"searchOnTable":"cmdb_ci_network_adapter"}]}],"relations":[],"logContextId":"775971d64f7383c0a4d3650f0310c7c1"}

Based upon what I'm seeing it appears that the SCCM Import doesn't have enough information to run the Identifier rules, but not sure how that is possible here, where I don't get this message during a normal import process.   If so, the missing fields are IP and MAC Address, which isn't something that is going to be easy to get out of SCCM in a normalized manner.

So now I am at a loss of what to do.   Anyone have any thoughts, ideas, suggestions, etc on what needs to be done here that would help get this resolved?

Thank you very much.

Nathan

[tenagra · Accepted solution]

This can be ignored.

First, I found that I copied in the wrong code into the onBefore script.   The script

1.   var cmdbUtil = new CMDBTransformUtil();  
2.       cmdbUtil.identifyAndReconcile (source, map, log);  
3.       ignore=true;  

Does indeed work as expected and follows the reconciliation rules.   The reconciliation definitions and data precedence rules were configured correctly.   They just weren't being activated due to the onBefore error.

The first example doesn't work as it set the target.data_source after the insert/updates have been made.   Per documentation, if data_source is blank/null, it takes precedence over everything and all reconciliation definitions are ignores.

Nathan

[priti14]

Hi Nathan,

I am having the same requirement, I followed your solution but its not giving me the proper output.

Could you please help me on this.

Thanks

Priti

[pflick7770]

Are you setting 'Discovery source' in an onBefore transform script? If not, where/when is:  target.discovery_source = 'SCCM Import' ?

Were you able to get the de-duplication functionality of the CMDBTransformUtil to work using this approach?  Our experience has been, and other posts have indicated that 'Discovery source' must be either "ServiceNow" or <blank> in order for the CI duplication check (and associated Duplicate metric and de-duplication task generation) to work.

[jatinder1]

Hi ,

How did you test the reconciliation and identification rules ?