CMDB Identification & Reconciliation and Domain Separation

Jon G · ‎04-26-2019

I'm trying to understand the logic behind the way domain separation was implemented into Identification and Reconciliation, and how to get around one of the poor design choices.

In most areas of the system, domain separation follows certain rules about what can and cannot be accessed by certain users. For example, in my domain, I can see data in all other domains. However, users in a child domain can only see data within their domain. This is the standard domain separation logic in almost every area of the system.

We are a services provider, and so we use domain separation to ensure our clients cannot access areas of the system outside of their domain. Each company has its own domain.

In Identification and Reconciliation, the engine uses the domain of the logged in user for any matching. I 100% understand the logic behind only wanting to match devices in the same domain - and in fact, I would want in that way. A device from organization A should never be matched against a device in organization B. I'm following the decision here so far.

As part of the services that we offer, we do monitoring on our clients' networks. I'm trying to import all of our monitored devices into CMDB. As part of the import, I have the company name as an element of each device that is imported.

Here's where the process breaks down: In any other area of the system, if I perform an import as my user that can see into other domains, and provide a company name, that device is imported into that company's domain. This is correct behavior - the device ends up in the correct domain. So far so good.

However, the logic of the identification and reconciliation engine observes the logged-in user's domain exactly and doesn't seem to respect domain visibility permissions and hierarchy. If the device CI is in my domain, it will match. If the device is in any other domain, it will not match an imported CI against an existing CI even though I am providing the company name, and even though the device ends up in the correct domain.

I've tried importing as system admin, and at the global domain level as well. Neither achieves the desired results. I've also tried running my import script in a scheduled job and letting the job run as "system" - still does not work.

The documentation outlines specifically that:

Domain IDs do not need to be explicitly sent in the input payload of the identification engine APIs. Internally, the identification engine causes the current domain ID of the user to call the identification engine APIs.
Only CIs that have the same domain ID as the currently logged-in user's domain display during matching.
During matching, if no records are found and a CI needs to be inserted, the CI’s domain ID is the same as the domain ID of the currently logged-in user’s domain.

(https://docs.servicenow.com/bundle/madrid-servicenow-platform/page/product/configuration-management/...)

I haven't tried yet, but my next step is to try providing the domain regardless of the instruction above and see what happens. I can't understand the reliance upon using the logged-in user's domain in this way.

Matching should be performed on the domain of the CI being imported, not the logged-in user. The engine can fall back to the logged-in user if no valid domain is provided. If the logged-in user can't access other domains, then they shouldn't be able to see or edit data in that domain anyway. This appears to be a significant limitation on the capabilities of this module.

So my question is:

How can I perform a mass import of CI data from our monitoring system, and have CMDB Identification work properly with domain separation?

Jon G · ‎04-30-2019

So here is what I found - I can impersonate the user that I want on-the-fly using gs.getSession()impersonate(); . The individual user accounts still need to be created, but once that is done, the script work as expected.

Here's a portion of my onBefore script used to do this:

var cmdbUtil = new C_CMDBTransformUtil();
	
	if (grUser.next()) {
		var runUser = gs.getSession().impersonate(grUser.sys_id);
		cmdbUtil.setDataSource('External DataSource Name Here');
		cmdbUtil.identifyAndReconcile_with_Class(source, map, log, targeted_table);
		gs.getSession().impersonate(runUser);
	}
	else {
		gs.log ('No Import User Found');
	}

Use a gliderecord to find the user that you want. In my case, it is a predictable user account for each company.

The nice thing about this is that if a client company isn't configured correctly in our system, the script ignores the record and moves on which avoids creating bad data.

C_CMDBTransformUtil(); is a modification to CMDBTransformUtil(); that I found here to allow us to select a CI class as we are importing instead of reclassifying after the fact.

SN did some really weird stuff with this Identification and Reconciliation module that I've had to work around.

View solution in original post

Deepak Ingale1 · ‎04-28-2019

Hi Jon,

1) You can create a MID Server Account user per domain, so that OOB behavior of SN is honoured when different CIs for different customers are being identified.

2) If you are using Import Sets to load these devices, I would suggest to use "Run as" field on scheduled import set to set the "run as" field value to MID Server User account created for that domain.

If you take care of these things, I believe you should be fine

Now, why SN does not use DOMAIN field OOB, reason being when Discovery tried to Identify CI, it is not aware of the domain attribute since it is not a part of payload coming from CI directly.

I hope above explanation will help you.

Note: Please mark reply as correct / helpful if it has answered your question

Jon G · ‎04-29-2019

Hi Deepak,

Thank you for reading through all of this and offering some suggestions. We are not actually using MID servers /built-in discovery for this process. We have a third party monitoring platform that we are using. I agree this process would probably be much simpler if we were doing that, but it isn't cost-effective for us at this time.

The goal is to bring in all device data from that third party platform and classify it during the import leveraging the identification engine.

We are indeed using import sets, however the "run as" field will limit us to importing for one domain at a time, which is the problem that I am trying to solve for.

Our import set contains information from many companies. Ideally, we should be able to take this information, map the company that each device belongs to, and everything should work fine in terms of domain separation. In fact, if we disable the "identification and reconciliation" engine and allow the transform map to insert the record directly into cmdb_ci, the insert works fine. Company and domain are applied as we would expect. This was the point I was making about asking why identification and reconciliation doesn't observe domain the way every other element of ServiceNow does.

The problem is that if we go back to directly inserting records into the CMDB, we lose out on the advanced features that come from identification and reconciliation.

We do actually have different user accounts for each company that align with this third party monitoring platform similar to your MID Server user suggestion. I haven't yet found a way to have the insert/import run as a different user based on the Company identified in the incoming record though.

So in order to do this based on the run-as user, we would need to apply filters and manually create and maintain 50 different scheduled jobs, all with their own run-as user? Unless there is a better process to do this that I'm not aware of?

I can't help but feel like I'm going about this the wrong way.

Deepak Ingale1 · ‎04-29-2019

Yes, one option would be to create 50 different users and run 50 import set scheduled job for against each user so system detects the domain correctly and you can leverage the power of Identification and reconciliation ( I & R )

If that is too much to do, I would suggest to give a try to gs.getSession().impersonate() API to impersonate the user grammatically in BEFORE transform script , this is just a theory and I have not tried with this, you may want to give it a try and see if this works. If it is working, that's the solution you might want to use with single import set schedule job but record being transformed is aware of the context of the user so that system can decide the domain appropriately and you can still leverage I & R engine capability of SN.

https://community.servicenow.com/community?id=community_question&sys_id=be41cfe5db98dbc01dcaf3231f96...

Note: Please mark reply as correct / helpful if it answered your question

Jon G · ‎04-30-2019

I like the impersonate idea, and in fact I had just found the same command yesterday. Let me give that a shot and I'll report back once I have done that.