Dynamic CI Groups and large groups of CIs

Alan Prochaska · Wednesday

Dynamic CI Groups appear to be a powerful tool. We want to set up DCIGs to connect all our infrastructure to the appropriate Technical Service Offering. DCIGs have a limit of 10K records. We have 150K printers and we want to get them associated with offerings and support groups.

Do we have to set up 15 DCIGs to cover the 150K CIs?

Similarly do we have to set up 5 DCIGs to cover 50K virtual servers?

Any guidance on how to break up large groups of CIs?

After assigning the CIs to a DCIG and relating that to a technical offering, will CMDB Query Builder see relationships from the printer to the offering and return them in a result?

Thanks for any help.

Alan Prochaska

Barry Kant · Wednesday

Hi Alan,

regarding those big numbers of assets it would be good to understand how the operational model for those assets is set up:
eg:
in a global company 150K printers are most likely spread around x number of sites (locations) which gives the option to capture all printers per location in a dynamic way.
50k servers in a data center could be done in a similar way if you can chunk them per zone.

Still sometimes it still is not easy to chunk it in logical groups.
For huge numbers I have seen occasionally that a TSO reference is added on the CI Class itself (kind of a reverse logic if you will), but it means that you need to manage this TSO data when ingesting the CI Class. Those are really edge cases for massive numbers in my opinion.

BR,
Barry

Alan Prochaska · Thursday

HI Barry,

Thanks for the response. I'm finding my way. I found a dataset < 10K members. Linux Virtual Servers - 8900 members.

I created the CMDB Group with an encoded query. 8900 members.

I created the Dynamic CI Group (DCIG) and attached the CMDB Group to it.

I created the Technical Service Offering (Linux Virtual Servers) and related it to the DCIG.

I created and ran a CMDB Query linking Service Offering to Dynamic CI Group. It returned no results.

I inspected CI Relationships (cmdb_rel_ci) and found a relationship between the technical service offering and the DCIG. So the relationship is there, but CMDBQ is not picking it up or returning a result for some reason.

The big picture is we want to home all infrastructure CIs to a technical service offering to a technical service so KPIs will compute and present in Digital Portfolio Management.

The problem case is ... given a server CI, trace it back to offering and service.

I was hoping I'd be able to trace it via server > DCIG > TSO > TS.

Is that an inappropriate expectation?

On the surface it looks like DCIGs allow you to set attributes on the member CIs but it doesn't create a relationship or link from the member CI (eg. server) back to the DCIG and service offering.

Thanks for any guidance.

Alan

Barry Kant · Thursday

Hi Alan,

yes, you are correct.
the structure is made of 3 different relating logics:

A reference (TSO to TS) --> References are used when there is 1 relation.
A CI relation (TSO to DCIG) --> CI relations are used for hierarchical models. (dependency view)
A CI association (CIs to DCIG) --> CI association is a flat view of the world. To speed up calculations. (In Dynamic Services x levels of the Hierarchical CI relations are also flattened to CI associations)

For impact analysis, reporting or visualizations this is a bit of a challenge as you need to use those relating logics to get a complete picture.

BR,
Barry

Mathew Hillyard · Thursday

Hi @Alan Prochaska

TLDR;

Don't create DCGs over the 10K CI limit and don't link multiple DCGs to a single Service Offering if the total number of CIs is over 10K - in both cases for performance reasons.
If you need manage a class of CIs that busts this limit, look deeper into the foundational data and break the CMDB groups, DCGs and Service Offerings down into smaller chunks. Yes, it can be painful, but it's necessary. Look at location, support type (gold/silver/bronze/VIP etc), user audience or SLAs to slice and dice.
The Dependency Views the CI form relationships section do not show which Service Offerings a CI in a DCG is connected to. Use the Unified Map (I think that's the correct name!) in CMDB Workspace.

Full answer:

There is no theoretical limit for a DCG. A system property sets the limit by default to 10,000 CIs but you could expand it.

However, you need to understand how this data is actually going to be used and the major pitfalls involved with large Dynamic CI Groups.

The DCG to CI relationship is stored in the Service Configuration Item Association [svc_ci_assoc] or SCIA table.
When filtering through CI relationships on an Incident, Change Request, or Problem, you will typically want to enter a Service Offering and have the CI field filtered to only show those CIs that are linked to that Service Offering, via any connected Service Instances. This includes both Application Services and Dynamic CI Groups.
The issue here is that you must go and find these relationships in the SCIA table. The only way to return the list of in-scope CIs is by using a reference qualifier on the Configuration Item field to traverse the Service Offering to DCG relationship, then search the SCIA table and return the in-scope CIs using a sys_idIN prefix. This is an inherently inefficient query method as it's basically picking each CI individually - when I tried this on a customer sub-production instance with a DCG with 40K CIs, entering a Service Offering then clicking the magnifying glass icon resulted in more than a 2 minute wait to bring up the list of filtered CIs.
A further issue arises when creating a Change Request and populating a DCG as the Configuration item. When you save the record a Business Rule "unpacks" the DCG's CIs and drops all of them into the Affected CIs related list. If you Refresh Impacted Services imagine the performance hit for individually calculating the impacted App Services, Service Offerings and Business Applications on (potentially) tens of thousands of CIs. I've tried this in a customer sub-production instance and it caused my user session to hang then timeout. I've yet to find a reasonable use-case for this functionality in DCGs with more than the low hundreds of CIs and have in some cases recommended deactivating this "unpacking" Business Rule.

There is also a technical limitation in the Dependency Views such that you can view which CIs are in a Dynamic CI Group but not see which Service Offerings a CI is related to via any of its Dynamic CI Groups. This is because the entire interface is hard coded to use the CI Relationship [cmdb_rel_ci] table - and as mentioned DCG to CI relationships are only stored in the SCIA table. I have raised an Idea with ServiceNow to fix it, and the response was to utilise the Unified Map in CMDB Workspace, and that the classic form/dependency view will not be fixed.

Regarding CMDB Query Builder, DCG to CI is not a CI relationship but a record in the SCIA table.

I hope this helps!

Mat