SG-SCCM v3.0.6 migration slowing entire instance

[Julie Gordon]

I recently migrated from the 2016 SCCM plugin to the Service Graph connector for SCCM v3.0.6 and the first pull of imports sets is making the entire instance so slow, it's unusable.  Anyone else run into this?  Know of a fix?

Please & thank you!

[Mike Marson]

Hi @Julie Gordon,

We are sorry that you are having issues migrating to SG-SCCM.  There are a number of items that could potentially impact performance and/or memory consumption, so it is difficult to diagnose without examining the specifics of your instance and SG-SCCM execution.  That being said, there are a few comments I can make.

(1) What we have seen in some cases is the presence of a significant number of active partial payload records in cmdb_ire_partial_payloads can affect both memory and performance.   If you have more than a handful of records in cmdb_ire_partial_payloads, that is a potential concern. As a temporary test, deleting all records in cmdb_ire_partial_payloads, cmdb_ire_partial_payloads_index and cmdb_ire_incomplete_payloads and then running SG-SCCM again (full run, see #2 below) would be an option. 

(2) We strongly recommend (and is a Guided Setup step) to clear the "Last Run Datetime" for at least the Computer Identity data source, to ensure a full load of computer data at least the first time.   First run should really do a full load for _all_ SG-SCCM data sources, to ensure data and reference integrity.   (As an ongoing configuration, you might want to consider disabling "Use Last Run Datetime" for the Computer Identity data source(s) specifically, to ensure there is no possibility of a gap in computer records from SCCM, that would have downstream implications in terms of creating partial payloads.)

(3) As SG-SCCM leverages concurrent import sets and multiple threads for processing, we have seen issues where misconfiguration of the cluster nodes and Import Set Transformer Jobs (in sys_trigger) can cause performance issues.  This is described more in KB1002706. 

(4) Due to the changes introduced with SG-SCCM 3.0 for multi-instance support, the Guided Setup process has changed significantly.   Misconfiguration can have various negative implications.  If you have not done so, we recommend reviewing/following KB1001248 to properly setup SG-SCCM 3.0.  This is true even if you are not leveraging multi-instance support.

Note for all of the above that it is strongly recommended to do any configuration changes and testing/validating in a sub-prod instance that is a recent clone of production, before actually changing a production instance. 

As I said, the above are not an exhaustive list.  If you have ongoing issues, the next recommended step would be to open a case with ServiceNow Support, who should be able to resolve your issues.

Regards,

Mike

[Ian Mildon]

Uncheck the "Concurrent Import" checkbox on each of the import records. While it may sound odd, I've found that doing this speeds things up.

However, the initial import/transform after setup and clearing the last run datetime field can take a large amount of time. The Software transform itself can run for days.

[Mike Marson]

@Ian Mildon,

If you are experiencing issues with Concurrent Import sets, you might be experiencing (3) above from my comment to Julie Gordon.   Depending on the characteristics of the instance you are running on, Concurrent import sets will normally be a significant performance boost.  Worst case (on a single node instance with 1 thread/Import Set Transformer Job) it will be the same behavior as non-concurrent import sets that only use a single thread for processing the imported data.     Worse performance should not happen and is indicative of a problem in the instance or the Service Graph Connector configuration (or customization if any).

The performance of Concurrent Import Sets is dependent on the # of nodes in the instance, where more nodes == more possible concurrency.  There are other factors too, primarily regarding the load rate from the external source (e.g. SCCM).

If you are seeing worse performance, please review the items described above in my message to Julie Gordon, and suggest opening a case for Support to review if you are unable to resolve your issues.

Regards,

Mike