CMDB Data Manager Archive Policy stuck at 93.95% / Sub-tasks pending

[KanteS]

Hi all,

I am running a CMDB Data Manager Archive policy in our DEV environment to archive retired Network Adapter CIs. The initial target scope was approximately 900,000 records.

The policy is processed automatically through the out-of-the-box scheduled job “CMDB Data Manager Archive/Delete Policy Processor.”

What We Tried

To improve performance , we modified the system property glide.db.archive.chunk_size and increased its value to 25,000.

However, after running the processor , no records appeared in the Archive Job Execution Chunks (sys_archive_run_chunk) table related to this Archive data manger poilicy. The execution successfully archived the vast majority of records but stopped at 93.95% completion, leaving approximately 2,100 records in the live table. In the backend Policy Execution Instance (cmdb_data_management_policy_execution), the execution appears to be stuck, with five sub-tasks remaining in a Pending state in the Policy Tasks (cmdb_data_management_task) table.

We would like to understand whether this behavior is related to the chunk size configuration, stuck policy tasks, or another issue within the archive processing mechanism.

Please help me with the following: 

1. What is the recommended approach for implementing and successfully completing a CMDB Data Manager Archive policy when processing a large volume of CIs (approximately 800,000 records)?

2. What troubleshooting steps should we follow to identify why approximately 2,100 CIs remain unarchived? How can we determine whether the issue is related to individual CI records, policy conditions, data inconsistencies, or stuck policy tasks?

3. What could be the possible reasons for no new records appearing in the Archive Job Execution Chunks (sys_archive_run_chunk) table when the scheduled archive processor runs? How can we identify the underlying cause?

4. What is the safest, ServiceNow-supported approach to recovering a policy execution instance when some policy tasks remain in Pending status? How can we resume processing without compromising the existing execution history or causing duplicate archive operations?

5. If we revert glide.db.archive.chunk_size to its default value of 1,000, how can we safely trigger the Data Manager engine to re-evaluate and process the remaining 2,100 CIs and complete the existing execution?

6. Are there any documented platform limitations, recommended chunk size values, database transaction limits, or timeout considerations when processing large archive volumes? Could increasing the chunk size significantly affect archive execution?

Any guidance on the recommended troubleshooting and recovery steps would be highly appreciated.

Thanks in advance!

[kmohammed]

Hi, a few pointers that should help you work through your questions in order:

Chunk size first (Q1, Q6): revert glide.db.archive.chunk_size to the default 1,000 before you retry anything. 25,000 makes each archive chunk a much larger single transaction - for CI tables with relationships that is exactly the pattern that times out / rolls back, which also explains Q3: if the chunk fails as a unit, no completed chunk rows survive in sys_archive_run_chunk even though the processor job ran. Bigger is not faster here; the default exists because archive work is deliberately done in small, restartable pieces.

Why ~2,100 remain (Q2): check the remaining records against the policy conditions again - do they still evaluate as in-scope (class, retired state and any other condition), or have some changed since the execution was created? Then separate a small sample (10-20 records) and test-archive just those with the default chunk size. If the sample archives cleanly, your problem is scale/transaction size, not the records. If the same records fail in isolation, look at those CIs individually - references/relationships that cannot be followed are a data problem on those records, not a job problem.

Recovery (Q4, Q5): do not edit cmdb_data_management_task or the execution states by hand, and do not try to force the pending sub-tasks to resume. The supported pattern is: revert the property, stop/cancel the stuck execution rather than leaving it Pending, and run the policy again so Data Manager creates a fresh execution. That is safe against duplicates because it re-evaluates the live table - the ~28k already archived are no longer in cmdb_ci_network_adapter, so the new execution will only pick up records that still match, i.e. roughly your remaining 2,100. Changing the property alone does not re-drive an execution that is already stuck.

For the volume generally: split very large archive runs into smaller policy executions (by date range or sub-class/condition) and run them off-peak, watching sys_archive_run_chunk grow steadily - steady chunk creation is your health signal; a long gap with Pending tasks and no new chunks means stop and fix rather than wait longer.

Since this is DEV you can prove the whole recovery safely there first. If, after reverting to 1,000 and creating a fresh execution, chunks still do not appear, that remaining behaviour is a platform/job issue - raise a Support case with the policy execution sys_id and the processor job's run times, rather than working around it with table edits.