The Only Way to Reduce Clone Times… Is to Stop Cloning So Much Data
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
42m ago
This is for large ServiceNow customers—50M+ Task records, many ServiceNow products, multiple sub-production environments, and regulatory requirements requiring years of record retention.
Here is a real-world scale example:
84.6M Task records
1.65M active
83M inactive
~98% of the Task population is historical
The answer to “Why do you need 83 million inactive records?” is often legitimate: regulatory and records-retention requirements.
But retention doesn't necessarily mean all that history needs to remain in the operational production workload.
Why clone tuning only gets you so far
At a simplified level, the ServiceNow clone process looks like this:
BACKUP SOURCE
↓
PROVISION TARGET DATABASE
↓
RESTORE SOURCE BACKUP
↓
APPLY EXCLUSIONS
↓
APPLY PRESERVERS
↓
CLEANUP / CUSTOM SCRIPTS
↓
TARGET READYThat's an important detail.
The source backup is restored before exclusions and cleanup processing occur. So when PROD contains 50M–100M+ records, you're starting the clone process with an enormous production database—even if the desired lower environment is much smaller.
ServiceNow documents this lifecycle here:
ServiceNow — Exploring Instance Clone
And the available clone options here:
At some point this stops being a clone-tuning problem and becomes a data-architecture problem.
One way to solve it: separate Operations from History
Use two production-purpose ServiceNow instances:
PROD_OPS — active records plus a short window of recently completed operational data.
PROD_DATA_LAKE — complete historical data required for regulatory retention, audit, reporting, dashboards, Performance Analytics, and analytics.
ServiceNow Instance Data Replication (IDR) continuously replicates the required data from PROD_OPS into PROD_DATA_LAKE.
Reporting, dashboards, PA, audit, and historical access can redirect users to DATA_LAKE. Operational links encountered there redirect users back to PROD_OPS.
The regulatory data is still retained. It just doesn't have to remain part of the operational workload forever.
Now normal lower environments can clone the much smaller operational dataset instead of decades of history.
And when the complete 84M+ record footprint is actually needed for performance and scale testing, a dedicated LOAD/SCALE environment can be sourced from PROD_DATA_LAKE.
The principle is simple:
PROD_OPS runs the business. PROD_DATA_LAKE retains the history of the business.
Every environment gets the data volume required for its purpose—not all the data that has ever existed.
For those working with customers at 50M–100M+ Task scale: How have you solved this with your customers?
- Labels:
-
Architect