Use PDIs? Take our 5-minute survey to help shape the PDI roadmap.

The Only Way to Reduce Clone Times… Is to Stop Cloning So Much Data

corbettbrasing1
Mega Guru

This is for large ServiceNow customers—50M+ Task records, many ServiceNow products, multiple sub-production environments, and regulatory requirements requiring years of record retention.

Here is a real-world scale example:

84.6M Task records
1.65M active
83M inactive
~98% of the Task population is historical

The answer to “Why do you need 83 million inactive records?” is often legitimate: regulatory and records-retention requirements.

But retention doesn't necessarily mean all that history needs to remain in the operational production workload.

Why clone tuning only gets you so far

At a simplified level, the ServiceNow clone process looks like this:

BACKUP SOURCE
     ↓
PROVISION TARGET DATABASE
     ↓
RESTORE SOURCE BACKUP
     ↓
APPLY EXCLUSIONS
     ↓
APPLY PRESERVERS
     ↓
CLEANUP / CUSTOM SCRIPTS
     ↓
TARGET READY

That's an important detail.

The source backup is restored before exclusions and cleanup processing occur. So when PROD contains 50M–100M+ records, you're starting the clone process with an enormous production database—even if the desired lower environment is much smaller.

ServiceNow documents this lifecycle here:

ServiceNow — Exploring Instance Clone

And the available clone options here:

ServiceNow — Clone Options

At some point this stops being a clone-tuning problem and becomes a data-architecture problem.

One way to solve it: separate Operations from History

Use two production-purpose ServiceNow instances:

PROD_OPS — active records plus a short window of recently completed operational data.

PROD_DATA_LAKE — complete historical data required for regulatory retention, audit, reporting, dashboards, Performance Analytics, and analytics.

ServiceNow Instance Data Replication (IDR) continuously replicates the required data from PROD_OPS into PROD_DATA_LAKE.

Reporting, dashboards, PA, audit, and historical access can redirect users to DATA_LAKE. Operational links encountered there redirect users back to PROD_OPS.

The regulatory data is still retained. It just doesn't have to remain part of the operational workload forever.

 

corbettbrasing1_1-1788196670299.png

 

 

Now normal lower environments can clone the much smaller operational dataset instead of decades of history.

And when the complete 84M+ record footprint is actually needed for performance and scale testing, a dedicated LOAD/SCALE environment can be sourced from PROD_DATA_LAKE.

The principle is simple:

PROD_OPS runs the business. PROD_DATA_LAKE retains the history of the business.

Every environment gets the data volume required for its purpose—not all the data that has ever existed.

For those working with customers at 50M–100M+ Task scale: How have you solved this with your customers?

0 REPLIES 0