The CreatorCon Call for Content is officially open! Get started here.

InfoSec Global integration crawling in PROD but very efficient in DEV & Stage

aislingo
Tera Contributor

yesterday

Something is affecting the speed of our ISG integration in PROD.

 

I have doubled the data sources and templates.

DEV = Zurich version, plug in 1.2.5

STAGE = Yokohama version 1.2.2

PROD = Yokohama version 1.2.2

 

DEV & Stage are fine. PROD is slow as hell, taking 3 days to process.

Error: Job exceeded processing time and was forced to complete status.

Any ideas? 

0 Helpfuls
  • 159 Views
1 REPLY 1

Connectmustaq
Kilo Guru

yesterday

Hi @aislingo ,

 

Greetings for the Day!!

 

The severe slowdown and processing timeout of your ISG integration job in PROD, compared to faster DEV and STAGE environments, despite similar Yokohama versions and plugin levels, indicates environmental or data-specific issues causing the performance.

 

Possible Causes:

  • PROD likely has much larger volumes of data and more complex dependencies than DEV or STAGE.

  • Increased data can exponentially increase processing time, especially if the data sources and templates have grown.

  • PROD servers might have stricter resource limits or slower performance (CPU, memory, I/O) than DEV/STAGE.

  • Review instance performance monitoring metrics to check for CPU spikes, memory contention, or database locks during the job.

  • Multiple jobs or integrations running concurrently in PROD could be causing contention.

  • Check if other heavy batch jobs run at the same time impacting resource availability.

  • This specific error means the integration job ran longer than the max allowed time, so ServiceNow forcibly ended it.

  • Investigate job logs for where the job is spending time—possibly in transforms, lookup, or script-heavy steps.

  • Confirm that PROD data sources and templates are identical in logic and configuration, not just count.

  • Complex templates or dependencies can cause inefficient processing.

  • Large data volumes require proper indexing in the underlying tables.

  • Check for missing or fragmented indexes that can slow queries in PROD.

  • Enable debug logging or telemetry for ISG integration jobs to pinpoint bottlenecks.

  • Compare query times, transform times, and staging vs. production differences.

Suggested Actions to Resolve

  • Reduce Job Batch Size:
    If configurable, reduce the batch/work chunk size to allow jobs to complete within time limits.

  • Optimize Templates and Transforms:
    Review and simplify templates or import set transforms consuming excessive time.

  • Add or Rebuild Database Indexes:
    Ensure frequently queried columns are indexed on tables used by ISG.

  • Stagger Job Runs:
    Schedule heavy jobs at different times to reduce concurrent load.

  • Scale or Upgrade PROD Resources:
    If possible, increase CPU/RAM allocated to PROD instance or database servers.

  • Contact ServiceNow Support:
    If performance issues persist, provide job logs and environment info to ServiceNow for deeper platform analysis.

If it is helpful, please hit the thumbs button and accept the correct solution by referring to this solution in the future it will be helpful to them.

 

Thanks & Regards,

Mohammed Mustaq Shaik