Question about Export Sets

[George Cutrell]

We are exploring ServiceNow's Export Set capability as a means to export customer specific data sets.  In our custom CSM application, multiple customers use the same instance and application.  We have a custom data model that segregates customer A's data from customer B's.  Customers want their data in their own data stores to support their own business intelligence reporting.  And we're thinking using Export Sets is an optimal way of delivering daily differentials.  We'll have many customers using the instance where each customer is generating upwards of 2K-4K new and daily differential records daily. 

Should we have any concern if we create scheduled daily Export Sets to run for each customer?  Might there be contention issues?  Would we need to consider staggering the customer export sets?  Note, while we already use a MID Server for LDAP integration of user and group data, we think it would be best to have a dedicated MID Server(s) to support Export Sets.

Thanks in advance.

[Ratnakar7 · Accepted solution]

Hi @George Cutrell ,

When using Export Sets in ServiceNow to export customer-specific data sets, there are a few considerations to keep in mind to avoid contention issues and ensure optimal performance. Here are some recommendations:

1. Contention Issues: When multiple customers are using the same instance and application, running multiple Export Sets simultaneously can potentially lead to contention for system resources. This contention can impact the overall performance of the instance. To mitigate this, you can stagger the scheduled export sets for each customer, ensuring they don't overlap in their execution times. This helps distribute the load and reduces the likelihood of contention.

2. Dedicated MID Server(s): Since you anticipate a significant number of daily differential records for each customer, using a dedicated MID Server(s) for Export Sets can help optimize performance. This ensures that the export operations are offloaded to the MID Server(s), reducing the impact on the instance's resources. By using dedicated MID Server(s), you can also have better control over resource allocation and scaling.

3. Monitoring and Performance Tuning: Keep a close eye on the performance of your Export Sets and monitor the system resource utilization during their execution. ServiceNow provides various performance monitoring tools and features that can help you identify any bottlenecks or contention issues. Additionally, consider optimizing your export set configurations, such as specifying selective queries to limit the data being exported and optimizing the export set filters and conditions.

4. Incremental Export: Since you mentioned daily differential records, consider using incremental export techniques to export only the changed or new records for each customer. This can significantly reduce the export set size and processing time. ServiceNow supports various methods for incremental exports, such as using date filters or tracking record modifications through system fields like sys_updated_on.

5. Load Testing: Before deploying the scheduled export sets for all customers, it's recommended to perform load testing in a non-production environment. Load testing helps simulate the expected concurrent usage and workload, allowing you to identify any performance bottlenecks, contention issues, or potential limitations. Based on the load testing results, you can fine-tune your configuration and make any necessary adjustments.

Thanks,

Ratnakar

[George Cutrell]

Thanks Ratnakar.  Lots of good advice there.  Will share with the rest of my team.