Data load without impacting system performance

[rambo1]

Hi Team,

I have a requirement to load huge data (more than 5 lakhs)

I can load it using Integration or manually via excel, I would like to know what would be the best approach to load without impacting system performance.

I know regading switching off 'run business rules', 'auditing target table' and 'notifications'

Avoiding complex maniplations during transform in transform script .

these would improve load performance but I would like to know some good solution like pagination, batching , I know how to use pagination while sending data out but I am not sure on how to use pagination for incoming data also I think its not possible as data is not present in system .

Also how do I distribute load on all nodes equally ? is it automatically done by servicenow.

Is importing and transforming done seperately , like one after other ? can we make it simultaneously batch wise.

Please suggest

[Swapna Abburi]

Hi @rambo1 

You can use Robust Transform Engine (RTE) in place of Transform maps as it processes data in batches.

https://www.servicenow.com/docs/bundle/xanadu-integrate-applications/page/administer/import-sets/con...

[Ankur Bawiskar]

@rambo1 

you can divide it in chunks and load manually during off business hours

node handling is not in your hand

If my response helped please mark it correct and close the thread so that it benefits future readers.

[rambo1]

we can divide data into parts if its a manual load , what is data is coming form external source via REST ? how do we manage ?

[AshishKM]

Hi @rambo1 ,

You must know all data and data type and some unique parameter which differentiate the data and you can make set of whole data in smaller chunks for upload. I assume this is one time activity.

Data is coming via REST, means either you are pulling from external system or some external system is pushing in ServiceNow, either case you have to deal with manual effort. Try to make it in smaller set of data, because if its fail, you have to do it again and again which will be time consuming efforts.

-Thanks,

AshishKM