Rest API: Import large number of Records

[chandukollapart]

Hi,

We are integrating with 3rd party application to import/create records in ServiceNow. In the 3rd party application, we have millions of records. So we are looking for best way to create/import those records. Also do we need to implement any pagination? If that is the case is there any sample pagination script.

Thanks in advance

[Anand Kumar P]

Hi @chandukollapart ,

Pagination—

https://www.servicenow.com/community/architect-forum/pagination-to-import-all-records-with-custom-da...

https://snprotips.com/blog/2016/9/1/gliderecord-pagination-page-through-records-with-ease

https://support.servicenow.com/kb?id=kb_article_view&sysparm_article=KB0727636

Events—

1. The sysevent table caches Elastic document blocks, storing the target table name in parm1 and a JSON array of index documents in parm2.

2. Import is handled by Script Actions triggered by events, with parallelization achieved through multiple event queues and using all application nodes via sys_trigger.

https://www.servicenow.com/community/now-platform-articles/how-to-import-4-million-records-in-3-hour...

Mark it as helpful and solution proposed if it serves your purpose.
Thanks,
Anand

[Ravi Chandra_K]

Hello @chandukollapart 

You can use the approach explained by Maik here, bit of background and work needed but it is effective without any performance loss.

https://www.servicenow.com/community/now-platform-articles/how-to-import-4-million-records-in-3-hour...

Please mark the answer as helpful and correct if helped.

Kind regards,

Ravi Chandra 

[Vamshi5]

HI @chandukollapart ,

Have you tried data stream?

https://www.servicenow.com/docs/bundle/xanadu-integrate-applications/page/administer/integrationhub/...