Concurrent import sets

[Hari24]

Hi,

How do we use concurrent import set with the existing Inbound webservice table, The data we are receiving it from third party application via ecc queue and inserting in import set staging table. Can we use the concurrent import set table.

Thanks,

Hari

[Sourabh26]

Hi,

To use concurrent import you need to create a Data source first as below - 

System Import Set --> Data Source

Once the data source is created in the related list create the Transform map (suppose in your case 3 maps has been created).

Define the order field to execute the transform map in that order. 

Then you can test it to load the first 20 records to test (from the related links in the Data Source "Test load").

Mark this as Helpful/Correct, if Applicable.

Regards,

Sourabh

[shloke04]

@Hari 

Yes, it can be used but it entirely depends on your use case which you are trying to implement.

Generically speaking, Concurrent Imports are used to Split incoming data into multiple import sets and transform the import sets concurrently to reduce processing time.

Running a concurrent import can be helpful when order does not matter and imports take a long time due to large data sets with time-consuming scripts. If order matters, you can split the import into multiple partitions to ensure that each partition is processed in order.

Note: Concurrent imports add processing and monitoring overhead. Use them only with large data sets.

Please refer the doc for more info on this:

https://docs.servicenow.com/bundle/paris-platform-administration/page/administer/import-sets/concept/concurrent-imports.html

Please refer to below video and blogs to learn on how this can be implemented:

https://pathwayscg.com/configuring-a-concurrent-import-set-in-servicenow/

https://support.servicenow.com/kb?id=kb_article_view&sysparm_article=KB0720801

https://www.youtube.com/watch?v=raiAXyvmG8w

Hope this helps. Please mark the answer as correct/helpful based on impact.

Regards,
Shloke

[Hari24]

Hi Sourab & Shloke,

Thanks for the reply!.  The existing import set webservice contains 7 staging table and more can we create datasource for all the seven staging table. if we are going head with the json file as attachment  for datasource, how the key value pairs are mapped from the json file to that of staging table. 

Regards,

Hari

[Sourabh26]

Okay, so below are some points that you can cross check from your end first -

Data Source and Transform Map (Import Sets) -

As you mentioned you have 7 different tables to which the data need to be created or updated then -

1. First you need to check the dependencies among the data to be loaded to different table. For ex. you need to update the user data in user table then first you need to load all the data of the user (location, department). It means you need to define the order in which the data will be loaded to different table as they will run concurrently once the data source will executed.

JSON as Attachment file -

If you are attached the file on Data Source then -

1. First check the type of the file (Excel,doc,csv,text) to retrieve the content from the file.
2. To map the staging table field from file you can go for onBefore Transform Map script. That will map the field before your transform map get processed.

Go through the below article for fetching the content from the attached file.

https://community.servicenow.com/community?id=community_article&sys_id=9ed305e2db63fb845129a851ca9619f7

Hope this helps you.

Mark this as Helpful/Correct, if Applicable.

Regards,

Sourabh