ServiceNow import data from multiple remote files (on MID server)

[RahulY00040]

Hello team,

We have a scenario in which we need to feed/import data from multiple files in ServiceNow staging table for transformation. Looks like current OTB configuration allows single remote file read to the data source.

Any option to load data from multiple remote files? Looking for least custom solution to avoid maintenance overhead.

Thanks in advance,

Rahul

[DineshS]

Can you clarify if thats multiple files with same structure going into the same import set table or is it is files of different structure feeding single import set table ?

[RahulY00040]

yes Dinesh. Those all files going to have same structure and if they can- can go to the same import set table.

Thanks,
Rahul

[DineshS · Accepted solution]

Hi Rahul,

You may push the files from MID Server into ServiceNow using the attachment API(or other methods as mentioned in https://support.servicenow.com/kb?id=kb_article_view&sysparm_article=KB0817437). Once it is in ServiceNow, you should be able to process them and load into target tables.

If you have IntegrationHub Pro(?), and if you are on a Windows MID server, you can run PowerShell commands from the Action in Flow Designer, to read, parse and load the file on to the table(s). In some cases, I have found this to be effective especially when we do not need all of the data from the files, but a subset. If you have the right license, the Powershell option will appear under Integrations when you add a step in the action in Flow Designer

[-O-]

What does "remote file" mean, better said, what is the protocol to "fetch" it and what is the format?
Depending on that you could:

- create a Data Source where Type is "Custom (Load by Script)" where you have the option to load any data you want in one go as long as the load can be scripted.

- create as many Data Sources as there are files and define a Scheduled Data Imports to make those run one after the other automatically, import data into the same Import Set Row table and be transformed by the same Transform Map.

As for the "least custom solution" part, usually data sources by definition require "custom solutions" unless they have a connector already developed and available in Store.