IntegrationHub ETL and DataSource from REST API

[anwie]

Hello together,

I am just trying out the possibilities with IntegrationHub ETL, because we want to fill the CMDB for our customer from different sources. And this relative new functionality seems nice.

As I understand, the source for an ETL Transform Map is a data source. My first steps for learning I have done with uploaded files (used the course in LeaningNow).

So now to my question: is the following scenario possible?

1. 3rd party sends data per scripted REST API into a defined table
2. this table should be the data source for an ETL Transform Map, which runs everey time, when new data is send per REST

But as data source I did not find a suitable option. So the scenario is not realizable? Or is there another way that we could use both scripted API in combination with IntegrationHub ETL?

Hope I could describe the situation understandable.

Thanks and kind regards,
anwie

[Avinash Dasari]

Hi Randy,

Could you please provide detailed steps?

Thanks,

Avinash

[Randy Fisher]

Avinash,

I am not sure to level of detail you are looking for but would be willing to help out.  Are you asking for more detail on the data source setup or IntegrationHub ETL?  ETL is fairly straightforward if you follow the guided setup and have some experience with transform maps but it would probably be too long to get into detailed steps on the setup and there is already some pretty good documentation and videos as well.  The data source setup was the only piece I was missing.  Let me know what specifically you are looking for and I can try to provide some more info.  Thanks

Randy 

[Avinash Dasari]

Thanks for response Randy!

I was actually looking some inputs around for below scenario.

1. 3rd sends the data into ServiceNow import set table using Scripted REST API.

2. I need to process that data using Integrationhub ETL.

Not sure we could use the combination of Scripted REST API and Integrationhub ETL.

Regards,

Avinash

[Randy Fisher]

Okay.  So I mentioned in my first reply, I started with an spreadsheet export from the 3rd party.  I would start with seeing if you can get something similar from yours or create it yourself assigning each field in the payload to a column in the spreadsheet. Once you have it, here is what I did:

1. Create a new data source with the type as File and the format to whatever your spreadsheet is.  Set the file retrieval method as attachment and attach the file to the data source. 

2. From the data source, "load all records" to build the import set tables.

3. Once the import set tables are in place, update the format on the data source to JSON and set the "path for each row."  For me, I set the path to /records/records

4. Have the payload from your 3rd party be in the format to match your path.  Again, for me it was:

{    "records":

[

     {

     "description":"CI Desc",

     "categories":"CI Category",

     "name":"CI name"

      }     

   ]

}

5. Have your 3rd party start sending the payload to the API

https://instance-name.service-now.com/api/now/cmdb/ingest/data_source_sys_id 

6. Update ETL to the new data source.  

That was pretty much how I did it.  Let me know.  Thanks

[Kenneth3]

Hi Randy,

I have exactly the same use case as Anwie, namely:

1. I import the 3rd party data via a REST API (with Flow Designer) into a custom table.
2. This table should be the Data Source for IntegrationHub ETL.

My question is: is it possible to import this data into this ServiceNow API?

Regards,
Kenneth