How To Use IntegrationHub ETL

[Siyou Li]

Articles to provide guidance on using IntegrationHub ETL (IH-ETL) application to integrate and ingest third-party data into CMDB.

In order to provide guided set-up experience while reading, we separated the content into below step-by-step sections: 

1. IntegrationHub ETL Introduction

2. Prepare your data before creating an ETL transform map

3. How to set up an ETL transform map using IH-ETL UI

     a. Input basic details for your transform map

     b. Preview your raw data and apply data transform operations

     c. CMDB Class mapping

     d. CMDB field mapping

     e. CMDB relationship mapping

4. Test and preview the sample integration results

5. Common error scenarios

In section 2 and 3, we will split each section’s content into two halves. First half presents an example configuration with step-by-step screenshots. Second half presents important IH-ETL function tips and common caveat using bullet points or check list style. 

In the last common error scenario section, we provide debug process and detail examples for common error scenarios. 

[Niclas]

Can Integration Hub ETL also be used to import Data to other Tables then CMBD? For Instance an API sends you a nested JSON Payload with Groups and Group Members. 

[Siyou Li]

Non-cmdb related features are not fully supported in current IH-ETL version as for now.

[Wiggins Timothy]

IntegrationHub ETL is the framework in which Service Graph Connectors populate the CMDB with data. ServiceNow continues to adapt the platform with new features and capabilities that allow us to populate the CMDB, leveraging new approaches that utilize the native Integration and Reconciliation Engine (IRE).

GreenSkyOnline Login

[newhand]

HI@Siyou Li 

I'm using integrationHub ETL to import data to CMDB(about 10000 records)

When i  clicked the “Run Integration” button，after about 5 miniutes later ,

finally the page shows "View results for sample integration test using the ETL Import Tables Data ETL Transform Map. You can return to any step to make changes." without any other information.

The system log shows "Transaction cancelled: maximum execution time exceeded".

I  don't know  know how to increase maximum execution time.

I need some help。Thank you！

[Gene7]

Hi @newhand 

I believe the sample testing only uses 2K records, maybe reduce your sample data source file to a good collection of your data, with a max of 2K records.  

I use 30K in my scheduled monthly import, and it works just fine.  

[newhand]

@Gene7 

Finally i successfully increaced the maximum execution time.
However my input can't be processed successfully.
I'm running ETL on my personal instance ,maybe the  instance specification is to low to support a large input....
But, it would be much better if it could give me an error like 'out of memory' or something, instead of nothing at all...

[Aravin_C]

Hello @newhand 

Im also facing the same issue in my PDI. Could you please help me out to solve this.

Thank you!

[jsb81]

Does this mean we can bring information from assets like firewall, switches wifi access point which is monitored by solarwinds or FortiManager into ServiceNow?