IH-ETL FAQ - Map loading is very slow or loading never completes

[Siyou Li]

1. Adjust preview size property and use small size import set

Check log to see if below error shows up:

api_name:sn_int_studio/studio, resource:sn_int_studio/studio/metadata/{appFeedSysId}

Transaction cancelled: maximum execution time exceeded

This error usually happens when customer has a really large import set and uses very complex structured nested payload. IH-ETL needs to first scan through raw data to determine nested payload schema and then transform by default 100 rows in provided import set.

We suggest to use a small size import set to test and set "sn_int_studio.preview.size" system property value to below 10. This will reduce the rows which IH-ETL needs to scan and process.

2. Check if there is large size column transform operation

Column transform operation such as "Cleanse Hardware Model" will query/insert into "core_company" table. If this table has really large amount of records, then the loading time could be really long due to processing these transforms. We suggest to check and clean these tables if necessary.

Additional Resources

Landing page

Common error scenarios