IntegrationHub - How to prevent a record from being inserted (ignore = true)

[midjoule]

Hello experts,

I've setup an ETL throught IntegrationHub and I'm wondering how I can ignore records with specific attributes.

The support documentation proposes this script, slightly enhanced to match our requirement (in yellow). However, it doesn't work:

 (function(batch, output) { 
                for (var i = 0; i < batch.length; i++) { 
                        var in0 = gs.nil(batch[i].input) ? '' : batch[i].input; 
                        output[i] = in0 + " modified by script”;

//do not import if the value of the field attribute = 'ABC'
if (batch[i].attribute = 'ABC')
   batch[i].ignore = true
 
                    } 
                } 
            })(batch, output); 

How could I achieve this ? Is there any additional supporting documentation in regards to scripting within the IntegrationHub ?

[Andrew Barnes -]

Greetings midjoule,

 I am not an expert in ETL at all! I would try output[i].ignore = true;

-AJB

[midjoule · Accepted solution]

Thank you Andrew for the proposal. Unfortunately it didn't resolve the issue. 

In fact, we managed it without using any script, which is good. In fact there is a condition builder which supports our requirement:

1. Click the button Edit Class
2. In our case, we want to exclude records were both LNX and @@@ vendor_monitor are set (value is not LNX and value is not @@@)
3. As the else condition remains empty, the class is not set.

Here is the screenshot:

Even if we found the solution, I would still be interested to know more about the methods available in the batch and output variables, and especially on how to ignore a record by script. If anyone has a proposal, please write it here.

Best regards

[Long6]

How to get this work with Robust transform map?

[midjoule]

Hi Long,

I'm not sure to understand your question, because these screenshots come from IntegrationHub ETL which makes use of Robust Transform Maps.

My screen shots come from the IntegrationHub ETL application, which comes with the IntegrationHub ETL plugin. So you first need to install the plugin to use Robust Transform Map.

The above screenshots correspond to the step coming after the robust transform map step. The Robust Transform Map step transform the imported data (import set) by creating new columns.

Then the class mapping (above screenshots) comes into the picture by importing the records into the corresponding CI tables.

Hope this help.