IH-ETL multi-input script transform operation cannot use batch[i].column_name to access input value

[Siyou Li]

1. Go to IHTEL and create a new map with a nested payload (Ensure nested payload has a JSON layer where data is not in an array)

For example
{"results":[
{
"computer_id": 123,
"properties": {
"property_name": "xyz"
}
}
]}
2. Create a multi input script transform operation on property_name column
3. Use batch[i].property_name to access the input value in property_name column
4. Check that above access returned null/undefined
5. Check that it has to use batch[i]["properties.property_name"] to access the value

[Siyou Li]

In IH-ETL 3.0 release, we enhanced how IH-ETL creates the metadata entities. Currently we do not create separate entity for temp.properties if it is an object. And lift all the value to the parent temp entity itself. In this way, users can drag columns in both first level temp and next level properties without copy operation. This cannot be done in previous IH-ETL release because there is a separate temp.properties entity. However, with above changes, all columns in previous properties entity now all prefixed with properties.column_name. Our expectation is that all previously created maps will not be affected at all, while newly created maps after this release will have the new metadata and new column names.

IH-ETL provides 2 ways to access column data in "batch" parameter in multi-input script operation.

First is the the normal method, IH-ETL puts all specified input value in the "batch[i].input" parameter and separates them by "|" symbol in given order. This method is designed to bypass the column name change mentioned above.

Second is the advanced method, user can identify the value they want by access the column name. IH-ETL has always been expecting the column name to be the value stated in the "input column 1", "input column 2" boxes etc. And the column name is clearly stated as prefix_name + dot + column_name in the input boxes (see screenshot attached). Essentially we did not change how we access the data in the "batch" parameter, but we changed the column name. 

When user sees the value in input column input box containing multiple parts such as separated by dot, then user should use Javascript convention i.e. batch[i]["prefix_name.column_name"] to access the column value. In short, the column name should be exactly the same as it displayed in input column input box and enclose it with brackets.

[scottl]

If there are two input fields and I want to test the output of one field;

output[i] = batch[i]["prefix_name.column_name"] 

It outputs both values, and not the value expected. 

InputValue1|InputValue2

Is this an expected outcome? As that is not how I would expect it to work based on the comments by SN on how to use it. 

Update: So after being led astray by the SN code comment, as the input field name (prefix_name.column_name) contains a 'dot' clearly stating it is part of an OBJECT, retrieving it through the object notation does not work as stated in the comment.  And is actually accessed through...

batch[i].column_name

Thus, making this statement is incorrect;

And the column name is clearly stated as prefix_name + dot + column_name in the input boxes (see screenshot attached). Essentially we did not change how we access the data in the "batch" parameter, but we changed the column name. When user sees the value in input column input box containing multiple parts such as separated by dot, then user should use Javascript convention i.e. batch[i]["prefix_name.column_name"] to access the column value. 

🙄