I have a really strange phenomenon involving import sets. This is on the Helsinki release. It seems that if I have two transform maps for the same import set, somehow the import set table produces a duplicate of all the data rows from the original import set!
A very simple CSV file with 9 rows containing machine names and some hardware data is loaded into a table named 'test_import'.
It looks like this:
Asset Tag,Appliance,Platform,Module,Component,Manufacturer,Model,PN,Man PN,Man SN
13752,ilauctsa01a,Dion,ilauctsa01a,Chassis,,1-u Intel D-Generation Edge Cache/SAS,,2602-556242,VM15AS001990
13752,ilauctsa01a,Dion,ilauctsa01a,Cpu/Processor1,Intel,Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz,,,
13752,ilauctsa01a,Dion,ilauctsa01a,Cpu/Processor2,Intel,Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz,,,
13752,ilauctsa01a,Dion,ilauctsa01a,Memory1,Samsung,8GB DDR4-1866 SDRAM DIMM Module,,M393A1G40DB0-CPB,418A74A0
etc...
I created two test transform maps that run scripts. They are called Test Import One and Test Import Two. They both look like this:
(function transformRow(source, target, map, log, isUpdate) {
// Do nothing
ignore = true;
})(source, target, map, log, action==="update");
After loading the data, if I look at the import set table named 'u_test_import' there are 9 rows.
Then I run the do-nothing transforms:
After the transform there are now twice as many rows in the IMPORT set table as before:
So the "transforms" I had did nothing (ignoring every row) and yet my INPUT table has doubled in size. As you can imagine this is not a good thing with massive data sets (which is unfortunately how I discovered this).
Can anyone explain what is going on here? Is there something I'm doing to cause this that I can work around?
Thanks in advance for any help!
