Data source import question

[Annika Aaltonen]

I have two import tables and two target tables. One import table is extending the other import table, so they are using the exact same columns. The target tables are completely separate.

I have a scheduled data source import set up for the "parent" import table which runs on the first of each month, + a transform which maps the data to target table 1. 

I am supposed to set up another scheduled data source import set up on the "child" import table which is supposed to run daily, + a transform which maps the data to target table 2. 

On both transform maps, I have the same field, located on the parent import table, set as coalesce: 'u_user_id'. The data that is being imported from another database to both my import tables is the same, except the scope of the daily imported data is a subset of the monthly imported data.

My concern is, will there be any unwanted updates to the records of the target table 1 (monthly) whenever the monthly import job runs due to this setup? I can't quite figure out how it will behave. It is important that the data on the target table of the monthly import stays unchanged as a "snapshot" between the monthly runs.

Therefore I wonder if I should revert my setup and keep also the import tables separate from one another instead of having one extend the other. 

Basically as it is set up now, while I have one field set as coalesce for both transform maps, on the parent import table this not a unique value. We will have "John Doe" exist with the same 'u_user_id' as part of the parent import table, and also coming from the child import table, since the daily source data is a subset of the monthly imported data. I do understand that transform maps are always specific to one import set, and they will not sweep the whole import table's content while triggered, but still I am not 100% sure this will work as intended. 

TIA,

Annika

[Ravi Peddineni]

@Annika Aaltonen 

There is no point in having the Parent & Child relationship for Import set tables. I would suggest you to create one more import set table for Target table2. There should not be any issue if you leave the current config as it is because the Import set groups will be different for both the import set tables while transforming the data. But still I would suggest you to have different import set tables.  

[Annika Aaltonen]

Hi Ravindranath,

Thanks for your reply! I have been digging into some documentation about import sets and I am also starting to think that this setup should probably work just fine and there won't be any crossovers, since the coalesce field is regarded only between the records within a specific import set and the target table data for each transform that runs. 

Doesn't that mean technically the setup would also work if I imported the data from both the monthly and daily imports to one and same import set table, as long as I keep two separate transform maps linked to each data source? Since the import sets would still make sure the right records are updated/created on the resp. target tables? Just trying to make sure I fully understand the ins and outs of importing data in SNOW.

Thanks,

Annika

[Ravi Peddineni · Accepted solution]

@Annika Aaltonen 

Yes, there will not be any issue technically if you use the same import set for both the transform maps.