IntegrationHub ETL updating instead of inserting

Ryan S
Mega Sage

3 weeks ago

I've inherited an IntegrationHub ETL configuration and am stuck on troubleshooting. I am importing new records to cmdb_ci_server and some of the records are 'updated' instead of 'inserted.' None of these records existed to begin with. I tried a test file with 4 servers. Obviously more attributes, but starting with what might be the culprit:

NameCompany
abcdAcme
abcdOceania
xyzAcme
xyzOceania

 

Yes, Acme and Oceania both have 2 servers with the same name. When I run the Robust Transform, I'm only getting 2 records inserted and 2 updated. What's weird is, if abcd for Acme gets created and the one for Oceania says updated, none of the values on abcd reflect Oceania so I don't know what it's actually updating.

 

How do I troubleshoot this? How do I get 4 records updated now, and later if I need to run an update (say, the status needs updated on Acme xyz) how can I run another import and ensure that record is updated?

0 Helpfuls
  • 631 Views
8 REPLIES 8

Vageesh Mishra1
Tera Contributor

3 weeks ago

Hi @Ryan S ,

You'll need to create a field using transform option available in preview and prepare data step, it will be combination of Name and Company, something like abcd||Acme and pass it as the source native key in your class mapping. 

Try this approach and you should find that it inserts all four records as per your requirement

 

Regards,

Vageesh

0 Helpfuls

Ryan S
Mega Sage
In response to Vageesh Mishra1

3 weeks ago

I'm not familiar enough with IntegrationHub ETL so if you have a moment to detail those steps I'd appreciate it, else I can research.

That said, IRE is supposed to be leveraged for this so why isn't the IRE taking care of it? It seems like this would be an unnecessary step.

0 Helpfuls

Simon Christens
Mega Sage
In response to Ryan S

3 weeks ago

Hi again

It should be fairly straight forward.

New transform - choose "Concantenation"
Defined output variable name.
Input variable 1 and 2 (name and company for ex)

Choose a char for concatenation - my ex uses underscore

SimonChristens_0-1769577700859.png

Then you do to the mappings and uses that variable for native key

SimonChristens_1-1769577874742.png

 

0 Helpfuls

Vageesh Mishra1
Tera Contributor
In response to Simon Christens

3 weeks ago

Hi @Ryan S ,

 

@Simon Christens has already shared the step to concatenate the field and pass it as Source Native key. Do follow the above steps and you should be able to see all the records being inserted as per your requirement.

 

Just FYI, their is a table called "sys_object_source" where you can see the entries for records being processed via the IntegrationHub ETL and their Source Native Keys.

Regards,
Vageesh

0 Helpfuls