Issue:
The import set data is partially processed and I could not see data in CMDB.
Root Cause:
There are couple of reasons for data going in to partial load (cmdb_ire_partial_payloads):
- ETL was not able to form well defined Source Native Key (aka. SNK) with partial data received from the source.
- The data doesn't have the key elements required for the identifying the CI. It varies for every CI as defined in the IRE Identifier Entries.
- The incoming data which has relationships with other CIs doesn't have key joining elements. For example, printer associated to a computer doesn't have computer's identity data for which IRE is not able to establish relationship.
- And many more ...
This article describe a particular case task for SG-JAMF where the Printer data coming with empty string. In one of the case task for SG-JAMF, we found the data is coming with empty string for which ETL was not able to form a valid source native key.
The source (JAMF) should either send a proper data or an empty array [], by which the application will handle it. But here the empty string is causing bad data fed in to IRE which puts it in to partial payload as IRE is not able to find a match.
Solution:
For this type of issue, we need filter the data in ETL so that bad data doesn't goes in to IRE. JAMF comes with nested payload (JSON) and current ETL UI doesn't provides support to perform null checks at the ETL UI. There is a feature request raised with the ETL UI team, which will be available in upcoming releases.
For now, here is how we can solve the to filter out the bad data going to IRE.
- Login in to Instance
- Go to cmdb_inst_application_feed.list
- Click on RTE Entity Mappings tab and search for "temp.hardware.mapped_printers[*]ToPrinter 1
- Check on 'Is Conditional' which prompts you to add code.
Based on the structure of the nested data, you need to perform appropriate checks and push the corrupted data to failingPaths, which will skip the data processed by IRE.
For more references, here are couple of community articles related to partial payload.
