Is it possible to control the row order visited by a Robust Import Set Transformer?

[Seong-HyunR]

Our product uses a Robust Import Set Transformer to ingest CI data from Kafka, and we noticed an issue which comes up when there are two rows of a single import set which feature the same CI but have conflicting data.

For example, we might have an import set which includes two rows for the CI "SYSG", one with the timestamp "2025-04-08" and the operational status of "Retired," and the other with a timestamp "2025-01-02" and the operational status of "Operational." The expected behavior is that the entry with the later timestamp should be ingested after the other, so that the most up-to-date entry is reflected in the CMDB. However, we haven't been able to find any way to designate an order for the rows within a single import set. 

We were able to find information regarding this feature within standard Transform Maps ("OrderBy" option in Advanced Options for Mapping), but this does not seem to be available in Robust Transformers. Is there any way to control the row order within an import set for a Robust Import Set Transformer?

[Shivalika]

Hello @Seong-HyunR 

But how is two updates regarding a CI coming at a time ? Shouldn't only the latest update be coming? 

We are also updating CIs but this never happened that we need to think previous update is coming. Like us this issue really happening in your system ? .

Kindly mark my answer as helpful and accept solution if it helped you in anyway. This will help me be recognized for the efforts and also move this questions from unsolved to solved bucket. 

Regards,

Shivalika 

My LinkedIn - https://www.linkedin.com/in/shivalika-gupta-540346194

My youtube - https://youtube.com/playlist?list=PLsHuNzTdkE5Cn4PyS7HdV0Vg8JsfdgQlA&si=0WynLcOwNeEISQCY

[Seong-HyunR]

I believe it is because our data source (Kafka connect) creates a record for every change detected within a database.

[amila]

Is this your custom solution? if you use asynchronous import sets, then transformation happen in the row order regardless of the transformation type. 

If your problem is to consume data from a customer side Kafka topic, better approach would be to use stream connect.

https://www.servicenow.com/docs/bundle/yokohama-integrate-applications/page/administer/integrationhu...

[Seong-HyunR]

Yes, this is our custom solution. I believe we are using synchronous import sets, and we are using REST API calls to obtain records from a Kafka consumer we create. Is there no way to change the row order for synchronous import sets?