The usage paradigm in case of transform maps with IRE is to actually cancel the transformation of all rows.

The only "role" played here by the transform map is to go through all the records in the import set transformed, call the IRE API for each row and than actually to cancel the transformation of the row. Therefore there is no need for coalesce either.

Consider that the transformation of one row might actually result in a CI created, updated or multiple CI being created or updated. So relating one import set row to a target row can be pretty problematic.

But if you are sure all rows in the import set will create or update a specific class, you could move the IRE API call into a sys_id scripted coalesce field and analyze the data returned by the IRE API call and based on that set the target record.

In order to coalesce based on scripts, one has to create a scripted field map for (target) sys_id and in that script return the sys_id of the matched record or -1 if a new record is to be created. SN in docs page Updating records using coalesce  describes it as "Conditional coalesce". The reason for moving the script from Before transform event script into a Scripted field map is because the latter (coalesce scripts) are actually executed even before the former (event scripts). In other words Scripted field maps that have as target field sys_id and are marked as coalesce "become" a special kind of Scripte field map.