Issue with Staging Table Record Creation & Running Only One Transform Map

[pgvaishnav]

Hi All,

I’m working on a requirement where records are sent to a staging table and then processed through a specific transform map.

However, I’m facing two issues:

1. When I insert a record into the staging table via script, it is being created twice.

2. When running the transform, all transform maps associated with the staging table are executed instead of just the one I want.

Here’s the script I’ve been working with:

var stagtable = new GlideRecord("staging_table");
stagtable.initialize();
stagtable.u_first_name = "test123";
stagtable.setWorkflow(false);
var it = new GlideImportSet("staging_table");
it.setSynchronous(true);
stagtable.sys_import_set = it.create();
var rgr = new GlideRecord('sys_import_set_row');
rgr.orderByDesc("sys_import_row");
rgr.setLimit(1);
rgr.addQuery('sys_import_set', stagtable.sys_import_set);
rgr.query();
if (rgr.next()) {
    stagtable.sys_import_row = rgr.sys_import_row + 1;
} else {
    stagtable.sys_import_row = 0;
}

stagtable.insert();

var importSetGr = new GlideRecord("sys_import_set");
importSetGr.get(stagtable.sys_import_set);
var importSetId = importSetGr.getUniqueValue();
var importSetRun = new GlideImportSetRun(stagtable.sys_import_set);
var importLog = new GlideImportLog(importSetRun, "SOAP Transform");
var ist = new GlideImportSetTransformer();
ist.setLogger(importLog);
ist.setImportSetRun(importSetRun);
ist.setImportSetID(importSetId);
ist.setSyncImport(true);
ist.transformAllMaps(importSetGr, stagtable.getUniqueValue());

So here I have two table transform map which is being used on two different target tables 
but when try to run the specific transform map it is not working

UseCase: The data is coming from 3rd party i have two add that in user table and hr profile table both are from different API's actually due to similar fields I am using single staging table 

1. When both the transform maps are active two records will be created on stagting table, import set run table as well with both transform maps.

How do I make sure the staging record is created only once? when i insert one record and is there any way we can run specific transform table for that record

Thanks in advance
Regards,
Prajwal

[Ankur Bawiskar]

@pgvaishnav 

you are inserting via script using insert() and import set will also create an entry

where have you written this script?

are you pulling data from some 3rd party API and then inserting data?

If my response helped please mark it correct and close the thread so that it benefits future readers.

[Ankur Bawiskar]

@pgvaishnav 

something like this will help

// 1. Create Import Set record for staging table
var importSet = new GlideImportSet('staging_table');
importSet.setSynchronous(true);
var importSetSysId = importSet.create();

// 2. Insert import set row(s) tied to the import set
var importSetRowGR = new GlideRecord('sys_import_set_row');
importSetRowGR.initialize();
importSetRowGR.setValue('sys_import_set', importSetSysId);
importSetRowGR.setValue('u_first_name', 'test123'); // Column in staging table
// Set additional staging table fields as needed
importSetRowGR.insert();

// 3. Load import set to process rows
var importSetGR = new GlideRecord('sys_import_set');
importSetGR.get(importSetSysId);

// 4. Prepare to run a specific transform map
var transformer = new GlideImportSetTransformer();
transformer.setImportSetID(importSetSysId);
transformer.setSyncImport(true);

// Replace with the sys_id of your specific transform map you want to run
var specificTransformMapSysId = 'your_transform_map_sys_id_here';

// 5. Run the specific transform map on the import set
transformer.transform(specificTransformMapSysId, importSetSysId);

// Optional: check status or logs after transform

If my response helped please mark it correct and close the thread so that it benefits future readers.

[Ankur Bawiskar]

@pgvaishnav 

Hope you are doing good.

Did my reply answer your question?

If my response helped please mark it correct and close the thread so that it benefits future readers.

[M Iftikhar]

Hi @pgvaishnav,

Both of the issues you’re hitting come from how the Import Set APIs behave:

1. Duplicate staging records

• In your script you’re doing both a manual stagtable.insert() and also calling GlideImportSet → create().

• This causes the same row to be created twice (once by you, once by the import set framework).
  To fix this, don’t call both. Either:

  • Insert into the staging table yourself without wrapping it in a GlideImportSet, or

  • Use the import set API end-to-end (preferred) and let it manage the staging record creation.

2. All transform maps running

• By default ist.transformAllMaps(importSetGr, stagtable.getUniqueValue()); runs all active maps for that staging table.

• If you want only one map, use GlideImportSetTransformer.transform(map, importSet, run) instead of transformAllMaps().

Example:

var mapGr = new GlideRecord("sys_transform_map");
mapGr.addQuery("source_table", "staging_table");
mapGr.addQuery("name", "My Specific Transform Map"); // map name
mapGr.query();
if (mapGr.next()) {
    var transformer = new GlideImportSetTransformer();
    transformer.setImportSetRun(importSetRun);
    transformer.setImportSetID(importSetId);
    transformer.setLogger(importLog);
    transformer.setSyncImport(true);
    transformer.transform(mapGr, importSetGr, stagtable.getUniqueValue());
}

• transformAllMaps() runs all active transform maps for the staging table.

• This script is the alternative when you want to run just one specific transform map.

Thanks & Regards,
Muhammad Iftikhar
If my response helped, please mark it as the accepted solution so others can benefit as well.