Number of time onStart transform event executes.

asoni1 · ‎08-14-2019

As per the docs for the onStart transform event ->"When: The onStart event script is processed at the start of an import run, before any data rows are read."

My understanding was it will execute only once despite any number of records I might insert to the import set. But when I tried it, I saw it executes once for each record insertion in import set.

Is my understanding correct that all the event executes as many number of times as input records?

Thanks

ARG645 · ‎08-21-2019

Great discussion so far.

As you might already know, WebService Import sets have two modes, 1.Synchronous 2.Asynchronous

An import set with a Mode of Synchronous will transform the data as soon as it is inserted (provided that the transform map already exists). This import set will also have a default State of Loading. By default, all Synchronousimport sets will automatically be modified to Processed (the state is set to process) at midnight. As a result, when a new insert happens to the same table after the previous importset is marked processed, a new Synchronous import set will be created. Web service import set mode

An import set with a Mode of AsSynchronous will be in a state of Loading until all the records are inserted. Once all the records are inserted [We dont know how to detect it in your case] we have to run the job manually or run it via script [script is what you need]. To run it via a script, you can write an Hourly/minutely scheduled data import. You can find how to create a scheduled data import in the link: Schedule a data import

Now that you already developed logic to make your importset mode as asynchronous. You can find a scheduled job named "Asynchronous Import Set Transformer", this may be inactive by default. You can activate it/create a copy of it and change the repeat interval to every 10 minutes[your desired iinterval] and change the script in such a way that it only checks for your one particular data source.

Try the below script in the scheduled job

transformAsyncIset();

function transformAsyncIset() {
var igr = new GlideRecord("sys_import_set");
igr.addQuery("mode", "asynchronous");
igr.addQuery("state", "loading");//Because in your case the state will never change to "loaded"
igr.addQuery("table_name","YOUR IMPORT SET TABLE NAME");
igr.query();
  while(igr.next()) {  
    sTransform(igr);
    igr.setValue('state','loaded');
    igr.update();//Set the state to Loaded , so that it wont be captured by the job again and again. 
  }
}

function sTransform(igr) {
  var mapGR = getMap(igr.table_name);
  if(mapGR.next()) {
    var t = new GlideImportSetTransformerWorker(igr.sys_id, mapGR.sys_id);
    t.setProgressName("Transforming: " + igr.number + " using map " + mapGR.name);
    t.setBackground(true);
    t.start();
  }
}

function getMap(sTable) {
  var mapGR = new GlideRecord("sys_transform_map");
  mapGR.addQuery("source_table", sTable);
  mapGR.addActiveQuery();
  mapGR.query();
  return mapGR;
}

Please mark my eligible responses as helpful/correct if applicable.

Thank you,

Aman Reddy Gurram

View solution in original post

Manish Vinayak1 · ‎08-14-2019

Well even my understanding is that it should have worked only once at the start of the transformation of the rows. Similarly onComplete is supposed to run after all rows transformation get complete.

As you mentioned, yes the documentation does say, it is processed at the start of an import run, before any data rows are read.

https://docs.servicenow.com/bundle/madrid-platform-administration/page/script/server-scripting/refer...

Would you be able to share what part of script you added in the onstart transform script, and what was the result you observed?

Thanks,

Manish

asoni1 · ‎08-14-2019

Thanks for responding.

I am new to service now and doing this as a part of a task. I was just checking it out how these event works and which one suits my requirement.

I just created a glideRecord object on the import set and tried to spit out a column from records.

var gs = new GlideRecord('u_item_option_new');

gs.addQuery('u_cat_item','=',source.u_cat_item);

gs.query();

gs.log('Record value is : ' + gs.getValue('u_record_name'));

So i observed, it ran as many times as the input rows.

DirkRedeker · ‎08-14-2019

Hi

Your variable is called gs for the GlideRecord. But gs is already defined for the GlideSystem.

Try to rename your GlideRecord variable to gr for example, and try again.

Let me know if that answered your question and mark my answer as correct and helpful, please.

BR Dirk

asoni1 · ‎08-14-2019

Thanks Dirk, but i wrote gs by mistake. Its a keyword I know and i am not using gs.