NullPointerException when record producer code trying to load import set

[richard_cha]

Hello I am trying to implement a record producer based on sys_data_source that will allow the automatic loading of an attached CSV file.

I have seen a number of other posts and web resources that suggested that the following record producer server script should be successful in doing this:

current.file_retrieval_method = "Attachment";

current.name = "import " + new GlideDateTime();

current.type = "File";

current.format = "CSV";

current.import_set_table_name = "x_1837147_leasem_0_bank_transactions";

current.header_row = "1";

current.sheet_number = "1";

var loader = new GlideImportSetLoader();

var importSetGR = loader.getImportSetGr(current);

var ranload = loader.loadImportSetTable(importSetGR, current);

importSetGR.state = "loaded";

importSetGR.update();

var transformerWorker = new GlideImportSetTransformerWorker(importSetGR.sys_id, "a3ac083b935a431494bb70aefaba10f4");

transformerWorker.setBackground(true);

transformerWorker.start();

on submit this code should initialise a record in the sys_data_source table with appropriate values to allow import of a CSV file attached to the record, then  populate the import set and apply the transform map.

I see that the record is created in the sys_data_source table and it has the CSV file attached (as expected).

However the loading of the import set is not succeeding. In the log file I see the following NullPointerException error when trying to call ImportSetLoader().loadImportSetTable function:

 Stack trace:
 at sc_cat_item_producer.3a4b061f937a43d094bb70aefaba103e.script ➚:11
 at sys_ws_operation.4b9f0a8967101200d22b794717415a30.operation_script ➚:60 (process)
 at sys_ws_operation.4b9f0a8967101200d22b794717415a30.operation_script ➚:73
Root cause of JavaScriptException: java.lang.NullPointerException 
: java.lang.NullPointerException: Cannot invoke "com.glide.db.impex.datasource.DataSource.getLoader()" because "dataSource" is null:  com.glide.system_import_set.ImportSetLoader.loadImportSetTable(ImportSetLoader.java:51)
 com.glide.system_import_set.ImportSetLoader.loadImportSetTable(ImportSetLoader.java:45)
 java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
 java.base/java.lang.reflect.Method.invoke(Method.java:580)
 org.mozilla.javascript.MemberBox.invoke(MemberBox.java:268)
 org.mozilla.javascript.NativeJavaMethod.call(NativeJavaMethod.java:300)
 org.mozilla.javascript.ScriptRuntime.doCall(ScriptRuntime.java:3570)

Note this is with Australia release. Is the code I have been using no longer compatible?

Has anyone else successfully implemented a similar solution?
Any hints how to resolve the exception error?

[richard_cha · Accepted solution]

ok i have found a solution! 

i moved the code that loads the import set and applies the transform map outside the record producer server script. This is because the current record had not completed creation at time the original code was being executed. So a null value was being passed to the functions instead of a complete Data Source record.

the record producer code is now:

current.file_retrieval_method = "Attachment";

current.name = "import " + new GlideDateTime();

current.type = "File";

current.format = "CSV";

current.import_set_table_name = "x_1837147_leasem_0_bank_transactions";

current.header_row = "1";

current.sheet_number = "1";

I then created a custom event "myEvent" and moved the code to loads the import set and applies the transform map into a new scripted action based on the "sys_data_source" that is triggered on the custom event. 

scripted action code:

var loader = new GlideImportSetLoader();

var importSetGR = loader.getImportSetGr(current);

var ranload = loader.loadImportSetTable(importSetGR, current);

importSetGR.state = "loaded";

importSetGR.update();

var transformerWorker = new GlideImportSetTransformerWorker(importSetGR.sys_id, "a3ac083b935a431494bb70aefaba10f4");

transformerWorker.setBackground(true);

transformerWorker.start();

Finally  the record producers script was modified (added to the end) to queue the custom event for further processing by the scripted action.  eg. gs.eventQueue('myevent', current, "", "");

The sys_data_source record created by the record producer is then passed into the scripted action in current.

[richard_cha]

its should be noted that it was necessary to put the record producer and scripted action into global scope.

when the record producer was in my scoped app it was not possible for it to successfully insert into the sys_data_source table.  even after reconfiguring sys_data_source table to have create privilege and a cross scope access  privilege for my scoped application.

ran out of time investigating this further but I would really like to move it back into the scoped app eventually!