We use Record Producers to generate fairly complex Change Requests which then kick off orchestrations, and it ends up being easier to just trigger the RP programmatically than it would be to construct a Change from scratch.   It allows us to abstract the behavior in a useful way.

But, I found the answer!   We had conveniently scheduled a working session with SN's ITOM team for yesterday, and they were able to fill me in on what's happening.

If you look at the submit() method of the RBA_Base_RecordProducer code I included in the original post, you'll see that the Change insert() happens before the question_answer data is added.   We originally did this to retrieve the sys_id (and if the sys_id is not found, we want to avoid adding question_answer data -- it had previously been created but with an empty target_sys_id column, which actually took down our instance after we added too many like that).

So, the insert() on change_request inserts, as you would expect, a record into change_request.   What we had not considered is that this kicks off the workflow engine, and triggers the associated workflow, immediately.   According to Jesse on the ITOM team, the workflow transaction can be thought of as very greedy, or very high-priority.   This results in the workflow transaction thread basically ignoring everything that happens after it was triggered, including the question_answer inserts we were doing immediately afterward.

In light of this, I've modified the RBA_Base_RecordProducer code like so, around what was line 45:

var v;


// Set mapped fields on target record


for (v in this.rpVariables) {


       if (this.rpVariables[v].mapToField == true) {


               targetRecord.setValue(this.rpVariables[v].field, this.userVariables[v] || "");


       }


}




// set the sys_id here and record it, but don't insert the change_request record yet.


// the insert() call triggers any associated workflows, and once a workflow has // been triggered, its transaction thread has a set environment -- one without


// the question_answer entries we're about to add.


targetRecord.setNewGuid();


var targetSysId = targetRecord.getUniqueValue();




// if there's no target sys_id, don't create any question_answer entries:


// prevents a bug that was slowing down the whole system for some reason


if (targetSysId)


{


       // One more loop - insert variables in question_answer table


       var qa;


       for (v in this.rpVariables) {


               qa = new GlideRecord("question_answer");


               qa.initialize();


               qa.question         = this.rpVariables[v].sysId;


               qa.order               = this.rpVariables[v].order;


               qa.table_name     = this.targetTable;


               qa.table_sys_id = targetSysId;


               qa.value               = this.userVariables[v];




               qa.insert();


               gs.log(self.type + ': sys_id of inserted question_answer data: ' + qa.sys_id);


       }


}


else


{


       gs.log(self.type + ': no target sys_id, skipping question_answer inserts');


}




targetRecord.insert();




return targetSysId;


},

Key changes here are that the original insert() call has been replaced by code to return the new sys_id without actually performing an insert, and then at the bottom, that the insert happens just before the return call.   This means that the question_answer data is populated before the workflow is triggered, and allows the workflow to see that data.

This seems to work, although I haven't had time to thoroughly vet the solution yet.

As far as Mark Stanger's message, it's true that we didn't have native access to easily access Record Producers, but that's the point of the RBA_Base_RecordProducer Script Include: it sets up easy access to RPs from server-side scripts.