Migration of live workflows

Phonsie Hevey1 · ‎11-13-2019

While working for Client Solutions on a recent project there was a decision to be made as to whether to go with a Greenfield implementation or to stick with the heavily modified pre-Aspen instance of ServiceNow. Of course reverting to OOTB was the best option but there was internal resistance from the client due to the existing JML requests which were in process on the old instance.The JML process was based on a number of requests from the Service Catalog with associated workflows. The client did not want to have two production ServiceNow production instances running in parallel and it would have taken a long time for all the JML processes to work through to the end. This was a show-stopper for the Greenfield implementation.

After searching on the community the closest I got to help on this was Migrating Workflow contexts which had most of the information required. After a few days of investigation the script below was developed. Basically everything in ServiceNow is just data so as long as the correct tables are identified it's possible to move data between instances using update sets. The script was also based on two articles from ServiceNow Guru which are in the code comments below.

It took a long time, perhaps 8 hours, to generate the update sets, move them between instances, preview them and commit them.This was just for one month's worth of data. There was not enough time on the go-live weekend to move all the data so all closed requests were moved in the week prior to go-live. This is why the query on the request table used sys_id as the query. A list of closed sys_ids were stored in Excel to manage which requests were already moved and which needed to be moved on the go-live weekend.

In the end the client was very happy with the decision to revert to the OOTB instance of ServiceNow. The Change Manager has told me since that after the upgrade to New York all the new functionality just worked as the instance was kept as close to OOTB as possible.

Points to note

1. Workflows over 180 days may disappear, see this question for more details. Initially I thought the code wasn't working as the workflow weren't there when I looked for them. They had been imported but then were deleted by the clean up script.

2. The approvals were not included in this but were exported and imported as XML separately. The script could be modified to include these as well.

3. If I remember correctly, there were 400,000 entries in one of the update sets! There didn't seem to be a limit on this number. About 5 - 10 records per update set failed to preview but they were mainly on the sysevent table.

4. There were a number of preview errors in the update sets which consistently appeared on each month's worth of data. These included missing records on the sysevent table. There seemed to be some cleanup of this table going on as well but not in a consistent manner. This was not investigated further due to time constraints. There was also some references to a user which was not present on the source instance anymore.

This code was used to move the workflows from a Jarkarta to Kingston instance. It has not been tested on later versions. Be sure to test on a dev instance before using. It would be best to clone production and export the data for closed requests on a sub-prod instance.

// Code adapted from https://www.servicenowguru.com/system-definition/exporting-service-catalog-items-step/ 
// and https://www.servicenowguru.com/system-definition/manual-update-set-inclusion/

// The general idea is that a Request is structured as follows: 
// Request
//  -> Workflow
// 	-> RITM
// 		-> Workflow
//		-> SCTASKs
//			-> Columns
// 				-> Instances

var um = new GlideUpdateManager2();
var grReq = new GlideRecord('sc_request');
// *********
// Specific queries can be added here to grab multiple requests
// *********
grReq.addQuery('sys_idIN');
grReq.query();
gs.log(grReq.getRowCount() + " requests found which will be saved in the current update set.");

// Grab the request and associated workflow
while(grReq.next()) {
	saveToUpdateSet(um, grReq);
	
	// Get the workflow context data
	var grWFContextReq = new GlideRecord('wf_context');
	grWFContextReq.addQuery('id', grReq.sys_id);
	grWFContextReq.query();
	gs.log(grWFContextReq.getRowCount() + " work flow contexts found which will be saved in the current update set.");

	while(grWFContextReq.next()) {	
		saveToUpdateSet(um, grWFContextReq);
		getWFData(grWFContextReq.sys_id.toString());
	}	
	
	// Grab the RITM and associated workflow
	var grRITM = new GlideRecord('sc_req_item');
	grRITM.addQuery('request', grReq.sys_id );
	grRITM.query();
	gs.log(grRITM.getRowCount() + " RITMs found which will be saved in the current update set.");

	while(grRITM.next()) {
		saveToUpdateSet(um, grRITM);
		
		// Get the workflow context data
		var grWFContextRITM = new GlideRecord('wf_context');
		grWFContextRITM.addQuery('id', grRITM.sys_id);
		grWFContextRITM.query();
		gs.log(grWFContextRITM.getRowCount() + " work flow contexts found which will be saved in the current update set.");

		while(grWFContextRITM.next()) {	
			saveToUpdateSet(um, grWFContextRITM);
		
			getWFData(grWFContextRITM.sys_id.toString());
		}	
		
		// Get any SCTASKs under this RITM
		var grSCTASK = new GlideRecord('sc_task');
		grSCTASK.addQuery('request_item', grRITM.sys_id);
		grSCTASK.query();
		gs.log(grSCTASK.getRowCount() + " SCTASKs found which will be saved in the current update set.");

		while(grSCTASK.next()) {	
			saveToUpdateSet(um, grSCTASK);		
		}		

		// Get the related variables for this RITM
		var grVariables = new GlideRecord('sc_item_option_mtom');
		grVariables.addQuery('request_item', grRITM.sys_id);
		grVariables.query();
		gs.log(grVariables.getRowCount() + " variables found which will be saved in the current update set.");

		while(grVariables.next()) {	
			saveToUpdateSet(um, grVariables);	
			
			// Save the answers to the questions
			var grAnswers = new GlideRecord('sc_item_option');
			grAnswers.addQuery('sys_id', grVariables.sc_item_option.toString());
			grAnswers.query();
			gs.log(grAnswers.getRowCount() + " answers found which will be saved in the current update set.");

			while(grAnswers.next()) {	
				saveToUpdateSet(um, grAnswers);
				gs.log("**** Saving following answer to the current update set." +  grAnswers.value);
			}			
		}	

		getJournalData(grRITM.sys_id);
	}
	getJournalData(grReq.sys_id);
}

function getJournalData(sysID) {
	
	// Get the Sys Audits data
	var grSysAudits = new GlideRecord('sys_audit');
	grSysAudits.addQuery('documentkey', sysID);
	grSysAudits.query();
	gs.log(grSysAudits.getRowCount() + " sys audit records found which will be saved in the current update set.");

	while(grSysAudits.next()) {	
		saveToUpdateSet(um, grSysAudits);

		// Get the History Line
		var grSysHistoryLine = new GlideRecord('sys_history_line');
		grSysHistoryLine.addQuery('audit_sysid', grSysAudits.sys_id);
		grSysHistoryLine.query();
		gs.log(grSysHistoryLine.getRowCount() + " sys history line found which will be saved in the current update set.");

		while(grSysHistoryLine.next()) {	
			saveToUpdateSet(um, grSysHistoryLine);				
		}		
	}

	// Get the history set as well which might be reference in the history lines above.
	var grSysHistorySet = new GlideRecord('sys_history_set');
	grSysHistorySet.addQuery('id', sysID);
	grSysHistorySet.query();
	gs.log(grSysHistorySet.getRowCount() + " sys history set found which will be saved in the current update set.");

	while(grSysHistorySet.next()) {	
		saveToUpdateSet(um, grSysHistorySet);				
	}		
	
	
	
	// Get the journal entries
	var grSysJournalField = new GlideRecord('sys_journal_field');
	grSysJournalField.addQuery('element_id', sysID);
	grSysJournalField.query();
	gs.log(grSysJournalField.getRowCount() + " sys journal field found which will be saved in the current update set.");

	while(grSysJournalField.next()) {	
		saveToUpdateSet(um, grSysJournalField);
	}		
	
	// Get the related emails
	var grEmails = new GlideRecord('sys_email');
	grEmails.addQuery('instance', sysID);
	grEmails.query();
	gs.log(grEmails.getRowCount() + " emails found which will be saved in the current update set.");

	while(grEmails.next()) {	
		saveToUpdateSet(um, grEmails);
		// Get the email log
		var grEmailLog = new GlideRecord('sys_email_log');
		grEmailLog.addQuery('email', grEmails.sys_id);
		grEmailLog.query();
		gs.log(grEmailLog.getRowCount() + " email logs found which will be saved in the current update set.");

		while(grEmailLog.next()) {	
			saveToUpdateSet(um, grEmailLog);	
			// Get the email event
			var grEmailEvent = new GlideRecord('sysevent');
			grEmailEvent.addQuery('sys_id', grEmailLog.event);
			grEmailEvent.query();
			gs.log(grEmailEvent.getRowCount() + " email events found which will be saved in the current update set.");

			while(grEmailEvent.next()) {	
				saveToUpdateSet(um, grEmailEvent);					
			}		
		}		
	}			
}

function getWFData(grWFContextSys_id) {
	
	// Get the Workflow Transition [wf_transition] and Workflow Activity [wf_activity] of this
	// Get the WF Executing Activities
	var grWFExecutingActivities = new GlideRecord('wf_executing');
	grWFExecutingActivities.addQuery('context', grWFContextSys_id);
	grWFExecutingActivities.query();
	gs.log(grWFExecutingActivities.getRowCount() + " WF Executing Activities found which will be saved in the current update set.");

	while(grWFExecutingActivities.next()) {	
		saveToUpdateSet(um, grWFExecutingActivities);
				
		// Check if there's any Scheduled items in the queue related to these executing activities. 
		var grScheduledItem = new GlideRecord('sys_trigger');
		grScheduledItem.addQuery('name', 'WFTimer' + grWFExecutingActivities.sys_id);
		grScheduledItem.query();
		gs.log(grScheduledItem.getRowCount() + " Scheduled items found which will be saved in the current update set.");

		while(grScheduledItem.next()) {	
			saveToUpdateSet(um, grScheduledItem);		
		}	
	}

	// Get the Workflow Transition [wf_transition] and Workflow Activity [wf_activity] of this
	// Get the WF Activity History
	var grWFActivityHistory = new GlideRecord('wf_history');
	grWFActivityHistory.addQuery('context', grWFContextSys_id);
	grWFActivityHistory.query();
	gs.log(grWFActivityHistory.getRowCount() + " WF Activity History found which will be saved in the current update set.");

	while(grWFActivityHistory.next()) {	
		saveToUpdateSet(um, grWFActivityHistory);
	}
				
	// Get the Workflow Transition [wf_transition] and Workflow Activity [wf_activity] of this
	// Get the WF Transition History
	var grWFTransitionHistory = new GlideRecord('wf_transition_history');
	grWFTransitionHistory.addQuery('context', grWFContextSys_id);
	grWFTransitionHistory.query();
	gs.log(grWFTransitionHistory.getRowCount() + " WF Transition History found which will be saved in the current update set.");

	while(grWFTransitionHistory.next()) {	
		saveToUpdateSet(um, grWFTransitionHistory);
	}

	// Get the WF Log
	var grWFLog = new GlideRecord('wf_log');
	grWFLog.addQuery('context', grWFContextSys_id);
	grWFLog.query();
	gs.log(grWFLog.getRowCount() + " WF Log found which will be saved in the current update set.");

	while(grWFLog.next()) {	
		saveToUpdateSet(um, grWFLog);
	}
}

// It's probably better to just bulk export all the record in wf_activity, wf_transition and wf_condition because 
// these are all interrelated and it will end up in a loop if it's not checked which sys_id have already been save to the update set
// Entries in wf_stage and wf_version need to be bulk exported also
/*
function getWFActivity(wfActivityID) {
	
	var grWFActivity = new GlideRecord('wf_activity');
	var foundit = grWFActivity.get();
	if (foundit) {
		saveToUpdateSet(um, grWFActivity);
		
		// Check if there's any related Workflow Conditions
		var grWFCondition = new GlideRecord('wf_condition');
		grWFCondition.addQuery('activity', grWFActivity.sys_id.toString();
		grWFCondition.query();
		gs.log(grWFCondition.getRowCount() + " workflow conditions found which will be saved in the current update set.");

		while(grWFCondition.next()) {	
			saveToUpdateSet(um, grWFCondition);		
			
			// Conditions can have transistions too
		}	
		
		// Check if there's any related Workflow Transition (from)
		var grWFTransitionFrom = new GlideRecord('wf_transition');
		grWFTransitionFrom.addQuery('from', grWFActivity.sys_id.toString();
		grWFTransitionFrom.query();
		gs.log(grWFTransitionFrom.getRowCount() + " workflow transitions (from) found which will be saved in the current update set.");

		while(grWFTransitionFrom.next()) {	
			saveToUpdateSet(um, grWFTransitionFrom);		
		}
		
		// Check if there's any related Workflow Transition (to)
		var grWFTransitionTo = new GlideRecord('wf_transition');
		grWFTransitionTo.addQuery('to', grWFActivity.sys_id.toString();
		grWFTransitionTo.query();
		gs.log(grWFTransitionTo.getRowCount() + " workflow transitions (to) found which will be saved in the current update set.");

		while(grWFTransitionTo.next()) {	
			saveToUpdateSet(um, grWFTransitionTo);		
		}
	}

}

function getWFTransistion(wfTransistionID) {
	saveToUpdateSet(um, grWFLog);
}
*/

function saveToUpdateSet(um, record) {
	gs.log(record.sys_class_name + ' ' + record.sys_id);
	um.saveRecord(record);
}