Is it OK to deactivate the Auto Flush (sys_auto_flush) record for workflow contexts?

GoBucks
Mega Sage

Is it OK to deactivate the Auto Flush (sys_auto_flush) record for workflow contexts?   I've read that older instances have this record.   Completed workflow contexts over 180 days old are deleted because of this record.   Newer instances do not appear to have this auto flush defined for workflow contexts, so I assume it is OK to set to inactive now?

Why did this exist at one time?

1 ACCEPTED SOLUTION

Mwatkins
ServiceNow Employee
ServiceNow Employee

Ok, Jeff, I have the answer now.



A long, long time ago, deleting the workflow context history led to an interesting problem with workflows starting afresh, years after the original flow completed. So, the Table Cleaner for Workflow Context was removed in Calgary - I did not realize it was so long ago. This was fine for a while but the flip side was that some heavy workflow customers started to see performance issues around the wf_context, wf_history, and wf_transition_history tables because they were growing so large. So, for Jakarta, we solved the original workflow restart issue and brought the workflow context cleaner back.



You can remove the Cleaner, but those tables will just continue to grow and the usefulness of the data in there is generally short lived.   If you delete it, or mark it inactive, you would own the record and wouldn't get our updates to it during upgrades.



So there is the long story.


Hope this helps!


View solution in original post

21 REPLIES 21

Matthew I appreciate your continued investigating on this issue.   So this record is present in older versions, not there in some of the more recent versions, returning in J...kind of confusing.   We just want to know, going forward, are wf_context records something that SN deems OK to keep on the platform, or only OK to keep for so long when completed (ie, 180 days)? Thanks.


For now I'm advising customers to put a "created on now" condition onto new workflows that are only valid on creation.   I still think it's certifiably insane that a workflow context can be flushed when the parent task is *ACTIVE*


Mwatkins
ServiceNow Employee
ServiceNow Employee

That makes sense, rfedoruk. Even if the workflow is finished, there is a legitimate business need to keep the workflow around until at least after the task is closed out. Here's a customization you can do to only clean wf_context records whose target record has been closed out. It works for tasks and task_sla records. It will only delete workflow contexts whose related task has been closed or whose related task_sla record has ended. I think that covers all use cases for workflow contexts. I have tested it out on my demo instance pretty thoroughly but please do your own testing to be on the safe side. It should be fairly performant, provided you have a database index on task.closed_at and task_sla.end_time. If you do not have these indexes I recommend adding them.



Robert, if you have not already, can you please open an incident with the HI team and reference this community thread and your other community post? We should open a Problem record in HI to track the issue. Given that the sys_auto_flush record comes back in Jakarta, I suspect that new customers will run into the issue that Robert described. I cannot be certain that just deleting the sys_auto_flush record is a safe option. However, I did not find any incidents in HI where this was found to cause performance impact and none of my colleagues were aware of this being a problem.



Alter (or create) the wf_context record as in the below screenshot:


Screenshot 2017-04-06 11.29.12.png



Create a global Script Include named "getWFContextID" with this code:


var getWFContextID = function(seconds) {


  //get all sys_id's of tasks that closed in the last 2 hours (i.e. since the last time the table cleaner ran)


  try {


  seconds = seconds || 15552000;


  var secondsAgo = new GlideDateTime();


  secondsAgo.addSeconds(seconds * -1);



  var result = "";


  var relativeTime = "RELATIVEGT@hour@ago@2";


  //grab all task records closed in last 2 hours


  var taskClosed = new GlideRecord("task");


  taskClosed.addEncodedQuery("closed_at"+relativeTime);


  taskClosed.query();


  while(taskClosed._next()) {


  result += "," + taskClosed.sys_id;


  }


  //now grab task_sla records


  var slaClosed = new GlideRecord("task_sla");


  slaClosed.addEncodedQuery("end_time"+relativeTime);


  slaClosed.query();


  while(slaClosed._next()) {


  result += "," + slaClosed.sys_id;


  }



  var wfIds = "";


  var workflowContext = new GlideRecord("wf_context");


  workflowContext.addQuery("id", "IN", result.substring(1));


  workflowContext.addQuery("ended", "<", secondsAgo);


  workflowContext.query();


  while(workflowContext._next()) {


  wfIds += "," + workflowContext.sys_id;


  }


  return wfIds.substring(1);


  } catch(e) {


  gs.log("getWFContextID failed: " + e);


  return null;


  }


};



I suppose this code might be better re-written to query all wf_context records over X age and then query their related task/task_sla records to see if they are still open but I would worry that that result set might start to get pretty large. The way I wrote it we only grab task/task_sla records that closed in the last 2 hours - should remain a fairly small, constant result set size.


rfedoruk what specifically are you using for the "created on now" condition?   We're on Helsinki, and I don't see a dynamic option for 'now' unless I'm missing something. I see 'created on current minute' for example, but not 'now'.   Perhaps 'relative - on - 0 - minutes - from now'? ..If that's valid...



Thanks.


Sorry, I actually used "Created on Today".   I did so knowing that any potential rogue workflow destruction would happen in the future, and I didn't want the workflow to re-trigger