There once was a customer that had tasks with unusually long lifecycles. These tasks would frequently be open and relevant between 6 months and 1 year. The tasks were associated with a complex approval / data population workflow that was, by comparison, very short. The effect looks like this...
THE FIRST HINT THAT SOMETHING WAS WRONG
For the first few months, the customer was elated at the business value being provided by this solution (it helped them ace a government audit). But suddenly a large number of tickets mysteriously "started over". A second set of approvals launched. Data overwritten. Notifications sent. On further inspection, it appears the workflow had launched a second time. Digging deeper, a large number of older Tasks simply had no workflow. Some of you may remember this from my The mysterious case of the disappearing workflows thread.
WORKFLOWS ARE FLUSHED ON OLD INSTANCES!!
The horrifying root cause to this issue is that on older instances of ServiceNow, completed workflows are flushed after a few months. Take a look at the visualization above. That workflow completes in a couple days. Here's the culprit. Workflow context table flushes "ended" workflows after 15 million seconds.
The extra scary fact is those workflows are GONE. There is *no* hope of repair. So imagine a large, complex task, which is critical for audits, that instantly has months of history wiped out. No joke people, this is the kind of thing that mobilizes whole teams of lawyers. The good news is that this only appears to exist on OLD instances of ServiceNow. On fresh instances of Geneva or Helsinki, this auto-flush record does not appear to exist.
HARDENING YOUR SOLUTIONS
After discovering the issue, our intrepid customer took two courses of action to harden their long-lifecycle Tasks against workflow flush/reset
1) De-activate sys_auto_flush records for wf_context
This appears to be a safe option, since later versions of ServiceNow don't even have the record.
2) Place a "now" condition on your Workflow conditions
Since de-activating the sys_auto_flush record only prevented new context flushing, the customer still had a terrifying problem of plentiful tasks with no workflow. They still had to prevent new workflow launches on existing Tasks with purged contexts! To accomplish this, they added a condition to the Workflow's triggering: created = Now. Since the tasks in question would already be months old, this condition would be false, and no new workflow would be run. It guarantees that a workflow "only runs once ever" no matter what other legacy issues exist.
TL;DR
- Sometimes tasks outlive their wf_contexts
- workflow contexts get flushed on old instances
- Check your sys_auto_flush table for a wf_context record, and deactivate it.
10 Comments
