automated update set batching and deployment

[kevinanderson]

Problem:  update set deployments are manual and time consuming. The processing of all the update sets in a release, multiple times a week, is a resource intensive operation.

Proposal:

automated update set deployment 

Design overview:

Using service now workflows, leverage the script activity, and timer activity to replicate the steps that a deployment requires.

Processes:

1. batch all update sets in a release

2. deploy a batch update set to 1 or more instances

Looking for community input on this process.  The end goal would be to connect this to a larger system to enable complete automation of the deployment process, in regards to update sets at first, and then eventually to other aspects of the release process for SN application code, configuration, and data changes.

A deployment is made up of multiple components, some of which, automation may be too much of a stretch for version 1.  At a minimum, a deployment process that handles batching and deployment of a list of update sets would be ideal. In addition, a mechanism to extract and install XML files, and a mechanism to execute a list of fix scripts in a timed, and sequential, manner would cover about 95% of the effort involved in most weekly deployments.

The installation of plugins or the automation of any additional manual configuration is out of scope for the first version of this proposed automated deployment process. 

I think I have pulled together enough resources from SN community support to automate all the update set deployment steps (load, preview, process errors, commit).  I have some concerns on the best approach to handling exceptions in update set previews. Some possible solutions for automation of update set preview processing I have come up with include:

1. Halt automated commit process for any preview errors.  Force all update sets to be clean (probably an un-realistic goal).

2. Provide a white list of preview errors. Compare all the preview errors against the white list.  Halt the auto-deployment on any errors not in the white list

3. Provide a per-update-set ability to accept or skip preview errors via an input parameter to the automation process. 

4. Provide a lookup table or API to allow per-update-set skip or accept behavior for preview errors.  

5. Provide a blacklist of preview errors.  Halt auto deploy process for any preview errors that match an entry in  the black-list of errors.

6. Accept all errors.  (scary!)

update set auto-commit process flow

1. start - provide update set name (batch update set - top most parent) as input

2. delete any uncommitted update sets in target instance

3. pull all completed update sets from remote instance

4. verify all update sets in release are in the target instance retrieved update set table

5. preview update set(s)

6. review and process preview errors

7. commit update set(s)

8. exit

update set auto-batch process flow

1. start: provide input : a list of update sets,  batch update set name 

2. search target environment local update set table, verify all update sets are in a completed state, and have no duplicates

3. create batch update set using name input, and date / time stamp

4. create batch update set for all application scopes represented in the list of update sets

5. process each update set, adding it to the matching application scope batch parent

6. add all application scope batch parents to the update set from step 3

7. set all update sets to the completed state

8. exit and return batch parent update set name

In summary, Service Now has newer tooling that does allow for fully automated deployments (CICD Plugin, SN Studio), but its a big paradigm shift from update set driven development. In my opinion, an in-between step is needed.  Some inputs will continue to be in the form of update sets for the near future, and having both the legacy (update set), and newer development model (SN studio and a repo) in the same orchestration environment will allow an easier transition from legacy to the newest technology from SN for continuous integration and continuous deployment.

I would appreciate any input on this proposal.  I think this is something that needs a community solution, and I hope this is a good starting point for that effort.

[Norm Link]

Hi Kevin,

I mistakenly posted a reply to the main thread instead of replying to this update.  Please see the main thread.

[Rohit Ladda]

But are API's available to retrieve update sets programmatically?  

[Norm Link]

Hi Rohit - yes.  Here is the OOB code in the UpdateSetAjax script include

retrieveUpdateSets: function() {
   var worker = new GlideUpdateSetWorker();
   var sys_id = this.getParameter('sysparm_id');
   worker.setUpdateSourceSysId(sys_id);   // sys_id for the sys_update_set_source record for your remote instance
   worker.setBackground(true);
   worker.start();
   var progress_id = worker.getProgressID();
   return progress_id;
},

[AlfredB]

Hello Kevin,

I am looking in to this area as well and I'm very interested. Did you get anywhere with this?

Regards

Alfred