Creating a batch installer for Store updates

[Adam Stout]

We have all been there, you just upgraded your instance, and you go to the Application Manager and the first thing you see is this:

199 updates available to install… Which ones do I need???  Where should I start??? Do I have to go through these one at a time???

In this article, we’ll explore how to review these updates in terms of major, minor, and patches in a list view and, most importantly, how to queue up batch installs. Once in place, you can quickly review all the patches available, select which you want to install, and then queue the install in just a few clicks.

Before diving into this walkthrough, I recommend watching the webinar that covers the general approach to Store updates and provides a detailed guide on the batch installer. Once you've watched it, this walkthrough will guide you through building the batch installer yourself. You can find the webinar here, with the batch installer walkthrough starting around minute 19.

Now that you have watched that, let’s get to the technical part of how to build it.

The batch installer has two main pieces. First, we start off with a remote table that helps us review the Store updates available. Then, there is a UI that allows us to choose which apps to update. Then, we call the CI/CD Batch installer to do the actual installation.

Disclaimer: The code provided in this blog post is intended as an example and is provided "as is" without any warranty of any kind, either express or implied. It is the responsibility of the user to thoroughly test the code in their own environment to ensure it meets their requirements and functions as expected. The author and publisher disclaim any liability for any damages or issues that may arise from the use of this code.

Creating the Remote Table to Review Available Updates

Create a new application

Start by creating a new application scope to house your custom app. This ensures that your app is self-contained and does not interfere with global settings. There are several ways to create an application.  I used ServiceNow Studio but you can use any method you prefer.

Subscription Note: This app will not have any physical tables (only a remote table) so it will generally not cause any entitlement issues. If you have any questions about your entitlement to create new applications, please reach out to your account team.

Docs: https://docs.servicenow.com/csh?topicname=create-application.html&version=latest

Create the remote table

Make sure you switch to the newly created app before creating any new objects. Navigate to the Tables module under Remote Tables and click New.

Create a table with the following fields:

Column label	Column name	Type	Reference
Application	application	Reference	Store Application [sys_store_app]
Avail. Major	major_count	Integer
Avail. Minor	minor_count	Integer
Avail. Patches	patch_count	Integer
Available Version	available_version	Reference	Application Version [sys_app_version]
Batch Level	batch_level	Choice
Installed Version	installed_version	String
Latest Major Version	latest_major_version	Reference	Application Version [sys_app_version]
Latest Minor Version	latest_minor_version	Reference	Application Version [sys_app_version]
Latest Patch Version	latest_patch_version	Reference	Application Version [sys_app_version]
Latest Version Level	latest_version_level	Choice
Level	level	Choice
Name	name	String

Write the script to populate the table.

Navigate to the Definitions module in Remote Tables and click New. There is a lot of room for customizations and enhancement, but this should get us close to what we are looking for:

(function executeQuery(v_table, v_query) {
      var res = {};
      var versionLevel = {'major': 3, 'minor': 2, 'patch': 1, 'none': 0};
      var versionLevelAscending = Object.keys(versionLevel).reverse();

      var getAvailableVersion = function (appSysId)
      {
          var vers = [];
          var v = new GlideRecord('sys_app_version');
          v.addQuery('source_app_id', '=', appSysId);
          v.orderBy('version');
          v.query();
          while(v.next())
          {
              vers.push({'sys_id': v.getUniqueValue(), 'name': v.getDisplayValue(), 'version': v.getValue('version'), 'publish_date': v.getValue('publish_date')});
          }
          return vers;
      };

      var compareVersion = function (currentVer, availableVer)
      {
          var currentVerParts = currentVer.split('.');
          var availableVerParts = availableVer.split('.');
          var diff = 'None';

          if (currentVerParts[0] !== availableVerParts[0]) {
              diff = 'Major';
          } else if (currentVerParts[1] !== availableVerParts[1]) {
              diff = 'Minor';
          } else if (currentVerParts[2] !== availableVerParts[2]) {
              diff = 'Patch';
          } else {
              diff = 'No';
          }
          return diff;
      };

      var storeAppVer = new GlideRecord('sys_store_app');
      storeAppVer.addQuery('active', '=', true);
      storeAppVer.addQuery('update_available', '=', true);
      storeAppVer.query();

      // Create scan finding
      while (storeAppVer.next())
      {
          var cur = storeAppVer.getUniqueValue();
          var availVersion = getAvailableVersion(cur);
          for(var v in availVersion)
          {
              var ver = availVersion[v];
              var diff = compareVersion(storeAppVer.getValue('version'), ver.version).toLowerCase();

              if(res.hasOwnProperty(cur) == false)
              {
                var versionTemplate = {'sys_id': null, 
                                        'application': null,
                                        'level': null,
                                        'available_version': null, 
                                        'installed_version': null, 
                                        'latest_major_version': null, 
                                        'latest_minor_version': null, 
                                        'latest_patch_version': null,
                                        'major_count': 0,
                                        'minor_count': 0,
                                        'patch_count': 0};
                res[cur] = {'major': Object.assign({}, versionTemplate), 'minor': Object.assign({}, versionTemplate), 'patch': Object.assign({}, versionTemplate), 'no': Object.assign({}, versionTemplate)};
              }

              res[cur][diff].sys_id = ver.sys_id;
              res[cur][diff].application = cur;
              res[cur][diff].level = diff;
              res[cur][diff].available_version = ver.sys_id;
              res[cur][diff].version = ver.version;
              res[cur][diff].installed_version = storeAppVer.getValue('version');
              if(diff == 'patch')
              {
                res[cur][diff].latest_patch_version = ver.sys_id;
                res[cur][diff].latest_minor_version = ver.sys_id;
                res[cur][diff].latest_major_version = ver.sys_id;
              }
              if(diff == 'minor')
              {
                res[cur][diff].latest_minor_version = ver.sys_id;
                res[cur][diff].latest_major_version = ver.sys_id;
              }
              if(diff == 'major')
              {
                res[cur][diff].latest_major_version = ver.sys_id;
              }

              res[cur]['patch'].latest_version_level = diff;
              res[cur]['minor'].latest_version_level = diff;
              res[cur]['major'].latest_version_level = diff;

              res[cur]['patch'][diff + '_count']++;
              res[cur]['minor'][diff + '_count']++;
              res[cur]['major'][diff + '_count']++;

              res[cur][diff].name = storeAppVer.getDisplayValue() + ' - ' + ver.version;          
          }
      }

      for(var app in res)
      {
        var lastLevel = res[app]['patch'];
        for(var l in versionLevelAscending)
        {
            var levelCheck = versionLevelAscending[l];
            if(l > 0)
            {
                if(gs.nil(res[app][levelCheck].application))
                {
                    //gs.addInfoMessage('checking - not set: ' + levelCheck);
                    res[app][levelCheck] = lastLevel;
                } else {
                    //gs.addInfoMessage('checking - set: ' + levelCheck);
                    lastLevel = res[app][levelCheck];
                }
            }
        }

        for(var lev in res[app])
        {
            res[app][lev].batch_level = lev;
            if(!gs.nil(res[app][lev].available_version))
            {
                v_table.addRow(res[app][lev]);
            }
        }
    }
  })(v_table, v_query);

Validate remote table

Navigate to your new remote table. In my example, mine is named x_snc_app_updater_st_available_updates.  This should look something like this:

Here, we can see every app available, the currently installed version, and the available version. In the example script, we chose the most recent patch, minor, and major release, so there is at most one of each type. If you need a specific patch and there are multiple available, you would want to use the standard Application Manager to pick that specific version.

Note: I created an inline UI action to view the app in the Store with one click.  This is not essential, but a nice to have.

Create a Subflow to do the Install

The key to the batch installer lies in utilizing the out-of-the-box Batch Install action within the Continuous Integration and Continuous Development spoke. This action should already be installed on your instance; however, if it's not, be sure to activate it before continuing.

Before you get started, be sure to configure a user with the sn_cicd.sys_ci_automation role, credentials, and credential alias so you can use the Batch Install action. You can find the latest documentation on how to do this here:  https://docs.servicenow.com/csh?topicname=cicd-api.html&version=latest

Navigate to Workflow Studio and create a new subflow.

 

Define the subflow inputs and outputs.

Inputs

Label	Name	Type	Description
Applications	apps	String	This is a list of sys_ids of apps to install

Outputs

Label	Name	Type	Description
Progress ID	progress_id	String	Progress ID of the worker that is doing that install
Status Message	status_message	String	Output message from the batch installer.

Flow Variables

Define a flow variable with the label Batch Manifest and name batch_manifest.  This is a string that will hold the JSON manifest of which apps to install for the Batch Installer action.

Actions

Lookup Application Versions

Add a lookup records action that will retrieve the application version records [sys_app_version] from the sys_ids we provided in the flow inputs.

Format Batch Manifest

Set the flow variable Batch Manifest to the required format. Use a script like this one:

var payload = '{"name": "Store App Batch Installer", "notes": "Installing available patches from the Store", "packages": [';
var loop = 0;
while(fd_data._1__look_up_records.records.next())	
{
    if(loop++ > 0)
    {
        payload += ',';
    }
    payload += '{"id": "' + fd_data._1__look_up_records.records.getValue('source_app_id') + '", "type": "application", ';
    payload += '"load_demo_data": false, "requested_version": "' + fd_data._1__look_up_records.records.getValue('version') + '", ';
    payload += '"notes": "' + fd_data._1__look_up_records.records.getDisplayValue() + '"}';
}
payload += ']}';
return payload;

Call Batch Install action

Add the Continuous Integration and Continuous Development -> Batch Install action.  The Batch plan will be the flow variable we set above.  The credentials will be what you have configured previously.  The instance URL is the instance you want to install the app on.  This is commonly the local instance you are running this on, but it could be a different instance if you have the proper credentials.

Assign outputs

Now that we have fired off the batch installer, we’ll output the progress worker id and status so we can display them to the user.

 

Here is what the complete flow should look like:

 

Save and publish this flow, and we can move on. This would be a good time to test the flow to make sure everything is correct. To install, you just need a sys_id or two from sys_app_version.

Create the UI to choose updates

Now comes the fun part.  In this example, I built the UI using Core UI but this could (and probably should) be written in Next Experience.

Create UI Pages

We are going to create two UI pages that will help us in the installation process. We will call them later via a UI action.

Updates to Install

Create a new UI page named updates_to_install. This will allow the user to review what they have selected to install. A confirmation screen is not essential, but it can prevent a simple clicking mistake, so I highly encourage adding it.

In the UI page, set the HTML to:

<?xml version="1.0" encoding="utf-8" ?>
<j:jelly trim="false" xmlns:j="jelly:core" xmlns:g="glide" xmlns:j2="null" xmlns:g2="null">
    <g:evaluate>
        var apps = new GlideAggregate("x_snc_app_updater_st_available_updates");
        apps.addQuery('available_version', 'IN', '${sysparm_app_versions}');
        apps.groupBy('name');
        apps.groupBy('level');
        apps.orderBy('level');
        apps.orderBy('name');
        apps.setGroup(true);
        apps.query();
    </g:evaluate>

    <h3>${gs.getMessage('The following applications will be upgraded:')}</h3>
    <ul>
    <j:while test="${apps.next()}">
        <li>${apps.getValue('name')} - ${apps.getValue('level')}</li>
    </j:while>
    </ul>
    <g:ui_form>
        <input type="hidden" name="app_versions" value="${sysparm_app_versions}" />
        <input type="hidden" name="app_count" value="${apps.getRowCount()}" />
        <div class="form-group pull-right" style="padding-right: 20px;">
            <g:dialog_buttons_ok_cancel ok="return actionOK()" cancel="return onCancel()" ok_text="${gs.getMessage('Install Updates')}" ok_id="ok_button" cancel_type="button" />
        </div>
    </g:ui_form>
</j:jelly>

And the processing script to:

UpBegin();
function UpBegin() {
        var progress_id = '-1';
        var status_message = '';

        try {
            var inputs = {};
            inputs['apps'] = app_versions; // String 
            var result = sn_fd.FlowAPI.getRunner().subflow('x_snc_app_updater.batch_install_updates4d5').inForeground().withInputs(inputs).run();
            var outputs = result.getOutputs();
            progress_id = outputs['progress_id'];
            status_message = outputs['status_message'];

        } catch (ex) {
            var message = ex.getMessage();
            status_message = message;
            gs.addErrorMessage(progress_id + ' - ' + message);
        }
    //var session = gs.getSession();
    //var URL = session.getUrlOnStack();
    var URL = 'x_snc_app_updater_installing_updates.do?';
    URL += "&sysparm_pworker_sysid=" + progress_id + "&sysparm_message=" + status_message;
    URL += "&sysparm_app_count=" + app_count;
    URL += "&sysparm_app_versions=" + app_versions;
    response.sendRedirect(URL);
}

 

Install Updates

In the UI page, set the HTML to:

<?xml version="1.0" encoding="utf-8" ?>
<j:jelly trim="false" xmlns:j="jelly:core" xmlns:g="glide" xmlns:j2="null" xmlns:g2="null">
    <g:evaluate var="jvar_reload_now" jelly="true">
        var progressWorker = new GlideRecord('sys_progress_worker');
        progressWorker.addQuery('sys_id', '=', jelly.sysparm_pworker_sysid);
        progressWorker.query();

        var reloadNow = false;

        if(progressWorker.getRowCount() == 0)
        {
            reloadNow = true;
        } else {
            progressWorker.next();
            if(progressWorker.getValue('state') == 'starting')
            {
                reloadNow = true;
            }
        }
        reloadNow;
    </g:evaluate>
    <g:evaluate>
        var remaining_msg = '';
        var remaining_apps = {};
        var remaining_app_count = 0;
        var hasApps = false;
        var apps = new GlideAggregate('sys_app_version'); // 
        apps.addQuery('sys_id', 'IN', '${sysparm_app_versions}');
        apps.addAggregate('COUNT');
        apps.query();
        if(apps.next())
        {
            remaining_app_count = parseInt(apps.getAggregate('COUNT'));
        }
        if(remaining_app_count)
        {
            remaining_msg = gs.getMessage('Installation of {0} apps has been scheduled, {1} remaining to install.', [ ${sysparm_app_count}, remaining_app_count.toFixed(0) ]);
        }
        var completed_msg = gs.getMessage('{0} apps have been installed.', [ ${sysparm_app_count} ]);
    </g:evaluate>
    <g:ui_form>
        <input type="hidden" name="reloadNow" id="reloadNow" value="${jvar_reload_now}" />
    </g:ui_form>
    <h3>
        <j:if test='${remaining_app_count > 0}'>
            ${remaining_msg}
        </j:if>

        <j:if test='${remaining_app_count == 0}'>
            ${completed_msg}
        </j:if>
    </h3>
    <j:if test='${sysparm_message}'>
        ${sysparm_message}<br /><br />
    </j:if>
    <g:ui_form>
        <table cellpadding="0" cellspacing="0" width="500px">
            <g:ui_progress_worker worker_id="${sysparm_pworker_sysid}" show_cancel="false" rows="grow"/>
        </table>
    </g:ui_form>
</j:jelly>

Client script:

if(gel('reloadNow').value == 'true') {
    location.reload();
}

Processing Script:

var gr = new GlideRecord('sys_progress_worker');
gr.addQuery('sys_id', '=', sysparm_pworker_sysid);
gr.query();

gr.next();

gs.addInfoMessage(gr.getValue('state'));
if (gr.getValue('state') == 'starting' || gr.getValue('state') == 'running') {

    var wait = 3;
    if(gr.getValue('state') == 'running')
    {
        wait = 15;
    }
    var pm = new GlideProgressMonitor(sysparm_pworker_sysid);
    pm.waitForCompletionOrTimeout(wait);
    var redirect = "x_snc_app_updater_installing_updates.do?sysparm_pworker_sysid=" + sysparm_pworker_sysid;
    response.sendRedirect(redirect);
}

Create UI Action

We are going to create a UI action to start the installation process.  Users will select which applications to update and then select the UI action to install them.

Set the following values:

• Name - Install Updates
• Table - Available Updates <or whatever the remote table is you created>
• Select checkboxes
  • Active
  • Show update
  • Client
  • List v2 Compatible
  • List banner button
  • List choice
• Onclick - openModalWithSelectedRecords()

Here is the script:

function openModalWithSelectedRecords() {
    var selectedRecords = g_list.getChecked();
    if (selectedRecords.length == 0) {
        gs.addErrorMessage('Please select at least one record.');
        return;
    }
    var sysIds = selectedRecords;
    var dialogClass = GlideModal ? GlideModal : GlideDialogWindow;
    var dialog = new dialogClass('x_snc_app_updater_updates_to_install');
    dialog.setTitle('Selected Apps');
    dialog.setSize(600, 400);
    dialog.setPreference('sysparm_app_versions', sysIds);
    dialog.render();
}

Add Modules

To make this easier to kick off the process, create 4 modules to quickly review the available updates.

Title	Application menu	Order	Link type	Table	Filter
Available Store Updates	Upgrade Center	2000	Separator
Major Updates	Upgrade Center	2010	List of Records	Available Updates (your remote table)	Batch Level = Major
Minor Updates	Upgrade Center	2020	List of Records	Available Updates (your remote table)	Batch Level = Minor
Patches	Upgrade Center	2030	List of Records	Available Updates (your remote table)	Batch Level = Patch

Try It Out

You have built out the app's basics now. Use the modules you built to review the updates available and use the UI action to start the batch update process.

What’s Next

This app shows the basics of how to build a batch installer, but it is certainly not the limit of what can be done.  Here are some ideas that have been discussed on how to improve it.

Build UI in Next Experience

This example uses Core UI, which works, but this would be better if we rebuilt it in Next Experience.

Add ACLs to allow others to review updates

In my example, I did not add any ACLs, but now that you have your table, you should add some read ACLs to decide who can read it other than admins. This is a great way to empower your business users to review what is available.

Create a catalog item to request updates

Now that you have a way for your business owners to review updates, allow them to request that they be installed. Delegating this responsibility will empower your users and ease your burden.

What else would you like to see an installer like this do?  Let me know in the comments.