Automatic Source Control and You - Save Time and Headaches

[jbauguess]

Today we're going to examine a build system I've been using, which I've dubbed "SNBuild".   Using it can help you accomplish the following:

1. Source Control
2. Update Set Backup
3. Code Quality Examination
4. Automatic Documentation

Sound too good to be true?   Well let's take a look at this: it's fairly simple.

Setup

Even if you aren't going to use a source control repository, which I'm not sure why you wouldn't want to do that, the NodeJS program I've developed will at a minimum extract files to a local folder.   If you haven't used NodeJS before, go to nodejs.org, download it, and try it out.   It's a very powerful engine, and very useful for a variety of tasks.   And you'll need it installed if you want to run my program locally.

ServiceNowScriptDocumenter

Not the most imaginative name, but at it's core, that's why I developed this application.   I needed a way to have documentation so I could easily see if I had already written functionality so I wasn't repeating myself.   DRY - Pragmatic Programming Tip #11 (and a tip in nearly every other resource that teaches you how to program).   Fork or clone this repository (git clone https://github.com/jmbauguess/ServiceNowScriptDocumenter).   There are just a few changes you'll need to make:

1. The "include" variable in index.js.   I set this to only extract code updated by my account in ServiceNow.   For starters, you don't want to extract and analyze OOB code.   Most of it isn't annotated with JSDoc, so your jobs to make documentation will ignore it.   Secondly, ServiceNow keeps their own code version controlled, so we don't need to help them out with that.   And if we are taking a piece of code and modifying it (which I don't recommend*), it will be caught by our app then.   Finally, much of it is rife with JSHint errors.   For your own sanity, just ignore them all.
2. Add any "requestsToMake" to that object.   I only extract Script Includes, UI Scripts, my unit tests, and the update set itself (which I use as an html file for documentation purposes).   Extracting the remote update set, the backup plan essentially, is done in a separate step because it's a little more complicated, as we'll see in the next section.
3. The "writeAFile" function.   I added some custom fields to my "sys_update_set" table, and you may not want those on yours.   Or you may already have similar fields on yours.   Either way, you may wish to modify this function, or delete it if you're not going to extract sys_update_set records with the requestsToMake array.
4. Any tasks in the gulp.js file.   You may not have your code with JSDoc, so you probably don't want those tasks to run.   You also may not care for the JSHinting, but I recommend it, as it will help you develop better code.   At a minimum you want the "extract", "add-files", "commit-files", and "push-files" tasks to run.
5. The sourceArray variable in the gulp.js file.   You may want to save different types of files.   And if you're not including the markdown or JSDoc output, you'll need to remove those.   Just know that if you want to add file types (say business rules or client scripts), you'll need to add corresponding records in the index.js file.   As I expand this, I'll likely extract the properties into a separate file, but since this was so small, it was easy enough to keep it in the source.   I know, bad practice...

Installing the program is a simple matter of running npm install from the directory where you cloned the repository.Running the program just requires a command line call: gulp --instance "$INSTANCE" --username "$USERNAME" --password "$PASSWORD" --reason 'A reason' --update_set Updatesetname.

Making Things Automatic

Once again, I'll be leveraging Jenkins to handle building my application for me.

1. Create a new generic job and give it an appropriate name.
   
2. Check the "This build is parameterized" checkbox.   Add a String parameter of UPDATE_SET, PASSWORD, USERNAME, and INSTANCE.   Give them default values (especially password, username, and instance!)
   
3. Under Source Code Management, enter your repository URL and any credentials you need to enter.
   
4. Under Build Triggers, check "Trigger builds remotely", and decide an authentication token.   You'll need this for use in ServiceNow in a little bit.
   
5. If you're like me, your Jenkins doesn't have the NodeJS plugin.   That's ok, because we're going to use a shell script to execute our build.
   

Now we have a job that will run via remote trigger.   Next, let's create a REST Message in ServiceNow that will serve our needs.

REST Message

If you haven't used REST Message before, it's a powerful way to integrate with other systems.   I'm currently working on integrations with Rally, Jenkins, and Sonar.

1. Create a REST Message.   Give it a good name like "SN Jenkins Builder".
2. Set the Endpoint to: ${jenkins}${job}${action}.
3. Set the basic authentication to be user Jenkins username and password, then save the record.
   

Congratulations, you've made a REST Message.   Of course, this one really won't do much of anything, will it?   After all, ${jenkins}${job}${action} isn't a valid URL.   Good thing we plan on using that a different way.

Business Rule

1. Go to the Update Set table, and right click to get to the Business Rules for the table.
2. Create a new Business Rule, and give it a good name like "Automated Push to Jenkins".
3. This rule needs to run "before update" on the condition of current.state.changesTo("complete")
4. The script needs to be the following (after the onBefore declaration):
   (function(current) {
var message = new RESTMessage('SN Jenkins Builder', 'post');
message.setStringParameter("jenkins", gs.getProperty("jenkins.url"));
message.setStringParameter("job", encodeURIComponent(gs.getProperty("script.documenter.job")));
message.setStringParameter("action", "buildWithParameters?token=" + gs.getProperty("script.documenter.token") + "&UPDATE_SET=" + encodeURIComponent(current.name));
var response = message.execute();
if (isAccepted(response.getStatusCode())) {
gs.addInfoMessage("Update set has been sent to Jenkins.");
} else {
gs.addErrorMessage("Failure sending update set.");
}function isAccepted(code) {
return (code == "200" || code == "201") ? true : false;
}
})();
5. Save the record.

System Properties

1. Go to System Properties, and create three properties:
   1. jenkins.url -> The base url of your jenkins server
   2. script.documenter.job -> The name of your job in Jenkins
   3. script.documenter.token -> The build token of your Jenkins job
2. To make them easier to find and maintain, let's add them to a page and module.
   1. From the property, click "New" in the Category related list
      
   2. Name the category and save it
      
   3. From the other properties, click "Edit" in the Category related list, and add it
      
   4. Create a module (I chose to do so under Update Sets).   Give it a URL of "system_properties_ui.do?sysparm_category=" and the name of the category.
      
   5. Now you can view and edit the properties from the left nav easily!
      
      Hat tip to Mark Amann of CloudSherpas for showing me this a long time ago.

Creating the Remote Update Set Automatically

If you right-click on the Export to XML UI Action on the Update Set once it's closed, you can see what happens behind the scenes to give you that remote update set record.   Now we need to take it, modify it slightly, and make it an action that occurs when the update set closes.

1. Create a new Business Rule for the Update Set table
2. Give it a meaningful name, such as "Automatically create remote update set"
3. It needs to run "before update", with a condition of current.state.changesTo("complete")
4. The script needs to contain the following code after the onBefore() function declaration:
   var retrievedUpdateSet = new GlideRecord('sys_remote_update_set');
retrievedUpdateSet.initialize();
retrievedUpdateSet.description = current.description;
retrievedUpdateSet.name = current.name;
retrievedUpdateSet.release_date = current.release_date;
retrievedUpdateSet.remote_sys_id = current.sys_id;
retrievedUpdateSet.application = current.application;
var scopeGr = new GlideRecord('sys_scope');
scopeGr.get(current.application);
if (scopeGr.isValid()) {
retrievedUpdateSet.application_name = scopeGr.name;
retrievedUpdateSet.application_scope = scopeGr.scope;
retrievedUpdateSet.application_version = scopeGr.version;
}
retrievedUpdateSet.state = "loaded";
var sysid = retrievedUpdateSet.insert();
var update = new GlideRecord('sys_update_xml');
update.addQuery('update_set', current.sys_id);
update.query();
while(update.next()) {
update.remote_update_set = retrievedUpdateSet.sys_id
update.update_set = '';
update.insert();
}
5. Save the record

Code Quality and Documentation

Now, to fully leverage these pieces, you need to do a few things:

1. Be willing to accept the fact that you need to refactor some code
2. Come up with rules based on what's at JSHint's website (which can mitigate some of your refactoring if you are less strict that the default configuration)
3. Learn JSDoc (which is simple) and use it on all of your code

JSHint

I predefine globals such as "gs", "g_user", "GlideRecord", since those are things ServiceNow has available in their scripts.   This is not uncommon among other frameworks such as jQuery. You may find a lot of problems with your code the first time you run this. That's ok. Take time as you go through cycles to fix those issues. If you have automated unit and functional tests, you don't need to be scared about breaking something. If you don't have those, do whatever you can to implement automated tests as soon as you can.

JSDoc

There are numerous comment annotation frameworks for javascript right now, but I chose JSDoc since it seemed to be easy to learn.   The main tags I use:

1. @description - At a bare minimum, use this to describe what your function is supposed to do
2. @param - The next bare minimum is to document your parameters by type and with a description of what they are
3. @return - If you are returning something, note what it should be both by type and description
4. @example - It's nice to include examples of your code in action
5. @memberOf - If you're extending a Class, be sure to annotate with this so the function knows to whom it belongs

Profit

Now that we have everything set-up, we can close an update set and have piece of mind that it's backed up safely in a repository, and our code is documented well (assuming we took the time to do that in the first place). I have an update set with all of the functionality listed above here. Feel free to install it in your instance.