Processing GlideRecords as a Series of Functions

[Javier Arroyo]

This article is a look at processing GlideRecord as a series of functions instead of the classical "monolithic" approach. 

Sample of Common GlideRecord Processing

(
getTraversalRules: function(usedBy) {
        var result = [];
        var gr = new GlideRecord('svc_traversal_rules');
        gr.addActiveQuery();
        gr.addQuery('used_by', usedBy);
        gr.addQuery('definition', 'ISNOTEMPTY', '');
        gr.addQuery('definition.rel_type', 'ISNOTEMPTY', '');
        gr.orderBy('order');
        gr.query();
        while (gr.next()) {
            result.push(this._toRuleObj(gr));
        }
		
		return result;
    }
}

The coding strategy shown above works similar to a run-on sentence where a series of thoughts are interrelated as if a single expression. Besides the technical debt incurred by such a strategy, a disruption of a more English-like grammar experience is introduced. By break it down into independent functions, a different code articulation can be created to clearly identify each independent thought of the whole. It will be transformed from interpreting how to get a GlideRecord result set into reading the steps lead to a result.

Some of the independent thoughts
While reducing the classical GlideRecord strategy is beyond this article, it's main steps listed below will be addressed at various depths:

1. GR with its filter criteria & querying it
2. Looping through the query result
3. Applying logic to each gr within the loop
4. Collecting the result after applying the looping logic

Step 1 is a collection of actions in itself (a process) that can be further separated. I will not be that granular, limiting this step to structurally separating it from the other steps. Steps 2 and 3 will be turned into reusable utility functions, while step 4 is only listed to demo that it is also an isolated process.

The goal is to allow coding various structures without excess syntax. Something like these:

getTraversalRules: function (usedBy) {
    var pipeline = pipe([getTraversalRulesUsedBy, whileGR, toRuleObj]);

    return pipeline(usedBy);

}


getTraversalRules: function (usedBy) {
    var traversalRules = getTraversalRulesUsedBy(usedBy);
    return whileGR(traversalRules, toRuleObj);

}

Prep work
Because the alternative strategy considers querying the db a collection of actions, some jargon and utility tools are in order for common understanding and base functionality which to leverage for all future db interactions.

Common language:
1. composition: putting functions together
2. process: a series of steps that make up a whole (large or not)
3. pipeline: an ordered list of functions that list each step within a process where the previous function's output is the next function's input.
4. pipe: the utility that transforms a list of functions into a executable pipeline

For a function to partake in a pipeline, certain rules must be followed: don't reach outside a functions parameters to carry out logic, accept only one parameter, and always return a value. This is because inside the pipeline the return value of one function becomes the input of the next. If a function does not return a value, the pipeline will break; just as it will if a function expects multiple arguments. The multiple argument exception in a pipeline is that the very first function can be of any arity. Going outside a function is not desirable as it can introduce unexpected behavior. Make all functions only act upon the values passed in. 

Sequential order is required for functions in the pipeline so that the output of the previous is the expected input of the next function. Upon completion, the pipeline returns the collective result of each function execution. 

In all: keep functions single purpose, unary, always return a value, and don't reach outside a function for additional data. From there, sequentially compose a pipeline matching output to input. The very first function will be fed whatever input(s) is required.

Refactoring the GR "Monolith"

Begin by separating a GlideRecord by responsibility: retrieving, applying logic, collecting the result. (Note: a mature implementation isolates GlideRecord to one function in the entire custom codebase). For demo purposes I will use the same OTTB builder strategy, except that it will be physically separated from while(gr.next()). 

A Script Include to host retrieving values from the DB

var GR = function buildGR (tableName) {

    var SET_LIMIT = 20;

    function _makeGR(tableName) {
        return new GlideRecord(tableName);
    }


    function someActiveRecords(amount) {
	var gr = _makeGR(tableName);
		
        gr.setLimit(amount || SET_LIMIT);
        gr.query();

        return gr;

    }


    return {'someActiveRecords': someActiveRecords, 'type': 'GR'};

};

GR SI stores reusable GlideRecords and contains one function:getSomeActiveRecords. It is used as GR(tableName).getSomeActiveRecords(amount);

Once an SI that builds GRs is in place, the next step is to separate record processing (the while statement) into it's own Script Include.

var Glidediator = {
 
  whileGR: function whileGR (callback) {

    return function applyWhile (gr) {
        var result = [];

        while (gr.next()) {
            result = result.concat(callback(gr));

        }

        return result;
    };
  }
}

Curried function whileGR in Glidediator SI will be used going forward to process all GlideRecord results. It accepts a function (callback) that is applied against each record once the returned function by whileGR is executed.

Gladiator.whileGR(callback) returns function applyWhile(gr) that expects a GlideRecord result set as an argument. It is during this second execution that while (gr.next()) will kick off, calling callback against each GlideRecord being looped. 

The implementation above is incomplete. Much like the classical fragment introduced at top, whileGR has a hard-binding to Arrays. This hard binding must be removed, something I will not showcase here. Post Working with Arrays, an alaternative has a solution that can be used as basis to extract that hard-binding. The basic idea is for a non type-binding implementation to inject strategies to whileGR: whileGR(callback, collectorCallback). This allows whileGR to collect a result any way it wants.

A drastic scalability increase has been introduced by passing a function as an argument rather than a non-function. Furthermore, thinking of callback not as a single function but, as a pipeline, is intriguing. An ability to execute a series of functions against each GR is not possible, allowing whilteGR to execute any number of functions against any GlideRecord result set.  

Building the pipeline
An utility function is required to leverage pipelines. The utility is what will allow callback to represent a pipeline.

function pipe (fns) {
    //    var fns = Array.prototype.slice.call(arguments);

    return fns.reduce(function (f, g) {
        return function applying () {
            return g(f.apply(null, arguments));
        };
    });
}

The pipe function above takes an array of functions, then waits for a seed argument that will passed to the very first function in the pipeline. Once given, the seed argument is passed through each function, first as in input where it gets "morphed" then, returned as an output to be used an the next function's input until a final state is achieved and returned.

**The commented line of code is for preference. left there for your reference for alternate usage.

Going forward, all functionality against a GlideRecord is addressed at the pipeline by adding or removing functions, rather than hardcoding behavior in the classical GR strategy. 

Review the newly introduced utilities
1. GR container to host GlideRecord queries
2. Glidediator container for GlideRecords specific helper functions
3. pipe standalone function SI to bundle a list of functions into a pipeline

(While I've placed functions inside script includes, there is no technical reason preventing them from being standalone.) Because SNow lacks a tree-like folder structure to store code, it leads to developers having to compensate for that structure through patterns

The next step
With pipe and whileGR in place, they can now be combined to run a series of functions against each GlideRecord. But, additional details still need to be addressed, such as a more trivial way to apply pipelines to individual fields, and entire records. This means nested pipelines, and some functionality that helps code it not to have to reinvent the inner pipelines each time.

A basic example that applies X number of functions to individual fields, and X number of overall functions to each record is the use case that will be showecased.

So, laziness leads to adding a function to Glidediator.

Updated Glidediator 

var Glidediator = {

    'whileGR': function whileGR(callback) {

        return function applyWhile(gr) {
            var result = [];

            while (gr.next()) {
                result = result.concat(callback(gr));

            }

            return result;
        };
    },

	
    'applyToEachField': function applyToEachField (fieldNames, callbacks) {
		
        function concat(gr) {
            return function red(reducer, field) {
                return reducer.concat(pipe(callbacks)(gr, field));

            };
        }

        return function processUsingGR(gr) {
            return fieldNames.reduce(concat(gr), []);

        };
    },

    'type': 'Glidediator'

};

Newly introduced curried function applyToEachField has been added to SI GR just for ease of use. Note that the array dependency is still there. This function is not required but, it can lead to an easier coding experience.

applyToEachField is a 2 arity function that accepts fieldNames and an array of functions that when called, returns function processUsingGR. The returned function accepts GlideRecord result set as an argument. Given a GlideRecord, it executes the pipeline against each field identified in fieldNames. It finally collects the values into by concatenating the result of each function execution into a final value.


Some functions to test the pipelines
With all that prep work out of the way. Now come the functions that satify specific uses cases. These are the ones that will be used to build the demo pipeline.

function up (value) {
    return value.toUpperCase();

}

function reverse (value) {
    return value.split('')
        .reverse()
        .join('');

}



function getValue (gr, fieldName) {
    return gr[fieldName] + '';

}


function debugStringified (logger) {
    return function debugit (value)  {
        logger(JSON.stringify(value));

        return value;

    };
}

Function up applies toUpperCase; reverse inverts a string; getValue collects a string field value. And, debugStringfied given a logging strategy (gs.debug/gs.log/gs.error) JSON.strings the value and prints it out.  

debugStringified is a continuation function meant to trigger a side effect (JSON.stringify). The continuation strategy is used to inject behavior into the pipeline: create another record, http post, send email, queue event, etc. In this example all it's doing is printing a value.

Putting it all together

From here on coding is limited to what should happen to a GlideRecord. There is no need to rebuild the classical GlideRecord experience. Instead, call the existing utility functions using pipelines. This also means that consideration should be given to code structure. Entry point functions in Business Rules, Action Scripts, Client Scripts, Widgets, etc that point to Script Includes where each function is a process using utility functions.

Example of Classical Style to be Decomposed

Classical ServiceNow Style

function getSomeChangeFieldsUpperedAndReversed(fields, amount) {

            var gr = new GlideRecord('change_request');
            gr.addActiveQuery();
            gr.setLimit(amount || 20);
            gr.query();

            var result = [];

            while (gr.next()) {

                for (var x = 0; x < fields.length; x++) {

                    var value = gr[fields[x]] + '';

                    var reversed = value.split('')
                        .reverse()
                        .join('');

                    var upped = reversed.toUpperCase();
                    gs.debug(upped);

                    result.push(upped);

                }

                gs.debug(result);

            }
            gs.debug(result);
            return result;
        }

    }

 The above function loops through a GlideRecord result set, extracting the value each field, reversing it and upper casing it.

As a Series of Functions

function upReversePrintEachField (fieldNames) {
    var upAndReversePipeline = [getValue, toUpperCase, reverse, debugStringified(gs.debug)];
    return Glidediator.applyToEachField(fieldNames, upAndReversePipeline);

}


function getSomeChangeFieldsUpperedAndReversed (fieldNames, amount) {

    var upAndReversedFieldsFn = upReversePrintEachField(fieldNames);
    
    var whileGRActiveChangesFn = Glidediator.whileGR(upAndReversedFieldsFn);
    var printInfoFn = debugStringified(gs.info);

    var upAndReverseEach = [GR('change_request').someActiveRecords, whileGRActiveChangesFn, printInfoFn];

    return pipe(upAndReverseEach)(amount);
}

The above getSomeChangeFieldsUpperedAndReversed entry point function is not fully refactored. Instead I chose to concentrate on building a pipeline for GlideRecords to save reading time. But, hopefully, I was able to achieve enough to show how functions got passed around.

var upAndReversedFieldsFn contains the field names from which to get record values and pipeline to apply against each one. Once passed to Glidediator.whileGR it waits to be called against a GlideRecord..

var upAndReverseEach is the pipeline to execute during each iteration of while(gr.next()). In the example above it means that some active changes will be grabbed, then functions getValue, toUpperCase, reverse, & debugStringified called in that sequence against each record. The last function of the pipeline is to print the cumulative result of calling those functions against each record will be printed.

The output is below. Number and short description were upped, reversed, then printed. The last print statement is from printInfoFn and it prints the final output of going through all the records in the while(gr.next()) If there was a need to add more functionality to each field value, or iteration of in the loop, create one then added it to the correct place.

getSomeChangeFieldsUpperedAndReversed(['number','short_description'],3);

--> "6401000GHC"
--> "TRELA PLD"
--> "9001000GHC"
--> "DESUBA SEGELIVIRP NIMDA"
--> "8201000GHC"
--> "NO-DDA LECXE NA DELLATSNI GNITNUOCCA NI NOSREP"
--> ["6401000GHC","TRELA PLD","9001000GHC","DESUBA SEGELIVIRP NIMDA","8201000GHC","NO-DDA LECXE NA DELLATSNI GNITNUOCCA NI NOSREP"]

From this point on is all about building functions which to put inside a pipeline. As well, structuring code to easily compose functions. Entry point functions, utility libs, curried functions, etc. Naming then is key. Come up with good names to lead to precise articulation. I wasn't able to achieve this clearly during this post as it was growing overly long. My apologies. 

A fully refactored functionality would lead to a few more SIs in which to place pipelines, and other utility functions that make it trivial and easily readable. Basically 2 or 3 line function calls.

Cheers and happy SNowing...

[Javier Arroyo]

Some script includes to play with

From background scripts type: GlidediatorTest2

It will run some functions which can be played around to test and play the functionality.

[Jon G1]

I love this content so much.  I've been trying to split some of my code out into functions to make it a more easy to read, and there's some interesting stuff here.

Admittedly this thread is a bit difficult for me to parse and I haven't yet had the chance to wrap my head around it all yet, but I think I understand the core concepts.

Following your example, it seems that you take one gr, and pass it through a series of transformations before returning the desired output.

What I'm not clear on is how to handle a nested loop like so using the method you described above or a similar method:

(function () {
    var contract = getCurrentContracts();
    while (contract.next()) {
        var sd = getServiceDetails(contract);
        while (sd.next()) {
            generateInvoiceItem(contract, sd);
        }
    }
})();

Perhaps you can provide some feedback?

Thanks for all of this information!