Introducing SNDoc - A better way to document your code

[Callum Ridley1]

At UP3 Services, we have been discussing the best ways in which we can work together as a team on projects, specifically for developers. We have all, at some point, found ourselves in a situation where our code bases get large and difficult to manage, with functional code spread far and wide across the ServiceNow platform, in some cases this code remains un commented and forgotten about.

We began to think about solutions to this problem, some way of keeping track of the often-reusable code, so we turned to Script Includes. Script Includes are a feature of ServiceNow that we at UP3 have been using for quite some time, but we recently made a conscious effort to try and use them wherever possible to contain application specific logic and therefore keeping our codebase in one place.

The next challenge was to make sure that the code, in what were growing Script Includes remained relevant and in a state that any developer could come along and know what the specific methods could and could not be used for, so we turned to comments, but of course in a large script include, it can still become difficult to find exactly what it is that you’re looking for. We needed a better solution.

Introducing SNDoc. After 6 months of iterations we are proud to be able to share with the ServiceNow Developer community a new way of keeping control over your script includes. I’m sure many, if not all, of you have visited the ServiceNow API documentation at some stage, whether it be to refresh your memory on the little used GlideRecord method, or to read up on a new API you’ve not used before. SNDoc aims to replicate the look and feel of the already familiar ServiceNow documentation. By now You’re probably wondering how this can be possible with custom written code, and the answer is structured commenting. Those with a background in JavaScript will no doubt have heard of JSDoc, the standard in JavaScript documentation. The issue with JSDoc though is that it’s offline, it’s certainly not built for the ServiceNow platform and setups people have documented in the past have always been tricky to get running and keep updated.

SNDoc is built for the ServiceNow platform, and while it doesn’t claim to be as advanced as JSDoc, it’s got some nifty features, including dependency discovery. It’s been built as an extensible framework, so if you have a need for a new tag in your comments, you can simply create one.

Head over to Share to download your copy. We would love your feedback on it, especially if you have feature requests.

[Chris M3]

Nice Job. We're going to be reviewing this and considering implementing this as our standard going forward.

One suggestion.  Private functions should maybe by default be hidden?  It's sorta nice being able to see them, but in general developers using your object shouldn't need to know about them.  Maybe a checkbox somewhere 'Hide Private Functions' that would be checked by default.

[Callum Ridley1]

Excellent idea Chris, thank you. I'll add this to the backlog and make it available in the next release.

[robwitty]

This promises to be an awesome tool for the team.   But I've bumped into some "gotchas" that you might investigate please?  There are times when I click on a script include (displayed in the API list on the left), and the parser just spins.  It never comes back.   I presume it's in a loop, and I've noticed some stack overflow errors in the log.

• If you have SNDOC keywords/tags (@param, @description, @return, etc.) embedded anywhere in your code (other than the doc block), it will cause the SNDOCs parser to loop.
• It appears that the @description tag can only handle 1068 characters after the @description tag appears in the doc block. Anything beyond that and the parser loops.
• The @example tag has the same limitation--1068 characters.   Any more than that and the parser loops.

Looking forward to your continued development of this tool!

[Callum Ridley1]

Hi Rob,

Thank you for your valuable feedback, it's very much appreciated. I have done a little bit of investigation to start this off.

Regarding your first point about tags placed outside of SNDOC comment blocks, I'm unable to replicate this, any chance you can provide an example for me to work with as all of the iterations I've tried have not resulted in issues?

The other two points appear to be a limitation of the Java platform upon which ServiceNow is built. The Regex used to parse over a SNDOC comment block is fairly complicated in the fact that it uses the 'positive lookbehind' syntax, Java seems to not like doing this when long strings are involved. I shall look to see if there is an alternative to using the current syntax, as this should hopefully stop the Java platform from encountering StackOverflows. I'll get back to you on this.

Callum

[robwitty]

Callum.

I think I narrowed it down to the presence of "/**SNDOC"....not one of the tags like @description.  This actually makes sense, since the parser is probably scanning the script lines for that very string to know it has a docBlock.

I've attached a text file of an object I created to convert our existing docBlocs to the SNDOC format.  In the convertScript() function there is a line that reads:

    lineOut = '/**SNDOC';

If I leave that line "as is" then the parser gets confused.  If I change it to:

    lineOut = '/**' + 'SNDOC';

...then the parser runs fine and renders the object. But it's strange... if I copy that line (lineOut = '/**SNDOC';) into another object function and run SNDOCs on it, it works fine.   ??  Go figure.

In any case, I don't perceive this as a shortcoming of the SNDOC parser logic.   It has to look for this keyword, and normal scripts that it parses will not have /**SNDOC embedded in the middle of a function anyway!   I tripped into it because I'm building a script to convert our scripts. 

Sorry for the confusion!  Thanks for the response!

[Callum Ridley1]

Hi Rob,

Yes you're right, that would cause issues. infact it's something I encountered myself during development.

Good news on the RegEx bug, this is now fixed and I have uploaded v1.1 to share. 

You'll find a patch in the "Related files" section which just includes the updates from v1 to v1.1.

Callum

[Callum Ridley1]

Hi Chris,

Good news, I've just released v1.1 on share, this includes the functionality that you asked for above with regard to hiding private methods as well as a bug fix.

You'll find a patch in the "Related files" section which just includes the updates from v1 to v1.1.

Callum

[jxsaxton421]

I love this idea, but does it only work for script includes? It would be nice to be able to document code elsewhere.

[Callum Ridley1]

Hi Joshua,

SNDoc designed for generating documentation for classes with reusable code so that others may easily understand how to use a reusable piece of code. It is not for code within Business Rules etc. which are single use by their nature, inline comments should suffice for those situations.

The code for SNDoc is freely available on Share, if you wish to create your own version which extends out beyond Script Includes, you're welcome to use the code I have created as a base, I currently have no plans to do this.

Callum

[Daniel Oderbolz]

This looks very promising!

I have noticed, that the URL in your post is no longer valid.

The correct one seems to be

https://developer.servicenow.com/app.do#!/share/contents/6426327_sndoc

Best
Daniel

[-O-]

Would it work to change the line that extracts tab content in function SNDocParser.prototype._getMatchingTags to:

var matchStr = new RegExp('(@' + tag_name + '\\s+([^@\\\\]*(?:\\\\.[^@\\\\]*)*)\\s*(@|\\*\\/))', 'gmi');

and to change the line that adds the content using instruction:

matches.push(match[2].replace(/\\(.{1})/g, this._replaceEscaped).trim());

where _replaceEscaped is a function that writes back the 1st capture in the match?

Thus one could escape non-tag denoting @s with a backslash.

[Tommy Jensen]

The resulting documentation live on the actual instance as a "live" service portal page.

Is there a way to export it as a fully contained website instead? Suppose you generate the doc on a PDI but want to publish the documentation on an intranet without linking to a live servicenow instance.

[Community Alums]

Well done guys! Having a standard comment tagging is the way and it would be defo worth and totally a game-changer if you include this library (as an extension) in the VS code to work together with sn-utils by @Arnoud Kooi. I know that JSdoc already does their job but you can take the opportunity to include SN application-specific stuff. 

[Javier Arroyo]

Great job guys.
Being allergic to JSDoc, If time allows, I plan to extend the utility to parse the functions themselves. In declarative programming, JSDoc style comments are an overkill and frowned upon. It turns neat terse and self documenting code into bloated scripts. But, there is still a need to document APIs at a very high level as you've done with this great util.

My pipe dream is to output the functions parsed in the expected format to hand to the portal. This came about from showcasing the tool to colleagues. I annotated a 60 line script include and it turned into 293 lines. The clutter is distracting and increases mental load when looking at functions. Now the developers are distracted by all the clutter, they have to weave in and out from comments to code, as well as being doubly task by re-reading comments and functions themselves just to ensure the code matches the comments.

With functions that are:

1. Single purpose
2. are unary
3. always return a value
4. have self documenting names and parameters
5. are pure
6. nearly always about 10 lines

this code structure make comments section unnecessary, as reading code becomes the same as reading the comments.

Example:

    /**SNDOC
    @name getRecordFromTableByName
    @description curried fucntion that retrieves a GlideRecord 

    @param {string} [table] - name of table to query
    @param {string} [name] - name of record to fetch

    @example
    var db = makeDBFor('incident');
    var coreCompany = db.getRecordFromTableByName('core_company')('Dr. Feel Good');

    @returns {GlideRecord} 
    */
    function getRecordFromTableByName (tableName) {
        return function withName (name) {
            var record = TableSupport(tableName).query('name=' + name);
            record.next();

            return record;
        };
    }

By parsing the functions (curried or not), with "self documenting code" comments would be minimal, following the approach stressed in declarative programming: only comment unclear code, or items not necessarily apparent at-first-glance. Something that hints what the developer was thinking when the approach was taken.

Never-the-less, bringing that function signature to a presentable form such as in the portal is the bigger deal. Perhaps the parsing of functions with limited annotations can do the trick in my case.

Fantastic start fellas, kudos!

[Javier Arroyo]

Some thoughts on SNDoc now that I've been using it for some weeks.

Instead of adding api usage documentation above function signatures, it's been moved to the bottom of the page under a section titled 

/*********************** SNDoc Support ****************************/

this helps keep code concise and reader friendly, while keeping the advantages offered by the tool. Our documentation has never been this easy to use. Even I, who built all the libraries, use it as reference. A++ guys

[Kingfapa]

Hello Javier,

Is this something your team has built for yourself? I too want to set this up for my team.

Do you have an example of how you set it up?

[Javier Arroyo]

Hi Kingfapa,

The tool was built by Callum Ridley and can be found here: https://developer.servicenow.com/connect.do#!/share/contents/6426327_sndoc?t=PRODUCT_DETAILS

[Kingfapa]

Hello,

Yes I understand that I have setup the tool in my test instance and it's great. Im just wondering how you got this line to work?

/*********************** SNDoc Support ****************************/

I have tried adding it to my script includes in the bottom of the script but it does not document my code. I still have to add /**SNDOC

[Javier Arroyo]

My bad. 
We created a syntax macro to add that line. It's not included with the too.

Cheers,
javier

[Can Althaus]

Is there a way to PDF-Export the generated Documentation?

Page printing apparently does not work. 

[Daniel Oderbolz]

Dear @Callum Ridley1 
Thanks a lot for writing this tool!

I was wondering if we could find an approach where we in fact use the real JSDocs code (& Syntax!!) and use it to directly create KB articles... I think that would be a very powerful approach - what do you think?

[Avro Guha]

https://developer.servicenow.com/connect.do#!/share/contents/6426327_sndoc?v=1.1&t=PRODUCT_DETAILS