Alex Coope - SN
ServiceNow Employee
This post has been updated to reflect multiple feature requests I've had since I originally posted this. 

The artifact now searches through multiple different aspects of a portal - Taxonomies, Topics, Menu's, Content Experiences,
Themes, Footers & Headers, Web Applications, Quick Links, Service Catalogs and their Categories as well as
KB Categories, widgets called from other widgets, custom Angular templates, if a variable is sent to getMessage it will check if it's in
a defined variable or if it's being sent from a function it will check for those function calls to find the variables in the calls and more...

A massive thank you to everyone who is providing feedback and suggestions, please keep them coming as you are all
helping to make this better for everyone - it is greatly appreciated by many!

Also, we are working on a better way to release this. Originally it was just a PoC and one to teach people how to make really powerful
artifacts. Now it's turned into something quite a bit more - watch this space...

 

So it seems I caused a bit of a stir with my last webinar at the Employee Center Academy. In that, it's become the single most asked question I now get - "how did you do it?" (referring to localizing the portal into Japanese).

If you haven't seen the session I highly recommend you check it out:

 

In this post I'm going to go through some concepts, some sneak peaks behind the scenes, how I rationalise table hierarchies for solution architecture and even share my prototype Localization Framework artifact script to get you started. 

This one is going to be quite technical and long so make sure you have a coffee at the ready and some snacks on hand because this is going to be a long one!

 

 

I have a portal, it needs to be in other languages

If we look at the new /esc portal, imagine we want to go from pure English:
find_real_file.png

 

To also have another Language (imagine Japanese in my example);
find_real_file.png

In this new portal, we now need to consider the "Mega Menu", "Topics" and even "Quick Links" (the widget title is in English as it's a pre-release version and will be fixed by the time you have access to it).
These new components will be super important, as we'll need to understand what tables they are in and therefore what their respective relationships are in respect to the Portal they are shown on. 

As a recap, you should remember from our training course (available here) that for custom widgets etc, we need to follow certain coding standards to ensure all end-user facing strings are properly externalised. Here's a quick recap slide to show what I mean, which details how text is calling the Message API based on it's string type:
find_real_file.png

 

To test if everything is indeed "externalised" as needed, if we remember from our training, we can leverage the "i18n debugger" which should show us a Prefix for each UI string representing that the system would expect a translation for it in one of the 5 tables;
find_real_file.png

 

If you're a regular reader of my blog, you'll know that whilst I'm a big proponent of the Localization Framework (possibly because I helped design it) I've also shown how you can write your own Artifacts, for example Portal Announcements and Surveys

Tip, if you haven't yet read those posts, please read them both before continuing, because they will act as foundational knowledge for the following aspects.

This then means, did I achieve my objective (in under 2 days I need for prep) by leveraging the Localization Framework? The simple answer is yes. I wrote a prototype Artifact to go and identify everything I could within that given portal - which required mapping out the table hierarchies of the types of records used in this new portal (the way I think about it is, it's a pyramid of data, so knowing how that pyramid is structured is the objective).

 

 

How did I do it?

For sake of time, lets assume we've built to coding standards (meaning everything is correctly externalised), let's consider making our new Artifact. The first consideration we would need to think about is where does a portal's hierarchy start?

The tip would be the [sp_portal] table, then we need to think about all of the known pages directly related to it, then the page route map, then all of the widgets (and their respective instances) in those given pages, as well as the new "mega menu" and it's entries, the new "topics" and their sub-values, as well as the new "footers". 

This therefore means, we need to follow the relationships to see if we can create suitable sub-queries from the [sp_portal]. In the most part we can, but due to the unique nature of how Service Portals are built, we can't necessarily obtain everything this way. And so, in my prototype it's not 100%, but it's pretty close.

As we're working on the Rome release, this artifact will be a "Processor Script" (aka a Script Include) and the UI action will be defined on the [sp_portal] table.

  • To learn all of the necessary steps make sure you read the two blog posts above and make sure you read the Product Documentation here (for what to do) and here (to understand what functions you have available).

Remembering this is a prototype, there's for sure many areas of improvement (optimization and efficiency), however this should cover the vast majority of scenarios. The idea here, is to show you that if you understand the table hierarchy of that proverbial thing you want to translate, then you too can write your own artifact should you need to.

 

 

[Update]
I thought it might be helpful to show how a portal is typically structured to explain the logic I applied to designing this POC artifact:

this is a simplification of what a Portal's structure looks likethis is a simplification of what a Portal's structure looks like

 

 

So here it is (it's quite a long one, you've been warned),

The artifact is called;

AlexCoopeSN_0-1687961785927.png

Important note - as with any Script Include make sure that the name of it is the exact same as the first declared object, in this case 
"LF_PortalProcessor"


With the processor script looking like this;

If you're revisiting this post, you might notice my code example is longer than before. That's because in a recent call with a Customer, they asked if I could also check for the associated Theme, hence the changes.

 

Note - this artifact will pick up a lot of strings. If you are translating into a net-new language, it may be more performant to use the Page level artifact further down below, but please be patient once you've selected your language at the request stage as it may take a couple of minutes to generate the task (even if it doesn't look like it's doing anything).

 

 

 

var LF_PortalProcessor = Class.create();
LF_PortalProcessor.prototype = Object.extendsObject(global.LFArtifactProcessorSNC, {
    category: 'localization_framework', // DO NOT REMOVE THIS LINE!

    /**********
     * Extracts the translatable content for the artifact record
     * 
     * @Param params.tableName The table name of the artifact record
     * @Param params.sysId The sys_id of the artifact record 
     * @Param params.language Language into which the artifact has to be translated (Target language)
     * @return LFDocumentContent object
     **********/
    getTranslatableContent: function(params) {
        /**********
         * Use LFDocumentContentBuilder to build the LFDocumentContent object
         * Use the build() to return the LFDocumentContent object
         **********/
        var tableName = params.tableName;
        var sysId = params.sysId;
        var language = params.language;
        var lfDocumentContentBuilder = new global.LFDocumentContentBuilder("v1", language, sysId, tableName);
        var pageArr = []; // for later
        var widgetArr = []; // we will need this later

        // what portal are we looking at?
        var portalCheck = new GlideRecord('sp_portal');
        portalCheck.addQuery('sys_id', sysId);
        portalCheck.query();

        if (portalCheck.next()) {
            // processString will look for the (portalCheck.getDisplayValue()) in sys_ui_message table and translated value too, if present
            lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(portalCheck, 'Portal', 'Name');

            // we need to do some specific AI search checks
            if (portalCheck.enable_ais == true) {
                var getSearchTabs = new GlideRecord('sys_search_filter');
                getSearchTabs.addQuery('active=true^search_context_config=' + portalCheck.search_application.sys_id);
                getSearchTabs.query();
                while (getSearchTabs.next()) {
                    lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(getSearchTabs, "Search", "Name");
                }
                // we also need to do some other filters checks
                var getSearchSorts = new GlideRecord('sys_search_sort_option');
                getSearchSorts.addEncodedQuery('active=true^search_context_config=' + portalCheck.search_application.sys_id);
                getSearchSorts.query();
                while (getSearchSorts.next()) {
                    lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(getSearchSorts, "Search", "Name");
                }

                // we also need to check some suggestion reader groups
                var getSuggestions = new GlideRecord('sys_suggestion_reader_group');
                getSuggestions.addEncodedQuery('context_config_id=' + portalCheck.search_application.sys_id);
                getSuggestions.orderBy('order');
                getSuggestions.query();
                while (getSuggestions.next()) {
                    lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(getSuggestions, "Search", "name");
                }

                // we also need to check the search facets
                var getSearchFacets = new GlideRecord('sys_search_facet');
                getSearchFacets.addEncodedQuery('active=true^search_context_config=' + portalCheck.search_application.sys_id);
                getSearchFacets.orderBy('order');
                getSearchFacets.query();
                while (getSearchFacets.next()) {
                    lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(getSearchFacets, "Search", "Name");
                }

                // exceptions
                lfDocumentContentBuilder.processString("Most relevant", "Search", "Name");
                lfDocumentContentBuilder.processString("Was this suggestion helpful?", "Search", "Name");
                lfDocumentContentBuilder.processString("Top result", "Search", "Name");
                lfDocumentContentBuilder.processString("Yes", "Search", "Name");
                lfDocumentContentBuilder.processString("No", "Search", "Name");

				// we need to also process the AIS macro for default strings
				var getAisMacro = new GlideRecord('sys_ui_macro');
				getAisMacro.addQuery('sys_id', '2ce09403fbc382900602ff88beefdc44'); // this is the "ais_sn_components_i18n" of stored default strings
				getAisMacro.query();
				if(getAisMacro.next()){
					// lets process it for getMessage calls
					lfDocumentContentBuilder.processScript(getAisMacro.xml, "AIS Search", "Default Strings");
				}
            }

            // we need to check through the Theme Header and Footer of this portal
            var themeHeader = '';
            var themeFooter = '';

            if (portalCheck.theme.header) {
                var getHeader = new GlideRecord('sp_header_footer');
                getHeader.addQuery('sys_id', portalCheck.theme.header.sys_id);
                getHeader.query();
                if (getHeader.next()) {
                    themeHeader = getHeader;

                    // let's now get the specifics in the theme's header
                    lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(themeHeader, "Theme", 'Header');

                    lfDocumentContentBuilder.processScript(themeHeader.template, "Theme", "Header - Template");
                    _getMessages(themeHeader.template.toString(), "Theme", themeHeader.getDisplayValue());
                    // we need to do a quick check to see if the "user_profile" widget is being used
                    if (themeHeader.template.includes('user_profile')) {
                        var getUserProfWid = new GlideRecord('sp_widget');
                        getUserProfWid.addQuery('id', 'user-profile');
                        getUserProfWid.query();
                        if (getUserProfWid.next()) {
                            lfDocumentContentBuilder.processScript(getUserProfWid.template, "User Profile", "HTML");
                            _getMessages(getUserProfWid.template.toString(), "User Profile", "HTML");

                            lfDocumentContentBuilder.processScript(getUserProfWid.script, "User Profile", "Server Script");
                            lfDocumentContentBuilder.processScript(getUserProfWid.client_script, "User Profile", "Client Controller");
                        }
                    }

                    lfDocumentContentBuilder.processScript(themeHeader.css, "Theme", "Header - CSS");
                    _getMessages(themeHeader.css.toString(), "Theme", themeHeader.getDisplayValue());

                    lfDocumentContentBuilder.processScript(themeHeader.script, "Theme", "Header - Script");
                    _getMessages(themeHeader.script.toString(), "Theme", themeHeader.getDisplayValue());

                    lfDocumentContentBuilder.processScript(themeHeader.client_script, "Theme", "Header - Client Script");
                    _getMessages(themeHeader.client_script.toString(), "Theme", themeHeader.getDisplayValue());

                    lfDocumentContentBuilder.processScript(themeHeader.link, "Theme", "Header - Link");
                    _getMessages(themeHeader.link.toString(), "Theme", themeHeader.getDisplayValue());
                }
            }

            if (portalCheck.theme.footer) {
                var getFooter = new GlideRecord('sp_header_footer');
                getFooter.addQuery('sys_id', portalCheck.theme.footer.sys_id);
                getFooter.query();
                if (getFooter.next()) {
                    themeFooter = getFooter;

                    // we have some specific /esc footer checks to make
                    if (getFooter.id == "employee-center-footer") {
                        // we need to double-check to see if the EC or EC pro is installed first
                        var checkEC = new GlideRecord('sys_package');
                        checkEC.addEncodedQuery('source=sn_ex_sp^ORsource=sn_ex_sp_pro');
                        checkEC.query();
                        if (checkEC.hasNext()) {

                            // now we need to go through the footer
                            var footerCheck = new GlideRecord('sn_ex_sp_footer');
                            footerCheck.addEncodedQuery('portalLIKE' + portalCheck.sys_id);
                            footerCheck.addQuery('active', 'true');
                            footerCheck.query();
                            if (footerCheck.next()) {
                                lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(footerCheck, 'Footer', 'Footer');

                                // now we need to find the menus for the footer
                                var footerMenuCheck = new GlideRecord('sn_ex_sp_footer_menu');
                                footerMenuCheck.addQuery('footer', footerCheck.sys_id);
                                footerMenuCheck.orderBy('order');
                                footerMenuCheck.query();
                                while (footerMenuCheck.next()) {
                                    lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(footerMenuCheck, 'Footer Menu', 'Footer Menu');

                                    // now we need to find the items in the menus
                                    var footerMenuItem = new GlideRecord('sp_instance_menu');
                                    footerMenuItem.addQuery('sys_id', footerMenuCheck.menu.sys_id);
                                    footerMenuItem.query();
                                    if (footerMenuItem.next()) {
                                        lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(footerMenuItem, 'Footer Menu Item', 'Footer Menu Item');

                                        // now we need to check each sub-item
                                        var footerMenuSubItem = new GlideRecord('sp_rectangle_menu_item');
                                        footerMenuSubItem.addEncodedQuery('active=true');
                                        footerMenuSubItem.addQuery('sp_rectangle_menu', footerMenuItem.sys_id);
                                        footerMenuSubItem.orderBy('order');
                                        footerMenuSubItem.query();
                                        while (footerMenuSubItem.next()) {
                                            lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(footerMenuSubItem, 'Footer Menu Sub-Item', 'Footer Menu Sub-Item');
                                        }
                                    }
                                }
                            }
                        }
                    }

                    // let's now get the specifics in the theme's footer
                    lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(themeFooter, "Theme", 'Footer');

                    lfDocumentContentBuilder.processScript(themeFooter.template, "Theme", "Footer - Template");
                    _getMessages(themeFooter.template.toString(), "Theme", themeFooter.getDisplayValue());

                    lfDocumentContentBuilder.processScript(themeFooter.css, "Theme", "Footer - CSS");
                    _getMessages(themeFooter.css.toString(), "Theme", themeFooter.getDisplayValue());

                    lfDocumentContentBuilder.processScript(themeFooter.script, "Theme", "Footer - Script");
                    _getMessages(themeFooter.script.toString(), "Theme", themeFooter.getDisplayValue());

                    lfDocumentContentBuilder.processScript(themeFooter.client_script, "Theme", "Footer - Client Script");
                    _getMessages(themeFooter.client_script.toString(), "Theme", themeFooter.getDisplayValue());

                    lfDocumentContentBuilder.processScript(themeFooter.link, "Theme", "Footer - Link");
                    _getMessages(themeFooter.link.toString(), "Theme", themeFooter.getDisplayValue());
                }
            }

            // we need to check if a cart widget is used for this portal and if so which
            var cartProp = gs.getProperty('glide.sc.portal.use_cart_v2_header');
            var cartId = '';
            if (cartProp) {
                cartId = 'sc-shopping-cart-v2';
            } else {
                cartId = 'sc-shopping-cart';
            }
            var getCartWidget = new GlideRecord('sp_widget');
            getCartWidget.addQuery('id', cartId);
            getCartWidget.query();
            if (getCartWidget.next()) {
                _getMessages(getCartWidget.template, "Shopping Cart", 'Cart Widget');
                _getMessages(getCartWidget.script, "Shopping Cart", 'Cart Widget');
                _getMessages(getCartWidget.client_script, "Shopping Cart", 'Cart Widget');
                lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(getCartWidget, "Shopping Cart", "Shopping Cart");
            }

            // we need to check for any Taxonomies used on this portal
            var taxPkg = new GlideRecord('sys_package');
            taxPkg.addEncodedQuery('source=com.snc.taxonomy');
            taxPkg.query();
            if (taxPkg.hasNext()) {

                // only progress if the Taxonomy plugins are installed
                var taxCheck = new GlideRecord('m2m_sp_portal_taxonomy');
                taxCheck.addNotNullQuery('active');
                taxCheck.addEncodedQuery('active=true');
                taxCheck.addQuery('sp_portal', portalCheck.sys_id);
                taxCheck.query();

                while (taxCheck.next()) {
                    // we have to use a while incase there is more than one taxonomy associated
                    // now we need to follow the taxonomy's path
                    var getTax = new GlideRecord('taxonomy');
                    getTax.addNotNullQuery('sys_id');
                    getTax.addQuery('sys_id', taxCheck.taxonomy);
                    getTax.query();
                    if (getTax.next()) {
                        lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(getTax, "Taxonomy", "Name");
                    }

                    var taxTopic = new GlideRecord('topic');
                    taxTopic.addNotNullQuery('taxonomy');
                    taxTopic.addQuery('taxonomy', getTax.sys_id);
                    taxTopic.query();
                    while (taxTopic.next()) {
                        lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(taxTopic, "Topic", "Name");

                        // Now we need to see what's connected
                        var conTopic = new GlideRecord('m2m_connected_content');
                        conTopic.addNotNullQuery('topic');
                        conTopic.addQuery('topic', taxTopic.sys_id);
                        conTopic.addEncodedQuery('content_type=07f4b6bfe75430104cda66ef11e8a9a9'); // we're looking for quick links
                        conTopic.query();
                        while (conTopic.next()) {
                            // we must have found a quick link
                            qls = true;
                            var qlRec = new GlideRecord('sp_page');
                            qlRec.addNotNullQuery('sys_id');
                            qlRec.addQuery('sys_id', conTopic.quick_link.page); // this is the page from the quick link
                            qlRec.query();
                            if (qlRec.next()) {
                                lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(qlRec, qlRec.sys_class_name.getDisplayValue(), qlRec.getDisplayName());
                                _getPage(qlRec.sys_id);
                            }
                        }
                    }
                }
            }

            // now we need to find the menu in this portal
            var menuCheck = new GlideRecord('sp_instance_menu');
            menuCheck.addNotNullQuery('sys_id');
            menuCheck.addQuery('sys_id', portalCheck.sp_rectangle_menu);
            menuCheck.query();
            if (menuCheck.next()) {
                lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(menuCheck, "Menu - " + menuCheck.getDisplayValue(), 'Menu');

                // now we have the menu, we need the options in it
                var menuItemCheck = new GlideRecord('sp_rectangle_menu_item');
                menuItemCheck.addNotNullQuery('sp_rectangle_menu');
                menuItemCheck.orderBy('label');
                menuItemCheck.addQuery('sp_rectangle_menu', menuCheck.sys_id);
                menuItemCheck.query();
                while (menuItemCheck.next()) {
                    if (menuItemCheck.label) {
                        // This will look for the (menuItemCheck.label)'s value in sys_ui_message table.
                        lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(menuItemCheck, menuCheck.getDisplayValue(), 'Menu Item');

                        // we need to check if we have "communities" installed or being used
                        if (menuItemCheck.label == "Employee Forums" || menuItemCheck.label == "Community" || menuItemCheck.label == "Forums") {
                            // we now need to loop through some specific records for the Communities actions and notification types
                            var comMessages = new GlideRecord("sn_actsub_activity_type");
                            comMessages.addEncodedQuery('activity_nl_string.messageISNOTEMPTY');
                            comMessages.query();
                            while (comMessages.next()) {
                                // now we need to get the specific message
                                var getComMessage = new GlideRecord('sys_ui_message');
                                getComMessage.addQuery('sys_id', comMessages.activity_nl_string.sys_id);
                                getComMessage.query();
                                if (getComMessage.next()) {
                                    lfDocumentContentBuilder.processString(getComMessage.message.toString(), "Activity Message", "Name");
                                }
                            }
                        }

                        // now we need to process a page if there is one
                        if (menuItemCheck.sp_page.sys_id) {
                            _getPage(menuItemCheck.sp_page.sys_id);
                        }

                        // lets check if there's a sub-menu also
                        var subMenuItemCheck = new GlideRecord('sp_rectangle_menu_item');
                        subMenuItemCheck.addNotNullQuery('sp_rectangle_menu_item');
                        subMenuItemCheck.orderBy('label');
                        subMenuItemCheck.addQuery('sp_rectangle_menu_item', menuItemCheck.sys_id);
                        subMenuItemCheck.query();
                        while (subMenuItemCheck.next()) {
                            lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(subMenuItemCheck, menuCheck.getDisplayValue() + ' - ' + menuItemCheck.getDisplayValue(), 'Sub-Menu Item');

                            // now we need to process a page if there is one
                            if (subMenuItemCheck.sp_page.sys_id) {
                                _getPage(subMenuItemCheck.sp_page.sys_id);
                            }

                            // lets check if there's a sub-sub-menu also
                            var subsubMenuItemCheck = new GlideRecord('sp_rectangle_menu_item');
                            subsubMenuItemCheck.addNotNullQuery('sp_rectangle_menu_item');
                            subsubMenuItemCheck.orderBy('label');
                            subsubMenuItemCheck.addQuery('sp_rectangle_menu_item', subMenuItemCheck.sys_id);
                            subsubMenuItemCheck.query();
                            while (subsubMenuItemCheck.next()) {
                                lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(subsubMenuItemCheck, menuCheck.getDisplayValue() + ' - ' + menuItemCheck.getDisplayValue(), 'Sub-Sub-Menu Item');

                                // now we need to process a page if there is one
                                if (subsubMenuItemCheck.sp_page.sys_id) {
                                    _getPage(subsubMenuItemCheck.sp_page.sys_id);
                                }
                            }
                        }
                    }
                }
            }

            // lets trawl the page route map before we figure out which pages are in the portal
            var porPageMap = new GlideRecord('sp_page_route_map');
            porPageMap.addNotNullQuery('portals');
            porPageMap.addEncodedQuery('portals=' + portalCheck.sys_id); // updated this query to not include route entries not associated to a portal due to higher chance of unecessary results
            porPageMap.query();
            while (porPageMap.next()) {
                var pages = [];
                pages.push(porPageMap.route_from_page.sys_id); // we need to process the from pages
                pages.push(porPageMap.route_to_page.sys_id); // we also need to process the to pages as well
                pages.sort();

                // we need to make the pages unique
                var cleanPages = new ArrayUtil();
                cleanPages = cleanPages.unique(pages);

                for (var i = 0; i < cleanPages.length; i++) {
                    // now we can process each unique page
                    _getPage(cleanPages[i]);

                    // now we need to check for any Request Filters
                    var reqFil = new GlideRecord('request_filter');
                    reqFil.addQuery('portal_page', cleanPages[i]);
                    reqFil.query();
                    while (reqFil.next()) {
                        lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(reqFil, reqFil.sys_class_name.getDisplayValue(), reqFil.getDisplayValue());
                    }
                }
            }

            // lets check for any categories
            var checkCatCats = new GlideRecord('m2m_sp_portal_catalog');
            checkCatCats.addEncodedQuery('sp_portal=' + portalCheck.sys_id.toString());
            checkCatCats.addQuery('active', 'true');
            checkCatCats.query();
            while (checkCatCats.next()) {
                // we might have more than one catalog associated to the portal, so we need to loop through each of them
                var getCatalog = new GlideRecord('sc_catalog');
                getCatalog.addQuery('sys_id', checkCatCats.sc_catalog.sys_id);
                getCatalog.addQuery('active', 'true');
                getCatalog.query();
                if (getCatalog.next()) {
                    lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(getCatalog, "Service Catalog", "Name");
                }

                // now we need to get the Categories in the Catalog, but we don't need the items in the Catalog (there's a seperate artifact for that)
                var getCatCat = new GlideRecord('sc_category');
                getCatCat.addEncodedQuery('sc_catalog=' + checkCatCats.sc_catalog.sys_id.toString());
                getCatCat.addQuery('active', 'true');
                getCatCat.orderBy('title');
                getCatCat.query();
                while (getCatCat.next()) {
                    lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(getCatCat, checkCatCats.sc_catalog.getDisplayValue() + " - Service Catalog Categories", "Category Name");
                }
            }
            // we also need to check for the KB categories associated to this portal
            var checkKBcats = new GlideRecord('m2m_sp_portal_knowledge_base');
            checkKBcats.addEncodedQuery('sp_portal=' + portalCheck.sys_id.toString());
            checkKBcats.query();
            while (checkKBcats.next()) {
                // we now need to loop through each KB for it's categories
                var getKBcats = new GlideRecord('kb_category');
                getKBcats.addQuery('parent_id', checkKBcats.kb_knowledge_base.sys_id);
                getKBcats.addQuery('active', 'true');
                getKBcats.query();
                while (getKBcats.next()) {
                    lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(getKBcats, checkKBcats.kb_knowledge_base.getDisplayValue() + " - KB Categories", "Category Name");
                }
            }

            // now we need to check the widgets on the defined pages for the portal in question, rather than the ones with the routing
            if (portalCheck.homepage) {
                // if we have a homepage value
                _getPage(portalCheck.homepage.sys_id);
            }
            if (portalCheck.kb_knowledge_page) {
                // if we have a kb page value
                _getPage(portalCheck.kb_knowledge_page.sys_id);
            }
            if (portalCheck.login_page) {
                // if we have a login page value
                _getPage(portalCheck.login_page.sys_id);
            }
            if (portalCheck.notfound_page) {
                // if we have a 404 page value
                _getPage(portalCheck.notfound_page.sys_id);
            }
            if (portalCheck.sc_catalog_page) {
                // if we have a catalog page value
                _getPage(portalCheck.sc_catalog_page.sys_id);
            }
            if (portalCheck.sc_category_page) {
                // if we have a Catalog category home page value
                _getPage(portalCheck.sc_category_page.sys_id);
            }
            if (portalCheck.sp_rectangle_menu) {
                // we need to fetch the widget record
                var getHeaderWid = new GlideRecord('sp_widget');
                getHeaderWid.addQuery('sys_id', portalCheck.sp_rectangle_menu.sp_widget.sys_id);
                getHeaderWid.query();
                if (getHeaderWid.next()) {
                    lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(getHeaderWid, "Header Menu", "Name");
                    lfDocumentContentBuilder.processScript(getHeaderWid.template, "Header Menu", "Body HTML Template");
                    lfDocumentContentBuilder.processScript(getHeaderWid.script, "Header Menu", "Server Script");
                    lfDocumentContentBuilder.processScript(getHeaderWid.client_script, "Header Menu", "Client Controller");
                    lfDocumentContentBuilder.processScript(getHeaderWid.link, "Header Menu", "Link");
                }
                // we also need to check if there's any ${} calls
                _getMessages(portalCheck.sp_rectangle_menu.sp_widget.template.toString(), "Header Menu", "Body HTML Template");
                _getMessages(portalCheck.sp_rectangle_menu.sp_widget.script.toString(), "Header Menu", "Server Script");
                _getMessages(portalCheck.sp_rectangle_menu.sp_widget.client_script.toString(), "Header Menu", "Client Controller");
                _getMessages(portalCheck.sp_rectangle_menu.sp_widget.link.toString(), "Header Menu", "Link");
            }

            // we need to check portal content categories in this way because otherwise it causes duplicates at the page level
            var pluginCheck = new GlidePluginManager();
            var cont = pluginCheck.isActive('sn_cd'); // this is to avoid an error for instances where this plugin is not installed
            if (cont == true) {
                var checkCDcats = new GlideRecord('sn_cd_content_category');
                checkCDcats.addEncodedQuery('active=true');
                checkCDcats.orderBy('order');
                checkCDcats.query();
                while (checkCDcats.next()) {
                    lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(checkCDcats, "Portal Content Experience", "Category");
                }

                // while we're here, lets also get the news from this portal
                var getNews = new GlideRecord('sn_cd_content_news');
                getNews.addEncodedQuery('state=published');
                getNews.query();
                while (getNews.next()) {
                    lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(getNews, "News Item", "Name");
                }

                // while we're here, lets also get the various Portal Content entries
                var getPorCont = new GlideRecord('sn_cd_content_portal');
                getPorCont.addEncodedQuery('content_type=41040dd7237e2300fb0c949e27bf652a^ORcontent_type=a673597a0b4303008cd6e7ae37673a6f^state=published');
                getPorCont.query();
                while (getPorCont.next()) {
                    lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(getPorCont, "Portal Content", "Title");
                }
            }

            // there is a scenario where we may have widgets with no ID's which can't be captured unless we specifically look for them but this will likely include widgets not in the scope of this portal, there is no otherway :(
            var getNoIDWid = new GlideRecord('sp_widget');
            getNoIDWid.addEncodedQuery('id=NULL');
            getNoIDWid.orderBy('name');
            getNoIDWid.query();
            while (getNoIDWid.next()) {
                _getWidgets(getNoIDWid);
            }
        }


        function _getPage(pageSYS) {
            try {
                if (pageSYS) {
                    // we need to check if this page has been already processed
                    if (!pageArr.toString().includes(pageSYS.toString())) {


                        // This might not be needed any more due to the Portal content artifact

                        // we need to pick up the Content Experience elements if this is an ESC Pro portal
                        // need to check if "Content Publishing" is installed first
                        var pluginCheck = new GlidePluginManager();
                        var cont = pluginCheck.isActive('sn_cd'); // this is to avoid an error for instances where this plugin is not installed
                        if (cont == true) {
                            // we only need to know it's installed, so we don't need the full record return

                            // let's now check for what Content is set up
                            var getContExp = new GlideRecord('sn_cd_content_visibility');
                            getContExp.addEncodedQuery('sp_page=' + pageSYS + '^active=true');
                            getContExp.query();
                            while (getContExp.next()) {
                                // we need to process the page directly related to this record
                                if (getContExp.sp_page.sys_id != pageSYS) {
                                    lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(getContExp, "Portal Content Experience", getContExp.title);
                                    _getPage(getContExp.sp_page.sys_id); // we need to send the page associated to the content, which would call this very function again... but we need to know...
                                }

                                // now we need the actual content record
                                var contBase = new GlideRecord('sn_cd_content_portal'); // this should represent the various sub-classes of content
                                contBase.addNotNullQuery('sys_id');
                                contBase.addQuery('sys_id', getContExp.content.sys_id);
                                contBase.query();
                                if (contBase.next()) {
                                    lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(contBase, "Portal Content Experience", getContExp.title);
                                    lfDocumentContentBuilder.processString(contBase.button_text.toString(), "Portal Content Experience", "Button Text");
                                }
                            }
                        }

                        var pageVal = instCheck.sp_column.sp_row.sp_container.sp_page.getDisplayValue().toString();
                        if (!pageVal) {
                            pageVal = "No Page Associated";
                        }

                        // if we have the page sys_id then we need to loop from the instances
                        var instCheck = new GlideRecord('sp_instance');
                        instCheck.addNotNullQuery('sp_column');
                        instCheck.addEncodedQuery('sp_column.sp_row.sp_container.sp_page=' + pageSYS);
                        instCheck.addQuery('active', 'true');
                        instCheck.query();
                        while (instCheck.next()) {
                            lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(instCheck, "Widget Instance - " + instCheck.sys_class_name.getDisplayValue(), "Page - " + pageVal + " > Widget - " + instCheck.sp_widget.getDisplayName());
                            _getWidgets(instCheck, pageSYS);
                        }
                        pageArr.push(pageSYS.toString()); // we need to feed the page just processed into the Page Array
                    }
                }
            } catch (err) {
                //gs.log('Error in _getPage - ' + err.name + ' - ' + err.message);
            }
        }

        function _getWidgets(instCheck, pageSYS) {
            try {
                // we need to check if this widget has already been processed
                var widgetCheck = '';
                if (instCheck) {
                    if (instCheck.sp_widget) {
                        // this covers instances, there are multiple types
                        widgetCheck = instCheck.sp_widget.sys_id.toString();
                    } else {
                        // this covers widget widgets
                        widgetCheck = instCheck.sys_id.toString();
                    }
                    // widget duplication check
                    if (!widgetArr.toString().includes(instCheck.toString())) {
                        widgetArr.push(instCheck.toString()); // now we need to push the widget processed so we can de-dedupe check it for each widget we go through
                    }
                } else {
                    return
                }

                if (widgetCheck == 'cf1a5153cb21020000f8d856634c9c3c') {
                    // this is the ootb carousel widget we now need to check the carousel slides to see if they are related to this portal
                    var checkCarousel = new GlideRecord('sp_carousel_slide');
                    checkCarousel.addEncodedQuery('carousel.sp_widget=' + widgetCheck);
                    checkCarousel.query();
                    while (checkCarousel.next()) {
                        lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(checkCarousel, "Carousel Slides", "Name");
                    }
                }

                if (!widgetArr.toString().includes(widgetCheck.toString()) && pageSYS != '') {
                    var pageVal = '';
                    var pageName = new GlideRecord('sp_page');
                    pageName.addQuery('sys_id', pageSYS);
                    pageName.query();

                    if (!pageName.hasNext()) {
                        if (instCheck.sp_column.sp_row.sp_container.sp_page.getDisplayValue().toString() != '') {
                            // we need to be super defensive
                            pageVal = instCheck.sp_column.sp_row.sp_container.sp_page.getDisplayValue().toString();
                        }
                    }
                    if (pageName.next()) {
                        pageVal = pageName.getDisplayValue();
                    }
                    if (!pageVal) {
                        pageVal = "No Page Associated";
                    }

                    // now we need to check through widgets
                    var widCheck = new GlideRecord('sp_widget');
                    widCheck.addEncodedQuery('sys_id=' + widgetCheck + '^ORsp_widget=' + widgetCheck);
                    widCheck.query();
                    while (widCheck.next()) {
                        lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(widCheck, widCheck.sys_class_name.getDisplayValue(), "Page - " + pageVal + " > Widget - " + widCheck.getDisplayValue());

                        // in some very rare scenarios we also need to check for other titles of a widget instance
                        var getOtherWidInst = new GlideRecord('sp_instance');
                        getOtherWidInst.addEncodedQuery('active=true^title!=NULL^sp_widget=' + widCheck.sys_id);
                        getOtherWidInst.orderBy('order');
                        getOtherWidInst.query();
                        if (getOtherWidInst.next()) {
                            lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(getOtherWidInst, widCheck.sys_class_name.getDisplayValue(), "Page - " + pageVal + " > Widget - " + widCheck.getDisplayValue());
                        }

                        // we need to see if this widget is the GetSupport widget
                        if (widCheck.id == 'ec-get-support') {
                            // this is if it's the new EC "get support" widget, then we need to go and query the sn_ex_sp_get_support for supporting elements
                            var getSupportCheck = new GlideRecord('sn_ex_sp_get_support');
                            getSupportCheck.query();
                            while (getSupportCheck.next()) {
                                // now we need to go to each record
                                var checkSupport = new GlideRecord(getSupportCheck.table.toString()); // this should be holding the table value
                                checkSupport.addQuery('sys_id', getSupportCheck.content.toString()); // this should be holding the sys_id of the linked record
                                checkSupport.query();
                                if (checkSupport.next()) {
                                    lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(checkSupport, "Support items", checkSupport.title.toString());
                                }
                            }
                        }

                        // we need to check the Angular provider as well
                        var checkAng = new GlideRecord('sp_ng_template');
                        checkAng.addQuery('sp_widget', widCheck.sys_id);
                        checkAng.query();
                        while (checkAng.next()) {
                            lfDocumentContentBuilder.processScript(checkAng.template, widCheck.sys_class_name.getDisplayValue(), "Page - " + pageVal + " > Angular Provider - " + checkAng.getDisplayValue());
                            _getMessages(checkAng.template.toString(), widCheck.sys_class_name.getDisplayValue(), "Page - " + pageVal + " > Angular Provider - " + checkAng.getDisplayValue());
                        }

                        // template field
                        lfDocumentContentBuilder.processScript(widCheck.template, widCheck.sys_class_name.getDisplayValue(), "Page - " + pageVal + " > Widget - " + widCheck.getDisplayValue());
                        _getMessages(widCheck.template.toString(), widCheck.sys_class_name.getDisplayValue(), "Page - " + pageVal + " > Widget - " + widCheck.getDisplayValue());

                        // Server Script
                        lfDocumentContentBuilder.processScript(widCheck.script, widCheck.sys_class_name.getDisplayValue(), "Page - " + pageVal + " > Widget - " + widCheck.getDisplayValue());
                        _getMessages(widCheck.script.toString(), widCheck.sys_class_name.getDisplayValue(), "Page - " + pageVal + " > widget - " + widCheck.getDisplayValue());

                        // we need to see if there's widget being loaded by script
                        if (widCheck.script.match(/(\$sp.getWidget)(\(\'|\").*?(\"|\')/gi)) {
                            var widLine = widCheck.script.match(/(\$sp.getWidget)(\(\'|\").*?(\"|\')/gi);

                            // we might have an array
                            var servWid = widLine.toString().split(',');
                            for (var widS = 0; widS < servWid.length; widS++) {
                                if (servWid[widS].toString() != '') {
                                    servWid[widS] = servWid[widS].match(/(?:['"].*?['"])/gi); // we want to strip any quotes to get a clean value
                                    // now we need to remove the quotes if there are any
                                    var cleanWid = servWid[widS].toString().replace(/['"]/gi, ''); // with nothing
                                    // now we need to get the widget before we process it
                                    var getWidID = new GlideRecord('sp_widget');
                                    getWidID.addQuery('id', cleanWid.toString());
                                    getWidID.query();
                                    if (getWidID.next()) {
                                        // process this widget
                                        if (getWidID.sys_id != instCheck.sys_id && (!widgetArr.toString().includes(getWidID.sys_id))) {
                                            _getWidgets(getWidID, instCheck.sp_column.sp_row.sp_container.sp_page.sys_id);
                                        }
                                    }
                                }
                            }
                        }

                        // Client Script
                        lfDocumentContentBuilder.processScript(widCheck.client_script, widCheck.sys_class_name.getDisplayValue(), "Page - " + pageVal + " > Widget - " + widCheck.getDisplayValue());
                        _getMessages(widCheck.client_script.toString(), widCheck.sys_class_name.getDisplayValue(), "Page - " + pageVal + " > Widget - " + widCheck.getDisplayValue());

                        // we need to see if there's widgets being loaded by script
                        if (widCheck.client_script.match(/(spUtil.get\(\'|\").*?(\'|\")/gi)) {
                            var widCLid = widCheck.client_script.match(/(spUtil.get\(\'|\").*?(\'|\")/gi);

                            // we might have an array
                            var clWid = widCLid.toString().split(',');
                            for (var widCL = 0; widCL < clWid.length; widCL++) {
                                if (clWid[widCL].toString() != '') {
                                    // now we need to remove the quote if there are any
                                    var cleanWidCL = clWid[widCL].toString().replace(/'|"/g, ''); // we want to strip any quotes to get a clean value
                                    // now we need to get the widget before we process it
                                    var CLgetWidID = new GlideRecord('sp_widget');
                                    CLgetWidID.addQuery('id', cleanWidCL.toString());
                                    CLgetWidID.query();
                                    if (CLgetWidID.next()) {
                                        // process the widget
                                        if (CLgetWidID.sys_id != instCheck.sys_id && (!widgetArr.toString().includes(CLgetWidID))) {
                                            _getWidgets(CLgetWidID);
                                        }
                                    }
                                }
                            }
                        }
                    }


                    // now we need to check instance specifics
                    // the JSON of the instance
                    var pageArr = [];
                    if (instCheck.widget_parameters != '') {
                        var valueObj = instCheck.widget_parameters.toString();
                        // we need to process the "widget_parameters" incase there's a message call in it
                        if (valueObj != "" || valueObj != " ") {
                            _getMessages(valueObj.toString(), instCheck.sys_class_name.getDisplayValue(), "Page - " + pageVal + " > Widget - " + instCheck.sp_widget.getDisplayValue());
                        }

                        // now we need to see if there's a page in the "widget_parameters"
                        if (valueObj.includes("page") || valueObj.includes("displayValue")) {
                            // we only need to look into the JSON values, if there is a mention of pages otherwise it could be completely unrelated to our needs.
                            var valueCheck = JSON.parse(valueObj, function(key, val) {
                                if (key == 'displayValue') {
                                    // we need this page's sys_id but we need to make sure we don't reprocess the same page we started with
                                    if ((val != pageSYS.id || !valueObj.toString().includes(pageSYS.id)) && val.toString().match(/[a-z0-9]{32}/g)) {
                                        // we don't want to cause a query with a null value and only for a sys_id
                                        var getPageID = new GlideRecord('sp_page');
                                        getPageID.addQuery('id', val.toString());
                                        getPageID.query();
                                        if (!getPageID.hasNext()) {
                                            /*
											// we might not need this after all
											// we might need to factor in a header
                                            if (val.toString() != 'true' && val.toString().match(/(?:^\{)|(^['"].*?['"])/gim) && val.toString() != 'false' && val.toString() != '' && val.toString() != ' ' && val.toString() != "" && val.toString() != 'number') {
                                                lfDocumentContentBuilder.processString(val.toString(), instCheck.sys_class_name.getDisplayValue(), "Page - " + pageVal + " > Widget - " + instCheck.sp_widget.getDisplayValue());
                                            }
											*/
                                        }
                                        if (getPageID.next()) {
                                            pageArr.push(getPageID.sys_id.toString());
                                        }
                                    }
                                    /*
									// we might not need this after all
									else if (val != '[object Object]' && val.toString().match(/(?:^\{)|(^['"].*?['"])/gim) && val.toString() != 'true' && val.toString() != 'false' && val.toString() != '' && val.toString() != ' ' && val.toString() != "" && val.toString() != 'number') {
                                        if (val.toString() != "" || val.toString() != " ") {
                                            lfDocumentContentBuilder.processString(val.toString(), instCheck.sys_class_name.getDisplayValue(), "Page - " + pageVal + " > Widget - " + instCheck.sp_widget.getDisplayValue());
                                        }
                                    }
									*/
                                }
                            });
                        }
                    }
                    var pageIDArr = new ArrayUtil();
                    pageIDArr.unqiue(pageArr);
                    for (var iD = 0; iD < pageIDArr.length; iD++) {
                        _getPage(pageIDArr[iD]);
                    }

                    // now we need to do some specific option_schema checks
                    var optMsg = [];
                    if (instCheck.sp_widget.option_schema != '') {
                        var widString = instCheck.sp_widget.option_schema.toString();
                        if (widString != '') {
                            var widJ = JSON.parse(widString, function(key, value) {
                                // if we send these to the array, we can clear out duplicates
                                if (key == "displayValue" && (value != '' && value != ' ')) {
                                    optMsg.push(value);
                                }
                                if (key == "label" && (value != '' && value != ' ')) {
                                    optMsg.push(value);
                                }
                                if (key == "hint" && (value != '' && value != ' ')) {
                                    optMsg.push(value);
                                }
                                if (key == "section" && (value != '' && value != ' ')) {
                                    optMsg.push(value);
                                }
                                if (key == "default_value" && value.toString().match(/(^\b).*(\b)/gim) && (value.toString() != 'true' && value.toString() != 'false')) {
                                    optMsg.push(value);
                                }
                            });
                        }
                        // we need to strip the option_schema array of any duplicates before we process it
                        optMsg.sort();
                        var optSchArr = new ArrayUtil();
                        optSchArr = optSchArr.unique(optMsg);
                        lfDocumentContentBuilder.processStringArray(optSchArr, instCheck.sys_class_name.getDisplayValue(), "Page - " + pageVal + " > Widget -  " + instCheck.sp_widget.getDisplayValue());
                        optMsg = [];
                    }


                    // we need to check to see if the EC or EC pro is installed first
                    var checkEC = new GlideRecord('sys_package');
                    checkEC.addEncodedQuery('source=sn_ex_sp^ORsource=sn_ex_sp_pro');
                    checkEC.query();
                    if (checkEC.hasNext()) {

                        // for the /esc portal, we also need to loop through any "web applications"
                        if (instCheck.sp_widget.id == 'web_applications' || instCheck.sp_widget.id == "app-launcher") {
                            var webApp = new GlideRecord('sn_ex_sp_pro_web_application');
                            webApp.addNotNullQuery('active');
                            webApp.addEncodedQuery('active=true'); // we only need active items
                            webApp.query();
                            while (webApp.next()) {
                                lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(webApp, "Web Applications", webApp.name);
                            }
                        }

                        // now we need the quick links
                        if (instCheck.sp_widget.id == "quick-links" || instCheck.sp_widget.id == "cd-quick-links" || instCheck.sp_widget.id == "quick_links_on_topic_page") {
                            var quickLinks = new GlideRecord('sn_ex_sp_quick_link');
                            quickLinks.addNotNullQuery('active');
                            quickLinks.addEncodedQuery('active=true'); // these will pick up everything connected via the m2m_connected_content table and any others associated
                            quickLinks.query();
                            while (quickLinks.next()) {
                                lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(quickLinks, "Quick Links", "Name");
                            }
                        }

                        // we need to get the Activity items of task records
                        if (instCheck.sp_widget.id == 'my-items') {
                            var activeItems = new GlideRecord('sn_ex_sp_activity_configuration');
                            activeItems.addNotNullQuery('active');
                            activeItems.addEncodedQuery('active=true');
                            activeItems.query();
                            while (activeItems.next()) {
                                lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(activeItems, "Active Items", "Name");
                            }
                            // we also need to manually add some due time messages for these tasks
                            lfDocumentContentBuilder.processString('overdue', "Active Items", "Name");
                            lfDocumentContentBuilder.processString('overdue {0} day', "Active Items", "Name");
                            lfDocumentContentBuilder.processString('overdue {0} days', "Active Items", "Name");
                            lfDocumentContentBuilder.processString('due today', "Active Items", "Name");
                            lfDocumentContentBuilder.processString('due in {0} days', "Active Items", "Name");
                            lfDocumentContentBuilder.processString('Quick tasks', "Active Items", "Name");
                        }
                    }


                    // now that we've processed this page, let's now process any others we've previously found
                    var cleanPage = new ArrayUtil();
                    cleanPage = cleanPage.unique(pageArr);
                    if (cleanPage.length > 0) {

                        for (var p = 0; p < cleanPage.length; p++) {
                            var pageID = new GlideRecord('sp_page');
                            pageID.addNotNullQuery('id');
                            pageID.addQuery('id', cleanPage[p]);
                            pageID.addQuery('sys_id', '!=', sysId); // just to ensure we're not re-looping
                            pageID.query();
                            if (pageID.next()) {
                                _getPage(pageID.sys_id); // call this function again with the new page
                            }
                        }
                        pageArr = []; // resetting the array
                        cleanPage = []; // resetting the array
                    }
                }
            } catch (err) {
                //gs.log('Error in _getWidgets - ' + err.name + ' - ' + err.message);
            }
        }



        function _getMessages(recField, name, label) {
            try {
                // ${} checks
                var MsgCheck = /\${(?:[^{}]|\{(?:[^{}]|\{[^{}]*\})*\})*}|(?:^\$\{)(.*)(\})/gim; // this is a lighter version but is designed to find ${something} under multiple scenarios
                var pgClean = /(^\$\{|^\{)|(\}$)/gi; // this is for cleaning out the flanking { } in the identified string
                var msgArr = [];

                var fieldCheck = '';
                fieldCheck = recField.toString().match(MsgCheck);

                if (fieldCheck) {
                    var fieldClean = fieldCheck.toString().split(',$');
                    for (var i = 0; i < fieldClean.length; i++) {
                        if (!fieldClean[i].toString().includes('getMessage')) {
                            // this is a defensive check to avoid unnecessary duplicates
                            if (fieldClean[i].toString() != '' && (!fieldClean[i].toString().includes('{{item.label}}'))) {
                                // this is a defensive check because there are some sometimes some MessageAPI calls that contain nothing
                                var cleanStr = fieldClean[i].toString().replace(pgClean, '');
                                if (cleanStr) {
                                    // we need to make sure we're not inadvertantly pushing a blank space
                                    msgArr.push(cleanStr); // send to the array to clean unnecessary duplicates
                                }
                            }
                        } else if (fieldClean[i].toString().includes('getMessage')) {
                            // this might not ever be needed due to use of .processScript() method used on the widget records
                            var msgStringReg = /(['"])(.*)(['"])/gi;
                            var msgString = fieldClean[i].toString().match(msgStringReg);
                            // now we need to move the flanking quotes (single or double)
                            var getMclean = /(\B['"])|(['"]\s)/gi;
                            var cleanGetM = msgString.replace(getMclean, '');
                            msgArr.push(cleanGetM); // send to the array to clean unnecessary duplicates
                        }
                    }
                }

                // getMessage(variable) checks
                var checkMsgReg = /(?:getMessage\()(?!['"])(.*)(?!['"])(\))/gi;
                var callCheck = recField.toString().match(checkMsgReg);

                if (callCheck) {
                    // we need to cleanup the call
                    var callValReg = /([^()]*)/gi;
                    var callValCheck = callCheck.toString().match(callValReg);
                    var varToCheck = '';
                    for (var cvi = 0; cvi < callValCheck.length; cvi++) {
                        if (!callValCheck[cvi].toString().includes('getMessage') && callValCheck[cvi].toString() != '') {
                            varToCheck = callValCheck[cvi].toString();
                        }
                    }

                    // now we need to see if there is a variable of this with a string associated to it
                    // the regex being built is - /(?:(var\s)(something).*(\=)).*(['"].*["';])/gi
                    var checkIfVarReg1 = "(?:(var\s)(";
                    var checkIfVarReg2 = ").*(\=)).*(['";
                    var checkIfVarReg3 = '"].*[';
                    var checkIfVarReg4 = '"';
                    var checkIfVarReg5 = "';])";
                    var buildCheckVarReg = checkIfVarReg1 + varToCheck + checkIfVarReg2 + checkIfVarReg3 + checkIfVarReg4 + checkIfVarReg5;
                    var VarRegChecker = new RegExp(buildCheckVarReg, "gim");
                    var VarRegCheck = recField.toString().match(VarRegChecker);
                    if (VarRegCheck) {
                        // this is for a variable with the string - which should find something like "var something = 'this is my text' or to that effect"
                        // now we need to identify just the pure string, our findings could be in an array
                        for (vriC = 0; vriC < VarRegCheck.length; vriC++) {
                            var VarStrCheck = /(['"].*["'])/gim;
                            var VarRegChecker = VarRegCheck[vri].toString().match(VarStrCheck);
                            if (VarRegChecker) {
                                // now we need to clean the flanking quotes (single or double);
                                var VarRegClStr = /(^['"])|(['"]$)/gim;
                                var VarRegCleanedStr = VarRegChecker.toString().replace(VarRegClStr, '');
                                msgArr.push(VarRegCleanedStr);
                            }
                        }
                    } else {
                        // so if there's no variable is there a function?
                        var checkIfFuncReg = "(?:function ).*?(\()(" + varToCheck + ")(\))";
                        var checkIfFuncChecker = new RegExp(checkIfFuncReg, "gi");
                        var checkIfFunc = recField.toString().match(checkIfFuncChecker);
                        if (checkIfFunc) {
                            // we have a valid find for a function call, now we need to see each call to this function
                            var funcCheckReg = /(?!\bfunction\s+)(?:(\w+)\s*)(?=\()/gi;
                            var nameOfFunc = checkIfFunc.toString().match(funcCheckReg);

                            // now we need to find every time the function is called and pull out it's string
                            // (error)(\(['"]).*(['"])
                            var nameOfFuncCheck = "(" + nameOfFunc + ")(\(['" + '"]).*([' + "'" + '"])';
                            var nameOfFuncChecker = new RegExp(nameOfFuncCheck, "gi");
                            // this will give us -> func = 'something' so we need to clean it up

                            var FuncNameChecker = recField.toString().match(nameOfFuncChecker);
                            // this will make an array of -> functionname('something')

                            if (FuncNameChecker) {
                                for (fni = 0; fni < FuncNameChecker.length; fni++) {
                                    // lets extract the clean text
                                    var cleanFNI = FuncNameChecker[fni].toString().match(/(['"]).*(['"])/gi);
                                    if (cleanFNI) {
                                        // now we need to clean the flanking quotes
                                        var cleanedFNI = cleanFNI.toString().replace(/(^['"])|(['"]$)/gim, '');
                                        if (cleanedFNI) {
                                            msgArr.push(cleanedFNI);
                                        }
                                    }
                                }
                            }
                        }
                    }
                }

                msgArr.sort();
                var cleanMsg = new ArrayUtil();
                cleanMsg = cleanMsg.unique(msgArr);
                if (cleanMsg.length > 0 && cleanMsg != ' ') {
                    lfDocumentContentBuilder.processStringArray(cleanMsg, name, label);
                }

            } catch (err) {
                //gs.log('Error in _getMessages - ' + err.name + ' - ' + err.message + " - " + label);
            }
        }

        return lfDocumentContentBuilder.build();
    },

    /**********
     * Uncomment the saveTranslatedContent function to override the default behavior of saving translations
     * 
     * @Param documentContent LFDocumentContent object
     * @return
     **********/
    /**********
        saveTranslatedContent: function(documentContent) {},
    **********/

    type: 'LF_PortalProcessor'
});

 

 

 

 

 

 

If for what-ever reason, the above doesn't pick something up on a specific page (this might happen when a page is in a forced href for example), I've also created a Page specific Prototype artifact that you can run on that particular page (I called it "LF_PortalPageProcessor", so be mindful of the Script Include name you'll need to make, and the Artifact name for the UI action):

var LF_PortalPageProcessor = Class.create();
LF_PortalPageProcessor.prototype = Object.extendsObject(global.LFArtifactProcessorSNC, {
    category: 'localization_framework', // DO NOT REMOVE THIS LINE!

    /**********
     * Extracts the translatable content for the artifact record
     * 
     *  params.tableName The table name of the artifact record
     *  params.sysId The sys_id of the artifact record 
     *  params.language Language into which the artifact has to be translated (Target language)
     * @return LFDocumentContent object
     **********/
    getTranslatableContent: function(params) {
        /**********
         * Use LFDocumentContentBuilder to build the LFDocumentContent object
         * Use the build() to return the LFDocumentContent object
         **********/

        // we need to trawl through this page for all widgets and widget instances
        var tableName = params.tableName;
        var sysId = params.sysId;
        var language = params.language;
        var lfDocumentContentBuilder = new global.LFDocumentContentBuilder("v1", language, sysId, tableName);
        var pageArr = [];

        var spPage = new GlideRecord('sp_page');
        spPage.addQuery('sys_id', sysId); // this is the page from the UIaction
        spPage.query();
        if (spPage.next()) {
            lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(spPage, "Page Name", "Name");
            // now we need to process the page accordingly
            _getPage(spPage.sys_id);
        }


        function _getPage(pageSYS) {
            var msgArr = []; // for later
            // if we have the page sys_id then we need to loop from the instances
            var instCheck = new GlideRecord('sp_instance');
            instCheck.addEncodedQuery('sp_column.sp_row.sp_container.sp_page=' + pageSYS);
            instCheck.addQuery('active', 'true');
            instCheck.query();
            while (instCheck.next()) {
                lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(instCheck, instCheck.sys_class_name.getDisplayValue(), instCheck.sp_widget.getDisplayName());

                // now we need to check through widgets
                var widCheck = new GlideRecord('sp_widget');
                widCheck.addQuery('sys_id', instCheck.sp_widget.sys_id);
                widCheck.query();
                while (widCheck.next()) {
                    lfDocumentContentBuilder.processTranslatableFieldsForSingleRecord(widCheck, widCheck.sys_class_name.getDisplayValue(), widCheck.getDisplayValue());

                    // template field
                    lfDocumentContentBuilder.processScript(widCheck.template, widCheck.sys_class_name.getDisplayValue(), widCheck.getDisplayValue());
                    _getMessages(widCheck.template, widCheck.sys_class_name.getDisplayValue(), widCheck.getDisplayValue());

                    // Server Script
                    lfDocumentContentBuilder.processScript(widCheck.script, widCheck.sys_class_name.getDisplayValue(), widCheck.getDisplayValue());
                    _getMessages(widCheck.script, widCheck.sys_class_name.getDisplayValue(), widCheck.getDisplayValue());

                    // Client Script
                    lfDocumentContentBuilder.processScript(widCheck.client_script, widCheck.sys_class_name.getDisplayValue(), widCheck.getDisplayValue());
                    _getMessages(widCheck.client_script, widCheck.sys_class_name.getDisplayValue(), widCheck.getDisplayValue());
                }

                // now we need to check the JSON of the instance
                if (instCheck.widget_parameters != '') {
                    var valueObj = instCheck.widget_parameters.toString();
                    if (valueObj.includes("page")) {
                        // we only need to look into the JSON values, if there is a mention of pages otherwise it could be completely unrelated to our needs.
                        valueCheck = JSON.parse(valueObj, function(key, val) {
                            if (key == 'displayValue') {
                                // we need this page's sys_id but we need to make sure we don't reprocess the same page we started with
                                if (val != pageSYS.id || !valueObj.toString().includes(pageSYS.id) || val != '') {
                                    pageArr.push(val);
                                }
                            }
                        });
                    }
                }
                if (instCheck.sp_widget.option_schema != '') {
                    var widString = instCheck.sp_widget.option_schema.toString();
                    var widJ = JSON.parse(widString, function(key, value) {
                        // if we send these to the array, we can clear out duplicates
                        if (key == "displayValue") {
                            msgArr.push(value);
                        }
                        if (key == "label") {
                            msgArr.push(value);
                        }
                        if (key == "hint") {
                            msgArr.push(value);
                        }
                        if (key == "section") {
                            msgArr.push(value);
                        }
                        if (key == "default_value") {
                            msgArr.push(value);
                        }
                    });
                }
                // message array duplicate check
                msgArr.sort();
                var arrayUtil = new ArrayUtil();
                arrayUtil = arrayUtil.unique(msgArr);
                lfDocumentContentBuilder.processStringArray(arrayUtil, instCheck.sys_class_name.getDisplayValue(), instCheck.sp_widget.getDisplayValue());
                msgArr = []; // we need to reset the array
            }


            function _getMessages(recField, name, label) {
				// ${} checks
                var MsgCheck = /\${(?:[^{}]|\{(?:[^{}]|\{[^{}]*\})*\})*}|(?:^\$\{)(.*)(\})/gim; // this is a lighter version but is designed to find ${something} under multiple scenarios
                var pgClean = /(^\$\{|^\{)|(\}$)/gi; // this is for cleaning out the flanking { } in the identified string
                var msgArr = [];

                var fieldCheck = '';
                fieldCheck = recField.toString().match(MsgCheck);

                if (fieldCheck) {
                    var fieldClean = fieldCheck.toString().split(',$');
                    for (var i = 0; i < fieldClean.length; i++) {
                        if (!fieldClean[i].toString().includes('getMessage')) {
                            // this is a defensive check to avoid unnecessary duplicates
                            if (fieldClean[i].toString() != '' && (!fieldClean[i].toString().includes('{{item.label}}'))) {
                                // this is a defensive check because there are some sometimes some MessageAPI calls that contain nothing
                                var cleanStr = fieldClean[i].toString().replace(pgClean, '');
                                if (cleanStr != '') {
                                    // we need to make sure we're not inadvertantly pushing a blank space
                                    msgArr.push(cleanStr); // send to the array to clean unnecessary duplicates
                                }
                            }
                        } else if (fieldClean[i].toString().includes('getMessage')) {
                            // this might not ever be needed due to use of .processScript() method used on the widget records
                            var msgStringReg = /(['"])(.*)(['"])/gi;
                            var msgString = fieldClean[i].toString().match(msgStringReg);
                            // now we need to move the flanking quotes (single or double)
                            var getMclean = /(\B['"])|(['"]\s)/gi;
                            var cleanGetM = msgString.replace(getMclean, '');
                            msgArr.push(cleanGetM); // send to the array to clean unnecessary duplicates
                        }
                    }
                }

                // getMessage(variable) checks
                var checkMsgReg = /(?:getMessage\()(?!['"])(.*)(?!['"])(\))/gi;
                var callCheck = recField.toString().match(checkMsgReg);

                if (callCheck) {
                    // we need to cleanup the call
                    var callValReg = /([^()]*)/gi;
                    var callValCheck = callCheck.toString().match(callValReg);
                    var varToCheck = '';
                    for (var cvi = 0; cvi < callValCheck.length; cvi++) {
                        if (!callValCheck[cvi].toString().includes('getMessage') && callValCheck[cvi].toString() != '') {
                            varToCheck = callValCheck[cvi].toString();
                        }
                    }

                    // now we need to see if there is a variable of this with a string associated to it
                    // the regex being built is - /(?:(var\s)(something).*(\=)).*(['"].*["';])/gi
                    var checkIfVarReg1 = "(?:(var\s)(";
                    var checkIfVarReg2 = ").*(\=)).*(['";
                    var checkIfVarReg3 = '"].*[';
                    var checkIfVarReg4 = '"';
                    var checkIfVarReg5 = "';])";
                    var buildCheckVarReg = checkIfVarReg1 + varToCheck + checkIfVarReg2 + checkIfVarReg3 + checkIfVarReg4 + checkIfVarReg5;
                    var VarRegChecker = new RegExp(buildCheckVarReg, "gim");
                    var VarRegCheck = recField.toString().match(VarRegChecker);
                    if (VarRegCheck) {
                        // this is for a variable with the string - which should find something like "var something = 'this is my text' or to that effect"
                        // now we need to identify just the pure string, our findings could be in an array
                        for (vriC = 0; vriC < VarRegCheck.length; vriC++) {
                            var VarStrCheck = /(['"].*["'])/gim;
                            var VarRegChecker = VarRegCheck[vri].toString().match(VarStrCheck);
                            if (VarRegChecker) {
                                // now we need to clean the flanking quotes (single or double);
                                var VarRegClStr = /(^['"])|(['"]$)/gim;
                                var VarRegCleanedStr = VarRegChecker.toString().replace(VarRegClStr, '');
                                msgArr.push(VarRegCleanedStr);
                            }
                        }
                    } else {
                        // so if there's no variable is there a function?
                        var checkIfFuncReg = "(?:function ).*?(\()(" + varToCheck + ")(\))";
                        var checkIfFuncChecker = new RegExp(checkIfFuncReg, "gi");
                        var checkIfFunc = recField.toString().match(checkIfFuncChecker);
                        if (checkIfFunc) {
                            // we have a valid find for a function call, now we need to see each call to this function
                            var funcCheckReg = /(?!\bfunction\s+)(?:(\w+)\s*)(?=\()/gi;
                            var nameOfFunc = checkIfFunc.toString().match(funcCheckReg);

                            // now we need to find every time the function is called and pull out it's string
                            // (error)(\(['"]).*(['"])
                            var nameOfFuncCheck = "(" + nameOfFunc + ")(\\(['" + '"]).*([' + "'" + '"])';
                            var nameOfFuncChecker = new RegExp(nameOfFuncCheck, "gi");
                            // this will give us -> func = 'something' so we need to clean it up

                            var FuncNameChecker = recField.toString().match(nameOfFuncChecker);
                            // this will make an array of -> functionname('something')

                            for (fni = 0; fni < FuncNameChecker.length; fni++) {
                                // lets extract the clean text
                                var cleanFNI = FuncNameChecker[fni].toString().match(/(['"]).*(['"])/gi);
                                if (cleanFNI) {
                                    // now we need to clean the flanking quotes
                                    var cleanedFNI = cleanFNI.toString().replace(/(^['"])|(['"]$)/gim, '');
                                    if (cleanedFNI) {
                                        msgArr.push(cleanedFNI);
                                    }
                                }
                            }
                        }
                    }
                }

                msgArr.sort();
                var arrayUtil = new ArrayUtil();
                arrayUtil = arrayUtil.unique(msgArr);
                if (arrayUtil.length > 0 && arrayUtil != ' ') {
                    lfDocumentContentBuilder.processStringArray(arrayUtil, name, label);
                }

            }
        }

        // now that we've processed this page, let's now process any others we've previously found
        var cleanPage = new ArrayUtil();
        cleanPage = cleanPage.unique(pageArr);
        if (cleanPage.length > 0) {
            for (var p = 0; p < cleanPage.length; p++) {
                var pageID = new GlideRecord('sp_page');
                pageID.addQuery('id', cleanPage[p]);
                pageID.addQuery('sys_id', '!=', sysId); // just to ensure we're not re-looping
                pageID.query();
                if (pageID.next()) {
                    _getPage(pageID.sys_id); // call this function again with the new page
                }
            }
            pageArr = []; // resetting the array
            cleanPage = []; // resetting the array
        }
        return lfDocumentContentBuilder.build();
    },

    /**********
     * Uncomment the saveTranslatedContent function to override the default behavior of saving translations
     * 
     *  documentContent LFDocumentContent object
     * @return
     **********/
    /**********
        saveTranslatedContent: function(documentContent) {},
    **********/

    type: 'LF_PortalPageProcessor'
});

 

 

 

 

 

 

For those who want to learn, have a look at some of my comments in the code to get an idea of my reasoning for certain concepts and the way I structured my queries. Which also serves as a good example of Coding Standards and ensuring you comment your code.

With regards to the UIaction (for the portal level artifact) it should be defined on the [sp_portal] table, and should look like this:
find_real_file.png

 

* NOTE - in the "condition" of the UI Action, we need to call the "internal name" of the artifact, your's may be different to my example, ultimately if nothing happens when you click the UI Action, it will be because the call isn't recognising your artifact's "internal name".

 

 

When it all comes together, and you're able to request a translation of the portal, the comparison UI will break out the components into the following sub-sections (there could be more or less, depending on your portal):
find_real_file.png 

 

Where-by you will be able to interact with the translations (just like you do with Catalog Items) like this:
find_real_file.png

 

 

Summary

What have we learned? Well, providing we understand how the 5 tables work and providing we understand the hierarchy of the "thing" we want to translate and therefore localize, it's very possible to make the method of actioning simpler. This method does not require a single spreadsheet or very complicated exports and imports.

Over time I can imagine some very clever Artifacts being made out there, but at least the 3 I've provided here will serve as a good starting point.

 

So, when it's all set-up you should be able to achieve this:


 

Feel free to tweak, modify, optimize this one as I only wrote it in a very short period of time so as to show the art of the possible, as well as help explain to others how to demystify the seemingly complex world of Localizing a Portal.

 

 

As always, if you liked this, please consider subscribing and like and share if you want to see more as it always helps

 

 

 

Comments
SowmiyaC
Tera Contributor

@Alex Coope - SN , No Nothing, I just make sure that UI action have internal name in it, Any other suggestion ?

Alex Coope - SN
ServiceNow Employee

@SowmiyaC,

best guess - you've possibly inadvertently also copied the page level artifact script into the same record of the Portal artifact - if you have, I'll update the post to make it much more obvious where the separation is because a few people have done that in the past,

 

Many thanks,

kind regards

Tonyskin
Tera Contributor

@SowmiyaC Go into Flow Designer and look at the Operations that ran for today right after you get the message.  Expand the steps and look at the responses you got for the API calls out to the translation provider.  Some providers have limitations on the number of characters that you can send in a single message (some as little as 5,000 characters which include all the HTML tags and stuff).  I have a feeling this is your problem as some pages can be quite lengthy.

SowmiyaC
Tera Contributor

@Alex Coope - SN , I just used processor script specific to the portal not for page .Thanks

Alex Coope - SN
ServiceNow Employee

@SowmiyaC, if it's failing to generate the task, then it's usually down to a few very specific things:
1. the name referenced in the UI action doesn't match the "internal name" of the artifact record,

2. typo / syntax error in the script - Script Log Statements should present an error either-way when a task generation fails,

3. one of the aspects of the artifact is in a different scope to the rest. E.g. the artifact record might be in global but is the Processor Script (Script include) also and the UI action?

4. Is the language selected and applied to the "setting" record for that artifact actually "active=true' in the [sys_language] table

 

 

Many thanks,
kind regards

Version history
Last update:
‎01-27-2026 05:15 AM
Updated by:
Contributors