Business Rules - Does the sys_override work outside of domain separation?

[Fabian Kunzke]

No. No it does not.

Now that we have that out of the way, you may ask yourself - quite rightfully so - why you have never heard of that field. It is present on other scripting parts on the platform such as UI Actions.

The sys_override for business rules does do exactly what you would think it does. It overrides - as in "Trigger this, not whatever i am overriding" - another business rule preventing the overridden one to be executed. BUT: Only if those business rules are within different domain scopes. It works with domain separation, but not outside of it (yet).

Now, why should you care?

Imagine a world where you would not have to touch ootb business rules. Where customization for a process could reside within an easy to maintain and deploy application. Where said application could have its own scope, its own functions and developers could be very restrictive with the way customizations are interaction with each other. Now imagine if you could have such customizations as a blueprint to "override" ootb processual behaviour.

Welcome to a world where the sys_override field actually works outside of the domain separation. A world where "going back to ootb" is just a click away - that click being called "uninstall" on the application itself. A world where you could put your "improved" yet customized incident process to the store and people could try it out in an instant. Welcome to a world which does not yet exist. 

Issue is: If you want your customization to reside within an application scope, said application scope cannot contain changes to the global scope or any other scope. Setting business rules to inactive does not work.

Why does it not work? I don't know. ServiceNow: Please fix.

[The SN Nerd]

I've observed this too.

However, I don't think it is an issue. In my view, it isn't always a bad thing to modify OOB Business Rules if there are a valid business requirement and justification. The update process will let you know if the vendor has made any changes to that object, and you can choose to adopt or merge. If you want to adopt later you can, the version changes are shown in Versions and you can rollback at any time.

Better to know than not, in my opinion. Not knowing can cause problems when dependent codes changes but you don't update yours. As long as you comment your changes, I think it is fine the way it is in the merge process.

In regards to scoping, the whole point of application scopes is to work within a confined scope and not mess with existing code/configuration. If you need to mess with another scope in your scope, then working in an Application Scope is not the correct design choice.

I'd be interested to hear the thoughts of others on this.

[Fabian Kunzke]

Great input, Paul!

Let me take a step back. I am working with multiple customers and i did observe, that the upgrade process almost always looks the same:

1) Check for any collisions
2) Fix the collisions
3) Retest the processes
4) -> No issues? Go live.

And i agree. In that case you actually do need the collision warning. However, this can result in very old customizations preventing you from using new features (which is why a lot of customers do start "Back to OOB"-Projects).

The alternative - and to me the preffered way of upgrading - would be to:

1) Check the upgrade change notes
2) Let process owners determine what they want to utilize
3) Deactivate customization which would override the utilized functionalities
4) Check for any collisions
5) Fix the collisions
6) Retest the processes
7) -> No issues? Go live

Now this has one major advantage: You are always using the "things you like" while removing whatever is unneeded. The question is, why is noone doing it?

And this is where i struggle with a clear answer. I do notice, that a lot of upgrades are rushed in between project go-lives. This results in a "no collisions, no problems" mentality to save time.

How could this be "fixed"? Maybe by seperating the "oh, we have a cool new feature here" and the "oh, we need to look into these collisions". An upgrade could run cleanly without any hassle, if all the OOB stuff stays untouched. An upgrade process could then be simplified from a technical perspective, while allowing stakeholders to regularly review their processes. However, this only works on 2 premises:

1) OOB stuff can be left untouched while customizing
2) customizations can be reverted easily

To revert cusomizations easlily in the first place (especially when it comes to script includes) it is absolutely neccessary to know the range of interaction (as in where everything is used). And applicationscopes are great for limiting the overall use.

To leave the OOB stuff untouched - which is possible in a lot of places - the following must be done:
- Use overrides to replace
- Use script include extensions for customization

And this is where i am coming from. If business rules could be overridden, they would not have to be touched. As such it finally would be possible to have most customizations done in scopes and decoupled from OOB code allowing for the least amount of collisions during upgrades.

BUT:

This is only important IF the process of upgrading an instance is changed from "Lets get rid of this collision warning" to "Lets get this feature and remove this customization". To me (and i may be alone with this opinion) the collisions while upgrading SHOULD NOT be used to structure the upgrade, but rather indicate "huh, thats weird, should'nt customization be decoupled". It should be the last indicator telling you "Oi, mate. Ur instance may have some stuff you want to review." But most times (from what i have seen at customers) it is the first indicator.

I would like to hear your view on this. You are absolutely right and i had the same discussion with some service now architects. The way upgrading is currently lived does not need (rather the opposite - it needs the collision warnings) customization to leave OOB untouched.

Regards and thanks for your insights
Fabian

Edit: Short side note about applicationscope and changes to other scopes: You are aboslutely right and i do agree. This is why having customizations in their own scopes does not currently work as you would have to change the OOB "stuff" in the global scope.
Regarding the merge process: On upgrades this would behave the same IF you leave the OOB stuff untouched. It does require you to create script includes which extend the oob script includes. I hope i was able to convey the point, that, if the customizations are overrides and extensions, merge conflicts are actually more unlikely to damage anything (as your customization only calls extensions and the OOB stuff still calls the now upgraded OOB stuff).

[Simon Westwood]

You make some interesting points and there’s a lot to unpack in your comments. 

Upgrades are not necessarily the time or place to repay technical debt or deploy new features, they should be about iterating the platform version and fixing immediate defects.  Sure, the collision assessments are valuable exercises but testing, preferably with ATF, is more likely to highlight issues which may manifest as collisions. 

I do feel strongly that simply hiding changes to baseline records in an application scope or via an override field doesn’t solve the underlying problem, it just means less collisions.  Does that mean faster upgrades?  It means less noise, but will there be any less defects to remediate?

[Fabian Kunzke]

Hello Simon,

Thanks for your insights.

I would like to make something very clear, which may have been drowened: regarding the approach to upgrades is purely philosophical as it currently cannot be practically archived.

That said, theoretically speaking, if you could encapsule all cusomization in apps an upgrade would not have any collisions. This is very much wish land, but it would be possible. What would that mean?

As you pointed out, an upgrade should not be the time to repay technical dept. I do disagree with the feature part though. OOB servicenow features will be deployed IF they are not addressed within the upgrade. On upgrade an instance which contains only encapsuled customizations would ONLY need you to check what you want from the upgrade and what you don't without relying on the collisions to decide.

On the other side, technical dept can be repayed easier at any time. As all you have to do is either reduce the customization overriding/extending the OOB parts.

Regarding "hiding" changes to baseline records: Within an application scope they are not hidden. Quite the contrary. Identifying extensions and "overrides" is a lot easier to find as they show distinct attributes (such as the "sys_override" being filled). Also, as doing so, the only place to look is in custom applications.

I cannot answer the question if there would be less defects. I would expect, but i do rely on non servicenow experience from that. None the less i fully agree on the testing part.

Now, looking into the "real world", what can be taken away from this? I do belive there is a lot of space to reconsider the approach to upgrades. I would like to move more into encapsuling customization and not "just" custom applications, but at the moment the plattform does not provide it. But, wherever it does, i like to use it (Script Include extensions, UI Actions...).

Regards

Fabian

[The SN Nerd]

I think upgrades are the best time to repay technical debt. New features arrive, and it is the perfect time to decommission customisations in favour of OOB.

However, that can compromise testing, as the 'accepted' system state changes. Usually, customers also want to upgrade as quickly and seamlessly as possible. 

What does the vendor recommend? I'm not actually sure.

[Fabian Kunzke]

It really depends. From ServiceNow (and this is just my understanding, as i may have an incorrect view here) the upgrade process should be driven from a use case perspective. More in the sense of "what is it that i need from this update and what is it i don't".

I see your point of repaying technical dept on the upgrade, but i know from experince, that this is highly unrealistic. The reason being that most instances are broadly customized. If customization and upgrade process would only focus on one specific part of the instance, it would be great. But since all of it is upgraded at the same time, each upgrade would demand a large amount of stakeholders to be involved.

Ideally, that is what a customer would do. Involve all stakeholders of the plattform, let them assess the changes within the upgrade and then repay the technical dept where needed and deactivate new OOB features wherever it's relevant. However, this demands two things from a plattform:

1) Buildbacks are simple
2) Stakeholders are very familiar with the plattform

The first step is architecturally not given at the moment. As a lot of customisation happens with replacements instead of extensions/overrides, reverting to OOB can be quite time consuming. But, and i wan to make it clear, it would be optimal to do so. However (and this is why i previously stated that i do not think, upgrades are the right place in time), for most customers this is a task of great investment.

The second "issue" is, that many stakeholders within customers are not familiar enough with the platform or are "happy with what they have and don't see why we should change it". This can be a struggle as time is a limiting factor.

My position about all of this can be summarized in "i am torn". I would love upgrades to run smoothly, be used as a chance to review customisations and needs and to repay technical dept. But, at least for now, i cannot see how this can happen without drastic change. And it is kind of a chicken and egg thing:

The better the customization is decoupled from OOB, the easier it is to roll back and it will lower collisions on upgrades making upgrades easier as well. Whenever upgrades and rollbacks are easier, the more they will be used by customers. Whenever rollbacks are used to go back to OOB, the easier upgrades and rollbacks will become.

Maybe, what i am trying to say is, that the first step into a upgrade-friendly world is to repay technical dept in a sense of making customisation more decoupled. Which is an undertaking i personally would not combine with an upgrade.

And this is also where i'd like to see business rules to be overridden instead of adjusted.

This thread took a turn into a very different discussion, which i like. Interesting views and insights. Thanks a lot!

Regards

Fabian

[The SN Nerd]

In my view, one of the biggest problems isn't the upgrade process itself, nor the rate of change, but that a lot of the OOB code is (mostly) not designed for extensibility. These biggest offenders are Business Rules, UI Actions, Script Includes, Client Scripts & Widgets, where most customisations lie.

If all Business Rule scripts were encapsulated in extendable Script Includes (using the SNCScriptInclude Pattern) with well-designed classes, where:

• All GR queries are placed into separate functions 
• All code that loops through GR records into seperate functions
• All code that process GR queries into seperate functions 
• Writing functions to do one thing and one thing only

Just as a crude example:

function buildQuery() { 
  var gr = new GlideRecord('table');
  gr.addQuery();

  return gr;
}

function processRecords() {
	var records = this.buildQuery();
	while(records.query() ) {
	   processRecord(records);
	}
}

function processRecord(gr) { 
	gr.field = 'hello';
	
	return gr.update();
}

Imagine how much easier it would then be to extend the platform, rather than 'customise'.

While an effort has been made with some new Script Includes and Business Rules, but often their underlying code is still not well-designed for extensability. The platform is riddled with code snippets that are not encapsulated at all, many that are almost always customised and almost always changed by the vendor with each upgrade.

And don't even get me started on Widgets. Even the smallest of changes to a Widget often requires a clone, which is very timely to merge in upgrades - perhaps one of the biggest time costs in the current upgrade process for customers with a mature Service Portal.

In a nutshell, the platform needs to lend itself more to extensibility, rather than customizability.

[Fabian Kunzke]

Hey Paul,

Yes! This is exactly what i mean. Although script includes can be extended (as they are "normal" javascript classes).

My issue also is not with the upgrade process itself, but rather how it presents itself. It is missunderstood as a "need-to-fix-list". And if the service portal is bothering you, don't take a look at workspaces...

Regarding your coding examples i did do another article on "clean architecture" where i (luckily) had a similar discussion with @Javier Arroyo who gave me a big insight of structuring business logic. I can only recommend reading through most - if not all - of his posts.

Regards
Fabian