Instance management - best practise

[Guled Yusuf]

Hi, the team and I discussed general best practices today but did not agree on a specific topic.

What is the right way to handle emergency updates on production instances?

Example - The update set is moved to production and for some reason, the change applied needs to be turned off until a complete fix is ready in the next update set.

1 - Is it right to just deactivate the introduced component in production? Like an ACL rule that was introduced.

2 - Or should you not deactivate it manually and just push a new update set where this ACL is deactivated?

Will there be any downsides if we manually deactivate the introduced component directly in production? Example Business rule, new ACL, etc. This would be the fastest way, but we need to understand if it goes against best practices and why. 

Thanks!

[Allen Andreas · Accepted solution]

Hello,

Some would say that you would backout the update set, but I don't necessarily agree with that unless the update set in question only contains that one piece. I actually created a video on my YouTube channel about the hidden risks of backing out the set.

Anyways, you can easily just deactivate the record for now, if it's sincerely causing problems, then have the team work on it properly and promote the record back up. When you go to commit it again in Prod, you'll most likely see a preview warning telling you that the record in Prod has been adjusted and so you'll just want to accept remote update anyway and it'll bring up your change successfully.

[Guled Yusuf]

I agree with the mindset of changes in production = possible development. But would you say deactivating the introduced components is on the scale of actual development? We are trying to understand where the line goes (by best practice), and what is an acceptable fix before we say "No go push the fix through the typical development process like dev-test-prod". This is why we need to understand the upsides/downsides of this situation. 

[Mark Roethof]

In ServiceNow terms, yes its development. We cannot argue on that 😀. Maybe not in your company terms, though in ServiceNow terms, yes its development.

Its just what mentioned before. Ofcourse do have to look at it from a practical point of view ofcourse. If it's a P1, whole organisation can't work etc.... you have to be a bit more flexible a break the "rules" perhaps. Though else, avoid doing so as much as possible. I see this over and over at customers, and eventually it becomes the norm and it's very normal to do a quick manual change in production. And yes, ofcourse at some point that does go wrong 😁.

Or for some type of companies such development is not allowed from a regulations point of view, how to explain than that such was done on purpose? While for other type of companies this is no issue it all.