Changes Getting Missed When Publishing/Promoting Custom Scoped Applications

[jmiskey]

Over the past few months, we have come across what appears to be a new issue.  When making changes in a Custom Scoped Application, and then publishing those changes to a new version, and then installing that new version on another ServiceNow Instance, we sometimes find some changes get "missed", and they do not show up as Skipped Updates in the System Upgrade History.

We opened up a HI Ticket, and it seems what may have happened is that somewhere along the way, someone made a change and promoted it via an Update Set instead of publishing the changes to a new version.  ServiceNow said that this can cause these kind of issues (though we have never seen the issue before until recently, leading me to believe that maybe something changed recently).  It is possible, we have had various consultants come through and perform work for us, and they did not always follow the same standards we did.

We are trying to come up with a game plan moving forward, and establish some "best practices".  ServiceNow said it is best not to mix change times for a Scoped App, either always publish, or always use update sets.  We agree, but that doesn't solve the issue of what may have happened in the past.  And from what we can see, there doesn't appear to be a really good way to identify which Scoped Apps may have had changes promoted via Update Set in the past.  So how do we proceed going further?  ServiceNow referenced a recently created KB0715422 called "How to go back to using the application repository after commiting changes to an application via update set", although they admit this is a difficult process, which we would like to avoid.

One idea/thought we had was to publish the initial Scoped Application to "establish" in the new environment, but only use Update Sets after that initial set-up going forward.  We we proposed that idea, ServiceNow just came back and recommended that we "not mix methods".  So that kind of leaves us in a tough spot, especially since we do not know which Scoped Applications may have been affected in the past.  So we really aren't getting good guidance on how to proceed with these Scoped Applications.

Has anyone else come across this?  Have you have handled it?