A guide to deleting records in scoped applications and retrieving them

[Pradeep Sharma]

Edit: For the New York release, we introduced a new concept to Scoped Applications, called the author_elective_update folder. When your application is packaged for the repo or for committing to Source Control, we will unload additional things into this folder that we would previously have ignored. The author_elective_update folder is where we will unload Deletes. When your application is installed on your own instances, we will automatically load these Deletes from the author_elective_update folder. Please refer to New York and Scoped Applications - some new features blog by Cory Seering.

Pre New York release:- If you are doing enhancement changes on an author instance, please be aware that deletion of records like business rules, client scripts, etc. will not be tracked in your scoped application file. In the past, I have seen a lot of vendors on store assuming that deleted entries will be shipped as part of their scoped app file. In this post, I will show you how to capture deletion of records in scoped application as well as what to do if you accidentally delete them. This is applied for those apps which have the first version of the app already published on the store or app published to company's target instance via make app available to other instances.

Deletion entries are not packaged as part of app file; for example, any delete action will not be recorded as part of scoped application file. Instead of delete, you need to set the active field of the record to false.This will ensure that the changes are captured and packaged as part of update file in the next version. For those records which don't have an active field on the form, please ensure to set the description field with message "This file is deprecated and is no longer required". Please refer the screenshot attached for reference.

Example of deleting records on a scoped app

Let's pretend we created a scoped application   called "Marketing Events" v1 which is already published on the store and is installed on customer instance. I need to make some additional changes on top of v1 and publish it to the store so that customers can install it as a part of the update file. My requirement in v2 is to delete a business rule and system property which was created earlier as part of the v1 app. We can do this tracking the changes related to deletion.

Steps to track changes related to deleting the business rule and system property

1. Navigate to the business rule you created in v1
2. Set an active field to false.

For "sys_properties" table, we don't have an active field in the base system. For those records which don't have an active field we recommend you to just update the description field with the message, "This file is deprecated and is no longer required." The same steps will be followed for any records which don't have active field in the base system.

The above changes will be captured as part of the update in version V2 and will be successfully migrated to customer instance as soon as update file is installed.

What to do in case records are deleted by mistake on source instance:

If you have already deleted the records, there are two ways to retrieve them.The first is restoring the deleted records, and the second is to ask ServiceNow for help.

1. Try to restore those records from deleted records module. In Using the Restore Deleted Records plugin to retrieve lost data the process to do so is outlined.
2. Get in touch with certification team at ServiceNow. We will provide you the report with diff results of all records between the current version and the already certified version on the store. This diff file is given to you so that you can write a fix script which will then delete those records on customer instance. Once the app is upgraded customer has to make sure that the script is triggered in the global scope.This option is only for vendor's who has the app already published on the store and is in use on customer instance. You can refer to Fix scripts for more information.   THIS OPTION IS NO LONGER SUPPORTED.

I recommend setting the records as active=false (most metadata types allow for active true/false). For those records which don't have an active field on the form, please ensure to set the description field with message "This file is deprecated and is no longer required."

[saxtonjoshu]

Apparently it's been like this from the beginning.

[JennyHu]

I've to agree with this. I got a lot of grief from how delete works in scoped applications. This design choice is assuming that developers would have nailed down the perfect database, and logic on the alpha release and that would never change in subsequent releases. What happens when requirements change and we need to alter the design? 

[saxtonjoshu]

I have found that update set will fix the issue. The problem then is managing what is going in the update set and the way I think Servicenow handles versions you might have to adjust.

However the update set will work.

[arpitt]

Hi Pradeep,

Please help:

https://community.servicenow.com/community?id=community_question&sys_id=90f5d23d1b9e7f40ada243f6fe4b...

[Kaushal283]

Here is a small utility to get the list of deleted files by comparing two git branches of your project (application): https://github.com/Kaushal28/SNOW-File-Comparator. 

For example, if you have certified an initial version of your application and its code is in "master" branch, and your upgraded application code which you are going to certify (newer version of the same application) is in "develop" branch, then you can compare those two branches using this utility and get the complete list of deleted files from branch "master" in your newer version. 

Hope this helps!

[Kenny Wimberly]

I'd like to submit an updated method of handling this. I've had to endure numerous headaches as a result of handling deletions due to agile development in scoped applications. One day, I thought about the fact that we have everything we need to make the deletes happen quickly. I tried this method and lo and behold, it worked. I definitely welcome feedback.

Here is what I do to handle a published scoped application that needs deletions AFTER the initial install on target instances:

1. Create an update set in the scope that you are working in. Name it something like "<scoped app> deletions v1.0". I always version mine so that someone knows what order they need to be applied, if I'm not applying them.
2. Do NOT "submit and make current". You don't want to capture anything in here inadvertently.
3. Navigate to sys_update_xml, locate the updates for your application. Filter to show only those updates.
4. Filter the updates further to show where action=DELETE.
5. Change the update set on each record from Default to the "<scoped app> deletions v1.0" update set. You can easily do this from a list view. Or, if you have many, just execute a fix script or background script to update them, but be careful doing this.
6. Go back to the update set you created and mark it complete.
7. Migrate the update set to the target instance(s).
8. Preview the update set and know that you will have some collisions. Review those and most likely you will just go ahead and apply the remote update for each. But, just make sure to be on the safe side.
9. Commit the update set and voila...the deletions have been processed.

Some may worry that this will impact the migration using the publish/install method. It won't, simply because you are deleting and not updating anything with the update set.

You could always create an update set at the onset of the application build that will capture updates and then select the update set any time you do a deletion, but who remembers to do that? 😉 I just find it easier to do it this way because it's a simple process to follow for every application.

Hope this helps at least one person.

[Pradeep Sharma]

Hi Kenny,

Thanks for your comments. To clarify, the steps you have mentioned via UPDATE SETS will not be applicable to Partners publishing their scoped application via the store.

- Pradeep Sharma

[Inactive_Us1705]

I totally agree with you Benjamin, could you imagine this type of design for apps like MS Word etc, no way. I remember this being a problem when I developed my first scoped application on ServiceNow years ago, I was brought in to fix a huge app that a couple of other people developed but it turned out to be a pile of poo! I had to delete about 6 tables and many, many BRs, CSs, SIs etc. I just assumed it would be promoted but was told after that it wasn't the case.

I hoped by now that ServiceNow would have changed that design because I can see NO reason at all to not include the deletions when installing from the repository (please can someone explain if there is).

But yes, I will now have to use the legacy Update Sets to "clean" up my apps. After all, after a deprecation period ServiceNow remove old objects don't they! I have never seen this "best practice" to not promote the deleted objects instruction.

[Inactive_Us1705]

Hello Pradeep,

One thing that is missing from this thread and more importantly on your initial post that I do not understand is why have ServiceNow chosen to have this design as "best practice", why do they not include deletions via the repository?

Thank you

[Tom Sienkiewicz]

Hi,

this is quite interesting. I imagine one reason why scoped apps have been designed like this is the following situation:

You develop Version 1.0 and publish it

Some clients install and start using that app

You then delete a bunch of tables and other stuff and publish that as Version 2.0

Clients install the updated app and suddenly nothing is working for them the way it used to, and half of their data is lost as it was stored in the tables you just deleted from 1.0.

Now that is only hypothetical, of course they would probably get some warnings but generally, how should then data migration be handled? 

I'm not sure there is a perfect solution that would cover all the situations but definitely a clean process would be nice. Perhaps some sort of Update script which handles data migration and then cleans up the stuff which should be deleted.