Maik Skoddow
Tera Patron
Tera Patron
MaikSkoddow_0-1678463268931.png

 

 

There are several thousand tables in ServiceNow, and the records in these tables are linked together as if in a net. Removing a node (a specific record) from this net has consequences for the surrounding nodes, which reference the record to be deleted. In the best case, the loose ends are either neatly obliterated (the reference is removed in the source record) or the referencing record is also deleted. In the worst case, the loose ends remain in the system and then represent so-called "broken references".

 

Even more "broken references" can occur for indirect references. These can be for example references in a condition builder or the records from the journal table, because there the Sys IDs to the referencing record are stored in a pure string field.

 

Either way, deleting records can lead to data corruption or incorrect as well as unexpected behavior, but ServiceNow does not provide any feature to get an overview of all referencing records and their type of relations. You only get a warning when deleting a record if there are referencing records that will also be deleted automatically (so called cascading deletion):

MaikSkoddow_0-1679024141949.png

But neither you can open the list of these records nor you can be sure, that really all referencing records are captured. Therefore, visualizing as many references as possible would be an extremely helpful feature.

 

For this reason, I built a feature "Generate References Report" which can be found in the context menu of a record's form view.

MaikSkoddow_1-1678463379889.png

It kicks in the report generation on server-side and visualizes the progress with a well-known progress bar.

MaikSkoddow_0-1678458351186.png

After finishing, the report can be opened by clicking on the respective button below the progress bar. In the last column of that report, you can find the number of records for the given table. By clicking on that number, you are redirected to the corresponding filtered list view.

MaikSkoddow_1-1678458781489.png

As you can see in the screenshot, even a simple Change Request can have many referencing records.

 

 

Notes

 

It was surprisingly difficult to implement a solution. The challenge arose from the concept of table inheritance, whereby both the parent tables and all child tables must be taken into account.

 

Another difficulty is the fact that a table can have several columns that (redundantly) reference the same record. This is especially the case with the child tables of the task table. Originally, there was only the field "Parent" defined on the task table. Later, however, many child tables (for example the "Change Task" table in the above screenshot) introduced an additional and individually named field such as "change_request" or "wm_order".

 

The values in "type" column represent the different reference types we have in ServiceNow. It would go beyond the scope of this article to go into detail about all of them. However, I will write a larger article on the topic of "references" later and link it here then.

 

The most important information is provided in the column "Delete Cascade Rule" as it represents the respected behavior when deleting the referenced record. Also that topic is too large to explain it in detail and I will cover this in the mentioned article.

 

And the real value of this report results in the last column, which not only outputs the number of referencing records, but also offers this number as a link to get to the list view of all these references.

 

 

Implemented Solution

 

The solution consists of 3 artifacts, and an Update Set containing them can be downloaded from the related Share project.

 

 

UI Action "Generate References Report"

 

It configures and opens an invisible and undocumented UI Page "hierarchical_progress_viewer". But there are many code examples in the OOTB scripts available and also the Community article How to make a custom progress bar - a guide to GlideScriptedHierarchicalWorker was a great help. Apart from some additional client-side code for providing a cancellation option, the JSON-based result data is consumed to build up the report in another modal.

 

 

Script Include ReferencingRecordsReportAjax

 

That Script Include acts as a mediator between client side and server side. Its method "start" (don't change the name!) is invoked by the previously mentioned UI page "hierarchical_progress_viewer" to instantiate a new GlideScriptedHierarchicalWorker() object and return its ID back to the client. With that ID, the client script then can periodically pull the progress from the server. This periodic communication is completely covered by the UI page "hierarchical_progress_viewer", so you don't need to care about it.

A second method "cancel" takes the cancellation request from the client and leverages the respective API to stop the progress worker.

 

 

Script Include ReferencingRecordsReportImpl

 

That Script Includes hold all the business logic for generating the report data. It is implemented it in a way which allows leveraging it independently if you want to test it in a different context:

 

var objGenerator = new ReferencingRecordsReportImpl();
var objReport    = objGenerator.createReport('change_request', '8ecd7552db252200a6a2b31be0b8f581');

gs.info(JSON.stringify(objReport, null, '    '));

 

Comments
CMDB Whisperer
Mega Sage
Mega Sage

Excellent utility, very useful!  Simple, effective, informative, and positive user experience.  One thing to note is that the Label used in the "Table" column does not match the actual table name, but rather, the label used is that of the root table that the actual table is derived from.  For example, it will show "Base Configuration Item (cmdb_ci_computer)" which is inconsistent.  It would be better if this included the label for the actual table.  

CMDBWhisperer_0-1678474381629.png

 

Maik Skoddow
Tera Patron
Tera Patron

Hi @CMDB Whisperer ,

thanks for raising that issue. It is one of the topics which drove me crazy. It seems, that there is really no JavaScript API which provides the label (dependent of the session language) of a extended table. Yes, there is a cool and unknown REST API 

/api/now/ui/meta/{tablename}

 

But I don't want to perform REST calls, so my workaround is sorting the result list from the sys_documentation table by sys_created_on in descending order. This should hopefully work in 99% of the cases. 


Or maybe somebody comes with a better idea. 

 

The new Update Set contains a version of the ReferencingRecordsReportImpl which also includes examining thte sys_journal_field and the sys_audit tables. 

CMDB Whisperer
Mega Sage
Mega Sage

@Maik Skoddow both of these worked for me for English only:

 

 

var grLabel = new GlideRecord('sys_documentation');
grLabel.addEncodedQuery('name=cmdb_ci_computer^element=NULL');
grLabel.query();
if (grLabel.next()) {
  gs.info(grLabel.label.getDisplayValue());
}

var grTable = new GlideRecord('sys_db_object');
grTable.addEncodedQuery('name=cmdb_ci_computer');
grTable.query();
if (grTable.next()) {
  gs.info(grTable.label);
}

 

 

But yeah, I think if you want to match the session langage I would use the Label approach as well, and just add that to the query.  I can't see why there would be more than one Label entry for the same language on a given table (where element is null)

Martin Kohler
Tera Guru

Very nice work, @Maik Skoddow, thanks for sharing! I previously used the "Find record references" UI action but I guess 10 years after it was published, it's now time for something new and better.

Vimal Mishra1
Tera Contributor

I am not sure if I am doing something wrong but I am getting this error when I clicked the UI Action "Generate References Report" on CI.

 

VimalMishra1_0-1678698408016.png

 

Maik Skoddow
Tera Patron
Tera Patron

Hi @Vimal Mishra1 

Basically you did not do anything wrong and without any more details I cannot tell you more. But I'm highly interested in that issue. If you want, you can connect to me on LinkedIn, and then we can schedule a video meeting where you can demonstrate what your situation looks like.

 

Update:
I found a bug, which could be the root cause of your issue. I fixed it and uploaded another Update Set version. Please tell me whether it is working now or not.

Vimal Mishra1
Tera Contributor

Hi @Maik Skoddow Thanks for the reply. I figured it out, there was nothing wrong with the code.

Paula Alejandra
Mega Guru
Mega Guru

This is amazing Maik!

 

 

I would like to create a Report to CHECK Assignment Groups prior to its deactivation, to the team run a due dilligence process, avoiding headaches down the road. Is there any hint you can share with me on where to start?

 

I started with reporting on some tables [INC, PRB, CHG, REQ, RITMs and TASKs] but that is not enough as I also need to include Knowledge Articles that might mention the Assignment Group into their body, Business Rules, Assignmemnt Rules and any other important linkage that may be damaged by the AG deactivation.

 

Thanks in advance, 

ravikmishracort
Giga Contributor

@Maik Skoddow @Vimal Mishra1 : I'm also getting same error. Please let me know the way to fix it.

Ingimar
Tera Guru

@Maik Skoddow , really excellent tool, thanks!!

 

Two observations after testing the new Share version on my PDI:

 

1. The UI Action did not work out of the box

I had to change line 8 in the UI Action to make it work on my PDI.  Changed this:

_objMainModal.setPreference('sysparm_ajax_processor', 'x_tsigh_i_monitor.ReferencingRecordsReportAjax');
to:
_objMainModal.setPreference('sysparm_ajax_processor', 'global.ReferencingRecordsReportAjax');
 
 

2. Warning logs from the UI Action condition

The usage of "InstanceMonitorUtils" in the UI Action conditions was logging warnings in the system log.  The UI Action still works, I just noticed this when looking for what was causing #1 🙂

 

 

Best regards,

Ingimar

Maik Skoddow
Tera Patron
Tera Patron

Hi @Ingimar 

many thanks for that finding. It was a left over from extracting that individual feature from a larger custom app.

Maik

_Damian
Tera Contributor

Hello @Maik Skoddow

Thanks for that tool, really useful but I had similar issue as Vimal. I just managed to fix it but just letting you know and asking for some kind of peer review 🙂 

in SI: ReferencingRecordsReportImpl you are excluding some tables before pushing them to array:

               //exclude non-existing tables and rotated tables

                if (gs.tableExists(_strTableName) && _arrRotatedTables.indexOf(_strTableName) === -1)
but few lines down you are adding some extended tables and you are not checking if they are existing:
               //we also have to check extended tables
                 var _arrTableExtensions = j2js(new TableUtils(_strTableName).getTableExtensions());
so I have reused part of previous if:
 if (gs.tableExists(_arrTableExtensions[_numIndex])) {
                            _objTableAndColumn.table_name = _arrTableExtensions[_numIndex];
                            _arrTablesAndColumns.push(JSON.stringify(_objTableAndColumn));
                        }
Not sure about rotated tables but above code works and now I'm able to generate report. In my case it looks like we have some entries in sys_dictionary  but table is not existing and script was throwing error in to log:
org.mozilla.javascript.EvaluatorException: GlideRecord.addQuery() - invalid table name: xxx (xxx; line 184)
 
Maik Skoddow
Tera Patron
Tera Patron

@_Damian 

Thanks for pointing me to the identified issue! Had not thought that a table which is returned by TableUtils API could not exist.

Steve from Dall
Tera Contributor

@Maik Skoddow ,

 

This is perfect. We needed a lot of cleanup. In one case, half of the references were pointing to the wrong department record. The report showed us where they were all hiding. A short script allowed me to correct the reference sys_id and we were good to go. I'm just happy I came across this feature. 

 

Thanks,

vams1
Tera Contributor

HI @Maik Skoddow  I have imported these script includes and UI action to my real time instance successfully but i could not able to fetch the report on incident table(i have changed from global to Incident table in Ui action ) ,its loading continually ,we have 5000+ records which need to be cleanup. 

vams1
Tera Contributor

Hello @Maik Skoddow ,it is great work around for table cleanup-i have tried this in my PDI and its working fine and I have replicated same thing in my real time incident and incid3nyt table but its loading /transection was not completing . am i missing anything .?

1__SteveS
Tera Explorer

Hello @Maik Skoddow ,

 

This has been working perfectly on all our instances. I love this tool. However now it stopped working on many of the tables I was able to search from before. It remains in a loading state, never getting to the status bar. For example I can run it from Location table, but not User, Group, Department. It also works on PDI. Any thoughts on where I can start looking for a possible issue?

 

Thanks, @Steve from Dall 

Maik Skoddow
Tera Patron
Tera Patron

Hi @Steve from Dall 

there is an infinite number of reasons why it is not working anymore. If you like, you can reach to me on LinkedIn to schedule a short meeting.

Maik

Kipp Gremillion
Tera Guru

@Maik Skoddow this is a great gift for you to share with us.  I will be replacing the older "Find Reference Records" UI action provided my Mark Stanger (SN Guru).

 

Thanks!

Kipp Gremillion
Tera Guru

@Maik Skoddow I discovered that it is missing references in conditions.  In ReferencingRecordsReportImpl, the operator at line 120 is LIKE.  Changing it to CONTAINS corrects it so it will find references in conditions.  While researching this I noticed that the addQuery documentation states "Use CONTAINS instead of the LIKE operator".

 

						operator:
							_strColumnType === 'glide_list' || _strColumnType === 'conditions' ? 
								'CONTAINS' : 
								'=',	

 

 

On another note, I find it is a little hard to find a table in the results when the results returned are large and not sorted.  I have added the below sorting at line 103.  I find this helps a lot at times and is my suggestion.

 

 

			_grDictionary.addQuery('name', 'DOES NOT CONTAIN', 'ts_');
			_grDictionary.orderBy('name');
			_grDictionary.orderBy('element');
			_grDictionary.query();

 

Thanks!

 

Version history
Last update:
‎12-13-2023 03:33 AM
Updated by:
Contributors