A new feature has arrived with the Vancouver release, related to the security area, called Data Separation. It sounds very interesting, but I need to highlight, that this application is supported only for Project Portfolio Management. I hope that this will be extended in the future. I tried this solution and collected all my experiences, which I would like to detail in this article. I'm going to write about all the important features and limitations, I believe it will help people to decide whether to introduce it to customers.
Data Separation application in general
The following types of entities are supported by the application:
- Demand
- Project
- Project Task
- Resource Plan
- Cost Plan
- Cost Plan Breakdown
It also supports records, related items, planning consoles, workbenches, and reports.
How it can be setup
Data Separation is available in the store, which has to be installed first.
Because the functionality is related to the Strategic Portfolio Management product, there are two pre-requisites:
- PPM (Project Portfolio Management) Standard
- Portfolio Planning
Roles
Two roles are coming with Data Separation:
- sn_ds.ds_admin
- This role is used for setup and maintaining the application
- sn_ds.ds_privileged_user
- Data separation rules are not evaluated, all records are visible for users who have this role.
Tables
The following two tables are created during the installation of Data Separation:
- Supported entity (sn_ds_data_separation_config)
- This table stores the entities where the Data Separation functionality can be enabled and used.
- Entity-Group mapping (sn_ds_entity_group_mapping)
- The records of this table connect the Lens entities with user groups, which manage the visibility of records.
Besides these two tables, the third important one is the Lens (sn_align_core_lens). The Lens object is used for Portfolio and Strategic Planning. It helps to build up different levels of entities as a parent-child format. This structure is used by Data Separation to give access to an object to somebody, who is at a higher level of the hierarchy.
Further information can be found in Lenses and portfolio plans and Lenses in Strategics planning.
Domain Separation and Data Separation together
The two features are not mutually exclusive. Domain Separation gives a higher level of data segregation and Data Separation adds another layer to it, which is set on the group membership level.
Limitations
The following list contains the limitations of the application, which I faced with and read about:
- Data Separation application can be used only for Strategic Portfolio Management.
- Data Separation does not ignore the ACLs and Query Business Rules. (For example, reports with aggregate numbers may not be data separated.)
- Based on some special circumstances the application can cause performance issues in the system.
- Only one active Lens can be in the system, which means that it is quite hard to implement complex data separation logic. ServiceNow does not allow enabling multiple Lens for data separation.
How it works
⚠️Important information
As mentioned before, Data Separation is based on ACLs and Query Business rules. To increase the performance, ServiceNow uses session variables in the Query Business rules. When the DS configuration changes, these session variables are not invalidated. To re-calculate the values, it is needed to close the browser and open it again!
The logic behind the functionality is simple. There is a key field, called Data separation groups (data_separation_groups). This is a list-type field, which is a reference to the Group (sys_user_group) table. The content of this field manages the visibility of object(s).
There are two specific groups, that are used for the Data Separation:
- DS Privileged Users Group
This group is added to the Data separation groups field automatically in all cases when the data separation logic is evaluated on the record. - Group for Non Data Separated Entities
This Group is added to all records, where the data Separation logic is not used or not interpreted.
These two groups have no members and it is highly recommended to NOT delete them if Data Separation is enabled because it can break system functionality.
If the DS is enabled, all records should contain one of these groups, if the functionality is enabled on the actual table.
The Data separation groups field is read-only and managed by business rules/scripts.
To summarise the purpose of these groups, they give access to the ticket (Group for Non Data Separated Entities) or restrict it (DS Privileged Users Group).
The second layer of the restriction is the evaluation logic, which consists of Query Business Rules and ACLs.
⚠️Important information
If the Strategic Portfolio Management product is heavily customised, my recommendation is to create a well-structured plan for the implementation taking into consideration the review of ACLs and Business Rules.
Configure Data Separation
As a first step, the entities have to be selected where Data Separation will be used on.
Then, the Lens structure has to be built up, which is followed by the Lens Entity and Group record relationship definitions. Finally, the configuration has to be synchronised with the Data Separation.
In order to understand how it works the best is to see some examples so the next part of the article is about going through some of them.
Base setup
The following configuration was set in ServiceNow for testing purposes:
Roles
Name | Description |
it_project_manager | This role gives full (read/write) access to PPM application. All projects can be opened and edited. |
sn_ppm_read | This roles gives read-only access to all PPM related objects |
Groups
Name | Role |
Project mgr test 1 | it_project_manager |
Project mgr test 2 | it_project_manager |
Project mgr test 3 | it_project_manager |
RO Project users | sn_ppm_read |
HR Project managers | it_project_manager |
Personas
Name | Department | Group |
Abraham Lincoln | IT Operations | Project mgr test 1 |
Alissa Mountjoy | IT Infrastructure | Project mgr test 2 |
Amos Linnan | IT Services Management | Project mgr test 3 |
Allyson Gillispie |
| RO Project users |
Andrew Jackson | HR Services | HR Project managers |
Organisational structure
Data separation Lens structure
Based on the organisational structure of Test Company, the following lens structure was created:
To use this data structure in the Data Separation logic, it has to be active, and the Data separation enabled checkbox has to be checked.
The configuration can be modified anytime. Once a modification is done, the value of ‘Is data synched’ becomes false, which means that the corresponding object visibility settings (in our case Projects / Project tasks) have to be synchronised.
The Lens setup is done. Now we have to create the mapping, which defines the relationship between Lens entity records and Groups. Based on our Lens structure, the following mapping definition was created:
As a first and most important action is synchronisation. Any modifications on the entity-group mapping definitions, have to be synchronised.
When the Sync data separation button is clicked, the system schedules an asynchronous job to update the value of the Data separation groups field on all affected records.
Need to wait a couple of seconds until the synchronisation is done. Now the configuration is ready to use. Let’s see how it works in the system.
As a fist action Abraham Lincoln submits a new project:
Because the content of the Department field was changed, the data separation logic was executed and two groups were set in the Data separation groups field, based on the Entity-Group mapping definition:
- DS Privileged Users Group
- Project mgr test 1
So the only users able to open the record are the members of the Project mgr test 1 group. There are two exceptions:
- The user has sn_ds.ds_privileged_user role
- The user is a member of a group, which is defined to a higher level of hierarchy in the Entity-Group mappings
Remember, the DS Privileged Users Group has no member, the purpose of this is to be an indicator of restriction.
To check that the separation functionality works well, using Alissa Mountjoy’s account the project record is not visible, by either opening the record directly or listing it:
⚠️Important information
I would like to highlight a dangerous factor. Users can easily close out themselves from the record. By default, the Department field is auto-populated, but editable. If the user changes the record, the system re-calculates the value of the Data separation groups field. If the user is not a member of any groups in the list, the record becomes hidden for the user.
Let’s take another example. Allyson Gillispie is a member of the IT Management group. Members of this group are managers of the IT Business Unit. Allyson has the sn_ppm_read role, which gives her access to all projects, but she should only see the IT BU related and the public projects.
In order to set it up, a new Entity-Group mapping record has to be added to the mapping table.
Now the mapping connects the group with the Business Unit record. It means that the users will be able to see all projects, which are part of departments under IT BU.
Let’s create a new project and add it to the HR Services department. Now we have two projects, one is related to the HR Services department the other to the IT Operations.
Each persona can see different records in the system:
Abraham Lincoln
Andrew Jackson
Allyson Gillispie
If a new project is created for the IT Service Management department, Allyson will also see that one.
Data separation on the Project Task level
In order to use Data Separation on the Task level, the first step is to enable it in the Supported Entities table.
When a Project task is created, the Data separation groups field is filled automatically, and the visibility logic is the same as in case of Project.
Closure
These examples are related to Project Management, but the functionality is the same in case of other types of objects as well.
From the configuration and maintenance point of view, we need to keep in mind, that there are two key fields used in the logic:
- Data separation groups, which is responsible for the visibility of records
- Lens identifier field (Bottom entity on planning item), which can be one of the following:
- Business application
- Business service
- Department
- Execution system
- External identifier
- Owner
- Portfolio
- Primary goal
- Service offering
One of these fields will be the key, to how the groups will be calculated. Not all fields are available on each type of record, selecting the correct one should be carefully considered at the design stage.
Data Separation can be set up easier on an out-of-the-box or not heavily customised SPM product. The lead time of implementation might be too long in other cases.
My opinion is that the functionality covers several use cases, but not everything. For example, it is not possible to ad-hoc involve multiple departments in the project using the OOTB available components.
The project object has a lot of relationship elements, but just a few are covered by Data Separation.
This is just the first version, I believe several great features will come with the later releases.
