Scoring Configuration for Model Risk Assessments

[Abhay Rathour]

Scoring Configuration for Model Risk Assessments

Required role: To configure scoring for model risk assessments, the sn_model_risk_mgmt.admin role is required.

Note: The scoring in model risk assessments is done only for model risk task of type "Assessment".

Overview

Model risk management systematically calculates risk scores, ratings, and tier classifications based on model complexity, business use, regulatory impact, and data sensitivity. Scores translate into standardised categories (Low, Medium, High, Very High) determining required governance levels and oversight intensity. ServiceNow's Model Risk Management application automates this scoring process through configurable assessment templates and custom scripts, ensuring consistent, objective risk evaluation across organisational models while supporting informed decisions about appropriate mitigation strategies. The configuration for scoring of model risk assessments can be created through following steps.

Step 1: Configure Smart Assessment Template Scoring

First, configure the scoring at the Smart Assessment Template level that you are using for model risk assessments. This step will generate numerical metrics for questions, sections and assessment based on user's responses.

Reference: ServiceNow Documentation - Scoring in Assessments

Step 2: Navigate to Assessment Score Configuration

1. Navigate to All > Model Risk Configuration > Assessment Score.
   
   
   
   
   
   
2. A list displays all configurations created for Model risk assessments.
   
   
   
   
   

Creating a New Configuration:

1. Click New to create a new configuration.
   
   
   
   
   
2. Select the Assessment Template for which you need to create the configuration (Select the smart assessment template configured in step 1)
   • Only templates belonging to the Model Risk Category can be selected
3. Save the form and reopen the page
   
   

Understanding Script Variables:

The Script Variable field lists all variables available for the script to calculate assessment metrics. Initially, two variables are available:

• modelRiskId: The sys_id of the model risk for which the assessment will be taken
• smartAsmtOverallScore: The overall assessment-level score generated by the smart assessment configured at step 1. Default value for this variable is zero.

Adding Additional Variables:

Additional variables can be added for use in the scoring script through related lists on the form. Questions and section metrics from the Smart Assessment Engine can be used as variables.

To add questions:

1. Navigate to the Questions related list
2. Click the Edit button
3. Add questions as per requirement
   
   
   

To add sections:

1. Navigate to the Section metric related list
2. Click the Edit button
3. Add sections as per requirements
   
   
   

For each question and section added through the related lists, a unique variable name will be generated. These names are visible in:

• The "Question Variable" and "Section Variable" columns in the related lists
  
  
  
  
• Auto-populated in the Script Variables field
  
  
  
  

All variables displayed in the variables field can be used in the script to calculate model risk and model risk task-related metrics.

Step 3: Script Development and Output Schema

Once all script variables are added, write a script using the provided variables and logic as per your requirements. The script output must be saved in the result object, which should follow the schema defined in the "Script output format" read-only field.

Required Output Schema:

1. Scoring outputs must be populated in the result object defined in the script

2. The following assessment metrics must be calculated and set in the result object:

   • model_tier: The final tier value of the model risk to be populated in materiality tier field. Valid values are those configured in the materiality tier field on sn_model_risk_mgmt_model_risk table. OOB values for this field are 'tier1', 'tier2', 'tier3', 'tier4'.

   • model_risk_score: The final risk score of the model risk to be populated in the risk score field.

     • Data type: Decimal

   • model_risk_rating: The final rating of the model risk to be populated in the Risk Rating field. Valid values are those configured in the risk rating field on sn_model_risk_mgmt_model_risk table. OOB values for this field are 'low', 'medium', 'high'.

   • model_risk_metrics: An array containing different types of  intermediate assessment metrics used in the final risk computation. These are stored at sn_model_risk_mgmt_model_risk_score with model_risk_level_metric field marked true.

     • Each array element is a metric object (schema defined below)

   • model_risk_task_metrics: An array containing different types of intermediate assessment metrics used in the calculation of metrics at the model risk task level. These are stored at sn_model_risk_mgmt_model_risk_score with model_risk_level_metric field marked false.

     • Each array element is a metric object (schema defined below)

3. Metric Object Schema:

   • metric_name: String containing the name of the created metric
   • metric_score: A numerical score corresponding to the metric
   • metric_rating: The choice value of the metric_rating field defined in the sn_model_risk_mgmt_model_risk_score table. OOB values for this field are 'low', 'medium', 'high', 'very high', 'tier1', 'tier2', 'tier3', 'tier4'.

The script field enables model risk rating calculations using ServiceNow scripting logic, including script includes, business rules, and flow designer steps. This JavaScript-based field provides flexible, customisable score calculation capabilities for organisational requirements, supporting complex calculations and conditional logic for establishing ratings and tiers through various implementation approaches.

Reference: Server-side scripting

Sample scripts demonstrating some methods to calculate assessment metrics and scores are available in the examples section below for practical reference.

Best Practices:

1. All custom scripts should be wrapped in a try-catch block to ensure safe execution.
2. Always read the script instructions thoroughly before editing or publishing any changes.

Step 4: Script Validation

1. Once the script is completed, save the form
2. Click the "Validate Script" UI action to validate the script.

NOTE: "Validate Script" only checks for reference or variable errors (e.g., undefined variables), and not the logical correctness.

Step 5: Activate Configuration

1. Make the script "Active" for this configuration to be used for score and rating calculation of model risks.
2. Active configurations execute automatically once an assessment is successfully completed and submitted.

NOTE: Only one configuration can be active at a time for each smart assessment template.

Model Risk Rating and Materiality Tier Computation Flow

1. When the smart assessment is submitted, smart assessment engine dispatches an event sn_smart_asmt.asmt_inst_completed which triggers a script action Calculate asmt score for model task.
2. This script action executes method calculateRiskRatingForModelRisk from ModelRiskManagementUtils script include which performs following operations:
   1. Internally calls method _executeAndGetScoresForModelRiskInstance method which takes input the model risk scoring configuration, model risk id and smart assessment id.
   2. The above method fetches all the corresponding scores provided by the smart assessment and assign the values to the input variables for the script.
   3. These inputs are passed to script evaluator for script evaluation and the result is returned.
   4. Based on the result returned _populateModelRiskTaskMetricScoreData method is executed which creates all the assessment metrics and saves in sn_model_risk_mgmt_model_risk_score table.
   5. The materiality tier, model risk rating and model risk score is also populated on the corresponding model risk task record.
3. When the model risk assessment task is approved and completed, Business rule Calculate rating for model risk on model risk task is executed which calls method populateModelRiskMetricScoreData from ModelRiskManagementUtils script include.
4. The above method copies the model risk materiality tier, model risk rating and model risk score from model task record to model risk record.
   
   

Example Script Structure

1.  Following sample script calculates the model risk materiality tier, model risk rating and model risk score based on the section level scores of risk assessment and materiality assessment section present in OOB model risk assessment questionnaire.
   

   // Example implementation
   /*************************************************************************************/
   
   /* 1. Use the predefined variables from the Variables field.
   /* 2. You can define your own script variables within the script itself. For example, var x = 1;  
   /* 3. Question response must be set to variable result;  
   /*************************************************************************************/
   
   try {
   
       var result = {};
   
       /***Start of Custom Code. Please write your scoring formula below. */
       result = {
           model_tier: '',
           model_risk_score: 0.0,
           model_risk_rating: '',
           model_risk_metrics: [],
           model_risk_task_metrics: []
       };
   
       // Calculate based on smartAsmtOverallScore and other variables
       var riskScore = SEC829895;
       var materialityScore = SEC590641;
       result.model_risk_score = riskScore;
       if (riskScore >= 4) {
           result.model_risk_rating = 'very high';
       } else if (riskScore > 3) {
           result.model_risk_rating = 'high';
       } else if (riskScore > 2) {
           result.model_risk_rating = 'medium';
       } else {
           result.model_risk_rating = 'low';
       }
   
       if (materialityScore >= 4) {
           result.model_tier = 'tier1';
       } else if (materialityScore > 3) {
           result.model_tier = 'tier2';
       } else if (materialityScore > 2) {
           result.model_tier = 'tier3';
       } else {
           result.model_tier = 'tier4';
       }
   
       // Add metrics as needed
       result.model_risk_metrics.push({
           metric_name: 'Total assessment score',
           metric_score: riskScore + materialityScore,
           metric_rating: result.model_risk_rating
       });
       /***End of Custom Code. Do not modify anything below. */
   
   } catch (ex) {
       result.error = ex;
   }

2. You could also implement rating assignments using Decision Tables configured within the Process Automation Designer, which allows you to retrieve ratings and tiers programmatically through Decision Table APIs. This modular approach enables you to separate business logic from calculation logic when appropriate. For detailed guidance on any specific implementation method, such as Decision Tables, refer to the corresponding ServiceNow documentation. 

   Reference: Decision Tables documentation.

   Following is a sample script to calculate the risk rating, materiality tier and risk score using the two decision tables that are shown below:

   
   

   // Example implementation
   /*************************************************************************************/
   
   /* 1. Use the predefined variables from the Variables field.
   /* 2. You can define your own script variables within the script itself. For example, var x = 1;  
   /* 3. Question response must be set to variable result;  
   /*************************************************************************************/
   
   try {
       var riskRating = '';
       var materialityRating = '';
       var tierRating = '';
   
       //Calculate Risk Rating
       var inputs = {};
       inputs['assessment_metric'] = "Risk Rating";
       inputs['score'] = SEC405211;
   
       var result_elements = this.callDecisionTable(inputs, "ae7265f0ffd22210db95ffffffffff91");
       riskRating = result_elements.qualitative_rating.getValue();
   
       //Calculate Materiality Rating
       inputs['assessment_metric'] = "Materiality Rating";
       inputs['score'] = SEC827551;
   
       result_elements = this.callDecisionTable(inputs, "ae7265f0ffd22210db95ffffffffff91");
       materialityRating = result_elements.qualitative_rating.getValue();
   
       //Calculate Materiality Tier
       inputs['materiality_rating'] = materialityRating; // Choice, Mandatory 
       inputs['residual_risk_rating'] = riskRating; // Choice, Mandatory 
   
       result_elements = this.callDecisionTable(inputs, "6f09adb8ffd22210db95ffffffffffc0");
       tierRating = result_elements.material_tier.getValue();
   
       //Set all metric on result object
       var result = {
           'model_tier': tierRating,
           'model_risk_score': SEC405211,
           'model_risk_rating': riskRating,
           'model_risk_metrics': [],
           'model_risk_task_metrics': [{
               'metric_name': 'Materiality Rating',
               'metric_score': SEC827551,
               'metric_rating': materialityRating
           }]
       };
   
   } catch (ex) {
       result.error = ex;
   }
   
   function callDecisionTable(inputs, decisionTableId) {
       try {
           var dt = new sn_dt.DecisionTableAPI();
           var response = dt.getDecision(decisionTableId, inputs);
   
           return response.result_elements;
       } catch (e) {
           gs.log("Couldn't run this script Error: " + e);
       }
   }
   

Conclusion

By following the steps outlined in this article, organisations can configure consistent and automated model risk scoring for model risk assessments. This approach supports regulatory compliance, aligns with business objectives, and enables flexible customisation while ensuring standardised and reliable assessment outcomes across all models.

A step-by-step walkthrough of this implementation is also provided in the accompanying video.
ServiceNow | Model risk management scoring configuration walkthrough demo