Tailor the Project Data Quality Index to your process: add, change and turn off rules

MahatmaK0342012 · 7 hours ago

The Project Data Quality Index scores each project on how complete and up to date its data is. Out of the box it checks 20 rules in six areas. Your organization may need rules that match its own process, or may want to retire a rule that doesn't apply. This article shows how to add, change and turn off rules without changing anything that ships with the product.

How the index works

Each project is checked against rules grouped into six dimensions:

Dimension	What it covers
Project Charter	Name, description, business case, investment type, scope
Schedule & Milestones	Milestone updates, baseline, owners, project manager
Resources & Team	Assignments and contractor end dates
Scope & Financials	Budget, actuals, cost and benefit plans
Reporting & Activity	Status reports, recent updates and notes
RIDAC Coverage	Risks, issues, decisions, actions, change requests

Each rule is a pass or a fail. A dimension's pass rate is the share of its rules that pass. That pass rate feeds a rating (Excellent, Good, Fair, Poor or Critical), and the dimension ratings feed the overall project rating.

Before you start

You need the admin role.
Work in a development or test instance first, then move the change through your normal process.
Work in the application scope ServiceNow Otto for Strategic Portfolio Management. Your changes are saved there.
Customize only the two customer-editable script includes below. Never edit the base versions with SNC in the name, because upgrades overwrite them.

Customer-editable script include	Use it to
`NADQIRuleCatalog`	Add, change or remove rules
`NADQIScriptedRules`	Write the logic for a custom scripted rule

To open one, go to System Definition > Script Includes and search by name.

Two ways to add a rule

	Simple rule	Scripted rule
Coding needed	A small configuration block	A short JavaScript function
Best for	Checking a field or whether related records exist	Any custom logic
Where it lives	`NADQIRuleCatalog` only	`NADQIRuleCatalog` and `NADQIScriptedRules`

If a simple rule can express your check, use it.

Add a simple rule

Open NADQIRuleCatalog. The dimensions are _getCharterRules, _getScheduleRules, _getResourcesRules, _getFinancialsRules, _getReportingRules and _getRidacRules. Override the one you want and add your rule to it. This example adds a rule to Project Charter:

_getCharterRules: function() {
    var dimension = NADQIRuleCatalogSNC.prototype._getCharterRules.call(this);
    dimension.rules.push({
        number: 27,
        name: 'Short description populated',
        type: 'simple',
        check: 'field_not_empty',
        field: 'short_description',
        table: NADQIConstants.TABLES.PROJECT,
        displayConfig: {
            displayName: 'Populate the short description',
            generalBenefit: 'Helps stakeholders identify the project quickly.'
        }
    });
    return dimension;
},

Property	Meaning
`number`	A unique number. The shipped rules use 1 to 26, so start at 27.
`name`	Internal name of the rule.
`type`	`'simple'` for this kind of rule.
`check`	Which check to run (below).
`displayName`	The action text shown to the project manager.
`generalBenefit`	One sentence on why the rule matters.

Available checks

`check`	Passes when	Extra properties
`field_not_empty`	Every listed field on the project has a value	`field` (one or more field names, comma-separated), `table`
`field_greater_than`	A number field is greater than a value	`field`, `value`, `table`
`field_within_days`	A date field is within the last N days	`field`, `days` (a number), `table`
`related_exists`	At least one related record exists	`table`, `linkField` (the field pointing to the project), optional `encodedQuery`

For example, this rule passes when the project has at least one risk created in the last 90 days. The link field name is a placeholder for your own schema:

{
    number: 28,
    name: 'Recent risk logged',
    type: 'simple',
    check: 'related_exists',
    table: NADQIConstants.TABLES.RISK,
    linkField: 'task',
    encodedQuery: 'sys_created_on>=javascript&colon;gs.daysAgo(90)',
    displayConfig: {
        displayName: 'Log or review a risk at least every 90 days',
        generalBenefit: 'Keeps the risk register current.'
    }
}

Add a scripted rule

Use this when none of the simple checks fit. It takes two steps.

Step 1: register the rule in NADQIRuleCatalog. Set type to 'scripted' and name the function that holds the logic in evaluator:

_getScheduleRules: function() {
    var dimension = NADQIRuleCatalogSNC.prototype._getScheduleRules.call(this);
    dimension.rules.push({
        number: 29,
        name: 'Project has a go-live milestone',
        type: 'scripted',
        evaluator: 'hasGoLiveMilestone',
        displayConfig: {
            displayName: 'Add a go-live milestone',
            generalBenefit: 'Gives stakeholders a clear delivery target.'
        }
    });
    return dimension;
},

Step 2: write the function in NADQIScriptedRules. The function name must match evaluator exactly. It must return { pass: true or false, detail: 'short message' }. Inside it, this.project is the project record and this.projectSysId is its sys_id.

var NADQIScriptedRules = Class.create();
NADQIScriptedRules.prototype = Object.extendsObject(NADQIScriptedRulesSNC, {
    initialize: function() {
        NADQIScriptedRulesSNC.prototype.initialize.apply(this, arguments);
    },

    hasGoLiveMilestone: function() {
        var gr = new GlideRecord(NADQIConstants.TABLES.PROJECT_TASK);
        gr.addQuery('sub_tree_root', this.projectSysId);
        gr.addQuery('milestone', true);
        gr.addQuery('short_description', 'CONTAINS', 'go-live');
        gr.setLimit(1);
        gr.query();
        if (gr.next())
            return { pass: true, detail: 'Go-live milestone found' };
        return { pass: false, detail: 'No go-live milestone defined' };
    },

    type: 'NADQIScriptedRules'
});

If NADQIScriptedRules already has your functions, add the new one to the existing object and keep what's there.

Tips for scripted rules

Keep it light. The rule runs for every project being scored. Use setLimit(1) when you only need to know whether a record exists.
Make detail useful. It explains the result, so say what was found or missing.
Don't write to data. A rule should only read.
Mind errors. If the function name doesn't match, the rule fails with "Evaluator not implemented". If the function throws an error, the rule fails with "Evaluation error". Either one lowers the dimension's score, so test before releasing.

Change an existing rule's text

You can reword the message and benefit shown to project managers without changing the rule's logic. Find the rule by number and update its displayConfig:

_getScheduleRules: function() {
    var dimension = NADQIRuleCatalogSNC.prototype._getScheduleRules.call(this);
    for (var i = 0; i < dimension.rules.length; i++) {
        if (dimension.rules[i].number === 7) {
            dimension.rules[i].displayConfig.generalBenefit = 'Your updated benefit text.';
            break;
        }
    }
    return dimension;
},

Turn off a rule

A rule has no active/inactive switch. To turn one off, remove it from the list in NADQIRuleCatalog. This works for shipped rules and for rules you added. This example removes rule 8, "Project baseline exists", from Schedule & Milestones:

_getScheduleRules: function() {
    var dimension = NADQIRuleCatalogSNC.prototype._getScheduleRules.call(this);
    dimension.rules = dimension.rules.filter(function(rule) {
        return rule.number !== 8;
    });
    return dimension;
},

If you are adding and removing rules in the same dimension, put both in the one override. Each dimension method can be overridden only once in the class. To bring a rule back, remove the filter.

Turn off a whole dimension. Clear that dimension's evaluation criteria in the skill configuration for the Data Quality Index. The dimension is skipped and left out of the overall rating. Don't remove all of a dimension's rules instead. A dimension with no rules scores 0%, which is rated Critical.

What happens to scores

Change	Effect
Add a rule	Its dimension gets one more equal-sized slice. A project that passed 4 of 5 rules (80%) and fails the new rule now passes 4 of 6 (67%).
Remove a rule	The remaining rules count for more. Removing 1 of 5 means each remaining rule is worth 25% instead of 20%.
Add or remove a rule	The dimensions' weights toward the overall rating do not change.

Within a dimension, every rule counts equally.
Dimension weights are written in the overall evaluation criteria in the skill configuration, where an admin can edit them. The defaults are Charter 20, Schedule 20, Resources 20, Financials 20, Reporting 15 and RIDAC 5.
Small dimensions react the most. RIDAC Coverage has a single rule, so adding a second makes each worth 50%.
Existing scores don't change until a project is scored again. Tell your project managers to expect some ratings to move.

How ratings are produced

Rating a project has two parts. The rules do the checking, and AI interprets the results.

1. Rules check your data. Each rule is evaluated against the project's data and returns a pass or a fail, with a short message about what was found. The percentage of rules that pass in each dimension is calculated at this stage.

2. AI rates and explains the results. The pass or fail results, the pass rate for each dimension, and your evaluation criteria are sent to the Now Assist rating capability. The model uses them to:

Rate each dimension and the project overall, using the scale and weights written in your evaluation criteria.
Explain its reasoning. Each dimension comes with a short rationale in plain language, so a project manager can see why it was rated that way.
Suggest improvements. For rules that failed, it can recommend what to fix first. Where it applies, it says which other rules fixing it would help.
Use your wording. If you relabeled the rating levels, such as "Poor" to "Low Quality", it uses your labels.

The AI doesn't decide which rules pass. That is always done by the rules. Its job is to turn those results into ratings and guidance.

Default scale

Pass rate	Rating	Value
85% to 100%	Excellent	5
70% to 84%	Good	4
55% to 69%	Fair	3
35% to 54%	Poor	2
0% to 34%	Critical	1

These are the default thresholds. You can change them, and the dimension weights, in your evaluation criteria in the skill configuration.

What this means for you

Ratings are guided by the thresholds and weights you set, but the AI applies them. A project close to a boundary could be rated differently on different runs, and the overall rating is a judgment based on the weights.
The rationale and suggestions can vary in wording from run to run.
Clear, specific evaluation criteria give more consistent ratings.
The checks are repeatable, so the pass or fail and the pass rates for a given project are the same each time.
If the AI service is unavailable or returns something unusable, the product falls back to a simple calculation. It places each dimension's pass rate on the scale above and averages the dimensions equally for the overall rating. In that case there is no rationale and no suggestions, only the ratings.

Test your change

Save your changes in the development instance.
Score a project you know well, one that should pass your rule and one that should fail it.
Open the project's results and check that your rule appears, in the right dimension, with the right pass or fail and the message you wrote.
If a rule shows "Evaluation error" or "Evaluator not implemented", check the system log for errors starting with DQI:.

Troubleshooting

Symptom	Likely cause
The rule doesn't appear	The dimension method name is wrong, the change wasn't saved, or the project hasn't been scored again.
"Evaluator not implemented"	The `evaluator` name doesn't match the function name in `NADQIScriptedRules`.
"Evaluation error"	The function threw an error, for example a wrong table or field name. Check the log.
The rule always fails	The check doesn't match your data, or the function never returns `pass: true`.
Two rules clash	They use the same `number`. Use a unique number for each rule.

Best practices

Use a number that has never been used for a rule, even a removed one, so old results aren't confused with new ones.
Keep the number of added rules modest. Many small rules in one dimension can change its scores a lot.
Document your customizations, because only your team knows about them.
Retest after every upgrade, since the shipped catalog can change.
Make changes in development first and move them through your standard process.