Demand Scoring: Why are Cost and Strategic Alignment Metrics Not Included?

[DavidB-ADT]

Question: Why are the Cost and Strategic Alignment metrics not being factored into the Demand Score? We were scratching our heads trying to figure out how the Score was derived from the 5 metrics before doing research/reverse engineering to then realize that only 3 of them are factoring into the score (risk, value, size).

I'm particularly curious about why I'd send out assessment questionnaires to Stakeholders about Strategic Alignment, to then not have that factor in any way to the score. Heck, the metric result attributes for Strategic Alignment and Cost aren't even shown on the form OOTB.

Am I missing something, or is this a huge fail with the OOTB metrics architecture/solution? Has anyone solved/fixed this in their deployment?

Context:
The OOTB Demand Assessment metric type has 5 categories:

1. Size
2. Strategic Alignment
3. Risk
4. ROI
5. Cost

The Populate Metric Results business rule and associated script include function DemandUtil.populateScore take the results of the script and question based metrics and populate them into their respective scoring fields:

1. score_size
2. score_strategic_alignment (not shown on demand form by default)
3. score_risk
4. score_value
5. score_cost (not shown on demand form by default)

Thereafter, as documented, the EntityScorer and ScoreCalculator script include are calculating the Score as:
score = ((10 - score_risk) + (10 - score_size) + score_value) / 3

So, based on this the score_strategic_alignment and score_cost values are just ignored? What are we supposed to do with them - eyeball the metric results for these categories, and manually adjust the priority on SPW or something?

--

I also have a separate rant about how all these OOTB metrics are HARD-CODED by sys_id into the business rules and script includes, but I'll probably save that for another post. This essentially means that if you want to create your own demand assessment metrics or questions, you pretty much have to clone and code your own business rules and script includes - seems like a poor solution architecture if it doesn't allow any customer enhancement to OOTB.

[pavani_paluri]

HI @DavidB-ADT,

I think OOB it wont consider the Cost and Strategic Alignment in score calculation. 

Create a custom Script Include (clone ScoreCalculator) and update EntityScorer to call custom script
Add score_cost and score_strategic_alignment to the Demand form so stakeholders see them during review.

Mark it helpful if this helps you to understand. Accept solution if this give you the answer you're looking for

Kind Regards,

Pavani P

[DavidB-ADT]

This is a low-effort, karma-farming response. No cookie for you, I'm afraid.

Echoing my observation as a thought - "I think it doesn't" and just stating the painfully obvious, blanket statement of a solution that's required. Of course I have to create my own custom script includes to fix this.

My question is mostly philosophical - why as a customer do I have to invest development time just to get the OOTB metrics working completely and accurately? I'd be twice as annoyed if I had to pay a vendor/implementation partner to fix this.

[DavidB-ADT · Accepted solution]

For anyone that does want complete and accurate demand scoring, my team and I have worked on some patched versions of the Business Rules and Script Include to correct the flaws mentioned above.

The attached Update Set includes the following items, which I'll explain in detail below.

The patched Business Rules are ordered to execute immediately after the OOTB ones, so that you can leave them untouched if you want. This also means that you can apply additional conditions if you only want the corrected scoring to work for specific types of demands, portfolios, etc. Also included is an additional Business Rule that leverages the new score calculator to write notes to the demand to show the score workings.

1. Business Rule: Populate Metric Results [Patched]

• Same as OOTB, but uses the patched x_DemandUtil script include instead.
• Ordered immediately after OOTB, so will overwrite initial scores. Can add additional conditions to only run for a specific scope of demands, leaving OOTB scoring intact for others.

2. Script Include: x_DemandUtil

• Makes use of constants and variables to define sys_ids for Metric Type and Category references, instead of repeated, hard-coded ones. This allows you to clone and update easily for any customer-specific metric types and categories.
• Variable-based choice between using the Rating or Normalized Value scores from assessment category results. Rating remains default (like OOTB), since you can't use Normalized Values if you intend to also use the Demand Workbench. But, you have the option to present Normalized Scores in the demand.
  
• Cleaner/more robust code for collecting and populating metrics per category. i.e. relies on variables and loops, instead of fixed IF statements with hard-coded values. Again, this allows you to add/remove/customize categories easily without having to worry about updating multiple parts of the code.

3. Business Rule: Populate Overall Score [Patched]

• Similar to OOTB, but uses the replacement x_ScoreCalculator script include.
• Ordered immediately after OOTB, so will overwrite initial demand score. Can add additional conditions to only run for a specific scope of demands, leaving OOTB scoring intact for others.

4. Script Include: x_ScoreCalculator

• Greatly simplified, but more robust (IMO) scoring mechanism than OOTB.
• Uses SCORE_FIELD constant from x_DemandUtils to dynamically change based on score preference (Rating / Normalized_Value)
• Top-level variables allow you to specify the use of Inverse scoring per category
  
• If using Rating, it's retrieves the configured Weight values from the Metric Categories records.
• Retrieves the Scale Factor from the Metric Type for use in the inverse scoring - no more hard-coded '10-<score>', even if you change the scale factor.
• Outputs overall score, as well as Breakdowns (used by score logging business rule to provide details)
• New scoring formula (assuming default inversions left configured):

(
     ( (Scale Factor - Risk Score) * Risk Category Weight ) + 
     ( Value Score) * Value Category Weight ) + 
     ( (Scale Factor - Size Score) * Size Category Weight ) + 
     ( Strategic Alignment Score) * Strategic Alignment Category Weight ) + 
     ( (Scale Factor - Cost Score) * Cost Category Weight ) 
) 
/
(
     Risk Category Weight + Value Category Weight + Size Category Weight + 
     Strategic Alignment Category Weight + Cost Category Weight
)​

Note: if using Normalized Values, then Category Weight is set to = 1. Thus the denominator becomes the sum of however many categories you're using i.e. Score = Average of Normalized Values summation.

5. Business Rule: Log Score to Work Notes [Custom]

• Business rule to retrieve Breakdowns from the x_ScoreCalculator and log the score workings to the Work Notes journal in the Demand.
• This has the additional benefit of tracking changes to the demand score, especially if the demand is manually adjusted on the Demand Workbench
  

Hopefully someone finds this useful. We're excited to use what we think is a more accurate and complete score to factor into our SPW prioritization. We also believe this makes explaining how the score was derived much easier to leadership/stakeholders, thus building trust in this data-driven process.

Welcome to any feedback or bug reports with the updated code.