If you want to improve something, you likely need to measure it first. When it comes to management, disposition and rationalization of a Business Application inventory, the Indicator and Scoring framework is a powerful and flexible tool.
On June 18, 2024, we have presented a workshop on this topic. Please see the link below to watch the recording.
Measure your Application Inventory with the Indicator and Scoring Framework
Indicators and scoring has been part of APM for years now and has grown and evolved with the product and concepts like CSDM. Here, we seek to help you configure and get best value from the framework to help govern your Application AND Capability inventory.
These are configurable business metrics that provide a key basis for evaluating both Applications and Capabilities. Indicators provide a normalized score for whatever condition they are evaluating.
There are five types of indicators:
- Assessments – indicator that uses a score derived from a configurable Assessment/Survey. Assessments are configurable in content, target user and evaluation.
- Query Condition – a query condition indicator is a configurable query against tables or other data sources such as performance analytics with ability to define differing
- Custom Script – utilize a custom script to generate an indicator score
- PA Analytics – Utilize Performance Analytics to gather historic data such as incidents, problems change to derive an indicator score.
- Indicators – calculated sum or average of a set of child indicators
There are now (as of Washington release) 14 out of the box indicators in the Washington DC release (see details below).
The chart below also show 7 older cost related indicators that used the ServiceNow IT Financial Modeling engine. That functionality is no longer available but we include these for completeness with legacy usage. However, as of Washington release, we have a new Total Cost of Ownership (TCO) indicator.
The system also provides a mechanism, Scoring Profiles, for Business Applications and Capabilities to provide a single score value derived from the weighted individual indicator values.
Out of the box Indicators
The following table details the out of the box Business Application indicators in Tokyo and above.
|
Indicator name
|
Type
|
Source
|
How is it calculated?
|
Description
|
|
Customer satisfaction (CSAT)
|
Assessments
|
Assessment Metric Type: Customer Satisfaction
Assessment Metric Category: CSAT
|
|
Template NPS
|
|
Technical risk
|
Assessments
|
Assessment Metric Type: Technical Risk
Assessment Metric Category: Performance
|
|
Technical risk captured through survey for the fiscal period.
|
|
Business value
|
Assessments
|
Assessment Metric Type: Business Value
Assessment Metric Category: Business Value
|
|
Business Value captured through survey for the fiscal period. |
|
Functional fit
|
Assessments
|
Assessment Metric Type: Functional Fit
Assessment Metric Category: Functional Fit
|
|
Functional Fit captured through survey for the fiscal period.
|
|
Usage
|
Query Condition
|
APM product. cmdb_ci_business_app table
|
Calculated from the Active User Count field
|
Number of user sessions and users for the application for a given fiscal period.
|
|
Number of Incidents via Service
|
Performance Analytics
|
Mapped to Performance Analytics > Indicators > Automated Indicators > Number of new incidents
Source = Business Application via Service
|
Utilizes the Impacted Business Applications related list on Incident
|
Indicator that gets the count of all incidents associated to the business application tied to the scoring profile of which the indicator is part.
|
|
Number of Changes via Service
|
Performance Analytics
|
Mapped to Performance Analytics > Indicators > Automated Indicators > Number of new changes
Source = Business Application via Service
|
Utilizes the Impacted Business Applications related list on Change
|
Indicator that gets the count of all changes associated to the business application tied to the scoring profile of which the indicator is part.
|
|
Number of Problems via Service
|
Performance Analytics
|
Mapped to Performance Analytics > Indicators > Automated Indicators > Number of new problems
Source = Business Application via Service
|
Utilizes Business application reference on Problem
|
Indicator that gets the count of all Problems associated to the business application tied to the scoring profile of which the indicator is part.
|
|
Application Operational Metrics
|
Indicators
|
Sum of
Incidents via Service
Problems via Service
Changes via Service
|
Sum of operational values with a registration date (change_request.opened_at) on collection date
|
Number of incident, problem, and changes opened over the period. Daily and historic data collection.
|
|
Total change hours
|
Performance Analytics
|
Mapped to Performance Analytics > Indicators > Automated Indicators > Summed duration of closed changes
Source = Changes.Closed (Change_Request table)
Fields: Opened, Closed
State = Closed, Business Application = any of the APM Business Applications, Closed today
|
Script: Change.CloseTime.Hours. All Change Requests closed today considered
|
Summed duration of closed changes for an application for the given fiscal period. Time taken to close the changes in hours. Daily and historical data collection.
|
|
Portfolio TCO
|
Custom Script
|
Total Cost of Ownership [sn_apm_tco]
|
Sums the costs within the specified fiscal period across various cost types
|
Sum of costs per application. the costs are provided by externa; source such as Proven Optics, Nicus, Apptio or other financial modeling systems including simple Excel import.
|
|
Technology Lifecycle Risk
|
Custom Script
|
Based on Business Application risk as evaluated in the TPM Technology Risk table [sn_apm_tpm_technology_risk]
|
Risk due to out of date or soon to be out of date technology in TPM are rolled up to the Business Applications
|
Normalized score of the Technology Portfolio Management (TPM) risk evaluation.
|
|
Application's Incident Count*
|
Custom Script
|
incident
|
Data will be available in the incident table only after the business application is associated to the incident. Not often used since Business Application is not OOB exposed.
|
Indicator that gets the count of all incidents associated to the business application tied to the scoring profile of which the indicator is part.
|
|
Application's Instance – Incident Count*
|
Custom Script
|
incident
|
Gets incident count attached to all Application Instances, which are mapped to a business application and rolls it up to application.
|
Indicator that gets the count of all incidents associated with application instances. The application instances, in turn, are associated to a business application tied to a scoring profile of which the indicator is a part.
The incident count is calculated first at the application instance or application service level, and then it is rolled up to the business application level.
|
* these Indicators are no longer best practice. They allow for only one Business Application and require some form of mechanism such as a business rule to set the Business Application on incident. They shoudl be disabled and removed from an scoring profile unless it is explicietly part of your process and you have made appropriate considerations in the Incident process.
Retired/Deprecated Indicators
These indicators rely upon an older and less mature data model where Business Application has to be directly specified on the Incident/Problem/Change. CSDM and latest maturity woudl advise use of Impacted Service/Business Application lists.
|
Indicator name
|
Type
|
Source
|
How is it calculated?
|
Description
|
|
Number of incidents*
|
Performance Analytics
|
Mapped to Performance Analytics > Indicators > Automated Indicators > Number of new incidents
Source = Incidents.New (Incident table)
|
Number of incidents opened today
|
Number of new incidents. Daily and historic data collection
|
|
Number of changes*
|
Performance Analytics
|
Mapped to Performance Analytics > Indicators > Automated Indicators > Number of new changes
Source = Changes.New (change_request table)
|
Number of changes with a registration date (change_request.opened_at) on collection date
|
Number of changes opened today. Daily and historic data collection
|
|
Number of problems*
|
Performance Analytics
|
Mapped to Performance Analytics > Indicators > Automated Indicators > Number of new problems
Source = Problems.New (Problem table)
|
Problems created today
|
Number of problems opened today. Daily and historic data collection
|
Cloud Readiness Indicators (from the Store download)
The APM product team provides a ServiceNow Store plugin designed to aid in analysis of Cloud Readiness. This plugin adds a few new attributes and the following indicators.
|
Indicator name
|
Type
|
Source
|
How is it calculated?
|
Description
|
|
Strategic Importance
|
Custom Script
|
APM product. cmdb_ci_business_app table
|
Calculated from the Emergency tier (emergency_tier) field of the business application record.
|
Evaluates the emergency tier of the business application.
|
|
Data Classification
|
Custom Script
|
APM product. cmdb_ci_business_app table
|
Calculated from the Data classification (data_classification) field of the business application record.
|
Evaluates data classification of the business application.
|
|
Cloud Version Available
|
Custom Script
|
APM product. cmdb_ci_business_app table
|
Calculated from the Cloud version available (cloud_version_available) field of the business application record.
|
Evaluates if there is an available version of this application that fits cloud deployment.
|
|
Core Architecture
|
Custom Script
|
APM product. cmdb_ci_business_app table
|
Calculated from the Core architecture (core_architecture) field of the business application record.
|
Evaluates if this business application is part of the core architecture.
|
|
Business Criticality
|
Custom Script
|
APM product. cmdb_ci_business_app table
|
Calculated from the Business criticality (business_criticality) field of the business application record.
|
Evaluates the business criticality of the business application.
|
|
Regulatory Implications
|
Custom Script
|
APM product. cmdb_ci_business_app table
|
Calculated from the Regulatory implications (regulatory_implications) field of the business application record.
|
Evaluates the regulatory implications of the business application.
|
|
Business Impact
|
Indicators
|
Not applicable
|
Calculated from the sum of the following indicators:
- Core architecture (core_architecture)
- Strategic importance (emergency_tier)
- Business criticality (business_criticality)
|
Evaluates the business impact score for the business application, based on other indicators.
|
|
Cloud Readiness
|
Indicators
|
Not applicable
|
Calculated from the sum of the following indicators:
- Data classification (data_classification)
- Cloud version available (cloud_version_available)
- Complexity of integration (integration_complexity)
- Regulatory implications (regulatory_implications)
|
Evaluates the cloud readiness score for the business application based on other indicators.
|
|
Complexity of Integration
|
Custom Script
|
APM product. cmdb_ci_business_app table
|
Calculated from the Integration complexity (integration_complexity) field of the business application record.
|
Evaluates the complexity of integration to the business application.
|
|
User Base
|
Custom Script
|
APM product. cmdb_ci_business_app table
|
Calculated from the User base (user_base) field of the business application record.
|
Evaluates the user base of the business application.
|
Old Financial Modeling (ITFM) indicators
These indicators are no longer provided though older APM installation might have them. The ITFM module for SPM (formerly ITBM) is no longer available. Our Partners, Proven Optics, Nicus, and Apptio each have solutions.
- Application TCO
- Facilities Cost
- Hardware Cost
- Software Cost
- Service Cost
- Labor Cost
- Other Cost
** note that as of March 2021, the Financial Modeling application and therefore the APM TCO and other cost indicators based upon the ITFM functionality are no longer available.
Operational Indicators
In order to support viable Operational scoring, you must first understand how Incident, Problem and Change are managed at your company. The key is to understand how Application Services are related in the Incident, Problem and Change processes and gather your metrics at the Application Service level then roll these up to the Business Application.
For example, if your company follows CSDM recommended practice, Application Services may be found as the Configuration Item Or, in the Impacted Services related list on Incident. Your configuration of the indicators must follow that data model.
For Incident, you might consider activating Populate the Business Application related list for incidents (com.snc.incident.populate_business_application ) system property and utilizing the related list of Business Applications.
Assessments
Assessments are a platform facility in ServiceNow. In the system, an Assessment is called a Metric Type. Within the assessment, you define Metric Categories and categories contain Metrics (questions). An Assessment, when completed, calculates a score on each contained category based on weighted values of the metric/question results and then derives and overall assessment score based on weighted category values.
Once you have defined the Assessment contents, you identify the SME(s) that will complete the assessment. Assessments can be automated to be sent on a schedule such as quarterly, bi-annual, etc. They may also be launched on demand. The user will see assigned assessments in Self Service > My Assessments & Surveys.
The four Application assessments are Customer Satisfaction, Business Value, Functional Fit and Technical Risk. Typical target users for the assessments are shown below.
- Customer Satisfaction – a select sub-set of users for the Business Application
- Business Value – Application Business Owner
- Functional Fit – IT Owner/Business Owner
- Technical Risk – IT Owner
The out of the box assessment questions will need to be defined as they are merely place-holders. Also, it is not required to use all four assessments.
Each Business Application is evaluated on a scale of 1(Low) to 5 (High) for each of the following example criteria.
- Customer Satisfaction
- What is the overall customer/consumer satisfaction?
- What is the customer/consumer perceived performance? Does the application perform well during peaks for the business process?
- What is the customer/consumer perceived availability?
- Is the data within the application current and up to date, trustworthy and readily available (per business goals)?
- Business Value
- Does the application deliver all of the requirements for the capability it supports?
- Does the business intend to maintain and scale this application for the future or get rid of it?
- How widely is the application used within the assigned user base?
- Is the application able to our changing business needs?
- Are all aspects of the application in use and fit for purpose?
- The application requires little in depth training and user support?
- The application positively impacts our bottom line?
- Functional Fit
- Does the application deliver all of the requirements for the capability it supports with none or minimal configuration and no customization?
- Does this application fully automate the processes it supports?
- Technical Risk
- Does the application adhere to established architecture (data, integration, infrastructure) guidelines/patterns?
- Is the application used out of the box or customized?
- Does introducing new functionality break existing functionality?
- Is the base technology on a current version that is supported by an internal team or vendor?
- Is the basic technology stack for the application allowed within the enterprise or similar to the greater ecosystem?
- Can the application be easily scaled to add more business functionality, users or integrations to other systems?
- Does the application execute functionality with efficiency (low error rates, average time to execute queries low resource utilization, etc. are all within desired thresholds)?
Business Capabilities and scoring
Business Capabilities are evaluated out of the box via Assessment. the three key, and traditional, evaluations are: People, Process, Technology. See the following examples of questions to configure for these assessments.
- People
- Is the Capability supported from a Knowledge / Skills Staffing perspective to deliver on business goals?
- Do people consistently have the same knowledge or awareness of the capability?
- Technology
- Rate the relative health of the technologies providing this Capability.
- Are there redundant Business Applications providing this Capability?
- Is the capability supported by the right technology in order to deliver the business goals?
- Process
- How mature are the processes underlying this Capability?
- Are these processes evaluated for continual improvement?
- Is the process related to the capability structured, streamlined and automated?
Each dimension is scored using the following scale:
1 – Capability exists and meets future-state business needs
2 – Capability exists and will need minor changes to meet future-state business needs.
3 – Capability exists and need major changes to meet future-state business needs.
4 – Capability does not exist but will be needed for future-state business needs.
5 – Capability exists but requires significant changes to meet current needs and/or future-state business needs.
Indicator Score Calculations
When an indicator is run, it captures a value that is called the Application Weight for each Business Application. Then the system will generate a Normalized Value.
Once the Indicator has processed all application for all date ranges (the system passes in the date ranges for which the indicator will evaluate the business Application that way things such as incidents and Changes don’t keep getting counted if they are outside the date range), it will calculate the Normalized value of the Application indicator.
This normalized value is essentially a “ranking” value from 1-10 of the Application’s value or weight compared to the Minimum and Maximum values found across the set of Applications evaluated. See the normalized calculation below:
Normalized Value = [ (app weight – minWeight)/ (maxWeight – minWeight) ] * 9 + 1
If the Indicator direction is set to Minimize (you want to minimize incidents and the like), the calculation is:
Normalized Value = (10 - Normalized Value(above)) + 1.
For example, if I get a score of 8 for # of Incidents meaning out of all the apps, I am ranking 8 out of 10 in highest number of incidents, I should have a bad Indicator score or at least worse than another app with a 2 or 3.
Indicator Score
After finding the Normalized Value, the system can now generate an Indicator Score which is the weighted value of the Normalized Value within the Scoring Profile for which the indicator score is being generated.
When an Application (or Capability) Scoring Profile is created, the user selects one or more Indicators and assigns the “Weight” it carries in the overall score for the Application.
The Indicator Score is the “weighted” value calculated as:
Indicator Score = Normalized Value * (Indicator Weight from Profile / Total of all Weights on Profile)
Application Score
Now that we have the Indicator Score, the system can calculate the Application Score for the fiscal period.
Application Score = per Application, Sum of all Indicator Scores for the Application during the fiscal period identified.
Application Profile
As you can tell from the discussion above, it is very important to consider the indicators in use and properly configure your Scoring Profile then appropriately set the correct profile on the Business Applications.
Important Note (Feb 2025): As of the February 2025 release, Enterprise Architecture Workspace Rationalization effectively only supports a single Scoring Profile at any given time. The Default Application Scoring Profile is referenced as that single profile out of the box. In order to specify another Scoring profile, you will need to create (or update) the system property, sn_apm_ws.app_indicator_scoring_profile, and set the sys_id of your new/custom scoring profile as the value of the system property. This will tell the bubble chart mechanism to use your scoring profile instead of the default.
