Balanced metrics: The key to success in DevSecOps transformation

  • Now on Now
  • Manshu Verma
  • Solutions
  • 2021
October 27, 2021

Two workers sit at a table with a laptop and a notebook.

When measuring the success of large-scale transformations—particularly in the technology space—it’s natural to look at hard metrics, such as cycle time, mean time to recovery (MTTR), and so on. In IT, for example, hard metrics are what we do all day long.

But within any organization, change is ultimately personal. In my experience, relying exclusively on hard numbers often leads you to ignore the human side of transformation, and sometimes even action the wrong things. That’s why we use both hard and soft metrics to guide our DevSecOps transformation at ServiceNow.

Measuring progress in 2 ways

In our transformation journey, we look at more than 20 data points, with equal weight given to soft numbers and hard numbers:

  • Hard metrics are quantifiable, such as the number of releases flowing through the pipeline, the frequency of deployment, and the number of teams using the pipelines.

  • Soft metrics are qualitative, subjective values, including Net Promoter Score (NPS) and Adoption Readiness surveys of our program leads, that assesses the engineering community’s satisfaction with the transformation. We often follow these surveys with retrospectives where engineers can share their thoughts on changes from a people (culture), process, and technology perspective.

To help us transform insight into action, we use a combination of both metric types to consider the human side of a project via satisfaction and feedback surveys. Including soft metrics brings team members into the transformation process, giving them the chance to voice their views and influence change.

Teams that show great self-reported metrics, such as satisfaction scores, and hard metrics, such as deployment frequency, are the ones that make steady progress and find new ways to improve week over week.

On the flip side, teams that rely only on the rigor of hard metrics and overlook user satisfaction often hit a plateau quite quickly. This results in frustration—pushing hard but not reaping rewards.

I must admit that although I love to see high satisfaction scores, I welcome the low ones because they yield a goldmine of feedback that will help us improve how we work.

"We use both hard and soft metrics to guide our DevSecOps transformation." Manshu Verma, Senior Program Manager, IT Program Management, ServiceNow


Gaining insights from divergence

Hard and soft metrics often track with one another; good progress against hard metrics often results in high satisfaction. Where it gets interesting is when the two metrics diverge.

For example, one of our teams’ high satisfaction scores showed they were very happy with their new process. Yet, they were used to their previous release schedule and saw no great need to increase their deployment frequency. By considering both metrics, I knew we needed to create encouraging conditions for them to deploy more frequently and experience the benefits themselves.

At first blush, it may seem like low satisfaction scores are purely a symptom of cultural challenges, but a deeper look can reveal whether it’s the process, technology, or culture that needs fine-tuning.

As an example, one of our teams had low satisfaction scores—not because they resisted change, but because their unique setup wasn’t supported by our technology. They listed their complaints, and the DevSecOps tooling team enabled features to accommodate them. As a result, the releases going through the pipeline and satisfaction increased not only for this team, but also for other teams that had this problem but not as severely. (Read more about using metrics to gain visibility and achieve desired outcomes.)

Transformation is a journey

Transformation is an ongoing process, from crawl to walk to run to sprint. Because of that, attitudes and metrics are going to change. Be sure to take the pulse of your community periodically—on both hard and soft metrics—to assess your progress and make any necessary course corrections to achieve your goals.

Find out more about how ServiceNow uses its own products.

© 2021 ServiceNow, Inc. All rights reserved. ServiceNow, the ServiceNow logo, Now, and other ServiceNow marks are trademarks and/or registered trademarks of ServiceNow, Inc. in the United States and/or other countries. Other company names, product names, and logos may be trademarks of the respective companies with which they are associated.

Topics

  • Future of low-code: female working on laptop
    Application Development
    5 trends driving the future of low-code
    Adopting low-code technologies can help businesses create purpose-rich careers for their employees. Find out five key trends driving the future of low-code.
  • ServiceNow Scholarship for diversity in tech: college students gathered around a laptop
    Careers
    Introducing the ServiceNow Scholarship to promote diversity in tech
    The ServiceNow Scholarship fund demonstrates ServiceNow’s deep dedication to supporting underserved students and building a more diverse tech industry.
  • 3 boomerang employees who returned to ServiceNow
    Careers
    Returning to ServiceNow: Insights from 3 boomerang employees
    Watch as three boomerang employees share their stories about why they came back to ServiceNow and how they’re excelling both personally and professionally.

Trends & Research

  • RPA: group of workers gathered around a conference table looking at a laptop
    AI and Automation
    Forrester report: ServiceNow debuts as a Strong Performer in RPA
  • #1 in ITSM, AIOps and IT Operations Market Share: Organizations around the world count on ServiceNow in times of change.
    IT Management
    ServiceNow is No. 1 for AIOps, IT operations, and IT service management categories by market share
  • ESG technology: green surrounding a river, woman smiling, 2 government employees in conversation
    Cybersecurity and Risk
    Survey says ESG technology drives results

Year