Balanced metrics: The key to success in DevSecOps transformation

  • Now on Now
  • Manshu Verma
  • Solutions
  • 2021
October 27, 2021

Two workers sit at a table with a laptop and a notebook.

When measuring the success of large-scale transformations—particularly in the technology space—it’s natural to look at hard metrics, such as cycle time, mean time to recovery (MTTR), and so on. In IT, for example, hard metrics are what we do all day long.

But within any organization, change is ultimately personal. In my experience, relying exclusively on hard numbers often leads you to ignore the human side of transformation, and sometimes even action the wrong things. That’s why we use both hard and soft metrics to guide our DevSecOps transformation at ServiceNow.

Measuring progress in 2 ways

In our transformation journey, we look at more than 20 data points, with equal weight given to soft numbers and hard numbers:

  • Hard metrics are quantifiable, such as the number of releases flowing through the pipeline, the frequency of deployment, and the number of teams using the pipelines.

  • Soft metrics are qualitative, subjective values, including Net Promoter Score (NPS) and Adoption Readiness surveys of our program leads, that assesses the engineering community’s satisfaction with the transformation. We often follow these surveys with retrospectives where engineers can share their thoughts on changes from a people (culture), process, and technology perspective.

To help us transform insight into action, we use a combination of both metric types to consider the human side of a project via satisfaction and feedback surveys. Including soft metrics brings team members into the transformation process, giving them the chance to voice their views and influence change.

Teams that show great self-reported metrics, such as satisfaction scores, and hard metrics, such as deployment frequency, are the ones that make steady progress and find new ways to improve week over week.

On the flip side, teams that rely only on the rigor of hard metrics and overlook user satisfaction often hit a plateau quite quickly. This results in frustration—pushing hard but not reaping rewards.

I must admit that although I love to see high satisfaction scores, I welcome the low ones because they yield a goldmine of feedback that will help us improve how we work.

"We use both hard and soft metrics to guide our DevSecOps transformation." Manshu Verma, Senior Program Manager, IT Program Management, ServiceNow


Gaining insights from divergence

Hard and soft metrics often track with one another; good progress against hard metrics often results in high satisfaction. Where it gets interesting is when the two metrics diverge.

For example, one of our teams’ high satisfaction scores showed they were very happy with their new process. Yet, they were used to their previous release schedule and saw no great need to increase their deployment frequency. By considering both metrics, I knew we needed to create encouraging conditions for them to deploy more frequently and experience the benefits themselves.

At first blush, it may seem like low satisfaction scores are purely a symptom of cultural challenges, but a deeper look can reveal whether it’s the process, technology, or culture that needs fine-tuning.

As an example, one of our teams had low satisfaction scores—not because they resisted change, but because their unique setup wasn’t supported by our technology. They listed their complaints, and the DevSecOps tooling team enabled features to accommodate them. As a result, the releases going through the pipeline and satisfaction increased not only for this team, but also for other teams that had this problem but not as severely. (Read more about using metrics to gain visibility and achieve desired outcomes.)

Transformation is a journey

Transformation is an ongoing process, from crawl to walk to run to sprint. Because of that, attitudes and metrics are going to change. Be sure to take the pulse of your community periodically—on both hard and soft metrics—to assess your progress and make any necessary course corrections to achieve your goals.

Find out more about how ServiceNow uses its own products.

© 2021 ServiceNow, Inc. All rights reserved. ServiceNow, the ServiceNow logo, Now, and other ServiceNow marks are trademarks and/or registered trademarks of ServiceNow, Inc. in the United States and/or other countries. Other company names, product names, and logos may be trademarks of the respective companies with which they are associated.

Topics

  • Career shift: two women in conversation at a table in an office
    Careers
    Automation is opportunity for a career shift into tech
    Teresa Ko considered making a career shift into tech, but she didn’t want to go back to school or lose income to do so. Find out how her dream came true.
  • Operational excellence strategy: a women’s rowing team on the water
    Cybersecurity and Risk
    A 6-step operational excellence strategy
    Forward-thinking business leaders understand that even in times of uncertainty, a sound operational excellence strategy is critical to competitive advantage.
  • Process mining: 3 workers in discussion over a conference table
    Now on Now
    Why process mining is a game changer for process optimization
    Vital to the push for automation is investing in process mining, which can help you do more with less, transform to increase revenue, and innovate to compete.

Trends & Research

  • RPA: group of workers gathered around a conference table looking at a laptop
    AI and Automation
    Forrester report: ServiceNow debuts as a Strong Performer in RPA
  • Digital innovation: three workers looking at a computer monitor
    AI and Automation
    Survey says digital innovation is the way to navigate macro uncertainty
  • Innovation is a top management imperative: man standing in a corner office overlooking a city
    IT Management
    Survey: Innovation is a top management imperative

Year