Developing a culture of observability

In the race to attract and retain customers, businesses must deliver great customer experiences, release reliable products fast, and scrutinize costs to achieve consistent growth.

That can either be a well-oiled machine or a tangle of disjointed communications and workflows that frustrate customers, employees, and management alike. By developing a culture of observability, you can have a framework that harmonizes the experience for everyone.

An example of observability

Observability is the ability to quickly and efficiently gain insight into the health of your tech estate by gathering, correlating, and interpreting metrics, distributed traces, and log data. This data is also called telemetry.

Let’s look at an example of observability in the world of healthcare. If you’re having a heart attack, first responders quickly gather metrics (blood pressure, heart rate, blood oxygen saturation) and log the event, including important information such as health history and recent meals.

To learn more about your heart’s health, a healthcare practitioner takes a trace of your heart with an electrocardiogram, or ECG. The ECG describes the health, timing, and physical condition of your heart.

The shape of the trace corresponds to where it was conducted through the heart and where the problem most likely exists. To a healthcare practitioner, the shape conveys your heart’s health.

A blood test can yield additional metrics to confirm a diagnosis and guide the care team. All of these elements are forms of observability.

The same logic applies to computer code and systems. As with life-saving medical treatments, time matters when it comes to the health of your tech estate. By tracing running computer code, we can understand where problems may exist. A deep, real-time inspection of running processes can yield fast resolutions, captured by metrics such as mean time to resolve (MTTR).

Ultimately, you want your customers and employees to have a positive, seamless experience. If an issue arises, you want a quick resolution and an easy way to examine the outcome so that everyone benefits.

How can you develop a culture of observability?

Your observability journey may begin with a few enthusiastic team members evangelizing the practice and enrolling stakeholders to adopt it. This is where proofs of concept, demos, and showcases come in. By helping key technical and organizational leaders understand why strong observability will have a positive impact on the business, you’ll more likely get buy-in and sponsorship.

Fostering a culture of observability can begin anywhere. When an executive asks the product or engineering team, “How could we have caught that?” it can trigger an observability exercise to determine the services involved and how to resolve issues. 

If a customer contacts support with an issue, the product or support team should lead the exercise. Invite people to the conversation and create modes of accountability through health targets such as service-level objectives. Reducing incidents and improving quality should be consistently encouraged and incentivized.

Why should you develop a culture of observability?

It’s important to ask, “Why would we do this?” Create a list of possible reasons, addressing three key areas: customer success outcomes, forces that can lead to burnout, and anticipated high-level business outcomes. Here are some answers to consider:

• Improved MTTR

• Increased operational efficiency

• Proactive issue detection

• Improved customer experience

• Ability to redirect resources to innovation

• Business and/or revenue growth

• Reduced employee burnout/ turnover

• Increased speed to market

• Predictable OpEx
  
  

Observability culture is a journey

Observability is a journey of continuous improvement. It starts by identifying and implementing critical workflows or services, followed by observing them over a specified time period to determine health targets (such as service-level objectives). Then, alert thresholds can be set.

Your goal is not to overwhelm product or support teams with “alert noise,” but to achieve a balanced number of accurate alerts with strong health targets. By taking time to observe, you can fine-tune the thresholds.

Think back to the heart attack example. Without metrics, logs, or traces, patients would face a higher mortality rate, which could lead to negative outcomes for providers. Institutional funding could be jeopardized by negative key performance indicators.

If this is happening in your organization, it’s time to evolve to be able to see the overall health of your organization improve. Observability should be seen as an opportunity cost, not a sunk cost. Adopting open-standard technologies and refining your observability strategy can help your organization grow.

Schedule a demo to see how you can increase operational efficiency and make your tech estate and your teams more resilient.

© 2023 ServiceNow, Inc. All rights reserved. ServiceNow, the ServiceNow logo, Now, and other ServiceNow marks are trademarks and/or registered trademarks of ServiceNow, Inc. in the United States and/or other countries. Other company names, product names, and logos may be trademarks of the respective companies with which they are associated.