How do you benchmark security?

Cybersecurity leaders urgently need standardized metrics to manage business risk

cybersecurity strategy metrics
  • Security chiefs have been in react mode for years, doing their best to respond to ever‑changing threats
  • Common security metrics often lack business context
  • Some companies practice “selective defense,” linking security risks with specific business goals or assets

“If you can’t measure it, you can’t manage it,” goes the old management bromide. But what if you aren’t measuring the right things? Or don’t even have a measuring stick?

That’s the position many companies are in when it comes to assessing and managing cyber risk. They might be able to count the number of malware incidents or attempted firewall intrusions, and they’re spending a fortune to block those threats. Companies will spend an estimated $124 billion globally on cybersecurity in 2019, according to Gartner.

Too often, security professionals either lack or aren’t using metrics that show top executives how those investments are lowering business risk or advancing strategic goals.

Companies that want to mature beyond simply reacting to the latest attack need standardized benchmarks for calculating true security risks. Benchmarking doesn’t just give security teams better tools for understanding threats and how to best deploy scarce resources. It’s also a way to deliver more meaningful metrics for the board and the C‑suite.

“Businesses want to know if they’re secure,” says Chris Hodson, chief information security officer at security company Tanium. “You have to give a view of the landscape in language that the board will understand.”

Benchmarking can help an organization compare its security profile with industry standards and best practices, as well as identify strengths and weaknesses. More important, it makes it possible to measure security functions against the broader goals of the business.

While other business sectors have used standard benchmarks for years—think of leverage ratios in finance and the Net Promoter Score for measuring customer satisfaction—cybersecurity has been slow to adopt standardized metrics for calculating risk.

This is partly because security pros have been in reaction mode for so long, responding to a growing cascade of threats. Risks also vary widely from business to business, and the threats themselves are constantly changing. One month it’s a denial‑of‑service attack. The next month your system has been hijacked by cryptocurrency miners. That makes it hard for metrics to keep up.

“Nothing stands still long enough to collect data,” says Bruce Schneier, a longtime cybersecurity technologist and fellow of the Berkman Klein Center for Internet & Society at Harvard University.

In spite of these challenges, there are efforts afoot to more clearly define and calculate security risks. The first step, experts say, is to identify what matters most to a business, where it invests security dollars, and whether the investments are working. Once companies have an effective toolkit of internal benchmarks, CISOs can start measuring their performance against industry peers.

Numbers without context

The metrics that security teams traditionally share with business leaders—such as the number of software patches applied, or how many email phishing threats were blocked by security software—don’t mean much by themselves. For example, it’s unhelpful to tell board members that your security software issued 300 million malware alerts in the past month, because the metric lacks context.

Bill Olson, technical director at Tenable, a Maryland‑based company that helps businesses gauge their exposure to cyber threats, uses a parable about two CEOs talking about cybersecurity. “I have 6,452 vulnerabilities,” says one. The other replies, “Well, I have 9,575.”

A company dealing with fewer vulnerabilities isn’t necessarily safer, Olson says. “What matters is whether the investments we are making to protect the things most important to the business are actually working.”

One answer is what Hodson calls “selective defense,” which links security risks with specific business objectives and assets. For instance, a company might risk the loss of valuable intellectual property to a hacker who has penetrated its computer systems. In that case, it could look at the proportion of unapproved outbound network traffic to spot malicious activity.

These metrics are repeatable and tied to business goals. They should help executives grasp the need to invest in improvements, such as reducing the number of unpatched vulnerabilities in order to keep deliveries running on time. “You get a baseline for trends,” Hodson says.

Ultimately, identifying and using the right benchmark means placing a dollar value on a company’s assets and the possible losses from a cyberattack. “The only way to understand risk is to have an absolute understanding of the monetary value of your world,” Olson says. “You have to get specific—if we lose a certain thing to hackers, it will cost us X dollars.”

Benchmarks don’t eliminate risk, but they can help companies understand how much they’re spending to protect critical assets and gauge whether that investment is working.

Consider a chain of fitness centers that wants to protect its customer database. In establishing benchmarks to measure how well it protects the asset, the company will first want to decide how quickly it needs to respond to an attack and add necessary protections to guard against future attempts.The security team might set an initial benchmark that, say, 80% of the corrupted files should be fixed within three days. If the goal is met easily, the team could raise the benchmark to 85% in the second month. And if one business unit meets the goal but another team doesn’t, analysts could study why the target was missed. Setting that initial benchmark is the key to charting progress.

Are industry comparisons possible?

Ideally, if enough businesses have internal security benchmarks, organizations can use them to measure their performance against leaders in their industries. But their widespread adoption has been hindered by the rapidly evolving nature of security threats and by the difficulty in comparing security risks among companies, even those in the same industry.

In one industry, protecting highly sensitive product designs might be the most important goal. In another, the goal might be to protect online payment systems. Attackers can spin up many different ways to breach systems, depending on the security systems protecting them.

With new security solutions being deployed to fight new kinds of attacks, it’s tough to collect comparable data and define measurable standards that hold up over time. “Products don’t solve the whole problem,” Olson says. In the end, the most useful security benchmarks are the ones specific to your business.