3 Metrics Every CISO Needs to Reduce Risk – and Build Budget

The security metrics that many CISOs traditionally use typically lack business context and often fail to provide a comprehensive and actionable view of how the security program decreases risk and enables the business.

This can leave security teams with a false sense of confidence and CISOs struggling to show ROI and build budget. However, in order to keep pace with the new threats that have surfaced over the past year, many CISOs feel more investment in security will be critical. According to an Enterprise Strategy Group survey of IT decision makers1, 66% of respondents intend to increase their cyber security budget throughout 2021.

In order to obtain the budget needed to properly secure their organizations, CISOs must be armed with the right metrics and communication tactics to demonstrate ROI, connect security investments to business outcomes, and prioritize a roadmap for reducing risk and highlighting operational efficiencies.

Examples of Common, Ineffective Security Metrics

Not all security metrics are equal. If a metric doesn’t provide any context as to whether it’s good or bad, or leaves you and your team unsure of how to derive meaning and act on it, then it’s not likely to help you make a case for more budget in the critical board-room conversations. Below are some of the commonly reported metrics that fall short in tying your security program back to business outcomes:

Consumption-based metrics: Consumption metrics like events-per-second or alarms-per-day are easy to pull from security tools, but they don’t account for the diversity (or lack thereof) of log sources, or the extent of geographic, cloud, or SaaS environments – nor do they capture increases and decreases in visibility that correlate to threat activity.

Mean time to detect (MTTD) and mean time to respond (MTTR): Everyone wants to reduce these metrics, but ultimately it doesn’t matter how fast your team is responding if they only have visibility into a small percentage of your environment or if you don’t have the proper detection capabilities in place.   While retrospectives are important, sharing mean time to detect and mean time to respond with board members – without proper context into visibility and coverage – raises more questions than answers.

Ratio of alarms, open to closed: The assumptions are that if the open alarm rate is high, your security team may not have enough people to respond adequately. If the alarm close rate is high, it’s good news. But this is likely an oversimplification of the true state of the security environment – and again, doesn’t offer action items.

The Security Metrics Every CISO Needs

1. Visibility

In a world where everyone wants to measure number of events and MTTR, there’s a critical question missing: Do you have the right level of visibility into your environment? This is a difficult question to answer, but you must consider this first before looking at any other metrics because you can’t protect what you can’t see.

Start by determining how many log sources you own; then, measure how many of those sources are actually logging.  After you determine how much of your environment you can see, you can then measure your detection content coverage mapped to industry frameworks such as MITRE ATT&CK^® to understand how much visibility you have into known attack techniques.

By identifying these gaps, you’ll be able to build a prioritized roadmap of log source integrations and new detection content to improve overall visibility.

2. Tool Efficacy

As enterprises continue to grow their technology stacks, it’s important to measure the tool’s usage and effectiveness to determine if you’re truly maximizing the capabilities and getting the appropriate ROI. 

Measure how well your tools are working by looking at metrics around the number of issues or outages within a tool. Then, determine if you’re taking full advantage of your tools’ capabilities by measuring integration and efficacy of the latest features. 

You can then work with your engineers and architects to drive ingestion of useful data sources or improve the reliability of alerting capabilities. To get the most out of your tools and enable cross-technology detection and response, you may consider an integrative platform such as an open XDR solution.

3. Team Performance

It’s important to gauge your team’s performance in order to identify any resource gaps, process improvements, or automation that could help them do their job more efficiently. Look at metrics like false positive rate, anomalous safe rate, and true positive rate to determine where your team is spending the majority of their time and how well they understand your environment.

From there, you can prioritize ways to resolve your team’s greatest challenges. The most effective way to improve team performance is through security automation, which provides an opportunity to eliminate the noise, reduce low-brain tasks, and increase alert fidelity so your team can focus more energy on what matters.

By measuring visibility, tool efficacy, and team performance, CISOs will be better equipped to answer the board’s toughest questions, identify and prioritize gaps, and build the budget needed to truly protect and enable the business.

To learn more about how to find and apply these metrics in order to show ROI, identify program gaps, and build budget,  view the CISO’s Guide to Metrics that Matter in 2021. 

Colin O’Connor is the Chief Operating Officer for ReliaQuest, one of the fastest-growing companies in the global cybersecurity industry. Over the past 11 years with ReliaQuest, he has played a key role in nearly every area of the company, helping to architect and enhance ReliaQuest’s solutions for its customer base of Fortune 1000 companies. He is an active member of the technology and information security community and has held roles with the Tampa Bay Technology Forum, ISSA, BSides, and InfraGard.