Measuring Security Risk vs. Success
Oftentimes, how organizations measure risk determines how they will prioritize investments. For IT professionals, building a set of metrics for security needs is often accompanied by feelings of anxiety, because if measurements look at the wrong data or indicators, they may lead to a false sense of security. Security programs are made up of many investments, including software, appliances, services, training and headcount, among others. For every business investment, there is an opportunity cost. The same principle applies to cybersecurity – for every mitigation investment, there is a threat mitigation return.
Measuring Success
Measurements serve to improve confidence in decisions and reduce cybersecurity risk. With this in mind, security professionals must first recognize that each measurement will be scrutinized both within the team and by external stakeholders who will review or rely on the measure. Second, IT teams must frame every measure in the context of the business needs or requirements. This may range from supply chain to regulatory needs to incident response resolution. Without providing the business context, these measures are just data points and have limited value. Finally, organizations may have both internal cybersecurity program measures and communicable measures used to communicate cybersecurity risk to the rest of the business.
While there is some data shared between the two, it’s often the case that teams manage two sets of metrics. I recall many conversations with CISOs over the last decade whose security teams do just that; one set of metrics to drive investments and improvements across the program, and another set that’s used in boardrooms for executive readouts. When asked why this was so, their answer was simple, “The context for the measurements was simpler to convey when crafted in the language of the audience.” In other words, know your audience. Security is a complex art, and the simpler the message, the easier to communicate outside of the immediate profession. Therefore, as security professionals craft measurements for their organizations, they should keep it simple wherever possible.
Building up to Risk
Building the right risk metrics to inform and support an organization takes time. Establishing a framework early on builds the structure, and from there, each facet of the program can further iterate their individual metrics. This process is rarely perfect right off the bat, because security professionals will continue to discover new learnings and make improvements at each stage of the process.
The best place to start when building risk metrics is by evaluating where you are now. IT teams should begin by asking a few simple questions:
- What do I know about what and who is on the network?
- Where are they and what is expected of the application or device being used?
- What controls are in place to assist in the discovery, validation, policy enforcement and auditing of all of these items?
You must be able to see the transaction or communication, know what it is supposed to do (or at least have guardrails to keep it reasonably well-behaved), and have a method to validate and take action if an unexpected or non-compliant item is identified. Quantity of events, time as a measure of efficacy or success, or programmatic triggers (i.e., weekly or monthly patch compliance reports on critical applications or a servers report) might be the right type of metric. Ultimately, the right metrics can track and inform better investments to reduce risk.
An example would be looking at how many critical patch events were required in a given period; say, over one month or one quarter, for data center applications to address critical security risks. This metric, in turn, could be used to inform and potentially support a case of application consolidation, or a reduction of attack surfaces. The ultimate goal is ensuring that teams are putting the right metrics in place that best suit the audience and serve to effectively communicate risk reduction or expense for their organization.
Avoiding Analysis Paralysis
Every effective risk metric starts with correlating its value or impact to key business or operational objectives. Whether that is uptime, customer loyalty or brand value, aligning the measurement to the risk exposure helps drive an attributable value. The sensitivity of the measure to the success of the security program will, in turn, drive update frequency. This process is then repeated until every business or program objective has been assessed and key metrics established. Risk metrics are not written in stone, and they should be periodically reviewed to ensure they remain relevant. Throughout this process, it is imperative to collaborate with other stakeholders, because they will rely on these metrics to serve as a window into the security risk of their objectives.
When building security risk metrics, the key is to not let analysis paralysis impair your progress. Jump in headfirst, and begin by having key conversations with the stakeholders who own key business objectives to understand their concern and the risks to their specific area(s) of responsibility. Then, craft plans to iterate. Managing security risk is often considered in terms of the abstract or high-level objectives desired. Translating these objectives and expectations into measurable and specific actions can be challenging, yet it is the simplest and most effective approach for most organizations.
