What does “good” cyber security look like? Sure, we can run a maturity assessment and see what good indicators are and we can create a baseline of our current state to establish where we are and what gaps we have (honestly in real terms this isn’t something to consider you should be doing this!) but how do we measure success in cyber security? Is every success an invisible outcome? Because one question that often comes to mind here is, just because we don’t see something, does that mean everything is ok? In the fast-paced world of cyber security, measuring success isn’t as easy as you would think. I’ll give an example of this, let’s say we don’t monitor, we get breached, but the threat actor just performs crypto mining (let’s say this is on premises) and we never really notice in the grand scheme of the world that our energy consumption costs have increased, if we didn’t know this had occurred, we might think our security is good.

In some sense the measure of success in cyber is absence. A bit like monitoring a black hole. This however doesn’t work in my mind. There are things we can measure, there are outcomes that are achieved that aren’t just invisible ghosts in cyber land. As to how this works for each organisation, I think that changes depending upon the culture, industry and its key stakeholders.

Measure all the things!

When it comes to security reporting, we have a range of potential stakeholders:

• The board
• A regulator
• Insurance companies
• The public (think public sector)
• Key stakeholders
• Employees

Each of these will have different viewpoints and views on what good cyber outcomes and measurements look like, but on top of that measuring cyber security isn’t that simple.

Let’s look at some examples of measurements:

• Number of emails received
• Number of emails marked as spam
• Number of emails blocked for malware
• Remaining Emails

This might look great and with modern systems this is fairly easy to measure. But this raises more questions:

• What about the emails that got through that are SPAM or malicious and we didn’t detect them?
• How does email volumes relate to business outcomes?

Again, hopefully this is starting to make a bit of sense as to why I’d write a blog about this.

Report all the things?

Ok so we know that some things are simple to measure, and some things are hard to measure, but even then what do we report? And to whom do we report it? And why should they care?

• Compliance Certifications Achieved/Maintained?
• Number of Lost Opportunities due to poor cyber hygiene
• Number of incidents over a period?
• Number of critical vulnerabilities in the environment?

What I think good cyber security measurement and reporting might look like could include metrics and measurements such as the following:

• Overall Posture Strength (weak/moderate/strong)
• Cyber Security Maturity Rating
  • Current State
  • Previous State
• Residual Risk Rating
• Number of gaps
• Number of vulnerabilities by category across the enterprise
• Number of exposed services to the public internet
• Number of public-facing vulnerabilities
• Percent of Servers with EDR deployed
• Percent of Endpoints with EDR deployed
• Number of incidents by category over a period
• Number of events prevented from becoming incidents
• Percentage of revenue spent on cybersecurity

I think it’s important to compare where we can so like we have shown with maturity rating, tracking historic views is also going to be important to consider.

Now when we get down into the weeds, we are probably going to want far more detail, we also are going to want to slice and dice this into views, for example we might want measures across the lifecycle:

Identify

Protect

Detect

Respond

Recover

We may also want to consider different perspectives:

• Customer
• Legal
• Staff
• Sales
• Cyber Operations
• IT/Technology/Digital

I’m going to be straight here and say that in my mind measuring security is NOT simple when it comes to business context. It’s easy to grab some metrics or measurements but making these relevant in a cyber world where you can be “secure” at 0900 and be highly at risk by 0930 is a challenging concept. I won’t sugar coat this. I think it’s also a moving, multi-faceted landscape.

Baselining the current state

There are tons of models, frameworks, and guidance materials for understanding a cyber security posture, commonly the following are widely adopted:

• NIST CSF
• NCSC CAF
• CMMC
• ISO27001:2013

You will also see control frameworks such as the CIS TOP Controls however be wary of control-based frameworks as they tend to not cover strategy, architecture and governance, which if it was me, I’d want to understand the full view.

Relating this to the organisation outcomes

One thing that should be obvious is that cyber security is an investment, sure it has a cost (all things do) but it’s a risk-based business investment, and it has value more than just “stopping malware”, it’s the digital defence and quality glue that ties modern digital organisation together. It’s both an enabling force and the last line of defence against significant digital threats.

So, with this in mind we should have some concept of how we can relate our cyber success view back through to our business goals. Digital services can both drive, deliver and support business outcomes as such so can security. I’d be thinking about the following linkages:

• Business Goals
• Business Outcomes
• Business Success Metrics
• Cyber Security Goals
• Cyber Security Outcomes
• Cyber Success Metrics

This doesn’t really fit into measurement, but I find creating diagrams, models, and visualizations important for both understanding and communicating. One way of doing this is with a benefit dependency network diagram:

This is relevant to measurement as you will want to ensure your measurements and reporting link back to business benefits and outcomes.

Summary

You might have come into this blog thinking you were going to walk away with a 3-step guide to measuring security, well I wish I had one, if we had to summarise measurement in an easy process it would be something like:

1. Baseline the current state
2. Measure Risk Exposure
3. Track Trends

The reality of cyber security is that it’s not simple, it’s not easy to measure and different people will have different views. Cyber security is about both reducing business risk and enabling the business, hopefully this post has helped people to see this in a different light and at least think about some of the perspectives, considerations and methods that can be leveraged. There’s lots of measurements that can be made, choosing the right ones to report on to the right people is often key to enabling the right security outcomes, and for that, I don’t have a cyber magic wand!