I was talking to a friend about a requirement to “measure” cyber essentials compliance. Now if you know a thing or two about standards and applying standards to complex technology environments you might come up with:

• Can’t we just script a checker?
• Don’t we have all the audit data in the *checks notes* 1000 inventory systems we have?

Well sure, you could write a massive set of rules which ignore any context and try and cater for a huge number of different scenarios. You could use the Q&A approach as well (which is how the standard workbook works anyway so that already exists). But let’s say you are an IT manager, and you want to KNOW how your environment stacks up!

The question is simple, it’s easy to ask, look:

• “How compliant are we against Cyber Essentials?”

See easy peasy, but the answer.. oh now there is a more complicated question. Why is it so complicated Dan? Well glad you asked (ok weird talking to myself blog, let’s roll with it!). It’s complicated beacuse of this reality.

Each device has a range of configuration models e.g. domain joined, workgroup, remote, agent managed, BYO etc. and each device class (PC, Server, Router, Switch, Printer) has different interfaces for interogation and management. Combine this with contextual questions such as: is all software authorised? are all firewall rules authorised with a documented business case? Are all critical patches deployed within 14 days of vendor release? Have default passwords been changed?

Now let’s add this auditing matra into it:

• Tell me
• Show me
• Prove it!

You hopefully are starting to see that diveristy of technology, deployment options, ease of access, phyiscal and logical limitations etc. paint a picture where it’s not se easy to “simply measure”.

So how do we attempt to measure the current state?

Well to do that i would suggest you need a range of activiites which inlude:

• Interviews
• Workshops
• Evidence Review
• Configuration Reviews
• System Reviews

You can use tools to suppor this but the process of assessment is a human one. Context is king and data alone doesn’t provide that. However, if anyone has tried to not only review but remiate an environment to meet a standard (or simpy the standard of good practise) you will know that it takes time, engergy, comms, planning and technical skills. To help support this process I’ve made a dashboard and tracking tool. It’s not a like for like reproduction of every CE requirement, it’s an abstraction designed to support mangement and tracking but also by being simple enough to not create an incredibly compex tracking overhead.

And because this takes time, we have a tracker over time to measure progress! Also don’t forget you might find more assets as you go, or configurarions may fall backwards against the standard! so we’ve got a way of tracking this:

So there you go, some ideas about how to track against standards in a meaningful manner without tracking every registry key, policy, mdm configuration etc.