Metrics that Matter: Evaluating Cybersecurity Performance
Measuring the return on investment (ROI) for cybersecurity investments can be challenging because it involves quantifying the potential cost savings or value generated from mitigating cybersecurity risks. But it is essential to communicate the effectiveness of security measures, justify investments, and inform decision-making. When reporting cybersecurity performance to management, it’s essential to focus on metrics that provide meaningful insights into the effectiveness of security measures and their impact on the organization’s risk posture. When measuring cyber security performance, organizations typically use a combination of quantitative and qualitative metrics to assess the effectiveness of their security measures. Gartner is benchmarking 16 cyber business value metrics that change the way organizations measure, report and invest in cybersecurity. In this blog, let us look at each of those metrics.
1. Incident containment time
It may take considerable time, from days to months to contain and mitigate a security incident from the moment it is detected. Containment time measures the efficiency and effectiveness of an organization’s incident response capabilities. The shorter the incident containment time, the faster the organization can minimize the impact and prevent further damage. To measure it, all the activities such as incident detection, incident response initiation, investigation and analysis, containment and mitigation need to be measured. By subtracting the incident detection time from the incident closure time, organizations can calculate the incident containment time. It provides a measurable metric that reflects the organization’s incident response efficiency.
2. Incident remediation time
It refers to the duration it takes to fully remediate and resolve a security incident from the moment it is detected and contained. It measures the effectiveness and efficiency of an organization’s efforts to address the underlying issues that led to the incident and restore normal operations. Along with containment, this also calculates the time to plan remediation, execute the remediation, testing and validation of the implemented remediation measures until the incident is fully closed.
3. OS patching cadence
Organization applies security patches and updates to its operating systems. OS patching cadence is the frequency and regularity with which patches are updated. It is measured by tracking the time between the release of patches by the operating system vendor and their deployment on target systems within a defined patching period. The percentage of systems that receive patches within the expected timeframe is calculated to assess compliance with the patching cadence. Monitoring patching cadence helps ensure the timely application of critical security updates, reducing the risk of vulnerabilities and improving the overall security posture of the organization’s operating systems.
4. Third-party risk engagement
There should be a process of assessing and managing the potential risks associated with partnering or engaging with external vendors, suppliers, contractors, or service providers. It involves evaluating the security posture, data protection practices, and overall risk profile of these third parties to ensure they align with the organization’s security standards and regulatory requirements. Measuring third-party risk engagement involves evaluating the effectiveness of the due diligence process, assessing the level of compliance with established risk mitigation measures, tracking the number and severity of incidents or breaches involving third parties, and monitoring ongoing risk assessments and audits of third-party relationships. This measurement helps organizations gauge the effectiveness of their third-party risk management program and identify areas for improvement to mitigate potential risks introduced by external parties.
5. Unassessed third parties
There may be external vendors, suppliers, contractors, or service providers with whom an organization has engaged but has not undergone a formal assessment of their security and risk posture. These vendors may or may not have impact on the organization’s security and compliance. Measuring unassessed third parties involves identifying the number or percentage of external entities that have not undergone an assessment and implementing a system to track and monitor the progress of assessments and due diligence efforts. This measurement helps organizations identify gaps in their third-party risk management program, prioritize assessments, and ensure comprehensive coverage of all third-party relationships to mitigate potential risks.
6. Expired policy exception
There could be situations where an organization’s security policy is no longer valid or effective due to its expiration. This can occur when security policies are not updated or reviewed regularly, rendering them inadequate in addressing new threats and vulnerabilities. To measure the expired policy exception, organizations can conduct periodic policy reviews, assess policy compliance, and track policy expiration dates.
7. Endpoint protection coverage
It refers to the extent to which an organization’s endpoints, such as desktops, laptops, and mobile devices, are protected against various threats and vulnerabilities. It encompasses the deployment and effectiveness of endpoint security solutions like antivirus software, intrusion detection systems, and firewalls. Measuring endpoint protection coverage involves assessing the percentage of endpoints with active and up-to-date security software, conducting vulnerability scans and penetration tests, monitoring endpoint activity for indicators of compromise, and evaluating the organization’s ability to detect and respond to endpoint-based attacks. Regular audits and metrics tracking can help gauge the overall effectiveness and improvement of endpoint protection coverage.
8. Ransomware recovery exercise
It is a simulated practice scenario aimed at testing an organization’s ability to respond to and recover from a ransomware attack. It involves simulating an actual ransomware incident using controlled malware or mock attack scenarios and evaluating the effectiveness of the organization’s incident response plan, backup and recovery processes, and overall readiness to handle such an event. The exercise can be measured by assessing factors like response time, containment and eradication of the ransomware, successful restoration of affected systems from backups, effectiveness of communication and coordination among teams, and the identification of areas for improvement in the organization’s ransomware recovery capabilities.
9. Ransomware downtime workaround
Organizations may implement temporary measures to minimize the impact and downtime caused by a ransomware attack. It involves alternative processes, or systems, that allow essential operations to continue while affected systems are being isolated, restored or recovered. Measuring the effectiveness of a ransomware downtime workaround can be done by assessing the time it takes to implement the workaround, the percentage of critical operations that can be sustained during the attack, the level of disruption experienced by users or customers, and the overall cost and productivity impact during the downtime period.
10. Cloud security coverage
It refers to the extent to which an organization’s cloud environment is protected against security threats and vulnerabilities. It encompasses the deployment and effectiveness of security controls, policies, and procedures to safeguard data, applications, and infrastructure hosted in the cloud. Measuring cloud security coverage involves assessing factors such as the implementation of strong authentication and access controls, encryption mechanisms, network security measures, vulnerability management, and incident response capabilities specific to the cloud environment. Regular audits, security assessments, and penetration testing can help evaluate the overall effectiveness and improvement of cloud security coverage, ensuring a robust and secure cloud infrastructure.
11. Multifactor authentication (MFA) coverage
Organizations need to check the percentage of user accounts or systems within an organization that have MFA enabled as an additional layer of security beyond traditional passwords. It measures the extent to which MFA is implemented across various access points to protect against unauthorized access and credential theft. To measure MFA coverage, organizations can assess the number of user accounts with MFA enabled, the percentage of systems or applications supporting MFA, and the overall adoption rate of MFA among users.
12. Access removal time
Organizations need to revoke or terminate access privileges of a user or entity when they no longer require access to a system, application, or data on a timely manner. It measures the efficiency and effectiveness of the access management process in promptly removing access rights to prevent unauthorized access. Access removal time is typically measured by tracking the time between the identification of access termination needs and the actual removal or revocation of access privileges from the relevant systems or applications, ensuring that access is promptly and securely disabled to mitigate the risk of unauthorized activities or data breaches.
13. Privileged access management (PAM)
PAM focuses on securing and managing the elevated or privileged access rights granted to users, accounts, or systems. It involves implementing controls, processes, and tools to limit, monitor, and control privileged access to critical systems and sensitive data. PAM solutions help enforce the principle of least privilege, monitor privileged activities, facilitate secure password management, and enable efficient provisioning and deprovisioning of privileged access. Measurement of PAM effectiveness can be done by tracking the percentage of privileged accounts that are actively managed and monitored, the average time taken to detect and respond to suspicious privileged activity, and the adherence to PAM policies and procedures through regular audits and assessments.
14. Security awareness training
Training initiatives are designed to educate individuals within an organization about cybersecurity risks, best practices, and their roles and responsibilities in protecting sensitive information and systems. It aims to enhance employees’ knowledge and awareness of potential threats and equip them with the skills to identify and respond to security incidents. Measurement of security awareness training can be done by evaluating the completion rates and participation levels of training modules or sessions, conducting pre- and post-training assessments to gauge knowledge improvement, monitoring the number and types of security incidents or breaches caused by human error, and soliciting feedback through surveys or quizzes to assess the effectiveness and relevance of the training content.
15. Phishing training click-throughs
How employees of an organization respond to simulated phishing emails or training exercises is a measure of cyber security effectiveness. It involves tracking the number of employees who click on a simulated phishing email link or perform actions that could compromise security during training exercises. This metric helps organizations assess the effectiveness of their phishing awareness training programs. To measure phishing training click-throughs, organizations can conduct regular phishing simulations, track the number of employees who click on simulated phishing emails or engage in risky behaviour, and analyse the results to identify areas for improvement in training content, awareness, and reinforcement strategies.
16. Phishing reporting rates
It refers to the measurement of the percentage of employees who report suspicious or phishing emails to the designated security team or reporting mechanism. It indicates the level of employee awareness and engagement in identifying and reporting potential phishing attempts, which is crucial for timely incident response and mitigation. To measure phishing reporting rates, organizations can track the number of reported phishing emails compared to the total number of simulated or real phishing emails sent to employees. This metric also helps evaluate the effectiveness of security awareness training, incident reporting processes, and the overall security culture within the organization.
In conclusion, the metrics discussed in this conversation provide insights into different aspects of cybersecurity performance. Gartner also suggest defining these metrics for specific organizations by writing down target question (about the security measure), in-scope elements, clock (for time-based metrics), definitions, and calculations. By consistently monitoring and analysing these metrics, organizations can identify areas for improvement, enhance their cybersecurity posture, and mitigate risks effectively.
Talk to our experts to know more.
