Measure, Share, Compare.

"When you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is a meager and unsatisfactory kind; it may be the beginnning of knowledge, but you have scarcely, in your thoughts, advanced to the stage of science."
--William Thomson and Lord Kelvin, 1883

Learn about Metrics and Scorecards

Security Performance Metrics provide valuable insight into the posture of your information security initiatives, effectiveness of security processes, and compliance to regulations.  Metrics are measurements that translate security policies and controls into actionable results. A few examples of Security Performance Metrics include:

• What percent of my business critical assets have severe vulnerabilities? 
• What is the average time it takes to remediate vulnerabilities of low, medium and high severity? 
• How many hours of exposure to known vulnerabilities did our critical assets experience last month?
  

What is the difference between a measurement and a metric?

An example of a measurement is the number of virus incidents.  While quantitatively, the measurement tells you the volume of virus incidents, it does not tell you the severity of these incidents, where these incidents occur, or if these incidents are increasing in frequency over time.  Knowing the volume of virus incidents does little to tell you if these events are an anomaly or if corrective action needs to take effect immediately.

Metrics are derived from measurements, but enrich them with statistical analysis and contextual information.  Metrics allow you to make better decisions in relation to organizational goals and structure. A metric is an indicator for the effectiveness of your security processes and compliance to corporate policies and legislative regulations. 

An example of a metric is the ratio of virus incidents originating from contractors versus employees.  By analyzing the origin of virus incidents, an organization can better understand the effectiveness of their anti-virus controls for virus signatures and scans of their user-base and identify if policy or process changes need to be made for a particular user-base.

To provide value and meaning, metrics have specific additional requirements that go beyond simply being a measurement of their security environment.

What is the composition of a Security Performance Metric?

A metric must be: 

• Useful.
  There must be a legitimate business need to assess the state and performance of a topic so that a decision can be made.  The purpose of a metric is to facilitate the decision-making process.


• Repeatable.
  A metric must be repeatable so that the impact of actions and decisions can be seen over time. Repeatability also allows the results to be independently tested. 


• Flexible.
  Metrics should constantly evolve to adapt to the needs of an organization.  If the organization's strategy, policy, or process changes, so must the metrics that analyze these topics.


• Accurate.
  Since decisions will be made based on the results of the metric, the calculation method should be accurate and yield results in the same way each time. 


• Accessible.
  A metric is only useful if stakeholders that need the information can access it; therefore, the results of a metric should be readily available. 


• Contextual.
  A metric must be meaningful and relevant to an organization. It should address a topic that the organization cares about. 


• Transparent.
  The purpose, algorithm, and analytics of a metric should be auditable and evident to the audience to ensure that appropriate decisions can be made. It should be clear what the metric is quantifying and what action should be taken based on the results.

How are Security Performance Metrics used?

Metrics enable you to better understand the impact that decisions and changes to security processes have on your organization's security posture.  No security program is complete without metrics. Metrics help you to:

• Determine the goals of the program
• Evaluate the current posture and forward momentum towards those goals
• Identify when a corrective course needs to be implemented
• Signify when the goals have been obtained
• Optimize the level of security investment

Without metrics these questions can only be answered intuitively, cannot be easily communicated, and are difficult to manage throughout the organization.

How are metrics used in a scorecard?

A scorecard is a collection of related metrics organized in such a way as to show correlation between your security process, initiative, or goal and the outcome of these processes, initiatives, and goals.  The purpose of a scorecard is to demonstrate the state and security posture.  A scorecard answers the who, what, where, and when of a security objective. 

Since scorecards are an integral part of your security program, it's important to publish and review them on a regular basis.  Reviewing scorecards on regularly scheduled intervals means that the context of actions and decisions are not lost when the impact is assessed.

The level of granularity of a scorecard depends upon the audience.  Scorecards are generally reviewed at several levels within your organization and contain varying levels of detail:

• Board Summary Scorecards
  Evaluate progress on key company-wide security initiatives. Examples include encrypting all laptops that contain sensitive data or having security reviews on all significant systems. These goals are updated annually and progress is reported annually and quarterly. 


• Executive Summary Scorecards
  Evaluate progress of key initiatives, and include additional levels of detail for programs that show interesting results or where significant activity has occurred. Includes additional scorecards as needed to communicate metric results on specific issues. Quarterly reviews and monthly updates on key indicators. 


• Line of Business Manager Summary Scorecards
  Grouped by operational responsibilities, and include additional levels of detail for areas under, or affected by, their operations. Additional scorecards that track the impact of specific system, process, or organizational changes on their operation are also provided.  Monthly reviews with weekly updates on key indicators. 


• IT/Security Operational Manager Summary Scorecards
  Detailed functional-area scorecards evaluating the outcome of processes. Weekly review of process and state key indicators and monthly summaries to communicate issues, decisions, and resolutions.