Security Fundamentals Series

If you find that the traditional security concepts of confidentiality, integrity, and availability are sound, don't read any further.

If you think that these concepts have shortcomings, and you are looking for an alternative, check our Security Fundamentals series of presentations:

Optimizing ISO/IEC 27001 using O-ISM3

The Open Group published a guide entitled "Optimizing ISO/IEC 27001:2013 using O-ISM3" that will be of interest to organizations interested in taking ISO27001:2013 ISMS to higher maturity levels.

O-ISM3 brings continuous improvement to information security management, and it provides a framework for security decision-making that is top down in nature, where security controls, security objectives, and spending decisions are driven by (and aligned with) business objectives. We have for some time now heard from information security managers that they would like a resource aimed at showing how the O-ISM3 standard could be used in managing information security alongside ISO27001/27002.

This new guide provides specific guidance on this topic. We view this as an important resource, for the following reasons:

  • O-ISM3 complements ISO27001/2 by adding the "how" dimension to information security management.
  • O-ISM3 uses a process-oriented approach, defining inputs and outputs, and allowing for evaluation by process-specific metrics.
  • O-ISM3 provides a framework for continuous improvement of information security processes Some of the specific guidance to be found in the guide include these items:
  • Maps O-ISM3 and ISO27001 security objectives.
  • Maps ISO27001/27002 controls and documents to O-ISM3 security processes, documents, and outputs.
  • Provides a critical linkage between the controls-based approach found in ISO27001, to the process-based approach found in O-ISM3.

If you have interest in information security management, we encourage you to have a look at Optimizing ISO/IEC 27001 using O-ISM3.

Using O-ISM3 with TOGAF

In order to prevent duplication of work and maximize the value provided by the Enterprise Architecture and Information Security discipline, it is necessary to find ways to communicate and take advantage from each other’s work. We have been examining the relationship between O-ISM3 and TOGAF®, both Open Group standards, and have found that, terminology differences aside, there are quite a number of ways to use these two standards together. We’d like to share our findings with The Open Group’s audience of Enterprise Architects, IT professionals, and Security Architects in this article.

Checking Application Logs

Can you think of performing a Forensic analysis on a system with no records, no logs? Neither can I. Logs contain events like startup, restart, abnormal termination of services, physical and logical thresholds being exceeded, access to resources, network connections, privilege and access rights changes, configuration changes, etc. Logs are generated everywhere, in a multiplicity of formats, with different transports, API and formats. There are quite a few standards, for example:

  • CEE (MITRE initiative)
  • CEF (ArcSight)
  • Extended Log File Format (W3C)
  • WebTrends Enhanced Log file Format.
  • WSDM Event Format (OASIS)
  • XDAS – Distributed Audit Service (The Open Group)
  • RFC3164 – syslog (IETF)
  • Events Logging Markup Language

It would be interesting to be able to check if all the important event are considered as part of the requirements of the design of an application. This can prevent nasty surprises when analyzing an incident. Using a good model of an information system can make this task relatively easy. Such a model would model an information system using the following elements:

  • Repositories (Credentials): Any temporary or permanent storage of information, including RAM, databases, file systems, and any kind of portable media;
  • Interfaces: Any input/output device, such as screens, printers and fax;
  • Channels: Physical or logical pathways for the flow of messages, including buses, LAN networks, etc. A Network is a dynamic set of channels;
  • Borders define the limits of the system.
  • Services. Any value provider in an information system, including services provided by BIOS, operating systems and applications. A service can collaborate with other services or lower level services to complete a task that provides value, like accessing information from a repository;
  • Sessions. A temporary relationship of trust between services. The establishment of this relationship can require the exchange of Credentials.
  • Messages (Instructions) . Any meaningful information exchanged between two services or a user and an interface.

For a log entry to be complete it should contain at least the following elements:

  • Every event can have an eventID.
  • If the event is not logged by the agent of the event, the "logger" can be identified using a "loggerID".
  • The "agent" of the event can be identified using a "sourceID".
  • The "agent" of the event can stay in different locations, identified using a "addressID".
  • The "credential" used by the source to perform a request can be identified using a "credentialID".
  • The "resource" (subject) of the event is identified using a "resourceID".
  • The "request" (access attempt) performed has a "RequestType" and a "Result". The reason for the "Result" is stated in the "ResultText".
  • The "payload" contains the information necessary to perform the request.
  • "dateTime" is the date and time when the request is performed.
  • "signature" is the digital signature of the event using the "credentialID".
  • "hash" is the digital summary of the event. It is recommended that the hash of the previous event in the Record is used to calculate it.

For example:

<sourceID></sourceID><addressID></addressID><loggerID>slacker proftpd[25530]</loggerID><Result>success</Result><ResultTextFTP session closed. </ResultText><dateTime>21/5/2007 20:22:14</dateTime>>">>

<sourceID></sourceID><addressID<</addressID><credentialID>abad</credentialID><loggerID> proftpd[31806]</loggerID><RequestType>login</RequestType><Result>failure</Result><ResultText>no such user found</ResultText><dateTime>21/5/2007 20:21:21</dateTime>

Using this scheme, it is possible to check how complete is a log, by checking:

  • If events need an unique identifier, or even a digital signature or a hash.
  • If there is a need to distinguish the process performing the action from the process logging the event.
  • if there is a need to identify the origin (agent) of the action.
  • If there is a need to identify the logical or physical location of the origin (agent) of the action.
  • If there is a need to identify the credentials used by the origin (agent) of the action.
  • If there is a need to identify the resource that is being accessed by the the origin (agent) of the action.
  • If there is a need to identify nature of the action (RequestType) performed on the resource.
  • If there is a need to identify the result of of the action performed on the resource.
  • If there is a need to identify the date and time of the action performed on the resource.

You can find a list of types of request and results here:

Using this list of type of request can very useful, as the RequestType indicates what is the type of resource being accessed, making it easier to read the log.

How do you check if your log designs are complete and contain all the information you might ever need?

O-ISM3 Challenge (Now closed for participation)

The current state of affairs is that most information security professionals use the concepts of Confidentiality, Integrity and Availability for audits, consulting projects, risk assessment and management, and development of new standards. These concepts present a series of problems that have yet to be solved:

  • They're incomplete. Some professionals suplement them with concepts like Possession, Utility, Risk, Authentication, Authorization, Audit, Non-Repudiation and Accountability. This means performance and delivery vary greatly depending on what professional or company you use.
  • They're ambiguous. Many professionals and even published standards give different definitions of Confidentiality, Integrity and Availability. This adds more undesirable variance.
  • They are not operational. Consequently, Threats, Incidents, Vulnerability and Weakness among other concepts can't be reliably defined in terms of Confidentiality, Integrity and Availability reliably, increasing the ambiguity of definition. Even seasoned professionals don't agree on just what an Incident or a Vulnerability is. If you can't agree on what something IS, how can you manage it?
  • They don't have units of measurement. This makes it impossible to manage information security quantitatively. Bye, bye, optimization of resources.

The use of ambiguous, incomplete, not operational concepts without units of measurement has created a number of problems for information security management. Communication with between specialists and non-specialists in information security is difficult. Demonstrating the value of information security is difficult. Generally speaking, the use of these proxy concepts that don't add value makes information security management more difficult that it needs to be. Time is wasted, security projects that need funding don't get it, and trendy projects with little return get the green light. Luckily, change is possible.

Reasons to participate

  • To prove you skills as an information security consultant and win the prize.
  • To show off the worth of the professional certificate you hold and win the prize.
  • To prove that Confidentiality, Integrity and Availability are indeed fundamental an useful concepts.
  • To prove that Confidentiality, Integrity and Availability are neither necessary nor useful concepts.


The O-ISM3 Challenge

The O-ISM3 Challenge presents participants with a Use Case scenario (see below). The Use Case is a fictional Travel Agency in Madrid, Spain. Participants will act as information security consultant who need to determine what are the information security needs of the Travel Agency. Please note that there is a difference between finding out what the Travel Agency needs and what the Travel Agency might do regarding information security. If we were to compare the security practices of the Travel Agency with some standard, we could find out that the Travel Agency is not doing everything that a standard says can be done. There is a difference between doing everything that is standards state is possible, and everything that meets the needs of the business.

Determining the security needs of the Travel Agency will enable the consultant to determine the reasonable security measures to be applied, which are likely to be different, and cheaper, than all the security measures that could be taken.

In order to prove that they can successfully determine the security needs, the participants have to create a list of Questions to ask the Travel Agency managers or employees. This should be easy, since all the answers are included in the Use Case, and they will be listed individually in a spreadsheet that the participants will have to fill out.

Participants have a choice:

  • CIA Option: Questions can ONLY ask about Confidentiality, Integrity and Availability.
  • O-ISM3 Option : Questions can NOT ask about Confidentiality, Integrity or Availability.

Development of the Challenge

  • Participants can sign-up between the 28th of February and the 14th of March (CET Timezone).
  • To participate there is a registration fee of 5 euro payable using the"Register via PayPal" button below.
  • Communication between the participants and Inovement will be only through the mail address used to register and
  • The challenge is limited to 500 participants.
  • Every participant will receive a copy of a spreadsheet with the Answers and a registration number within 24h of registration.
  • In the spreadsheet, the participant must choose either the CIA Option or the O-ISM3 Option, and mention any information security professional certifications they hold.
  • Each participant must fill in Questions that would give the Answers provided and send the spreadsheet attached in an e-mail to, with their registration number in the subject of the mail.
  • Questions will be evaluated. A single ambiguous question, where the answer is not a perfect match causes the participant to FAIL the challenge (Example of FAIL: "What is the Integrity of the Data?· for the answer "Data needs to be kept for 5 years", example of PASS: "When does the system needs to be Available? for the answer: "Between 8 and 5 Monday to Friday") .
  • Grammatically, logically inconsistent questions, not in the English language, or longer than 255 characters or inclusion of malicious macros will result in a FAIL.
  • For the CIA Option failure to use at least one of Confidentiality, Integrity and Availability (or Confidential, Integer, Available) in any question will result in a FAIL.
  • For the O-ISM3 Option using any of Confidentiality, Integrity or Availability in any question will result in a FAIL.
  • All answers must be sent by midnight on 16 of March (CET Timezone).
  • Those who have PASSED will be announced on 24 of March. The best ones will be published.
  • The results of the entry evaluations may not be contested.
  • The names of the participants will not be published without their permission.
  • A 500€ prize and a free seat for an O-ISM3 Course will be awarded. The winner will be chosen from those who PASSED and will be given to the one with the most significant digits (two at least) in their registration number matching the first prize of Loteria Nacional of 29 of March (a random number). The prize will be paid within seven days via Paypal to the same account used to register.
  • The number of participants both the total number and those with correct answers for both Options will be published, along with an analysis of the results comparing how the CIA Option and the O-ISM3 Option fared, and what that means for the use of concepts like Confidentiality, Integrity and Availability.

If you want to get posted on the development of the Challenge, follow us on Twitter, or request begin added to our mail list.


If the number of participants who PASS using the CIA Option is higher than the number of participants who PASS using the O-ISM3 Option, this will indicate that Confidentiality, Integrity and Availability are valid concepts in order to analyze the security needs of a company like Ambiguous SL. If the number of participants who PASS using the O-ISM3 Option is higher than the number of participants who PASS using the CIA Option this will indicate that Confidentiality, Integrity and Availability are not particularly useful in order to analyze the security needs of a company like Ambiguous SL. As a secondary result, we will compare how different information security professional certificate holders performed.

The Use Case

Ambiguous SL is a travel agency located in Madrid, Spain. Their business is selling retail travel packages both online and through their offices, which are street level on a main street. The most important system they own and operate is the Package Sales System, which they use for advertising, sales, and bookings. This system interfaces with the Amadeus GDS system (checking availability and bookings), with VISA (payments), and with an equivalent system of a Moroccan partner (MTravel), as it is a popular destination for Spanish tourist and represents a significant part of the company's business.

The owner of Ambiguous SL has put Myrna in charge of IT, among other responsibilities. Myrna has hired you do find out which security measures (controls or processes) would provide the highest return of investment for Ambiguous SL. Myrna will take care of implementation. Your first (and only) task is to make an assessment of Ambiguous SL security needs.

Myrna has named Ignatius as the project manager for the Package Sales System. He is an employee of the company (Confederacy SL) that develops and maintains the Package Sales System for Ambiguous SL.

The Package Sales System functionality is as follows (please note this a Use Case, so it is simpler than a real life case):

  • Create, Modify and Delete Travel Packages.
  • Sell Travel Packages both online and at the office.
  • Receive feedback from customers and the public in general.
  • Send Travel Package offers to subscribers.
  • Manage Claims and Issues.

A high level view of the Package Sales System Database reveals the following data resources:

  • Travel Package Archive
  • Sales Archive
  • Feedback Archive
  • Offers Archive
  • Claims, Feedback and Incidences Archive

The following list of actions can be performed on each data resource:

  • Travel Package Archive: Create, Update, Retire, Publish, Unpublish.
  • Sales Archive: Book, Release, Sell, Refund, Update.
  • Feedback Archive: Create, Update, Close.
  • Offers Archive: Create, Update, Retire, Publish.
  • Claims, Feedback and Incidences Archive: Create, Update, Close
  • Sales Statistics Report Archive: Create, Close

There are certain requirements about who can do what, and where they can do it:

  • Only the sales manager can Create, Update and Publish Travel Packages.
  • Each salesperson can only view the personal information of his or her own clients.
  • Only the sales manager and the person assigned to Feedback and Claims can view the personal information of all clients.
  • Only the owner of the company can access the Sales Statistics Report.
  • Only the sales manager can create Offers

Certain parts of the Package Sales System are licensed, namely the Operating System, Application Server and Database.
As the company and systems are located in Spain, the Package Sales System needs to comply with the Spanish Privacy Law (LODP). Since the Package Sales System manages VISA payments, it needs to comply with PCI-DSS.

Some of the users of the Package Sales System are employees of Ambiguous SL, some are temps from Adecco. The administrators of the Package Sales System are employees of Confederacy SL. The general public of Spain is a user and they can purchase Travel Packages through the application. The application does not serve the public of countries other than Spain. Persons under the age of 18 can ask for feedback and signup for offers, but they can't purchase Travel Packages.

The system is located in a properly conditioned room inside the office. The system interfaces with Internet via a high speed fiber optic connection. The system interfaces with the interconnected systems and users via mail, file transfers and a VPN that connects directly with the MTravel network.

The system is expected to work 24x7, but because of maintenance stoppages of no more than one hour per week during no business hours (from 9 to 5 from Tuesday to Sunday) are acceptable. The longest time that the system can be offline during business hours is 2 hours, because sales can be performed with TPV and handwritten notes can partially replace the use of the system. In case of a major malfunction of the system, it would be acceptable to lose one day of data, since most data could be reconstructed checking with VISA, Amadeus and Mtravel. It is understood that all "live" transactions would be lost in case of an incident.
Data needs to be archived for 5 years in order to meet tax regulations. After ten years data should be deleted permanently, as customer behaviour changes over time and data is no longer useful for Business Intelligence.
Sales representatives and customers sometimes make mistakes entering data. This is acceptable as long as there is no more than one percent of the records contain innacurate information.

In order to create an account in the Package Sales System, potential clients can login using Facebook or create an account linked to their email address. They can unlink or delete the account at any time, but that does not delete any data in the database if they have purchased a Travel Package, even if they cancelled the purchase. In order to create an account in the Package Sales System, the Sales Manager sends an email to the Administrator. The email states what functions the user should be able to perform. The general public doesn't need an account to provide feedback or sign up for the Offers newsletter.

Customer who lose their passwords to the Package Sales Systemcan request a new one and a link will be sent to their email address. Users who lose their password to access the Package Sales System need to physically visit the Administrator, who resets the password and give it to them in a written note.

As some Offers expire at midnight, the Package Sales System should prevent customers from purchasing Travel Packages after they have expired, even by a few seconds.

There is a development environment, that Confederacy SL maintains in their own data center, and a pre-production environment, at Ambiguous SL office.
The current administrator is subscribed to email lists that notify him of security updates. The Administrator has configured the system using security guidelines found on Internet for every component. Security patches have not been applied since a patch caused a half day downtime.
The Administrator changes about once every six months.

The system has no malware protection.
The domain has been registered with The digital certificates used by the system are from Thawte. No one has been assigned with the responsibility to manage the domain or the certificates.
The systems logs all the sales activity, but not any other activity.
There is no Firewall. The internet connectivity provider (Telefonica) provides a service that is supposed to provide "clean" traffic.

No part of the Package Sales System is located in a publicly accesible location.
No part of the Package Sales System is accesible via Mobile application, but there are plans to incorporate a solution for this.
No part of the Package Sales System is exposed to extreme environmental conditions.

The O-ISM3 Challenge arises, the first security consultants competition

This initiative is launched by Inovement Spain and tries to catch the attention into the importance of the concepts used in security consultant’s activities

Inovement launches the O-ISM3 Challenge, an equivalent in the consulting arena to the popular “Capture the Flag” competitions of the ethical hacking scene. In order to solve the Challenge participants have to solve a Use Case where they play the role of a consultant who is in charge of finding the security needs of a company. In order to solve the Challenge, participants have the option of using traditional concepts like confidentiality, integrity, availability, or new concepts like O-ISM3’s security objectives. The purpose of the Challenge is comparing the success solving the Use Case of participants using either option, which are mutually exclusive.

The Challenge will develop between the 28th and 14th of March, and the winners will be published on the 24 of March. Among those who pass the test, a prize of 500 euro and a spot in an O-ISM3 course will be randomly assigned. Furthermore, the conclusions of the Challenge will be published, depending on the number of participants, their approach to solve the Challenge, and their comparative success solving the Challenge.

About Inovement
Inovement is a European consulting company. We provide consulting services in Security Architecture, IT governance, Risk Management, Information Security, Business Continuity and Compliance (ISO 27001, PCI-DSS, O-ISM3)

About O-ISM3
The Open Group Information Security Management Maturity Model (O-ISM3) Standard defines security processes for managing an enterprise’s Information Security Management System (ISMS). The O-ISM3 Standard places the onus on the business to define its required business security targets in its Security Policy, and then offers a set of security management processes from which the business selects which ones to deploy in a coherent ISMS. Each security control process in the ISMS then returns metrics to indicate how well that process is contributing towards achieving the business's security targets.
The O-ISM3 Standard metrics feedback is a major differentiating feature compared with other ISMS systems, because it enables the ISMS manager to present quantitative evidence (as opposed to qualitative subjective judgments) to show:

  • Which security control processes are revealing IT operational areas that are under-achieving regarding security targets
  • Which processes need to be tuned to improve performance to achieve or exceed critical targets
  • Which processes are not contributing sufficient to justify continuing to use them

The ISMS manager is thereby informed on which processes to retire and which to add to their ISMS. (More) Importantly, they are also informed with metrics that they can report as objective evidence to their CxO-level management on how well their ISMS is performing (based) on their security targets, and how effective their investment in security is.

O-ISM3 Risk Assessment Webminar

I introduced the O-ISM3 Risk Assessment Method and SpreadSheet. We learnt how to model the Business, Model the Information Technology, the dependencies between them, the Threat level, the Protection level, arriving at a Qualitative evaluation of the Risk, using the SpreadSheet Tool.

Security Metrics

A metric is a quantitative measurement that can be interpreted in the context of a series of previous or equivalent measurements. Metrics are necessary to show how security activity contributes directly to security goals; measure how changes in a process contribute to security goals; detect significant anomalies in processes and inform decisions to fix or improve processes. Good management metrics are said to be S.M.A.R.T:

  • Specific: The metric is relevant to the process being measured.
  • Measurable: Metric measurement is feasible with reasonable cost.
  • Actionable: It is possible to act on the process to improve the metric.
  • Relevant: Improvements in the metric meaningfully enhances the contribution of the process towards the goals of the management system.
  • Timely: The metric measurement is fast enough for being used effectively.

Metrics are fully defined by the following items:

  • Name of the metric;
  • Description of what is measured;
  • How is the metric measured;
  • How often is the measurement taken;
  • How are the thresholds calculated;
  • Range of values considered normal for the metric;
  • Best possible value of the metric;
  • Units of measurement.

Security Metrics are difficult to come by
Unfortunately, it is not easy to find metrics for security goals like security, trust and confidence. The main reason is that security goals are “negative deliverables”. The absence of incidents for an extended period of time leads to think that we are safe. If you live in a town where neither you nor anyone you know has ever been robbed, you feel safe. Incidents prevented can’t be measured in the same way a positive deliverable can, like the temperature of a room.

Metrics for goals are not just difficult to find; they are not very useful for security management. The reason for this is the indirect relationship between security activity and security goals. Intuitively most managers think that there is a direct link between what we do (which results or outputs) and what we want to achieve (the most important things: our goals). This belief is supported by real life experiences like making a sandwich. You buy the ingredients, go home, arrange them, and perhaps toast them and voilá: A warm sandwich ready to eat. The output sought (the sandwich) and the goal (eating a home made sandwich) match beautifully.

Unfortunately, there is no direct link every time. A good example can be research. There is not direct relationship between goals (discoveries) and the activity (experiments, publication). You can try hundreds of experiments and still not discover a cure for cancer. Same thing happens with security. The goals (trust, confidence, security) and the activity (controls, processes) are not directly linked.

When there is a direct link between activity and goal, like the temperature in a pot and the heat applied that pot, we know what decision to take if we want the temperature to drop: stop applying heat But, how will we make a network safer, adding (more accurate filtering), or summarizing (less complexity) filtering rules? We don’t know.  If a process produces dropped packets, more or less dropped packets won’t necessarily make the network more or less secure, just like a change in the firewall rules won’t necessarily make the network safer of otherwise.

The disconnect present in information security between goals and activity prevents goal metrics from being useful for management, as you can never tell if you are closer to your goals because of decisions recently taken on the security processes.

Goal metric examples:

  • Instances of secret information disclosed per year. What can you do to prevent people with legitimate access to disclose that information.
  • Use of system by unauthorized users per month. What can you do to prevent people from letting other users to use their accounts.
  • Customers reports of misuse of personal data to the Data Protection Agency. Even if you are compliant, what can you do to prevent a customer to fill a report?
  • Risk reduction per year of 10%. As risk depends on internal an external factors, what can you do to actually modify risk?
  • Prevent 99% of incidents. How do you know how many incidents didn’t happen?

Actually useful security metrics
If metrics for goals are difficult to get, and are not very useful; what is a security manager to do? Measuring process outputs can be the answer. Measuring outputs is not only possible but very useful, as outputs contribute directly or indirectly to achieve security, trust and confidence. Using output metrics you can:

  • Measure how changes in a process contribute to outputs;
  • Detect significant anomalies in processes;
  • Inform decisions to fix or improve the process.

There are seven basic types of process output metrics:

  • Activity: The number of outputs produced in a time period;
  • Scope: The proportion of the environment or system that is protected by the process. For example, AV could be installed in only 50% of user PCs;
  • Update: The time since the last update or refresh of process outputs.
  • Availability: The time since a process has performed as expected upon demand (uptime), the frequency and duration of interruptions, and the time interval between interruptions.
  • Efficiency / Return on security investment (ROSI): Ratio of losses averted to the cost of the investment in the process. This metric measures the success of a process in comparison to the resources used.
  • Efficacy / Benchmark: Ratio of outputs produced in comparison to the theoretical maximum. Measuring efficacy of a process implies the comparison against a baseline.
  • Load: Ratio of available resources in actual use, like CPU load, repositories capacity, bandwidth, licenses and overtime hours per employee.

Examples of use of these metrics:

  • Activity: Measuring the number of new user account created per week, a sudden drop could lead to detecting that the new administrator is lazy, or that users started sharing user accounts, so they are not requesting them any more.
  • Scope: In an organization with a big number of third party connections, measuring the number of connections with third parties protected by a firewall could lead to a management decision not to create more unprotected connections.
  • Update: Measuring the update level of the servers in a DMZ could lead to investigating the root cause if the level goes above certain level.
  • Availability: Measuring the availability of a customer service portal could lead to rethinking the High Availability Architecture used.
  • Efficiency / Return on security investment (ROSI): Measuring the cost per seat of the Single Sign On systems of two companies being merged could lead to choose one system over the other.
  • Efficacy / Benchmark: Measuring backup speed of two different backup systems could lead to choose one over the other.
  • Load: Measuring and projecting the minimum load of a firewall could lead to taking the decision to upgrade pre-emptively.

There is an important issue to tackle when using output metrics; what I call the Comfort Zone. When there are too many false positives, the metrics is quickly dismissed, as it is not possible to investigate every single warning. On the other hand, when the metric never triggers a warning, there is a feeling that the metric is not working or providing value. The Comfort Zone (not too many false positives, pseudo-periodic warnings) can be achieved using an old tool from Quality Management, the control chart. The are some rules used in Quality Management to tell a warning, a condition that should be investigated from a normal statistical variation (Western Electric, Donald J. Wheeler's, Nelson rules), but for security management the best practice is adjusting the multiple of the standard deviation that will define the range of normal values for the metric until we achieve the Comfort Zone, pseudo-periodic warnings without too many false positives.

Using Security Management Metrics
There are six steps in the use of metrics: measurement, representation, interpretation, investigation and diagnosis.

Measurement: The measurement of the current value of the metric is periodic and normally refers to a window, for example: “9:00pm Sunday reading of the number of viruses cleaned in the week since the last reading” Measurements from different sources and different periods need to be normalized before integration in a single metric.

Interpretation: The meaning of a measured value is evaluated comparing the value of a measurement with a threshold, a comparable measurement, or a target. Normal values (those within thresholds) are estimated from historic or comparable data. The results of interpretation are:

  • Anomaly: When the measurement is beyond acceptable thresholds.
  • Success: When the measurement compares favourably with the target.
  • Trend: General direction of successive measurements relative to the target.
  • Benchmark: Relative position of the measurement or the trend with peers.

Incidents or poor performance take process metrics outside normal thresholds. Shewhart-Deming control charts are useful to indicate if the metric value is within the normal range, as values within the arithmetic mean plus/minus twice the standard deviation make more than 95.4% of the values of a normally distributed population. Fluctuations within the “normal” range would not normally be investigated.

Investigation: The investigation of abnormal measurements ideally ends with identification of the common cause, for example changes in the environment or results of management decisions, or a special cause (error, attack, accident) for the current value of the metric.

Representation: Proper visualization of the metric is key for reliable interpretation. Metrics representation will vary depending on the type of comparison and distribution of a resource. Bar charts, pie charts and line charts are most commonly used. Colours may help to highlight the meaning of a metric, such as the green-amber-red (equivalent to on-track, at risk and alert) traffic-light scale. Units, the period represented, and the period used to calculate the thresholds must always be given for the metric to be clearly understood. Rolling averages may be used to help identify trends.

Diagnosis: Managers should use the results of the previous steps to diagnose the situation, analyse alternatives and their consequences and make business decisions.

  • Fault in Plan-Do-Check-Act cycle leading to repetitive failures in a process -> Fix the process.
  • Weakness resulting from lack of transparency, partitioning, supervision, rotation or separation of responsibilities (TPSRSR) -> Fix the assignment of responsibilities .
  • Technology failure to perform as expected -> Change / adapt technology.
  • Inadequate resources -> Increase resources or adjust security targets.
  • Security target too high -> Revise the security target if the effect on the business would be acceptable.
  • Incompetence, dereliction of duty -> Take disciplinary action.
  • Inadequate training -> Institute immediate and/or long-term training of personnel.
  • Change in the environment -> Make improvements to adapt the process to the new conditions.
  • Previous management decision -> Check if the results of the decision were sought or unintended.
  • Error -> Fix the cause of the error.
  • Attack -> Evaluate whether the protection against the attack can be improved.
  • Accident -> Evaluate whether the protection against the accident can be improved.

What management practices become possible?
A side effect of an Information Security Management System (ISMS) lacking useful security metrics is that security management becomes centered in activities like Risk Assessment and Audit.  Risk Assessment considers assets, threats, vulnerabilities and impacts to get a picture of security and prioritize design and improvements while Audit checks the compliance of the actual information security management system with the documented management system with an externally defined management system or an external regulation. Risk Assessment and Audit are valuable, but there are more useful security management activities like monitor, test, design & improvement and optimization that become possible with output metrics. Theses activities can be described as follows:

  • Monitor—Use metrics to watch processes outputs, detect abnormal conditions and assess the effect of changes in the process.
  • Test—Check if inputs to the process produce the expected outputs.
  • Improving -  Making changes in the process to make it more suitable for the purpose, or to reduce usage of resources.
  • Planning - Organizing and forecasting the amount, assignment and milestones of tasks, resources, budget, deliverables and performance of a process.
  • Assessment -  How well the process matches the organization's needs and compliance goals expressed as security objectives. How changes in the environment or management decisions in a process change the quality, performance and use of resources of the process; Whether bottlenecks or single points of failure exist; Points of diminishing returns; Benchmarking of processes between process instances and other organizations. Trends in quality, performance and efficiency.
  • Benefits realisation. Shows how achieving security objectives contributes to achieving business objectives, measures the value of the process for the organization, or justifies the use of resources.

While audits can be performed without metrics, monitoring, testing, planning,  improvement and benefits realisation are not feasible without them.

What needs to be done?
S.M.A.R.T security managers need metrics that actually help them performing management activities.

While it is not necessary to drop goal metrics altogether, the day to day focus of information security management should be on security monitoring, testing, design & improvement and optimization using output metrics, which are the ones which will show what are the effect of management decisions, if things are getting worse or better, if processes work as designed, and if there are changes out of our direct control that cause abnormal conditions in security processes. All these activities are perfectly feasible using outputs metrics and control charts.

SABSA mapped to O-ISM3

Enterprise Architecture is a very effective approach to undestand with detail how the success of an organization depends on Information Technology. In order to leverage the use of SABSA and TOGAF, The Open Group recently published Combining The Open Group Standards, O-ISM3 and TOGAF®, with the SABSA® Framework

In order to help SABSA and O-ISM3 practicioners use both standards, Inovement Spain and Seccuris just published this mapping of both standards:

Swiss Armed Forces using ISM3 - Aligning Security and Business

by Lars Minth 29.03.2011

The usage of ISM3 within the Information Assurance program of the Swiss Armed Forces is threefold:

  • first there is the necessity to comply with a couple of regulations inter alia the ISO 27k family, ISO 31000 and ISO 20000.
  • then the development of measurable and achievable security processes is very demanding in such a high security environment while the pure implementation of ISO 27k, especially its controls, is not sufficient to prove a Return of Security Investment (ROSI)
  • at last the governance of security in a highly decentralized organization needs a clever structuring.

Basically in all of these action areas ISM3 is giving us a helping hand and therefore saving us time and money to develop an own interpretation of ISO 27k. ISM3 came into the focus of the Swiss Armed Forces during a study about a business-driven implementation of an ISMS in order to regain management attention and acceptance for the restructuring of security. ISM3 itself is not a new invention but a straight forward and enabling approach to comprehend existing security frameworks in order to make security understandable for the rest of the (business) world. During the long process of aligning diverse security initiatives within the Swiss Armed Forces ISM3 is and will be the central repository and helping cornucopia to establish security processes which are measurable, acceptable and achievable in the sense of ROSI. The methodology ISM3 provides is helping to achieve ROSI while the ISM3 security processes in detail are helping to focus on the servicing and main-tenance of security at all levels: operational, tactical and strategic.


Subscribe to O-ISM3 Consortium Blog RSS