Bankia’s experience with O-ISM3

CajaMadrid (now Bankia) started using O-ISM3 in the security process of ethical hacking of systems and applications. Several enhancements were made in order to measure the metrics of this process, and a Service Level Agreement was established based on those metrics. Using O-ISM3 criteria, the classification of information systems, which determines how frequently the systems are tested, was improved.

As a result of O-ISM3 implementation, the team’s productivity doubled during the first year. The follow-up reports with metrics made collaboration between developers, system administrators and security personnel easier and more productive. The information available is so detailed that CajaMadrid now uses objective criteria to give an award to the manager whose applications or systems present the fewest vulnerabilities, and vulnerabilities that are found are fixed faster.

The methodology’s orientation on deliverables makes the entire evidence-generating activity possible, which means that it is auditable, measurable and manageable. As the management system is metrics based, daily operations do not require audits for improvement, speeding up the improvement cycle significantly. When metrics correlate well with pursued goals, process improvement leads directly to the achievement of goals with greater effectiveness, efficiency and quality. Metrics make the status and progress of activities clearly visible, making it easier to reach agreements with our internal client and partners, and communicate our achievements to upper management. Ultimately, this methodology allows information security to be managed using best practices that apply to business in general. Based on the success of the methodology, we are extending the use of the method to other processes.

This is what the CISO had to say: "We are very interested in the method, and once it has been set up, it is simple to use.”

Bankia is ISO27001 and ISO20000 certified, and uses CMMI-3 and standards like ITIL, OSSTMM and many others. O-ISM3 integrates seamlessly with all of them and we have even received a positive note about our ISMS certification thanks to ISM3.

Compliance vs Continuous Improvement

In this video the pros and cons of the Compliance approach and the Continuous Improvement approach are weighed.

Most ISMS standards emphasise Risk Assessment and Audit. These management practices leave other information security management practices in shadows, which is specially unfair if you consider the limitations risk assessments face. Creating a risk assessment method is very easy, as you can make many choices:

  • The scope (what's in, what's out)
  • The depth (think OSI levels and above to business processes)
  • The way you model the parts/objects of the organisation, their relationships, and the states of their life cycles.
  • Your threat taxonomy (there is not a single one widely accepted one at all depth levels)
  • The way you score the impact on assets (dollars, high-medium-low or 1-5 Confidentiality, Integrity, Availability scales and expansions or combinations thereof)
  • Controls taxonomy (there is not a single one widely accepted one at all detail levels. Many use the ISO27001 list)
  • How you combine threats, their probability, controls, their quality and impact to reach a Risk figure.

The multiplicity of risk assessment methods and standards  makes exceedingly difficult to reuse or compare risk assessments, problem compounded with changes in the method design or even the way it is used. Very seldomly it is possible to compare this year's RA with the last years one, and comparing RA from different companies becomes an unattainable Saint Grail. A good risk assessment standard should meet the following criteria:

  • Reproducible. This means two different independent practitioners should get virtually the same work products and results.
  • Productivity (Added value) This means the work products should serve as inputs for:
  • Gauge how safe is the organisation;
  • Identify threats and weaknesses;
  • Choosing what processes are appropriate for fulfilling the security objectives;
  • Prioritising investment in security processes;
  • Quantifying investment in security processes.
  • Cost-effectiveness. Setting up a ISM system should be cheaper than operating it, just like the cost of choosing a security tool should be small in comparison with the cost of purchasing and using the tool.
  • Added value. This means the result of the process selection should be learnt from the process selection itself. If the process selection result is known beforehand, and the process selection is just a justification for a previously taken decision, the added value is nil,which negates any cost-effectiveness.

O-ISM3 considers  the following management activities:

  • Risk Assessment (part of GP-3) - Considers assets, threats, vulnerabilities, impacts to get a picture of security and prioritise design and improvements.
  • Audit. Using the GP-2 ISM System and Business Audit process, checks are made on whether the process inputs, activities and results match their documentation.
  • Certify: Certification it evaluates whether process documentation, inputs, outputs and activities comply with a pre-defined standard, law or regulation. The certificate provides independent proof of compliance that third parties can trust. This practice is also performed using GP-2 ISM System and Business Audit.

Additionally, and in equal footing:

  • Testing. Assessment of whether process outputs are as expected when test data is input. This is an aspect of TSP-4 Service Level Management.
  • Monitoring. Checking whether the outputs of the process and the resources used are within normal range for the purpose of detecting significant anomalies. This is also performed using TSP-4 Service Level Management.
  • Improving. Making changes in the process to make it better fit for the purpose, or to lead to a saving in resources. The removal of faults before they produce incidents, bottlenecks that hamper performance and making trade-offs are examples of process improvements.. This management practice needs information gained from evaluating, testing or monitoring the process. The gains from the changes (if any) can be diagnosed with subsequent testing, monitoring or evaluation. GP-3 ISM Design and Evolution provides a framework for monitoring.
  • Planning. Organising and forecasting the amount, assignment and milestones of tasks, resources, budget, deliverables and performance of a process. This is performed using TSP-4 Service Level Management.
    Evaluation. Required periodically to assess the outcomes of the ISM system.
  • Assessment. Using the GP-3 ISM Design and Evolution process, the following areas are assessed:
  • How well the process matches the organisation's needs and compliance goals expressed as security objectives.
  • How changes in the environment or management decisions in a process change the quality, performance and use of resources of the process;
  • Whether bottlenecks or single points of failure exist;
  • Points of diminishing returns;
  • Bench marking of processes between instances and other organisations.
  • Trends in quality, performance and efficiency.
  • Benefits realisation. Shows how achieving security objectives contributes to achieving business objectives, measures the value of the process for the organisation, or justifies the use of resources. This is performed using TSP-4 Service Level Management

So, do Audit, and assess your risks, but don't let this drain all you energy from real management.