Choosing the Risk Framework with the Best Fit

February 20th, 2012

I was reviewing the changes to NIST SP 800-30 today, and have been discussing with several colleagues how one chooses the right risk framework given that there are so many known flaws or limitations with the ones commonly used today.  That got me thinking about what an organization who is just starting out needs to do day 1 to get their program off the ground and showing value.  Otherwise they are never going to get the opportunity to implement a mature or elaborate program.

The following are some initial thoughts on that initial selection process as part of building a program from scratch, but making the best use of what is out there knowing it isn’t even close to perfect:

  1. Matching the company culture / industry
  2. What exists already?
  3. Map to business’ strategic objectives
  4. Articulate the organization’s risk tolerance

When looking at the ‘frameworks’ that are commonly used in information security circles, you will find that they all came about to address slightly different problems, and therefore may be more suited to one environment or another.  For example, the OCTAVE framework’s emphasis on workshops, interviews, collaboration, and detailed worksheets might not be a good fit for a federal government agency, but it might be well suited for a software development company.

Beyond the frameworks themselves, it is essential to consider what risk processes and practices already exist within your organization.  You might look to the operations, legal, insurance, finance, business continuity, compliance, business strategy, or human resources teams to identify if they have a framework for risk assessment or management already.  If something does already exist, be sure to first determine if it can be expanded to meet your needs or how the information security risk frameworks would interoperate with models in other business units.

The next step before you select a framework is to look at the core objectives of your organization.  Make a list of your company’s annual strategic objectives, read the mission statement or vision statement, and look where the company is investing resources most heavily.  This is an important activity, because it can inform the style of framework that works best for you.  For example, if your organization is expanding into international markets you may want to consider the ISO framework because it will be more accepted than NIST outside the U.S.  Similarly, if your organization may have new regulation on the horizon, you would want to look at the frameworks that are recommended by that regulator to avoid unnecessary audit challenges later.

Finally, before you select any framework or even a risk model, you need to guide the executive team through the exercise of articulating the organization’s risk threshold.  Have them describe in words only, what level of risk they want escalated to them.  For example, if the organization’s mission includes a focus on being highly available to clients, then this might be their risk tolerance statement:

“At a minimum, any risk that is likely to result in service outage for all clients longer than the published recovery time objective should be escalated to the executive management team.”

Then you would want to select a framework that could articulate risks in a way that would allow you to compare each risk to this tolerance statement easily.  The framework would need to include the same risk factors that your organization is focused on.

If this topic interests you, you may want to check out my upcoming session at RSA Conference 2012 later this month: Taking Information Security Risk Management Beyond Smoke & Mirrors (GRC-107).  You can get a feeling for the content of the session by listening to this podcast I did last month: GRC-107 Sneak Peak.

Stop Asking for a Copy of My Pen Test Report

December 31st, 2011

Clearly vendor due diligence reviews are an essential part of any risk management program, but the consistency and quality of the assessments being done today are really appalling.  Anyone who is working for a service provider today is probably getting inundated with requests ranging from a 50 item checklist of yes/no type questions all the way up to requests to review internal policies and procedure documents, or even an on site walk-through.  You might think that the variety of request types and levels of detail demonstrates that the customers are taking the time to rank each  provider based on the sensitivity of the service, and assess them accordingly, but sadly when you read through the due diligence requests it becomes obvious that most organizations have no clue how to assess their vendors.  Many have clearly just redistributed their own internal risk assessment questionnaires and don’t even bother to remove the internal acronyms and references.  Others clearly don’t know where to focus and try to audit every possible control one might implement in an environment.  This brings up a fundamental point, risk assessments and audits are not the same, but when you have people running a risk management program who don’t understand the distinction, you get a lot of activity with little actionable results.  The intention of a vendor risk review is to gauge whether that service provider presents an unacceptable risk to your organization, not to document every practice, procedure, and technology that may differ from your own environment or an industry framework like ISO.

Some of the worst offenders will waste your and their own time following up on questions about disclaimer banners on systems or why you only prevent the reuse of the last six passwords instead of the last twelve.  You can’t tell me that they are prioritizing based on risk!

Now a growing trend is to collect volumes of documentation like an auditor would, but this is problematic in many ways.  The first concern is whether it puts the service provider at risk to provide this data to all their clients.  In a shared service provider model in particular, am I really going to give you a copy of all my firewall rulesets?  If I do, then I think that should be the risk.  Same with the increasing demand for copies of penetration test reports.  These are some of the most sensitive documents an organization has, and we are supposed to just hand them over to every client just because there is an NDA in place?  Given that these reports are basically step by step instructions for how to compromise an application or environment, it really doesn’t seem responsible to be distributing them.  Instead of asking for the details on every single vulnerability found, you should be assessing the frequency and scope of the testing being performed, and ask for metrics on remediation of the highest risks.  There are plenty of ways to assess the quality of a security program without knowing the particulars of every vulnerability that has been found.  It comes back to a basic discussion about meaningful metrics.  You can certainly ask all your providers how many critical and high risk vulnerabilities have been found and not yet remediated, but that is pretty meaningless unless the answer is zero.  And even then that really doesn’t tell you much about your risk posture tomorrow, or next month.  If you want to assess a provider’s risk posture, ask for metrics on turnaround time between identifying a high or critical vulnerability and remediating it.  Or take it one step farther and ask them for their metrics around the time to implement some kind of mitigating control once a critical risk has been identified.  If these metrics aren’t good, or they have never gathered them, then you know something about the maturity of their program that is more than a point in time vulnerability discussion.

Of course in a dedicated provider model, this level of audit might be appropriate, but organizations need account for the differences in the service provider model and realize that they give up something for the cost savings of shared services.  Same with requesting copies of detailed network diagrams or the names of technologies being used.  You need to assess a vendor’s maturity and process, not every single control possible.

Another issue with the “give me everything” documentation requests is that is distracts the assessor from finding the real significant risks.  If a clients asks for a copy of every single security policy, standard, and procedure document, how can they possibly digest all that information without a team of analysts, and is it even worth it?

Organizations need to take a hard look at their practices for performing vendor risk reviews, and be honest about whether they are really focusing on risk or turning it into a blind checklist based audit.  Frankly, some of the due diligence requests I have seen from organizations recently make me start to question their own internal program’s effectiveness.

Shrinking Training Budgets

February 27th, 2011

One of the alarming trends that I have heard lately is that budgets for Information Security seem to be recovering from the economic slump, but training budgets are continuing to shrink.  We have always suffered from the artificial distinction between companies setting aside money for higher education that can’t be touched, versus money for training which is some of the first areas for cuts when times get tough.  I talk to a lot of young professionals who are frustrated because they can’t get money from their companies for training directly related to their job unless it is part of a degree program.  I have always been a proponent of higher education, but I have never really understood the chinese wall between funds set aside for college education and professional training.  Higher education isn’t for everyone, and professional skill development should never be discouraged if it is related to your current position.  Even many of the most accepted industry frameworks of Information Security emphasize keeping the skills of your team up to date.  Of course professional training courses aren’t always the answer, on the job training and other free training opportunities are available, but eventually you will want to invest in some commercial training courses.

Earlier this month at the RSA Conference I attended an interesting professional development session titled “The CISO of The Future: Building A Competitive Skill Matrix.”  The presenter, Lee Kushner, spoke about several aspects of the CISO position and how to position yourself for it.  The point that made the biggest impression on me was “If you do not invest in yourself, do not expect anyone else to.”  Although there might not be equal funding in organizations for professional training, you really do have to take your career into your own hands sometimes.  Just relying on your employer to fund all your training and certification options could end up setting you back several years in your career development plan.  Good technical training isn’t cheap, and maintaining certifications can also get expensive, but if you choose the right ones it will be worth it in the end.  The trick is making the most of your training dollars.  Look for options that don’t require travel and register early to receive discounts.  For example, the SANS Northern Virginia conference offers a wide selection of Security, Audit, and Management courses taught by some of the top experts in the field, and if you register before March 2nd you can get $400 off the normal cost.  Many conferences and courses will even offer discounts if you register with your co-workers as a group.

Another disturbing trend that I have seen is the “get a CISSP for everyone” mentality.  Imagine a Windows system administrator comes to you with an interest in information security.  That is a good moment, right?  What I am seeing is that everyone’s first reaction is to send them down the CISSP path.  What value does the CISSP have for a system administrator?  Sure it gives you a good management level overview of security domains, but it isn’t going to give that administrator one single skill that he/she can apply in their daily work.  I would really like to see more Information Security programs sponsoring targeted security training for staff outside of the security team.  For example, the Northern Virginia conference has a great course, Virtualization Security, which would be much more appropriate for a systems administrator than a generic CISSP.

The rise in the number of CISSPs is happening at an alarming pace.  For someone who specializes in information security, the CISSP is on its way to becoming the equivalent of a high school diploma.  Not having one raises questions, but having one doesn’t really prove anything about your abilities as a security professional.  With so many CISSPs (over 70,000) it is going to dilute the value of the certification eventually.  In a way this is good because it will open the door for more specialized certifications, but as a community we need to stop recommending the CISSP to anyone who shows an interest in security.  I would rather have my operational teams learning the skills that they can use everyday to protect our organization, than trying to memorize some esoteric details from ten domains of information security.

If you want to voice your own opinions about the value of security certifications, you might want to check out the survey being run by the Information Security Leaders website.  This survey seems like it has the potential to provide a good independent assessment of the value of certifications.

Future Security Leaders at the RSA Conference

February 17th, 2011

Back in November, I mentioned my concern about the lack of educational opportunities for future security leaders, and I am thrilled to report that we have started to change that this year at the RSA Conference.  Anyone who has been to the RSA Conference before knows that it officially kicks off on Tuesday each year, but the number of pre-conference options has been growing each year.  This year they added a new session in the Professional Development track, Building and Managing a Successful Information Security Program, which was very highly attended (450 attendees, not bad for 8:30am).  As a speaker (and organizer for the morning series), I was surprised by the attendance, which validated the premise for the sessions: there are no formal educational forums for aspiring security leaders.  Basically when you get the position of leading a security program for the first time, you have to figure it out as you go.  Wouldn’t it be nice to learn from the mistakes that have already been made by past leaders?

I thought that I would share a few points that I found most interesting from the sessions.  When someone joins an organization as a CISO or CSO, the tendency is to jump right in and start fixing things, but the reality is that it is far more important to spend the first six months on the job listening and profiling the organization.  Sure you could make a list of the critical holes in the organization’s security program and start forcing change, but that is going to set the wrong tone for your program.  Instead, the advice from CISO’s is to get as much information about how the business functions and what their priorities are, before you try to change anything.  Observe and document to start.

Another important topic was how to promote (or evangelize) the security program, and ultimately affect the culture of the organization.  Of course this is going to require support from the top executives, but this approach needs to be combined with context from the bottom up.  All too often the most senior management will agree to security initiatives, but that gets communicated down the chain to execution with no context.  Without context, the folks on the ground are bound to go through the motions even if the activity provides no security benefit.  It is important to make sure that the engineers and administrators understand why they are performing a task, so that they are empowered to identify when something is wrong.  Otherwise they will diligently perform a task everyday, with no hope of achieving its original purpose.  So as a security leader, you need to focus on both approaches to security education and awareness.

I received a lot of feedback after the session about how organizations are trying or have tried to decentralize some of the operational roles of the security team into other business functions.  For example, firewall administration and maintenance doesn’t need to be directly managed by the security team.  A network administration team can perform this function with oversight from the security team.  However, someone pointed out that it isn’t as easy as just moving the responsibility to another group.  There needs to be a transition plan, complete with training and oversight.  This also needs to be a formal part of their objectives, not a best effort responsibility.  Many organizations have found it beneficial to even give up headcount to other groups to support them bringing security professionals onboard.  One of the side benefits of this approach is that a security engineer who is embedded in an operational group often isn’t perceived as the bad guy like the security sometimes can be.  They are often seen as part of that team, which promotes trust and cooperation.  This can really start help  expand the reach of the security team with no additional cost.

Overall, I think that this opportunity for attendees to hear straight from leading CISO’s what is important in their role, is an invaluable dialogue that needs to continue if we have any hope of preparing our future information security leaders for the demands of the job.

I have posted a copy of my presentation, Organizational Structure, What Works (PDF), with speaker notes on my website.

The Next Generation of CISO

November 28th, 2010

One question keeps coming up in my discussions with other peers in our field, and that is where do aspiring Information Security leaders learn the necessary skills to run a mature Information Security program? There is really no professional certification or academic program that really prepares you for the in’s and out’s of building and running an Information Security program. There are certainly classes that claim to be for security leaders, but these are really geared more on security middle management, not the top positions. We are also starting to see some university programs starting to market their course towards the up and coming security executive, but from what I have seen most don’t have experienced leaders on staff. Rather they are just taking some computer science staff and throwing them together with faculty from the business school, and calling it a program. Certainly the CISO community is not enormous, but we need to find a way to educate the next generation of CISO so that we don’t keep making the same mistakes over and over. It is still way to common for another leader in the organization to get thrown into the top information security position with often little to no security expertise.

I was talking to colleague just the other day about the CISO of a major corporation, and he told me that this CISO had gone right from being an IT manager to the CISO position. How did he learn what was necessary to run a fairly complex security program and bring it up to current expectations? Well he learned on the job as he went. Imagine how much more effective and how much further along the maturity scale this program could have been if its leader wasn’t stumbling over all the same pitfalls that every modern CISO has already experienced. It seems like the best you can hope for these days is to find a great mentor to help you develop the needed skills for the CISO position and try to learn from their style and mistakes. I know that in my own career I have been fortunate to find some amazing mentors who were willing to give up their time and set aside their own agendas to help with my career development. In fact a big motivation for my last job move was to work under my current mentor, and I have seen my career opportunities grow and expand like I never imagined in that environment.

You can pick up a book and get maybe 20% of what you need to know to be a great information security leader, and if you are lucky enough to find the right mentor, you might be able to fill in the other 80%, but you really only get one perspective that way. What we need is a real structured executive leadership program for aspiring CISOs.

Not that it will totally fill this need, but I am happy to announce a new session at the RSA Conference 2011 that will hopefully inspire further development in this area. Along with some real all stars in the field, I will be participating in a professional development session on the first day of the conference titled: Information Security Leadership Development: Building and Managing a Successful Information Security Program. The idea is that the best way to create an effective crash course in security leadership would be to recruit existing leaders in the field to teach students the various topic areas needed to be a run a security program. The format will be several short modules throughout the morning, each presented by a different security leader. This will allow students to experience many different perspectives about what it means to be a security leader. You can see the full agenda on the RSA Conference website:

We have organized this half-day session into modules focusing on a few of the essential security leadership topics:

  • Building Blocks of a Security Program
  • Making Regulations & Audit Work for You
  • Managing the Breach

If this session is successful, and really think it will be a hit, then I hope that we can cover some additional topics in future years or other venues:

  • how to base your security program on risk management principles
  • navigating the relationship between internal audit or external regulators
  • measuring the success of your program and communicating metrics to executives
  • presenting security to a Board of Directors or other senior management
  • how to influence and change the culture of your company to support security initiatives
  • how to align security initiatives with the objectives of the business
  • building a long-term strategy and short-term objectives
  • maturing your program to account for privacy considerations
  • ethical decisions and scenarios for security leaders

Getting to learn first-hand from real leaders and pioneers in the field is just such a great opportunity. I hope it will inspire even more learning opportunities like it.