Clearly vendor due diligence reviews are an essential part of any risk management program, but the consistency and quality of the assessments being done today are really appalling. Anyone who is working for a service provider today is probably getting inundated with requests ranging from a 50 item checklist of yes/no type questions all the way up to requests to review internal policies and procedure documents, or even an on site walk-through. You might think that the variety of request types and levels of detail demonstrates that the customers are taking the time to rank each provider based on the sensitivity of the service, and assess them accordingly, but sadly when you read through the due diligence requests it becomes obvious that most organizations have no clue how to assess their vendors. Many have clearly just redistributed their own internal risk assessment questionnaires and don’t even bother to remove the internal acronyms and references. Others clearly don’t know where to focus and try to audit every possible control one might implement in an environment. This brings up a fundamental point, risk assessments and audits are not the same, but when you have people running a risk management program who don’t understand the distinction, you get a lot of activity with little actionable results. The intention of a vendor risk review is to gauge whether that service provider presents an unacceptable risk to your organization, not to document every practice, procedure, and technology that may differ from your own environment or an industry framework like ISO.
Some of the worst offenders will waste your and their own time following up on questions about disclaimer banners on systems or why you only prevent the reuse of the last six passwords instead of the last twelve. You can’t tell me that they are prioritizing based on risk!
Now a growing trend is to collect volumes of documentation like an auditor would, but this is problematic in many ways. The first concern is whether it puts the service provider at risk to provide this data to all their clients. In a shared service provider model in particular, am I really going to give you a copy of all my firewall rulesets? If I do, then I think that should be the risk. Same with the increasing demand for copies of penetration test reports. These are some of the most sensitive documents an organization has, and we are supposed to just hand them over to every client just because there is an NDA in place? Given that these reports are basically step by step instructions for how to compromise an application or environment, it really doesn’t seem responsible to be distributing them. Instead of asking for the details on every single vulnerability found, you should be assessing the frequency and scope of the testing being performed, and ask for metrics on remediation of the highest risks. There are plenty of ways to assess the quality of a security program without knowing the particulars of every vulnerability that has been found. It comes back to a basic discussion about meaningful metrics. You can certainly ask all your providers how many critical and high risk vulnerabilities have been found and not yet remediated, but that is pretty meaningless unless the answer is zero. And even then that really doesn’t tell you much about your risk posture tomorrow, or next month. If you want to assess a provider’s risk posture, ask for metrics on turnaround time between identifying a high or critical vulnerability and remediating it. Or take it one step farther and ask them for their metrics around the time to implement some kind of mitigating control once a critical risk has been identified. If these metrics aren’t good, or they have never gathered them, then you know something about the maturity of their program that is more than a point in time vulnerability discussion.
Of course in a dedicated provider model, this level of audit might be appropriate, but organizations need account for the differences in the service provider model and realize that they give up something for the cost savings of shared services. Same with requesting copies of detailed network diagrams or the names of technologies being used. You need to assess a vendor’s maturity and process, not every single control possible.
Another issue with the “give me everything” documentation requests is that is distracts the assessor from finding the real significant risks. If a clients asks for a copy of every single security policy, standard, and procedure document, how can they possibly digest all that information without a team of analysts, and is it even worth it?
Organizations need to take a hard look at their practices for performing vendor risk reviews, and be honest about whether they are really focusing on risk or turning it into a blind checklist based audit. Frankly, some of the due diligence requests I have seen from organizations recently make me start to question their own internal program’s effectiveness.