SAS 70 Reports: A sheep in wolfs clothing.

SAS 70 reports (particularly "Type II") are seen as some kind of certification that a (service) company can be trusted and has good processes and controls in place. A company that gains a favorable SAS 70 report often claims "bragging" rights about "getting SAS70 certified" or "passing the audit".

OK, there are many things wrong with that first paragraph. It seems there is a huge misconception about SAS 70 and what it is, and for me this is dangerous because a lot of folks overestimate the usefulness of SAS70 reports and indeed it is generally seen as a means of assuring compliance with Sarbanes-Oxley section 404 - which it does not, but we'll get to that.

Let's start with the idea that companies comply with SAS 70.

POINT 1: YOU CANNOT COMPLY WITH SAS 70
The way a SAS 70 audit works is this;

• the service provider states some objectives they hope to achieve using "controls";
  
• the service provider states the controls that they use to meet their objectives;
• an independent auditor looks at evidence that the stated controls are operating and effective at meeting the stated objectives.

So basically the audit verifies that a company does what it says it does, and that what it does meets its own objectives.

POINT 2: SAS 70 DOES NOT ASSURE QUALITY OR BEST PRACTICE
As noted in point 1, the audit reports a companies compliance with their own procedures/controls. Now those controls may be slow, cumbersome, expensive, outdated, etc. But if they meet the objective stated by the company then there is nothing to report.

POINT 3: SAS 70 DOES NOT REQUIRE A MINIMUM SET OF CONTROLS
As stated on the SAS70.com website:

"Since service organizations are responsible for describing their controls and defining their control objectives, there is no published list of SAS 70 standards. Generally, the control objectives are specific to the service organization and their customers."
However, there are certain headlines that do need to be addressed

1. Control Environment.
2. Risk Assessment.
3. Control Activities.
4. Information and Communication.
5. Monitoring.

So if a company defines 5 controls, that is what gets audited.
I think there is some inferred assumption that the "independent auditor" who performs the SAS 70 audit will use some best practice framework as a benchmark (such as COSO/COBIT), but this is not required.

POINT 4: THERE IS NO PASS OR FAIL...THERE IS NO CERTIFICATION
So as we can see from the preceding points, you cannot pass or fail because there are no independent criteria against which to score, and you cannot be certified for the same reason.
What can happen is that the report will describe some control that is operated incorrectly or ineffectively such that its defined objective is not met.


So finally what use is a SAS 70 report
Well it is promoted as a solution to confirming Sarbanes-Oxley compliance

The Wikipedia article even says:
"SOX heightened the focus placed on understanding the controls over financial reporting and identified a Type II SAS 70 report as the only acceptable method for a third party to assure a service organization's controls."
This is nonsense. A SAS 70 report is a very useful tool for communicating audit results. It is not a statement of compliance with SOX or any other regulation .

SAS 70 is useful and misleading at the same time - and which it is depends on how it is sold to you and how much you understand about the process.
A "white paper" highlight many of the failings of SAS 70 even exists; SAS 70: The Emperor Has No Clothes. This was written by an ISO17799 consultancy, so they have their own agenda to push, but I do not and have independently come to the same conclusions (as have other bloggers).

So what is the benefit of a "clean" SAS 70 Type II report. Here are a couple of thoughts:

1. From sas70.com; "Many SAS 70 audit engagements result from user organizations making repeated requests of the service organization in order to gain an understanding of the internal control environment at the service organization." So it can save time when end user organisations want to audit a service provider. It is worth remembering that a SAS 70 report is meant to be an auditor-to-auditor communication tool.
2. If a service company is actually making an effort to apply CoBIT or some other defined best practice, the audit can serve to highlight to them areas for improvement, but only if they ensure that they provide the appropriate criteria to the auditor (see point 2 above).
   

I believe the one thing to really remember is that a "clean" SAS 70 Type II report guarantees nothing. The end user company needs to define what it wants in terms of quality and control from its suppliers, and then it can use the SAS70 report to confirm if their criteria are met by the supplier's controls.

Regulatory: FDA Warning Letter Top Citations

Looking at the past 3 years of warning letters we can count the number of letters that cite a specific regulation at least once. In this way we can see which part are cited in most letters by FDA and therefore the area in whioch companies most commonly fail.

As we can see, there are three areas that really stand out for continual citation;

• 820.100 Corrective and preventive action...
• 820.198 Complaint files....
• 820.30 Design controls....

It is clear that the first two are co-dependant, and a failure in managing design controls will also feed into Complaints and CAPA.

Drilling down into the detail, we can see in individual warning letters that time and again, companies do not manage complaints correctly. Key failures include:

• Not assessing complaints at all,
• Dismissing complaints as not critical or not investigating fully (no risk assessment or justification),
• Not implementing corrective actions,
• Not describing preventative actions or managing risk from failures,
• Lacking procedures,
• Having procedures but not following the processes.

So the key message is treat complaints seriously, and if you do decide they do not require CAPA, make sure there is a documented rationale including risk analysis for that decision.