Differences Between Audits, Evaluations and Investigations
Audits, evaluations and investigations are three distinctively different and powerful tools for supporting
decision-making within any organization’s proactive safety management system (SMS).
Understanding the differences of the SMS “big three” safety assurance (SA) elements can help us utilize each tool at the right time with minimal resources and result in the quality decision-making required within the SMS risk management framework.
Defining each of the three SA elements can help us explore their mechanics, goals, applications and objectives. As an aid to learning the true purpose of detailed safety assurance practices, we might want to begin by blowing up the paradigm that an audit and an internal evaluation program (IEP) always default to pre-baked regulatory data collection tools or standards and recommended practices.
Audits, at best, are a snapshot in time and begin aging immediately after their formal finding report and corrective action requests (CAR) are issued. A fully completed internal audit will assure minimum standards of care but has no means to adapt. Then, up [A1] your proactive SA management approach and use the following discussion and tables to plan and treat a much different series of ongoing business management analyses and evaluations.
Be mindful that there may be some IEP/IAP terms that fall outside our too-familiar SMS lexicon.
Audits (Internal/External)
The audit has been present in business for thousands of years. There are different types (according to their objectives), but in a generic way, audits are objective and systematic assessments of how well an organization (or certificate holder) documents, implements and executes (complies with) industry external regulations, industry standards and recommended practices.
In the ISO 19011:2018 world, this is defined as an “independent" process for obtaining evidence of existence—records, statements of fact or other information—that are relevant and verifiable toward predetermined audit criteria.
Audits are scheduled activities carried out by external, third-party teams of auditors, specialty[A2] -trained industry auditors or government inspectors. Conversely, internal audits are conducted by company employees using first-party direct department employees or second-party indirect department employees working on the company’s behalf.
Evaluations (Internal/External)
Evaluations involve assessing the strengths and weaknesses of programs, policies, personnel, products, organizational design, behavior, and culture, to improve their resiliency and effectiveness. Their purpose is to make judgements based on selected corporate performance criteria while providing near real-time status of program success.
Analyzing interprets data as it deals with meanings and implications, while evaluating assesses something's worth. Hence, results are more compulsory for the evaluation process. Analyzing precedes evaluating. Analysis largely involves a longer thinking process as compared to evaluation.
Based on the results of external evaluations, organizations can focus their efforts and resources on those actions that proved most useful and effective to achieve the maximum possible impact. Evaluation allows us to:
Learn from experience and identify best practices to continually improve the impact of projects.
Deepen the history behind project results and explain why an intervention was successful or failed.
Observe the results of the project from multiple angles and consider all categories of beneficiaries and stakeholders involved.
Increase the transparency of an organization and demonstrate the relevance, effectiveness and efficiency of its work in the eyes of financiers, stakeholders and process owners.
Easily communicate the value of an intervention, including quantifying its results and social and economic impact.
Investigations (Internal/External)
Investigations are usually undertaken in response to reports of deviation from protocol or procedure Or they involve potential violations of operating regulations and specifications and focus on persons or teams instead of policies, processes or procedures.
Investigation reports may be prepared to refer to matters of personal licensing and certification or correct serious deficiencies. They may also inform other government or local agencies of the need for action, discipline, letters of warning or formal prosecution.
For this discussion there are 12 key questions/areas of discussion to compare Internal Audit - Standards audit (Compliance to external regulations and company-adopted standards) of WHAT we should be doing.
Internal Evaluation - Measure effectiveness of procedures of HOW we should be doing something. (Risk assess degree of Conformance to internal company policy) - Business case analysis supporting WHY we should be doing something. (ID Gaps in conformity and errors in current policy design)
Investigations
The following questions may help guide a discussion of the control elements of effective SMS safety assurance tools:
Key purpose?
Key objectives?
Who performs it?
Who oversees its results?
When is it done? Why is it done?
How is it carried out?
How are policies, programs, procedures involved?
What are the key oversight requirements for internal departments or external agencies?
How is it continually monitored?
How are key Indicators, status, and trends reported?
What are measures of prevention, effectiveness, and success?
For responses to each question, relative to Internal Audit, Internal Evaluation, and Investigations, download the chart in this link.