EHS Audits – Have We Lost Our Way? A Sequel

Aug 13th, 2011 | By | Category: Auditing

On July 11, 2010, an article titled “EHS Audits—Have We Lost Our Way?” was published in EHS Journal.[1]  It has since elicited a number of quite thoughtful and extensive comments and much general discussion.  The premise of the article was that perhaps environmental, health, and safety (EHS) audit programs have evolved, particularly in the United States, into mere checks of administrative requirements and not evaluations of risk.  This trend might have a number of causes, including

  •  The constant growth of enforced EHS regulations, which now encompass more than 25,000 pages at the federal level in the United States
  • The propensity of some regulatory agencies to assign considerable focus on recordkeeping and similar administrative documentation requirements as part of compliance evaluations and inspections
  • An assumption that achieving compliance equates to elimination of risk
  • An assumption that establishing sustainability goals, tracking progress against the goals, and receiving sustainability recognition and awards also equates to elimination of risk[2]
  • The belief that mature audit programs have already identified major risks
  • The inability of EHS auditors, because of a lack of time or experience, to focus on anything other than what’s on their “ten thousand question” checklist

Thus, we find that typical audit findings are often quite minor deviations from compliance requirements, and rare high-risk audit findings may not have precise regulatory drivers. This situation is demonstrated below by Table 1, which is drawn from the original article.

 Table 1: Focusing on Compliance and Not Risk

Compliance Area Typical Audit Finding “Rare” Audit Finding
Wastewater Minor exceedances of pH in a wastewater discharge from a minor source No knowledge of the integrity of 50 year old process sewers
Above Ground Storage Tanks Secondary containment wall that is 2 inches too short and not proximate to a navigable water 40 year old AST never tested for integrity
Confined Space Entry No expiration date on one CSE permit Attendants at active entry not always attentive
Employee Safety Guarding on seldom-used grinder in the shop is not set to the correct gap Operators on production line are clearing debris while the machine is running

A more value-added audit report would include those “rare” audit findings that pose the greatest risk, regardless of whether there is a specific regulatory requirement governing the activity or operation.  Alas, this is mostly not the case.

The remainder of this article first proposes a methodology that could be used to determine if an individual audit program is “broken” in this regard.  It then discusses the results of applying the methodology to two audits.

 

Overview of the Methodology

The essence of the methodology is that each finding on an audit can and should be evaluated for the potential consequences of noncompliance with regulations or company standards and for the inherent EHS risk posed and the likelihood of an incident (see Figure 1). Findings with a low potential for noncompliance consequences and a low potential for EHS risk and incidents (quadrant I) are common, but their discovery probably does not add significant value to the audit or the organization.  Findings with a high potential for noncompliance consequences and a high potential for EHS risk and incidents (quadrant IV) are rarer but add significant value to the process and the understanding of the audited facility’s operational status.  Other findings might present either a high likelihood of regulatory compliance consequences or a high likelihood of EHS, business, or other risks, but not both (quadrants II and III); these findings should be evaluated accordingly.

 

 

The outcomes of the methodology can be used in a number of different ways.  First, if 95 percent of an audit’s findings are in quadrant I, then either the site is performing remarkably well or the audit team did indeed focus only on the detailed administrative requirements.  If this result continues for facility after facility, for example in a highly regulated and very risky business, then maybe the auditors need some additional training or the program needs to be reworked.

A second way to utilize the outcomes is to look closely at the outliers.  Obviously, the outliers in quadrant IV are worthy of scrutiny.  However, it’s not so obvious that outliers in quadrant III, in particular, warrant some attention.  In fact, programs that focus only on regulatory compliance may miss quadrant III findings entirely.  These findings are high-risk findings for which there is little in the way of a regulatory or corporate requirement driver.  A good example, which is discussed later, is a large water tank that is 50 years old and has never been tested for wall thickness or integrity.  This oversight might be quite important, as many water tanks at manufacturing facilities are located in the utilities area, often near an electrical substation.  If the tank ruptured, the plant could very easily lose power.  Not a good thing.

These examples are only two of the numerous ways in which the information gained from using this methodology can be used to benefit the organization.  In the examples provided later in this article, one can see a real-life application of the methodology and imagine other analyses that could be undertaken.

Examples of Findings for Each Quadrant

The question might be posed—Don’t we need a definition of what findings go in what part of each quadrant?  In a perfect world, that definition would be straightforward.  Unfortunately, we do not live in a perfect world.  The best way to provide guidance on quadrant placement is by way of examples.  Provided below are examples based on the authors’ experiences.  If an organization decides to utilize the methodology, the idea would be for the developer to provide additional examples based on a particular company’s operations and compliance challenges.  Definition by example is probably the best approach.

Quadrant I—Low Noncompliance Consequences, Low Risk

  • Minor exceedances of pH in wastewater discharge
  • Secondary containment wall two inches too short and not proximate to sensitive receptors
  • No expiration date on one confined space entry permit
  • Guarding on seldom-used grinder in shop is not set at correct gap
  • Community Right-to-Know Tier II inventory report submitted late

Quadrant II—Medium to High Noncompliance Consequences, Low Risk

  • No tracking of refrigerant leak rates and repairs for units containing more than 50 pounds of ozone-depleting substances (ODSs)
  • Expired wastewater discharge permit; no reapplication on record
  • Missed Toxics Release Inventory (TRI) chemical on last five reports
  • Incomplete hazardous waste manifests for wastes that are disposed of properly
  • Lack of documentation for hazard communication training that was conducted

Quadrant III—Low Noncompliance Consequences, Medium to High Risk

  • No knowledge of the integrity of 50-year-old process sewers
  • Forty-year-old aboveground storage tank never tested for integrity
  • Storage of incompatible materials together in containers and tanks
  • Hydrogen storage trucks parked and unloaded onsite, where the quantity of hydrogen does not exceed the Process Safety Management (PSM) thresholds, but there are no process safety controls
  • Grandfathered landfill for which there is no historical knowledge of what was disposed of in the landfil

Quadrant IV—High Noncompliance Consequences, High Risk

  • Attendants at active entry not always attentive or present
  • Operators on production line clearing debris while line is active
  • Large hazardous material tank with no secondary containment on the banks of a high-quality designated stream
  • Unpermitted discharges of industrial wastewater
  • Unreported and mitigated releases of hazardous substances to the environment

 

Example Application I—A Food Plant Environmental Audit

Environmental Resources Management (ERM) recently conducted a food plant audit in which the auditors received permission to utilize this methodology informally.  Each of the fourteen environmental findings was given two scores, ranging from 1 to 10.  One score rated the potential noncompliance consequences, and the second rated the level of risk.  The scores were based on the judgment of the lead environmental auditor and the examples provided in the previous section.  The results were then plotted on a graph, which is provided in Figure 2 below.

 

How can the results be evaluated?  If one takes a closer look, as in Figure 3 below, it can be seen that 50 percent of the findings (seven findings) were placed in or on the line of quadrant I, and there were at least two findings in each quadrant.  This distribution is a fairly even one, which suggests that the auditor did not simply focus on the checklist items.

 

 

Now, let’s review four of the outliers and maybe draw some conclusions from that analysis:

  • Finding A (Quadrant III)—The site had a several hundred thousand gallon fire water tank that had never (to site knowledge) been integrity tested, despite the fact that it was many decades old.  The tank was located near electrical and critical process equipment.  Catastrophic failure of the tank could result in substantial operational interruption, risk to human health, and risk to storm water outfalls.
  • Finding B (Quadrant II)—The site had several large ODS-containing units, and the leak rates, repairs, repair verifications, and so forth on these units were not being tracked.  Because these records are an EPA enforcement priority, not having them could result in substantial regulatory enforcement, even though site maintenance personnel had knowledge that leak repairs had not been necessary.
  • Finding C (Quadrant IV)According to the site’s air permit, compliance with the particulate matter emission requirements is met by monitoring the pressure drop on baghouses to ensure the systems are maintained within an acceptable range. Site records routinely showed very low to zero pressure drop readings, which suggest that the baghouses may have failed and the facility was not in compliance with its particulate matter emission requirements.  Emissions of large quantities of particulate matter and documented permit noncompliance posed a substantial potential risk to the environment and a substantial potential risk of regulatory enforcement.
  • Finding D (Quadrant I)—This finding had the lowest overall risk. One of the daily pH recordings was not taken at the site’s wastewater outfall but was recorded nonetheless in the outfall data sheet.  To avoid confusion, it was recommended that the site report only outfall data on the outfall sheet or clearly note that the pH reading recorded was associated with an internal monitoring point.

 

Example Application II—A Fire Safety Audit

For comparison purposes, the following chart includes the results from a recent fire safety audit in a jurisdiction where National Fire Protection Association (NFPA) standards had not been incorporated into the fire code as a mandatory requirement.  The results show multiple findings with significant risk but not significant potential for regulatory enforcement.

The most significant risk findings were associated with the following:

  • Inoperable dust detectors in the heating, ventilation, and air conditioning (HVAC) system, which created a significant risk for fire at this facility
  • Missing ceiling tiles, which caused bypassing of smoke detectors and sprinklers
  • No routine inspections of smoke detectors and fire alarms

All of these findings presented significant risk to the facility but had neither an associated regulatory requirement nor the potential for agency enforcement.

Conclusion

EHS audit programs continue to be a critical part of most organizations’ governance and assurance efforts.  In many cases, however, the audits have evolved into a pro forma evaluation of administrative procedures.

Despite this trend, the prospects are not all “gloom and doom.”  Some in industry have identified this issue and developed new initiatives to improve their internal audit processes to better focus on risk. For example, David Cummings, DuPont’s Corporate Process Safety Competency Leader, has undertaken a pilot program to improve the focus of PSM audits at his company. Starting in 2009, DuPont added mechanical integrity/quality assurance (MIQA) specialists to specific PSM audits at thirty higher hazard process sites in the U.S. This was initiated because: MIQA is a critical component of an effective PSM program, and auditing MIQA is a niche specialty that in many cases requires significant practical experience, detailed technical knowledge, and an understanding of the proper application of not just OSHA requirements but also Recognized and Generally Accepted Good Engineering Practices (RAGAGEPs) developed by API, ASME, ISA and others. Utilizing MIQA audit specialists also enables expanded data sampling and field evaluations as part of each review, which ultimately provides an improved risk based assessment. Your typical EHS auditor does not have these skill sets.  The DuPont MIQA auditing pilot program has proven to be very beneficial and is being applied in other regions starting in 2011.[3]

Finally, recent incidents (e.g., the Gulf of Mexico oil spill, the Massey Energy mine disaster, and the Tokyo Electric Power Company [TEPCO] nuclear accident) suggest that a risk-based approach would add more value to audit programs.  Historical and current audit findings can and should be evaluated for the value added to the organization’s understanding of EHS risk.  If that evaluation suggests that audit programs are not achieving the objective of better identifying and managing EHS risks, then programs should be reworked to accomplish that objective.

About the Authors

Lawrence B. Cahill, CPEA (Master Certification), is a Technical Director at Environmental Resources Management in Exton, Pennsylvania, U.S.A.  He has more than 30 years of professional EHS experience with industry and consulting.  He is the editor and principal author of the widely used text Environmental, Health and Safety Audits, published by Government Institutes, Inc., and now in its ninth edition.  He has published more than 60 articles and has been quoted in numerous publications, including The New York Times and The Wall Street Journal.  Mr. Cahill has worked in more than 25 countries during his career.  He holds a B.S. in Mechanical Engineering fromNortheasternUniversity, an M.S. in Environmental Health Engineering fromNorthwesternUniversity, and an MBA from theWhartonSchool of theUniversity ofPennsylvania.

Robert J. Costello, P.E., Esq., CPEA, is a Senior Project Manager at Environmental Resources Management in Exton, Pennsylvania, U.S.A.  He has more than 17 years of professional environmental resource management and consulting experience.  Mr. Costello manages global regulatory compliance, management systems, and sustainability assurance programs and participates personally on-site in typically 30 or more audits and assessments per year.  He holds a B.S. in Environmental Engineering from Wilkes University, an M.S. in Environmental Engineering from Syracuse University, and a J.D. from Syracuse University.  Mr. Costello is admitted to the bar in Pennsylvania, is a licensed professional engineer in Pennsylvania and Delaware, and a Certified Professional Environmental Auditor.

Photograph: The Dome of Rome’s Pantheon by Vladimir Fofanov, Moscow, Russia.

Other Articles by Lawrence Cahill in the EHS Journal

Using Risk Factors to Determine EHS Audit Frequency

Measuring the Success of an EHS Audit Program

EHS Audits – Have We Lost Our Way?

Statistically Representative Sampling on EH&S Audits: Expectations Established by Third Parties

Outsourcing EHS Audits: Does it Make Sense?

 

Notes


[1] Cahill, L.B., “EHS Audits—Have We Lost Our Way?”, EHS Journal, July 11, 2010.

[2] Note that the Tokyo Electric Power Company (TEPCO) was listed on the Dow Jones Sustainability Index (DJSI) on March 11, 2011, when a tsunami hit Japanese shores and severely damaged TEPCO’sFukushima nuclear power plant.  TEPCO was removed from the DJSI list on May 13, 2011.  BP was similarly recognized by the DJSI prior to the April 20, 2010 Gulf of Mexico Deepwater Horizon incident.  BP was de-listed on May 31, 2010.

[3] This information has been taken from discussions with Mr. Cummings and his presentation “Mechanical Integrity Challenges in PSM Audits – It’s All About the Details,” given at the Philadelphia National Meeting of The Auditing Roundtable, on September 9, 2010.

Tags: , , , , , , , , , , ,

7 Comments to “EHS Audits – Have We Lost Our Way? A Sequel”

  1. [...] titled “EHS Audits – Have We Lost Our Way?”  That article was followed a year later by a sequel that explored the topic more fully.  The articles elicited numerous thoughtful comments and much [...]

  2. [...] EHS Audits – Have We Lost Our Way? A Sequel (Cahill and Costello) [...]

  3. The value of any audit is really based on two primary factors. 1) The skills of the audit team in identifying and effectively communicating risks 2) The action taken by the organization with the information provided. This is really the ying and yang of the audit process. If the audit is weak or reports only trivial issues, management is being misinformed as to the state of the business. If managment fails to act of effectively delegate findings with effective contain, corrective and preventative action programs, then what is driving the audit team to raise the bar at the next audit?

    The two factors need to continually challenge each other and be involved in both aspects of the process. Finding things wrong is easy, ask any consultant! Fixing things that are wrong so they can’t go wrong ever again (fool proofing) is the opportunity that needs to be embraced by all parties. Management teams should never accept a weak audit (except maybe by a regulator!) just as they won’t accept substandard work from a supplier. Audits if used properly should be able to add value to a business, if they are not then the audits are either weak or the managmeent team is keeping their head in the sand.

    When I hear people say things like “We need to eliminate all risks” I like asking them how they plan on stopping people from sneezing while they drive to work. We need to assess risks and manage them accordingly based on the businesses we are in and the level of risk that is acceptable to the management team.

    So on your next audit – do as Emerald Lagasse does “kick it up a notch”, put a little “bam” into it, dig a couple steps deeper, challenge the management team to identify and prevent the root cause from happenening again. Facilitate the entire audit process, not just the audit itself. Show your company the value of your audits.

  4. [...] EHS Audits – Have We Lost Our Way? A Sequel (Cahill and Costello) [...]

  5. Thanks for the thoughtful article. Is the role of determining the level of potential risk associated with both environmental aspects and health and safety hazards more appropriately placed in the domain of consultancy rather than that of the auditor? If the work to identify EH&S risks is done skillfully by the organization, and appropriate controls established to mange the EH&S risk to tolerable levels, the role of the auditor is then to verify the effectiveness of the controls, not to determine the need for them.

    The role of the auditor should be to understand the audit criteria, collect and review audit evidence against those criteria, make findings regarding conformity with the criteria based on the review of the evidence, and then draw audit conclusions about the effectiveness of the organizations management system to control the risks to a tolerable level.

  6. Excellent points without doubt, but a couple thoughts. The article defines “risk” in a somewhat circular manner, indicating that “risk” consists of two dimensions – “consequences” and “risk”. In the traditional risk management (RM) world, “risk” is indeed generally viewed as having two components, one component being a time element (frequency or likelihood) while the other reflecting a magnitude of effect (severity or impact). Using the terms “consequences” and “risk” may not align with companies’ internal RM Departments, possibly making communications with them a little difficult or complicated.

    This is becoming increasingly important in my opinion as I am frequently asked to help clients adapt their own internal RM methods/frameworks to be used by internal EHS staff. This also includes adapting risk assessment methods that are somewhat standardized for traditional risk management, such as Failure Mode Effect Analysis (FMEA) – a state of the art risk assessment process widely used in property risk assessment/insurance applications (and to a lesser extent in casualty risk).

    Two fundamental aspects of EHS risk assessments that need to be addressed before launching such approaches are:
    – defining the parameters/benchmarks for “severity” and “frequency” – what does “high” and “low” mean? What dollar values should be used? What other indicators might also apply?
    – “gross risk” versus “net risk” – should the process quantify the risk profile assuming controls do not exist or will fail, thus indicating the worst case scenario? Or should the output reflect the existence and effectiveness of controls that are in place?

    Of course, risk maps are extremely helpful and intuitive graphical devices to show the results of the assessment, as the article clearly shows. Typically, the first question posed is “How do we move the risks from the outer edges to the lower left hand corner of the map?” Again, using a frequency/severity structure can be instructive in answering the question. Consider a risk point that is high severity but low frequency (a Black Swan event like Fukushima). There may be no practical way to reduce the frequency, therefore a solution aimed at frequency reduction will provide little value. A more appropriate response is to reduce the severity of the event when (or if) it occurs. In this context, the more appropriate control may be a financial solution such as insurance. Conversely, low severity events that happen frequently (like first aid cases) would be better addressed through a management system solution (such as training or making additional PPE available) rather than a financial solution.

    The integration of “risk” into EHS and sustainability is still a work in progress in our field. As I have espoused for a number of years, EHS staff not define risk on their own – but do so using existing benchmarks, standards and definitions that have been validated many times over within internal RM departments. EHS professionals should try to work with their company’s RM staff to achieve this, or engage external support with significant first hand experience in EHS and traditional risk management.

  7. Laura McHugh says:

    As auditors, we need to help our audited organizations understand the value of performing meaningful root-cause analyses and risk maps like those presented in this article. Audits should be part of an organization’s compliance management system, the “check” in the Plan-Do-Check-Act process of continual improvement toward identification and mitigation of organizational risks from E and HS risks.

    Great articles! Thanks for publishing.

Leave a Comment