On July 11, 2010, an article titled “EHS Audits—Have We Lost Our Way?” was published in EHS Journal.[1] It has since elicited a number of quite thoughtful and extensive comments and much general discussion. The premise of the article was that perhaps environmental, health, and safety (EHS) audit programs have evolved, particularly in the United States, into mere checks of administrative requirements and not evaluations of risk. This trend might have a number of causes, including

• The constant growth of enforced EHS regulations, which now encompass more than 25,000 pages at the federal level in the United States
• The propensity of some regulatory agencies to assign considerable focus on recordkeeping and similar administrative documentation requirements as part of compliance evaluations and inspections
• An assumption that achieving compliance equates to elimination of risk
• An assumption that establishing sustainability goals, tracking progress against the goals, and receiving sustainability recognition and awards also equates to elimination of risk[2]
• The belief that mature audit programs have already identified major risks
• The inability of EHS auditors, because of a lack of time or experience, to focus on anything other than what’s on their “ten thousand question” checklist

Thus, we find that typical audit findings are often quite minor deviations from compliance requirements, and rare high-risk audit findings may not have precise regulatory drivers. This situation is demonstrated below by Table 1, which is drawn from the original article.

Table 1: Focusing on Compliance and Not Risk

A more value-added audit report would include those “rare” audit findings that pose the greatest risk, regardless of whether there is a specific regulatory requirement governing the activity or operation. Alas, this is mostly not the case.

The remainder of this article first proposes a methodology that could be used to determine if an individual audit program is “broken” in this regard. It then discusses the results of applying the methodology to two audits.

Overview of the Methodology

The essence of the methodology is that each finding on an audit can and should be evaluated for the potential consequences of noncompliance with regulations or company standards and for the inherent EHS risk posed and the likelihood of an incident (see Figure 1). Findings with a low potential for noncompliance consequences and a low potential for EHS risk and incidents (quadrant I) are common, but their discovery probably does not add significant value to the audit or the organization. Findings with a high potential for noncompliance consequences and a high potential for EHS risk and incidents (quadrant IV) are rarer but add significant value to the process and the understanding of the audited facility’s operational status. Other findings might present either a high likelihood of regulatory compliance consequences or a high likelihood of EHS, business, or other risks, but not both (quadrants II and III); these findings should be evaluated accordingly.

The outcomes of the methodology can be used in a number of different ways. First, if 95 percent of an audit’s findings are in quadrant I, then either the site is performing remarkably well or the audit team did indeed focus only on the detailed administrative requirements. If this result continues for facility after facility, for example in a highly regulated and very risky business, then maybe the auditors need some additional training or the program needs to be reworked.

A second way to utilize the outcomes is to look closely at the outliers. Obviously, the outliers in quadrant IV are worthy of scrutiny. However, it’s not so obvious that outliers in quadrant III, in particular, warrant some attention. In fact, programs that focus only on regulatory compliance may miss quadrant III findings entirely. These findings are high-risk findings for which there is little in the way of a regulatory or corporate requirement driver. A good example, which is discussed later, is a large water tank that is 50 years old and has never been tested for wall thickness or integrity. This oversight might be quite important, as many water tanks at manufacturing facilities are located in the utilities area, often near an electrical substation. If the tank ruptured, the plant could very easily lose power. Not a good thing.

These examples are only two of the numerous ways in which the information gained from using this methodology can be used to benefit the organization. In the examples provided later in this article, one can see a real-life application of the methodology and imagine other analyses that could be undertaken.

Examples of Findings for Each Quadrant

The question might be posed—Don’t we need a definition of what findings go in what part of each quadrant? In a perfect world, that definition would be straightforward. Unfortunately, we do not live in a perfect world. The best way to provide guidance on quadrant placement is by way of examples. Provided below are examples based on the authors’ experiences. If an organization decides to utilize the methodology, the idea would be for the developer to provide additional examples based on a particular company’s operations and compliance challenges. Definition by example is probably the best approach.

Quadrant I—Low Noncompliance Consequences, Low Risk

• Minor exceedances of pH in wastewater discharge
• Secondary containment wall two inches too short and not proximate to sensitive receptors
• No expiration date on one confined space entry permit
• Guarding on seldom-used grinder in shop is not set at correct gap
• Community Right-to-Know Tier II inventory report submitted late

Quadrant II—Medium to High Noncompliance Consequences, Low Risk

• No tracking of refrigerant leak rates and repairs for units containing more than 50 pounds of ozone-depleting substances (ODSs)
• Expired wastewater discharge permit; no reapplication on record
• Missed Toxics Release Inventory (TRI) chemical on last five reports
• Incomplete hazardous waste manifests for wastes that are disposed of properly
• Lack of documentation for hazard communication training that was conducted

Quadrant III—Low Noncompliance Consequences, Medium to High Risk

• No knowledge of the integrity of 50-year-old process sewers
• Forty-year-old aboveground storage tank never tested for integrity
• Storage of incompatible materials together in containers and tanks
• Hydrogen storage trucks parked and unloaded onsite, where the quantity of hydrogen does not exceed the Process Safety Management (PSM) thresholds, but there are no process safety controls
• Grandfathered landfill for which there is no historical knowledge of what was disposed of in the landfil

Quadrant IV—High Noncompliance Consequences, High Risk

• Attendants at active entry not always attentive or present
• Operators on production line clearing debris while line is active
• Large hazardous material tank with no secondary containment on the banks of a high-quality designated stream
• Unpermitted discharges of industrial wastewater
• Unreported and mitigated releases of hazardous substances to the environment

Example Application I—A Food Plant Environmental Audit

Environmental Resources Management (ERM) recently conducted a food plant audit in which the auditors received permission to utilize this methodology informally. Each of the fourteen environmental findings was given two scores, ranging from 1 to 10. One score rated the potential noncompliance consequences, and the second rated the level of risk. The scores were based on the judgment of the lead environmental auditor and the examples provided in the previous section. The results were then plotted on a graph, which is provided in Figure 2 below.

How can the results be evaluated? If one takes a closer look, as in Figure 3 below, it can be seen that 50 percent of the findings (seven findings) were placed in or on the line of quadrant I, and there were at least two findings in each quadrant. This distribution is a fairly even one, which suggests that the auditor did not simply focus on the checklist items.

Now, let’s review four of the outliers and maybe draw some conclusions from that analysis:

• Finding A (Quadrant III)—The site had a several hundred thousand gallon fire water tank that had never (to site knowledge) been integrity tested, despite the fact that it was many decades old. The tank was located near electrical and critical process equipment. Catastrophic failure of the tank could result in substantial operational interruption, risk to human health, and risk to storm water outfalls.
• Finding B (Quadrant II)—The site had several large ODS-containing units, and the leak rates, repairs, repair verifications, and so forth on these units were not being tracked. Because these records are an EPA enforcement priority, not having them could result in substantial regulatory enforcement, even though site maintenance personnel had knowledge that leak repairs had not been necessary.
• Finding C (Quadrant IV)—According to the site’s air permit, compliance with the particulate matter emission requirements is met by monitoring the pressure drop on baghouses to ensure the systems are maintained within an acceptable range. Site records routinely showed very low to zero pressure drop readings, which suggest that the baghouses may have failed and the facility was not in compliance with its particulate matter emission requirements. Emissions of large quantities of particulate matter and documented permit noncompliance posed a substantial potential risk to the environment and a substantial potential risk of regulatory enforcement.
• Finding D (Quadrant I)—This finding had the lowest overall risk. One of the daily pH recordings was not taken at the site’s wastewater outfall but was recorded nonetheless in the outfall data sheet. To avoid confusion, it was recommended that the site report only outfall data on the outfall sheet or clearly note that the pH reading recorded was associated with an internal monitoring point.

Example Application II—A Fire Safety Audit

For comparison purposes, the following chart includes the results from a recent fire safety audit in a jurisdiction where National Fire Protection Association (NFPA) standards had not been incorporated into the fire code as a mandatory requirement. The results show multiple findings with significant risk but not significant potential for regulatory enforcement.

The most significant risk findings were associated with the following:

• Inoperable dust detectors in the heating, ventilation, and air conditioning (HVAC) system, which created a significant risk for fire at this facility
• Missing ceiling tiles, which caused bypassing of smoke detectors and sprinklers
• No routine inspections of smoke detectors and fire alarms

All of these findings presented significant risk to the facility but had neither an associated regulatory requirement nor the potential for agency enforcement.

Conclusion

EHS audit programs continue to be a critical part of most organizations’ governance and assurance efforts. In many cases, however, the audits have evolved into a pro forma evaluation of administrative procedures.

Despite this trend, the prospects are not all “gloom and doom.” Some in industry have identified this issue and developed new initiatives to improve their internal audit processes to better focus on risk. For example, David Cummings, DuPont’s Corporate Process Safety Competency Leader, has undertaken a pilot program to improve the focus of PSM audits at his company. Starting in 2009, DuPont added mechanical integrity/quality assurance (MIQA) specialists to specific PSM audits at thirty higher hazard process sites in the U.S. This was initiated because: MIQA is a critical component of an effective PSM program, and auditing MIQA is a niche specialty that in many cases requires significant practical experience, detailed technical knowledge, and an understanding of the proper application of not just OSHA requirements but also Recognized and Generally Accepted Good Engineering Practices (RAGAGEPs) developed by API, ASME, ISA and others. Utilizing MIQA audit specialists also enables expanded data sampling and field evaluations as part of each review, which ultimately provides an improved risk based assessment. Your typical EHS auditor does not have these skill sets. The DuPont MIQA auditing pilot program has proven to be very beneficial and is being applied in other regions starting in 2011.[3]

Finally, recent incidents (e.g., the Gulf of Mexico oil spill, the Massey Energy mine disaster, and the Tokyo Electric Power Company [TEPCO] nuclear accident) suggest that a risk-based approach would add more value to audit programs. Historical and current audit findings can and should be evaluated for the value added to the organization’s understanding of EHS risk. If that evaluation suggests that audit programs are not achieving the objective of better identifying and managing EHS risks, then programs should be reworked to accomplish that objective.

About the Authors

Lawrence B. Cahill, CPEA (Master Certification), is a Technical Director at Environmental Resources Management in Exton, Pennsylvania, U.S.A. He has more than 30 years of professional EHS experience with industry and consulting. He is the editor and principal author of the widely used text Environmental, Health and Safety Audits, published by Government Institutes, Inc., and now in its ninth edition. He has published more than 60 articles and has been quoted in numerous publications, including The New York Times and The Wall Street Journal. Mr. Cahill has worked in more than 25 countries during his career. He holds a B.S. in Mechanical Engineering fromNortheasternUniversity, an M.S. in Environmental Health Engineering fromNorthwesternUniversity, and an MBA from theWhartonSchool of theUniversity ofPennsylvania.

Robert J. Costello, P.E., Esq., CPEA, is a Senior Project Manager at Environmental Resources Management in Exton, Pennsylvania, U.S.A. He has more than 17 years of professional environmental resource management and consulting experience. Mr. Costello manages global regulatory compliance, management systems, and sustainability assurance programs and participates personally on-site in typically 30 or more audits and assessments per year. He holds a B.S. in Environmental Engineering from Wilkes University, an M.S. in Environmental Engineering from Syracuse University, and a J.D. from Syracuse University. Mr. Costello is admitted to the bar in Pennsylvania, is a licensed professional engineer in Pennsylvania and Delaware, and a Certified Professional Environmental Auditor.

Photograph: The Dome of Rome’s Pantheon by Vladimir Fofanov, Moscow, Russia.

Other Articles by Lawrence Cahill in the EHS Journal

Using Risk Factors to Determine EHS Audit Frequency

Measuring the Success of an EHS Audit Program

EHS Audits – Have We Lost Our Way?

Statistically Representative Sampling on EH&S Audits: Expectations Established by Third Parties

Outsourcing EHS Audits: Does it Make Sense?

Notes