“It is better to know some of the questions than all of the answers.” James Thurber

Introduction

In recent years there has been considerable discussion of establishing a more risk-based focus on environmental, health, and safety (EHS) audits. The concern is that audit results had devolved over time and several audit cycles into an inventory of minor infractions and administrative regulatory deficiencies. Line managers were not so sure that the effort put into the audits was identifying true and important risks and/or that maybe the sites were asymptotically approaching full compliance. On the other hand, EHS incidents experienced by BP, TEPCO, Duke Energy, Volkswagen, and other major corporations suggested that there were still major risks out there that needed to be anticipated or identified.

There are any number of ways to re-engineer an audit program to better focus on risks and achieve more value from the process. Examples might include:

• Increase the frequency of audits for inherently higher-risk facilities
• Increase the frequency of facility audits in highly regulated and/or high-profile settings
• Assure that non-traditional operations and remote locations are included as part of the audit universe and that these are audited at the appropriate depth and frequency, even if by an independent, third-party
• Conduct single-focus, deep-dive audits on specialized, high-risk topics such as hazardous waste management
• Require that root-cause analysis be conducted by auditors on perceived high-risk observations
• Add subject matter experts to audit teams on high-risk, specialized topics such as the mechanical integrity aspects of process safety management

One additional approach often implemented to help managers and executives better understand and interpret the results of facility audits is to classify EHS audit findings by level of risk: high, medium, or low; significant, major, or minor, and so forth. The goal is to help to separate “the wheat from the chaff”, so to speak. Yet, even with articulated definitions of risk levels, it is often difficult for even the most experienced audit professional to anticipate the relative consequences and potential impacts of different findings and observations. Why is this the case? Because like many things in life – it’s complicated. This article focuses on the reason for these complications.

The Challenge of Assessing Relative Risks

Over many years of auditing, I have had to work with audit teams to determine risk levels for individual audit findings. This has not always been a straightforward experience as two trained audit professionals can interpret the very same set of facts in widely divergent ways. Ten examples of some of the more troublesome comparisons I’ve experienced are listed in Tables 1 and 2. Take a look, and see what you think. Your task is to select the higher risk observation in each case. And think about the choices as Jack Welch, the former Chairman of General Electric, did with his managers when evaluating staff performance. He required them to force-rank all staff, and the bottom 10% were vulnerable to re-assignment or even termination. Managers were not allowed to conclude that all their staff were above average.

So, in line with Mr. Welch’s philosophy, force-rank each of the scenarios provided below by choosing either Scenario A or Scenario B. You are not allowed to conclude that the risks are roughly equal. You must choose one over the other. When choosing try to keep in mind not just the observation itself but what the observation might say about the underlying controls or management systems in place or not. This might say something about whether that very same issue might be observed yet again on the subsequent audit conducted three years later. After evaluating the tenth scenario, it should become evident just how difficult it is to indeed assign risk on EHS audits. But that is exactly what auditors are expected to do.

Table 1: Making EHS Risk Choices for Environmental Topics

Table 2: Making EHS Risk Choices for Health & Safety Topics

I can only hope that you were as perplexed as I was with choosing the higher risk finding in each of the scenarios posed above. And this has not been a “trick exercise” where I am about to provide the magic answer. There is no magic answer to these choices.

Calibrating Relative Risks

One useful exercise would be to have a group of auditors in an organization each make the choices for the 10 scenarios given above. Then the overall percentage distributions could be calculated and entered into Table 3, Evaluating Risk Choices. For the cases where the result is that either Choice A or Choice B was made by close to 100% of those polled, that would suggest that there is a commonality of thinking amongst colleagues about relative risks posed by EHS issues. Where the results are closer to 50/50, then some organizational calibration might be required. Auditors working for the same organization should view risks pretty much the same way. A 50/50 split would suggest that they do not hold the same risk perceptions. In these cases, the program is at risk of being viewed as being inconsistently applied. And that is never a good thing for an audit program.

Table 3: Score Sheet

Closure

It is indeed a good thing that EHS audit programs are generally moving towards more risk-based approaches. Line managers and executives both really need to know the difference between real potential problems and administrative glitches. Yet assigning and evaluating risks on individual findings can be a challenging exercise. This is particularly true on the safety side of the business where even the most mundane of deficiencies can be viewed as life-threatening under a worst-worst case scenario. Training and calibration among a company’s professional auditors on relative risks posed by EHS findings of deficiency can help to lead the way to making any audit program more useful to the organization.

References

1. Cahill, L.B. and R.J. Costello, “EHS Audits-Have We Lost Our Way, Part III,” EHS Journal On-Line, March 12, 2013.
2. Cahill, L.B. and R.J. Costello, “EHS Audits-Have We Lost Our Way? A Sequel,” EHS Journal On-Line, August 13, 2011.
3. Cahill, L.B., “EH&S Audits – Have We Lost Our Way?” EHS Journal On-Line, July 11, 2010

About the Author

Lawrence B. Cahill, CPEA (Master Certification) is a Technical Director with Environmental Resources Management and has over 35 years of professional EHS experience with industry and consulting. He is the editor and principal author of the widely used text, Environmental, Health and Safety Audits, 9^th Edition and its 2015 follow-up text EHS Audits: A Compendium of Thoughts and Trends, both published by Bernan Press. He has published over 70 articles and has been quoted in numerous publications including the New York Times and the Wall Street Journal. Mr. Cahill has worked in over 25 countries during his career. He holds a B.S. in Mechanical Engineering from Northeastern University where he was elected to Pi Tau Sigma, the International Mechanical Engineering Honor Society. He also holds an M.S. in Environmental Health Engineering from the McCormick School of Engineering and Applied Science of Northwestern University, and an MBA from the Wharton School of the University of Pennsylvania. He is a Certified Professional Environmental Auditor, Master Certification.

Photograph: Drawing Serie by Miguel Ugalde.

Return to the EHS Journal Home Page