Evaluation of the Effectiveness of Pollution Prevention Planning in NJ

II. Key Results
Although the findings of this study indicate generally that planning was successful, not all elements of planning were a success, nor did all facilities benefit equally from planning. Unless otherwise stated, these findings and conclusions apply to the 115 facilities visited by HRA and NJDEP. An association or correlation between planning factors and measures indicates that the relationship is statistically significant to the 95 percent confidence level. See Section III for a discussion of plan completeness and plan quality.

II.A. Analysis of Findings

II.A.1. Elements of Pollution Prevention Plans

Process-Level Materials Accounting. Detailed, process-level materials accounting is the main feature of the New Jersey Planning Rule. Facilities are required to provide materials accounting data for each process and quantify sources of NPO within each process. Clearly, the process-level materials accounting requirement was successful in terms of plans produced. Only 25 percent of facilities had tracked NPO per unit product in some form before planning was required, and usually in the form of process-level yield, so for many facilities, the New Jersey plan was their first efficiency calculation.

Specific findings:

• Knowing more about NPO generation correlates to better results. The more knowledge about how NPO is generated within processes, the better the end product - in this case, the facility's pollution prevention plan.

1. Tracking NPO to individual sources was associated with greater reduction goals. Attributing NPO generation to individual sources within each process was associated with more complete plans and larger average percent NPO reduction goals. The average NPO reduction goal was 15.6 percent for facilities that attributed NPO to individual sources vs. 8.4 percent for those that did not.
2. Tracking NPO per unit product prior to planning was also associated with greater reduction goals. Facilities that had tracked NPO per unit product prior to planning set percent use reduction goals slightly higher than facilities that had not tracked NPO per unit product prior to planning.

Cost Analysis. The planning rule currently requires facilities to do a complete cost analysis of all processes at the facility, including those not targeted for reduction. The intention was to provide facilities with a different view of NPO generation: in terms of what it costs them, not just the amounts generated (the largest NPO streams may not cost the facilities the most money to manage, and this information might cause the facilities to target different processes than they would using amounts of NPO). It should be stated from the outset that nearly half of the facilities (48 percent) did not do a complete cost analysis as required in the rule, so the cost requirement failed, de facto, to achieve this aim. Beyond this, however, completing the cost analysis was not associated with greater reduction goals (plan quality and completeness used cost analyses as part of their criteria, and so could not be tested). Facilities perceived the cost analysis to be too difficult and unnecessary. The main stumbling block seemed to be in calculating cost savings for personnel assigned to environmental tasks: Since these employees would still be working at the facility, it was difficult to postulate cost savings from reducing their time spent on environmental activities.

Specific findings:

• A majority of facilities that did the required cost analyses report that cost analyses provided them with different views of NPO, but such analyses are not associated with increased pollution prevention goals. Although one-third of the 48 facilities visited by HRA thought that assigning environmental costs to NPO generation within each process as required by the planning regulation was the most difficult part of completing their plans, 11 of the 21 facilities that did a financial analysis of NPO generation (52 percent) stated that the analysis yielded a different picture of NPO generation than other NPO analyses. However, none of these facilities reported targeting different processes or selecting different reduction options based on their environmental costs.
1. Attributing environmental costs to each process was not associated with greater percent reduction goals.
2. When facilities calculated cost savings from reduction options, they mostly calculated raw material cost savings, not savings in environmental costs.
3. Calculating environmental cost savings beyond those required in the plan was not associated with more complete plans or greater reduction goals.

Grouping and Targeting Processes. The latitude given facilities by NJDEP in preparing their plans should be counted as a success, both in simplifying the planning process and in the completeness, quality, and percent reduction goals in the plans themselves.

Specific findings:

• Selection of options to group processes or sources and to target processes does not appear to hinder effectiveness of pollution prevention planning. New Jersey gave facilities significant latitude in completing two parts of the planning process - grouping of processes and/or sources and targeting on the basis of use, NPO, or releases - and facilities took advantage of this latitude with no negative effects on plan completeness, plan quality, or percent reduction goals.
• Most facilities report that grouping simplifies planning. Seventy-four percent of the 115 facilities grouped processes or NPO sources, and 73 percent of these reported that grouping reduced the number of processes and sources that had to be examined in planning. For the 48 facilities visited by HRA, 67 percent grouped processes or NPO sources, and 81 percent of these reported that grouping reduced the number of processes for planning purposes. Also, 72 percent of the facilities visited by HRA that opted for grouping reported that grouping saved them time or money in the planning process.
• High-end targeting is associated with higher use reduction goals. For all facilities, targeting 90 percent or more of use, NPO, or releases was associated with a larger average percent use reduction goal, as opposed to not targeting (i.e., planning for all processes) or selecting processes that comprised less than 90 percent of use, NPO, or releases (use reduction goals of 2.6 percent vs. 1.5 and 1.6 percent, respectively).
• Nearly all facilities appear to use targeting appropriately. Virtually none of the 115 facilities seemed to use targeting as a way to avoid developing reduction options on processes that are important or used targeting on use or releases to "hide" processes with large amounts of NPO.

II.A.2. Previous Planning and Reduction Activities

Previous Planning Experience. The pollution prevention plans created under the New Jersey planning rule are significantly different from environmental plans created prior to the planning requirement, since none of the planning factors examined in this study - previous pollution prevention planning, formal environmental planning, and business reviews which included environmental costs - showed any association with plan quality, completeness, or reduction goals. If there is a learning curve for planning activities as there appears to be for source reduction activities (see below), then one would expect to see that previous planning activities had some effect on facilities' plans. Since they did not, this finding reinforces the view that process-level scrutiny in the New Jersey planning rule is both effective and different from previous planning activities.

Specific findings:

• Most facilities have environmental planning experience, but environmental planning experience does not correlate with more effective pollution prevention planning. A majority of the 48 HRA facilities reported previous environmental planning activities, including waste minimization planning, pollution prevention planning, and business reviews which included environmental elements.
• Of the 48 HRA facilities, 83 percent had undertaken some form of environmental planning in the seven-year period prior to being interviewed. Almost two-thirds (62 percent) had undertaken waste minimization planning, and over one-half (56 percent) had undertaken some form of pollution prevention planning, either alone or in the context of other planning. Facilities with no pollution prevention planning prior to the New Jersey requirement constituted 44 percent, and 17 percent of facilities had no experience with environmental planning.
• Almost 60 percent of facilities interviewed had undertaken business reviews that included environmental costs over the previous seven-year period, and 46 percent had some formalized approach to environmental planning, as opposed to strictly responding to compliance requirements or emergency situations.
• Although prior pollution prevention experience was associated with better planning measures such as completeness and increased average percent reduction goals, prior planning experience was not. Of the three planning factors tested - previous calculation of environmental costs in business reviews, formal environmental plans (as opposed to one-time responses to emergencies or compliance requirements), and previous facility-wide plans - none was associated with more complete plans, better plans, or increased percent reduction goals.

Previous Source Reduction Activities. Interestingly, facilities that had undertaken source reduction activities prior to planning created greater reduction goals in their plans. This finding has several implications with regard to planning and source reduction activities. First, there may be a learning curve associated with source reduction, so that facilities become better at spotting opportunities once they've learned more about source reduction by undertaking source reduction activities. This conclusion is supported by the fact that more facilities described previous source reduction activities to HRA in interviews than reported source reduction activities on state and federal reporting forms (see Section II.A.4). Many of these reduction activities were in fact source reduction, although the facilities may not have considered them to be source reduction when filling out the forms, mainly when the result was unintended or the activity concentrated on chemicals other than those for which the source reductions were primarily achieved. Another possible explanation is that facilities may be more familiar with source reduction in general as more guidance has become available from industry associations, government, and NGOs - in other words, a rising tide floats all boats.

The major conclusion arising from this finding is that source reduction opportunities are still plentiful, even for facilities that have undertaken previous source reduction activities. If, as some industry associations claim, facilities had exhausted the "low-hanging fruit" and now required more capital investment for diminishing returns in reduction projects, it would follow that facilities that had undertaken previous source reduction activities would also find fewer opportunities in their plans. The planning rule does not require facilities to create non-zero reduction goals, and 47 percent of all reduction goals submitted to NJDEP on plan summary forms were zero. The fact that facilities identified source reduction goals through planning beyond the activities already undertaken indicates that many source reduction opportunities still exist. It may also imply that future reductions will increase as a result of the planning requirement.

Specific findings:

• Hands-on experience helps. Previous pollution prevention experience was helpful in creating facility-wide plans.
• For the 115 facilities visited by HRA and NJDEP, undertaking source reduction activities prior to planning (as reported on New Jersey DEQ-114 forms and U.S. TRI Form Rs) was associated with more complete plans and larger average percent use and NPO reduction goals. That is, facilities that had previously reported source reduction activities on state and federal reporting forms were more likely to have more complete plans and larger average percent use and NPO reduction goals than facilities that had not previously reported source reduction activities (percent use reduction goals of 2.7 percent vs. 0.8 percent and percent NPO reduction goals of 15.3 percent vs. 2.1 percent).
• HRA also asked its 48 facilities if they had undertaken source reduction in the past five years. More facilities described source reduction projects to HRA than had reported source reduction activities on state and federal reporting forms. For these facilities, previous source reduction activities were associated with more complete plans and better plans.

II.A.3. Pollution Prevention Goals

Previous Reduction "Goals." Percent NPO reduction goals set by facilities in their pollution prevention plans are, on average, much higher than previously reported reduction projections. HRA examined facilities' 1991 and 1992 DEQ-114 forms, looking at projections for reductions in NPO by source reduction over a five-year period and comparing them to reduction goals in their plan summaries. Facilities created NPO

reduction goals over twice as great in their plans as they had projected on their earlier DEQ-114 forms. Although it can be argued that these reduction projections are not "goals" per se in the way that planning requires facilities to make reduction goals and that they may be conservative estimates of reductions, they do represent facilities' ideas of what they expected to achieve by source reduction (see Section III.D for an explanation of the limitations of the data).

It might also be argued that facilities were climbing the learning curve for source reduction in 1991 and 1992 and may have come up with greater goals in 1993 and beyond without required planning. However, nearly one-third of facilities also reported that planning caused them to examine more processes than they would have without planning. Two-thirds reported identifying more reduction options and setting higher reduction goals through planning, indicating that the planning process was key to setting those higher goals.

Specific findings:

• Planning is associated with greater projected reductions in NPO. All facilities projected greater percent reductions in NPO, on average, in their pollution prevention plans than they did in their DEQ-114 reports prior to such plans.
• Fourteen of the 48 facilities visited by HRA (29 percent) stated that planning was a spur to examine more processes, identify new reduction options, or make greater reduction goals than they would have without the planning requirement.
• Thirty-two facilities (67 percent of those visited by HRA) reported that they identified some or all of their reduction options through planning. Identifying reduction options through the planning process was associated with greater average percent NPO reduction goals in the pollution prevention plans than finding no new options through planning (18.7 percent vs. 3.7 percent).
• Of the 115 facilities visited by HRA and NJDEP, facilities projected percent NPO reductions at least twice as great on average in their plans as they had previously reported to New Jersey on their DEQ-114 forms (19.0 percent vs. 8.1 percent), and the differences in percent reduction goals were statistically significant at the 95% confidence level.
• Overall, NPO reductions projected by all 405 facilities that submitted plan summaries to NJDEP by July 1, 1994, were similarly greater on average after required planning (20.0 percent vs. 7.7 percent), and the differences in percent reduction goals were also statistically significant at the 95% confidence level.

II.A.4. General Outcomes of Planning

NPO Generation and NPO/Use Ratio. Planning appears to have been more effective for facilities generating larger amounts of NPO and facilities with a higher NPO/use ratio. This is not surprising - it makes sense that facilities generating greater amounts of NPO would find greater percent reductions, and generating more NPO might necessitate better internal accounting procedures that would also lead to greater plan completeness. A higher NPO/use ratio indicates that more of a given material on site ends up as NPO, therefore less is consumed in process or shipped in product. Typically, these types of applications where this occurs - coating, degreasing, cleanup, etc. - are more amenable to reduction options with more immediate results than those requiring reformulation.

Specific findings:

• The more that NPO is generated and the less that chemicals are consumed on site or shipped in product, the more effective the plan. These facilities set higher percent reduction goals, as well as producing more complete and better plans.
• Facilities that generated more than 2.0 million pounds of NPO in 1992 set slightly higher percentage use reduction goals in their plans than facilities that generated less than 2.0 million pounds of NPO, and the difference was statistically significant to the 95% confidence level. The 2.0 million pound level reflects a natural break in the DEQ-114 database for 1992 rather than an arbitrary NPO level for facility classification. (see III.A.)
• Facilities for which more than 25 percent of chemical use ends up as NPO also set higher use reduction goals than facilities with an NPO/use ratio less than 0.25.

Value of Planning to Facilities Themselves beyond Fulfilling Requirements. More facilities than not found planning worthwhile. Realistically, facility planners were reluctant to denigrate the results of a process that took their time and effort, although those who did were outspoken about it. Finding planning worthwhile was associated with greater reduction goals for the 115 facilities, begging the question of which came first - setting greater reduction goals and deciding that it was a good experience or deciding that planning was a worthwhile activity no matter what the outcome. Finding planning worthwhile did not correlate with a more complete or better plan, so it is difficult to say that prior attitudes toward planning affected the outcome. More importantly, facilities found other benefits to planning beyond creating the plan itself, indicating that their finding planning worthwhile was genuine.

Specific findings:

• Planning takes time, but most facilities find it worthwhile. While 52 percent of the facilities visited by HRA stated that planning activities took key personnel away from important activities, 63 percent of facilities believed that planning was worthwhile and had made important contributions to facility operations.
• Seventeen facilities (35 percent) stated that planning increased their awareness of operations, including processes, NPO generation, and individual sources of NPO. For six facilities (17 percent), the New Jersey plan was the first facility-wide examination of NPO other than required reporting they had done. Two facilities also stated that the New Jersey plan was the first facility-wide long-term plan done on site.
• Seven facilities (15 percent) stated that they could use the pollution prevention plan as background information to convince management to fund source reduction projects.
• Other benefits recounted by facilities included increased employee good will and an increased pace of reduction compared to previous reduction activities.

Planning Costs, Cost Savings, and Time Requirements. NJDEP's preliminary results indicated that planning paid for itself eight-fold, including administrative costs of the program by the Department. HRA also found that planning pays for itself. There are two ways to calculate planning costs: taking facility dollar estimates of planning costs as is or taking facility time estimates and multiplying them by an average salary. Facility dollar estimates usually have a time and salary component in them unless the plan was prepared by consultants, in which case facilities reported the amount paid to the consultant. Dollar costs of planning as reported by facilities were always higher than costs calculated by average salary. Either method of calculating cost shows that average planning costs, while not exactly trivial, reflected the time necessary to gather process level materials accounting data, since these data were unavailable at most facilities prior to planning. HRA found a higher average planning cost than NJDEP - not unexpected, since HRA examined more large facilities as a percentage of total visits than NJDEP.

It is important to mention that most facilities calculated only the direct savings in raw material costs for their reduction projects (see II.A.1, above). Since it was difficult for facilities to quantify environmental cost savings, and even more difficult to quantify the other benefits of planning, estimates of cost savings are probably low.

Specific findings:

• Half of all facilities visited found planning pays. Fifty-three percent of the 115 facilities visited found a net cost savings in their plans, while 46 of 115 facilities (40 percent) found no net cost savings.
• Twenty-eight of 115 facilities were able to calculate their annual cost savings, for a total of $7.1 million. These figures yield an average of just under $254,000 in annual savings.
• Averaging the $7.1 million in savings for the 61 facilities that found a net cost savings in their plans yields an average annual savings of $116,000.
• Averaging the $7.1 million in savings for the 107 facilities that could definitely state that they either did or did not find savings in their plans results in an average annual savings of just over $66,000 per facility.
• Eight of the 48 HRA facilities had calculated the costs/amounts of both capital investment and annual cost savings. These eight facilities together showed a total capital investment of $7.2 million and a total annual return of $1.7 million, for an average payback period of 4.2 years.
• Potential capital investment to complete all of the projects in facilities' plans ranged from zero to $19.0 million for the 48 HRA facilities (nearly half of the facilities visited by HRA were large, generating more than 2.0 million pounds of NPO in 1992). Eight facilities stated that no capital investment would be required.
• Average savings from reduction projects outweigh planning costs. Although some facilities spent a great deal of time and money on their pollution prevention plans, the average cost of planning was less than the average annual savings per facility found through planning, including those facilities that found no cost savings.
• The 49 of 115 facilities that could place a dollar figure on the cost of their planning spent an average of $35,000 each.
• The 95 of 115 facilities that could quantify the amount of time necessary to prepare their plans spent, on average, 6.5 weeks of one person's time to develop the plan. Taking an average salary of $50 per hour yields an average planning cost of $13,000 per facility.
• Twenty-three of the 48 facilities visited by HRA stated that their time or cost estimate for preparing the plan included time for research and development and for personnel other than environmental or management staff.

Observations on Pollution Prevention and Data Reporting. Facility site visits provided the opportunity to gain insight into facilities' conceptions of pollution prevention and their practices in data reporting. It is safe to say that there is still learning about pollution prevention and reporting to be done.

Specific findings:

• There is still considerable confusion about just what constitutes pollution prevention. Many facilities still include all waste management activities in their corporate definitions of pollution prevention and may not recognize reduction projects as source reduction activities. However, virtually no facilities included reduction projects in their pollution prevention plans that were not source reduction.
• Thirty-eight percent of the facilities visited by HRA include activities other than source reduction and in-process recovery in their corporate definitions of pollution prevention, although this figure is down from 59 percent five years ago.
• While 83 percent of facilities described to HRA real source reduction projects implemented before planning was required, only 60 percent reported undertaking source reduction activities on state and federal reporting forms.
• Site visits uncovered numerous data discrepancies. Often, facilities have one person completing federal TRI forms and another person completing state DEQ-114 forms. Even when the same person or group of people complete both sets of forms, inconsistencies occur, both within the DEQ-114 forms and between the two sets of data.
• Thirty percent of facilities visited by HRA and NJDEP had data discrepancies between TRI and DEQ-114 reporting.
• Fourteen percent of facilities had an imbalance between inputs and outputs on DEQ-114 forms.
• Forty-eight percent of the 115 facilities found incorrect DEQ-114 or TRI data either through the site visit or the planning process.

II.B. Conclusions

The majority of facilities appear to have set higher reduction goals, on average, than they would have without required planning. As a result of the planning requirement, many facilities examined processes that would not ordinarily have been examined, investigated new reduction options, and, on average, set reduction goals that far outstripped other reduction projections made before planning was required.

Most facilities found planning worthwhile and found benefits beyond reduction goals and fulfilling regulatory requirements. These benefits included inventories of processes that would not previously examined, greater understanding of processes, and a background/cost/benefit framework to propose capital investment projects.

Process-level materials accounting was a successful (and, in many cases, new) planning tool for identifying reduction opportunities. The heart of materials accounting - attributing NPO to individual sources within a process - was key to successful planning, especially in terms of setting higher reduction goals.

Pollution prevention opportunities continue to be plentiful even for facilities with previous reductions, and planning was a useful approach to identifying them. Facilities that had undertaken source reduction activities prior to planning set greater percent reduction goals in their plans, even though setting no further pollution prevention goals was an option.

The cost accounting requirement did not achieve its purpose. Since most facilities did not perform a cost analysis as specified in the planning rule, the cost analysis requirement could not achieve its purpose. In addition, those facilities that did undertake the full cost analysis did not appear to benefit more than those that didn't in terms of creating a better plan or greater reduction goals. Although eliminating the cost analysis requirement is one option in response to this finding, a more productive approach might be to modify the requirement to include cost analyses for targeted processes only in the form of complete cost analyses for reduction options (Part II of the plan requires facilities to compare the environmental costs of reduction options to those of the original process). It makes sense to include Part II cost analyses, since facilities would have to do a cost comparison to receive funds for reduction projects.

Most facilities did not appear to understand the benefits of cost accounting. Although NJDEP's guidance document provides useful information on beginning total cost accounting, and NJDEP offered facilities software to assist them in preparing that portion of their plan, facilities might benefit from "real world" examples. Fifty-two percent of facilities visited by HRA stated that planning took key personnel away from "important" activities - that is, productive activities - but they did not make the connection between this effect and the value of time recaptured from reductions that lead to decreased reporting, labeling, manifesting, etc. "I get paid anyway," was the most common response. Similarly, although some facilities were willing to undertake source reduction projects that did not meet their normal cost criteria because they would demonstrate the facilities' commitment to pollution prevention, plant personnel were reluctant to attempt to place a dollar value on those benefits.

"Small" facilities benefited less from the planning process than larger facilities. Planning appeared to be less effective for facilities with smaller amounts of NPO. Many are reluctant to "tamper" with operations that are running well and are profitable, even though small reductions in NPO generation can provide cost savings.

Site visits assisted most facilities as well as NJDEP. Most facilities visited by HRA gained concrete information about pollution prevention and/or data reporting during the visit itself, both as described in the findings above and in terms of general information. NJDEP staff reported the same experience. At least one-sixth of the facilities visited by HRA were unaware of changes to TRI reporting thresholds that provided added incentives to reduce NPO generation to less than 500 pounds per year per chemical. As facilities explore more pollution prevention options, even more reporting issues will arise.