Satisfaction Is Not the Same as Policy Success

Regulatory policies that satisfy the interests involved in policy-making are often assumed to be better policies. Numerous studies purport to evaluate the success of public participation by asking individuals what they think of the regulatory processes and outcomes in proceedings in which they were involved. Although data on participant satisfaction can potentially provide useful feedback to those who facilitate various kinds of policy deliberations, satisfaction is not an appropriate basis for evaluating the overall value of public policies or even ultimately of the processes by which these policies are made.

A policy decision may satisfy those who participate in making it, but this does mean it is a good policy decision. Regulatory policies that satisfy everyone involved in the decision-making process may simply be based on the lowest common denominator of the policy-making group, or they may result from the predominance of a “group think” mentality that leads participants to believe the policy is a good one. The reality is that any policy may prove to be effective or ineffective, efficient or inefficient, and just or unjust. Policies that greatly satisfy participants can still be prone to error. Consider two examples of policies that were satisfactory to participants when they were created, but later turned out to be significant policy failures.

The first example is from the early 1990s, when the EPA convened a negotiated rulemaking to develop standards for new, reformulated blends of gasoline that would reduce urban smog. The agency selected representatives from the automobile and petroleum industries, as well as from the environmental community, in an effort to agree on standards for the new fuels. Political scientist Edward Weber has written that “the entire process met with considerable success” because “[s]atisfaction with the negotiated outcome led all participants to sign a consensus agreement.” Despite the participants’ presumed satisfaction, the resulting negotiated regulation turned out to be one of the most problematic of all of the EPA’s rules in the history of the agency. Following implementation of the rule, the key chemical additive used in reformulated gasoline—MTBE—was discovered to be leaking into groundwater across the country, causing substantial public alarm about drinking water contamination and resulting in legislative and regulatory efforts to ban the use of this fuel additive. A little more than a decade ago, Discover magazine included the reformulated gasoline decision in its list of “Twenty of the Greatest Blunders in Science in the Last Twenty Years.”

The second example is another enormous policy blunder that was remarkably satisfactory to participants at the time the policy was adopted: the restructuring of California’s electricity markets in the 1990s. State legislators adopted electricity restructuring in California after extensive public participation, structuring the new market in a way that effectively achieved a compromise among the major players. Because wholesale rates were low, this compromise, which among other things capped retail rates but allowed wholesale rates to float with the market, was generally satisfactory to the participants when the legislation passed. Indeed, at the outset of the restructuring, there were surprisingly few critics of the plan in California. Only later, under a taxing set of circumstances, did California’s “satisfactory” compromise reveal itself to be seriously flawed, wreaking havoc on Californians and utility companies as wholesale rates and demand for power increased.

In both of these examples, it should be obvious that the resulting policies failed, no matter what the original participants may have thought about the policies when they were adopted. Many of the participants in these policy-making processes might well even agree with such an assessment now, as they presumably are no longer as satisfied with the policies they helped create.

These examples starkly demonstrate that just because many people may be satisfied with a policy when it is created this does not necessarily mean the resulting decisions constitute good public policy. When evaluating the effectiveness of different forms of public participation, researchers must focus squarely on the underlying outcomes of policy decisions that result from different participatory processes rather than rely on measures of participant satisfaction.

Of course, participant satisfaction could very well be a relevant consideration to take into account if policies that garnered satisfaction were also—at the margin or on average—higher-quality policies. Yet, how likely is this? Regulation is needed to solve problems of externalities, information asymmetries, and monopoly problems that arise in situations where private interests are not well aligned with overall social interest. The goal of regulation is to change the behavior of firms and individuals to support improved social welfare. As such, the targets of regulation presumably should not find it very satisfying to be regulated.

We know that business firms, as the principal target of regulation, are typically better organized than other groups in society and they tend to participate more regularly and intensively, and in greater numbers, in public policy-making. If these targets of regulation come away satisfied with a policy-making process, it may well be that the resulting policy decision has not been as effective as it needs to be. For this reason, participant satisfaction in many cases could actually indicate that a policy is of a lesser quality, not greater.

Even though data on participant satisfaction can be collected relatively easily by distributing surveys to participants or by interviewing them, researchers, policy analysts, and public decision-makers should look elsewhere for a meaningful basis for evaluating regulatory policies and the process of public participation in regulatory policy-making. Ultimately outcomes matter.

Cary Coglianese is the Edward B. Shils Professor of Law at the University of Pennsylvania Law School, where he teaches administrative law. His most recent book is Does Regulation Kill Jobs?.