Problems with automaker’s diesel emissions reveal fundamental caution over popular approach to regulation.
Volkswagen’s diesel emissions scandal has properly raised questions about the character of that company’s management or corporate culture. But the crisis should also raise questions about a popular approach to regulation known as performance-based regulation. Although widely heralded for its flexibility and cost-effectiveness, performance-based regulation confronts a fundamental limitation: it can only be as good as a regulator’s ability to monitor outcomes.
A performance-based regulation is one that mandates the achievement or avoidance of a specified outcome, but then leaves it up to the regulated firm to figure out how to satisfy the required standard. An automobile emissions limit is a classic example of a performance standard. It specifies an outcome – a maximum level of emissions from a car’s tailpipe – but then leaves it up to each automaker to find a way to keep emissions below the limit. The automaker can then choose whether to adjust the engine’s fuel mixture or compression ratio, change the overall vehicle’s size and weight, install pollution control devices, or pursue some combination of techniques.
Giving automakers and other regulated firms flexibility makes a good bit of sense. After all, regulated companies know more about their products, services, and processes than any regulator ever can. If the alternative is for the regulator to choose a single solution and then mandate it across the board, that approach will likely prove costlier for some firms. At other firms, the regulator’s solution might not very work well at all. When compared with such a one-size-fits-all approach, performance-based regulation looks pretty good.
That is why policymakers from across partisan lines favor performance standards. President Bill Clinton directed federal agencies to try to “specify performance objectives, rather than specifying the behavior or manner of compliance that regulated entities must adopt.” President George W. Bush’s Office of Information and Regulatory Affairs declared that performance standards are “generally superior” to alternative forms of regulation. President Barack Obama not only explicitly affirmed his predecessors’ preference for performance standards but he took an added step to encourage agencies to pursue “flexible approaches” to regulation.
Flexibility is certainly laudable. In regulation, it can sometimes make results more sensible and cost-effective. But just as infatuated lovers fail to see each others’ flaws, bipartisan enthusiasts for performance-based regulation tend to overlook its fundamental weakness.
The case for performance-based regulation depends on a crucial assumption, namely that the outcomes it achieves will be close to the outcomes it mandates. Only if that assumption holds will performance-based regulation’s flexibility yield greater cost-effectiveness. When it does not hold, any cost-savings obtained by performance standards’ flexibility may come at the expense of benefits to society. In Volkswagen’s case, it appears that the company’s cars emit up to 40 times as many pollutants as everyone assumed based on the legal limits.
The Volkswagen case is hardly unique. The EPA went through a similar experience with manufacturers of heavy-duty diesel truck engines in the 1990s. Just as with Volkswagen, truck engine manufacturers had to meet an emissions limit in a laboratory test that checked for emissions under more than a dozen speed and torque conditions and then averaged the results. The engine manufacturers built engines that satisfied the test. But after a period of time on the road, the engines’ onboard computers allegedly adjusted the fuel mix automatically to optimize for responsiveness rather than emissions control.
As the late Yogi Berra said, déjà vu all over again.
Over the years, problems have arisen with other performance standards too. In New Zealand, for example, a performance-based approach to building regulation contributed in recent decades to one of that country’s major public health and financial crises involving up to nearly 90,000 leaky and moldy buildings.
When automobile manufacturers in the United States first designed air bags to meet performance-based safety standards, the systems worked to protect many people; however, they also actually caused numerous fatalities to children and adults smaller than the average adult male who occupied front seats in otherwise minor accidents. Why were smaller individuals put in danger? Because the mandated performance test called for crash test dummies the size of the average adult male and the airbags deployed at speeds meant for someone of that body size and mass. (Fortunately, regulators learned and now have standards calling for crash test dummies of different sizes.)
In yet another example, high-stakes testing of elementary and high school students, a type of performance-based approach to oversight of education, has seen about a dozen educators in Atlanta prosecuted for cheating. Even more worrisome are claims that such educational testing actually worsens the quality of instruction, as students are taught strategies to score better on tests rather than working to develop deeper academic knowledge and cognitive skills.
As should be apparent, the Volkswagen case is just the latest example of the “teaching to the test” phenomenon.
Performance-based regulation’s flexibility is both its principal advantage and its chief challenge. Flexibility gives firms room to find ways to comply with the letter of the law while still advancing their private goals – but sometimes they do so at the expense of public goals.
In the case of automobile emissions, the law did anticipate this possibility and expressly forbade the use of “defeat devices.” In Volkswagen’s case, that seems to be what the company did that was illegal, as its onboard computers apparently detected when cars were connected to laboratory emissions monitoring equipment. (The truck engine manufacturers in the 1990s contested the EPA’s claim that they had installed defeat devices and the agency eventually settled out of court with the firms.)
It does not deny the reprehensibility of what Volkswagen is accused of doing to recognize that the performance-based regime in this case needed much greater enforcement scrutiny. Given how extensive Volkswagen’s transgressions appear to have been, it is shocking that it took so many years for regulators to discover the problem. And they eventually did so only because a small but enterprising non-profit group paid a university researcher to test two Volkswagen vehicles on the road.
Recognizing the limits of performance-based regulation does not mean that governments should never adopt performance standards. It also does not deny their cost-effectiveness. Rather, recognizing their limits means that regulators everywhere must be more vigilant. It means the unbridled enthusiasm for performance-based regulation needs a reality check. Yes, in theory, performance-based regulation might be “generally superior” to other approaches; in reality, it only works when regulated firms’ performance can be assured.
The Volkswagen case makes clear that regulators opting to use performance-based regulation need to pay careful attention to how they will monitor compliance once firms start to take advantage of the flexibility that performance standards offer.
Professor Coglianese’s previous essays in The Regulatory Review on performance-based regulation include “The Performance of Regulatory Performance Standards” and “Is Flexible Regulation an Oxymoron?” His other related publications include “The Limits of Performance-Based Regulation” and “The Law of the Test: Performance-Based Regulation and Diesel Emissions Control” (with Jennifer Nash).
