Regulatory Trial and Error

Trial-and-error decision-making could provide great benefits to regulators.

Over numerous millennia of cultural evolution, human beings have devised few if any methods of decision-making more effective than old-fashioned “trial and error.” At its most basic level, the scientific method is an applied form of this concept: formulate a hypothesis, test it against observed reality, and retain or reject it depending upon the results. So, too, are democracy and free market economics: a “good” result is most likely to emerge from millions of individualized actors making decisions on the basis of limited information, and that result should improve over time as better information becomes available.

The modern regulatory state, which first took shape in the United States during the Progressive Era of the late nineteenth and early twentieth centuries, has traditionally favored a more deductive, less empirically-minded approach to decision-making. President Woodrow Wilson, one of the intellectual architects of the regulatory state, argued that the “science of administration” would extract government policymaking from the confused melee that characterized democratic republicanism, turning it over to well-trained experts who would ply their superior knowledge to carry out the popular will while maximizing efficiency, fairness, and other important policy goals.

Regardless of whether one sees the modern regulatory state as a godsend or an anathema, scholars of the political left, right, and center would all likely agree that the science of administration has received a mixed reception among the American public. Calls to rein in regulators have proven to be a highly effective political strategy, deployed by Republican and Democratic politicians alike, suggesting that the American people have limited faith in the wisdom and efficacy of administrative science.

In one sense, the unpopularity of modern bureaucracy is not an especially compelling argument against it. It is of course entirely possible that a large segment of the populace or even elected representatives may vote for policies that undermine their own interests, especially when a small but highly influential group of citizens or businesses has a strong incentive to turn public opinion in a particular direction. But the sustained popular angst surrounding regulatory decision-making more broadly, rather than specific policy decisions that create winners and losers, suggests something more profound than overweening skepticism is at play.

This angst, I would argue, is at least partly caused by very pervasive and fundamental philosophical doubts concerning the soundness of the science of administration. And the numerous calls for regulatory reform over the past five or so decades, I would suggest, are at least partly geared toward pushing power down from centralized bureaucracies and reintroducing a more trial-and-error-based system of policymaking. A quick survey of some of these efforts is illustrative:

Sometimes, power has been taken away from the regulatory state completely and turned back to the free market, such as with airline deregulation in the ‘70s.
At other times, power has been decentralized, with policymaking authority at least partly turned over to the states, as with the cooperative federalism regime of the Clean Air Act, or to private actors, as with the National Technology Transfer and Advancement Act.
Still other times, federal regulators are directed to learn from lower-level governments, such as with Executive Order 13,132, or foreign governments, such as with Executive Order 13,609.
Throughout the past several decades, Presidents have attempted to introduce economic tools such as benefit-cost analysis into regulatory decision-making and to encourage or force agencies to clear out old rules when adopting new ones.

These efforts have undoubtedly promoted a more empirical, evidence-based system of regulatory policymaking. But the reforms have largely proceeded pell-mell, characterized by a series of one-off efforts of the President or Congress to improve some discrete aspects of the regulatory state.

As members of Congress and the President contemplate further reform to the regulatory state, they should strive to capture the enormous benefits of trial-and-error decision-making. Two basic principles should guide this effort.

The first is that regulatory policy never can nor should be fixed in stone. Trial and error is a never-ending process that does not yield a “right” answer, only marginally better ones. As such, a robust system of retrospective review is fundamental to any well-functioning regulatory system. And a forcing mechanism such as a regulatory budget may be needed to overcome the profound disincentives to effective retrospective review, including “the Ikea effect” (in which people tend to overvalue things they themselves created) and the reluctance of regulators and regulatory parties alike to shift course once they have incurred sunk costs.

At the same time, regulatory dynamism is not an unalloyed virtue. The rule of law is partly premised on the notion that some degree of stability is needed to allow private actors to plan their future conduct. Indeed, Professor Aaron Nielson has even argued that some measure of regulatory “ossification” can be a good thing, ensuring that the rules are not constantly changing and disrupting settled expectations. Retrospective review efforts should therefore be sensitive to both the benefits and costs of regulatory change.

The second principle is that regulatory policy should be designed to promote learning. It is valuable to think of regulatory learning as a hierarchy. Agencies should, at the very least, always be aware of relevant precedents. In certain cases, they may go much further, actually designing a rule to allow them to test its efficacy and easily readjust it if a better course of action arises.

It is possible, as I outline below, to provide a very basic sketch of what such a regulatory learning hierarchy may involve. The four tiers of such a hierarchy track in many important respects a recent recommendation on regulatory learning issued by the Administrative Conference of the United States.

Tier One: Regulatory Awareness. Before regulating, the agency should ask whether regulation is indeed necessary, considering existing efforts of state governments and self-regulatory regimes erected by private actors. This question is similar to the one posed by the European concept of subsidiarity. Currently Executive Order 12,866 and Circular A-4 call upon agencies to ask if regulation is truly needed, but federal agencies often pay inadequate attention to this critical first step, simply assuming the existence of a problem. If the agency decides federal intervention is needed, in determining how to craft its intervention it should consider the successes and failures of existing regulatory efforts, including its own past actions, state and international precedents, and self-regulatory regimes.

Tier Two: Natural Experiments. Regulations will often naturally be designed in a way to allow comparative learning. For example, consider a regulation in which firms with annual revenues below $100 million are exempt from its requirements. The agency can take advantage of this threshold effect, looking at firms just above ($101 million) and below ($99 million) the cutoff to determine how the regulation affects virtually identical parties. Similarly, when only one or a handful of regulated entities requests and receives a waiver or exemption, the agency can compare the results in regulated and unregulated firms. This information should prove valuable in deciding whether to later modify or eliminate the regulation.

Tier Three: Built-in Variation. Agencies should, wherever possible, strive to use performance standards rather than design standards, setting an overarching goal but leaving it up to state regulators or private parties to determine how that goal will be met. Sometimes, this may involve allowing variation for a period of time until evidence emerges indicating that one approach is superior to another, at which point the agency may enshrine that approach in a rule. At other times, the agency might allow for ongoing variation and adaptation, as it does in programs involving cooperative federalism.

Tier Four: Policy Pilots. Agencies can actually design some rules as experiments, with random assignment of regulated entities to a control group or to one or more treatment groups. This is the most ambitious approach to regulatory learning and the most analogous to what a scientist does in a laboratory. It raises the most issues too, including legal concerns about equal protection and reasoned decision-making, and potential problems related to the fairness and efficiency of treating similarly situated businesses or regulatory beneficiaries differently for arbitrary reasons. And, of course, conditions will change over time, such that the agency may need to reconduct the experiment at a future date.

At the end of the day, any system of governmental regulation cannot match the level of dynamism that prevails in an unregulated market, which much more effectively captures the benefits related to trial and error but which may sometimes be less effective in protecting other societal goals such as equality or fairness. But government regulators can make much more effective use of trial and error than what they currently do, learning from existing variations and explicitly designing rules to allow for variation that will promote ongoing improvement. Although human ingenuity may yet devise a more effective means of learning than humble trial and error, that day has not yet arrived, and regulators would do well to design their rules accordingly.