Evaluating Regulations after the Fact

ACUS calls for incorporation of retrospective analysis into the regulatory process.

Ever since the Reagan Administration, federal regulatory agencies have routinely conducted benefit-cost analyses of significant new regulatory proposals before they adopt them. However, these same agencies only infrequently look back, after adopting new rules, to determine whether these rules actually serve their intended purposes. That will start to change if agencies act on recommendations adopted late last year by the Administrative Conference of the United States (ACUS), a federal agency charged with finding ways to improve the work of regulatory agencies.

Although every president dating back to Jimmy Carter has called for some type of evaluative process where agencies identify – and, where necessary, modify or eliminate – existing regulations that are no longer effective, ACUS found that “retrospective review of regulations has not been held to the same standard as prospective review.”

ACUS’s recommendation aims to address this imbalance by helping agencies foster a “culture of retrospective review.” If agencies can more regularly evaluate rules’ effectiveness, they can use that information to address current problems and guide future decision-making, according to ACUS. As a result, ACUS urges agencies to approach retrospective review as a valuable, mission-advancing practice, rather than “an episodic, top-down reporting and compliance obligation.”

In explaining its recommendation, ACUS identified buy-in from top agency officials as a key factor in institutionalizing retrospective review. If agency leaders view retrospective analysis not as a separate, additional requirement that is solely backward-looking but rather as part of an “ongoing process,” then a “durable commitment” to retrospective review will start to develop, according to ACUS.

Agency-level commitment will have an even greater effect, ACUS has suggested, if agencies collaborate with one another and with the Office of Management and Budget (OMB) on retrospective review. To facilitate such collaboration, ACUS recommends creating a high-level government body dedicated to “promot[ing] a coherent regulatory scheme that maximizes net benefits.”

Although buy-in and collaboration may be critical to making regulatory “look-backs” a core component of the regulatory process, ACUS’s recommendation makes it clear that further action is needed for agencies to capture the full benefits of retrospective review. ACUS has proposed that each agency tailor retrospective review to fit with its mission. Rather than creating a one-size fits all model, ACUS identifies common themes of successful efforts and suggests measures that regulatory agencies and the Office of Information and Regulatory Affairs (OIRA) can take to create and support a culture of retrospective review.

ACUS has also encouraged agencies to incorporate plans for retrospective review into the design of new regulations. To the extent that they can, agencies should identify not only the objectives of a new rule, but also the methods they will use and the data they will need to measure progress towards those objectives.

Because regulators face resource constraints and competing priorities that make it impossible to review all regulations, ACUS recommends that each agency devise clear selection criteria for deciding which rules to evaluate. These criteria will vary to some degree by agency; however, the recommendation suggests several factors that all agencies may want to consider in the priority-setting process. For example, agencies may find retrospective review particularly useful in cases where there was a high degree of uncertainty about the accuracy of assumptions or estimates underlying the regulation.

Whatever factors an agency considers in prioritizing rules for retrospective analysis, ACUS stresses that transparency and receptivity to external input will be important. The recent recommendation urges agencies to leverage the expertise and experience of outside parties, including academics, regulated entities, and non-governmental organizations. According to ACUS, these individuals and groups “may possess valuable information concerning both the impact of individual regulations and the cumulative impact of a body of regulations issued by multiple agencies to which to which individual agencies might not otherwise have access.”

The core elements of ACUS’s Recommendation are largely consistent with scholarly work on retrospective review. For example, Professor Cary Coglianese at the University of Pennsylvania has identified three critical actions for building “a lasting culture of serious regulatory evaluation.” First, he has recommended that OIRA create guidelines for agencies on how to conduct rigorous retrospective reviews; specifically, the Office could issue guidance comparable to Circular A-4, which sets the parameters for prospective regulatory review.

Second, Coglianese has recommended that agencies be required to include plans for retrospective analysis for all the significant new rules that agencies must submit to OIRA.

Finally, Coglianese has suggested that OIRA should help agencies identify regulations that would benefit from retrospective review, such as rules projected to produce only marginal net benefits, rules for which prospective cost-benefit estimates were uncertain, and rules that implicate common issues or assumptions about cost-benefit estimation.

Recent actions by the Obama Administration suggest that efforts to integrate retrospective review into the regulatory processes have already started to take root in the federal government. Executive Orders 13563 and 13610 require agencies to submit a plan to OIRA for periodic review of significant agency regulations. Each agency must specify in its plan exactly how it will conduct retrospective reviews, and must make the plan available to the public for review and comment. In Executive Order 13579, President Obama encourages independent agencies to take similar measures.

ACUS’s recent recommendation endorses the Obama Administration’s efforts, and calls for ongoing collaboration between agencies, OMB, and research teams composed of experts from statistical, program, and policy evaluation offices across government. To the extent that these and other retrospective review activities require additional resources, ACUS urges Congress to appropriate the necessary funds.