Obama’s next OIRA Administrator should act to embed retrospective evaluation into the regulatory process.
It’s a truism that good decision makers should look before they leap. But good decision makers should also look back after they leap, to see how far they moved and in what direction. Only by looking back can they know if they are on course and whether they need to leap again.
Over the last thirty years, the look-before-you-leap principle has become a fixture in the federal regulatory process. Before regulatory agencies like the Food and Drug Administration and the Department of Commerce can adopt significant rules, they must produce detailed regulatory impact analyses that force decision makers to think carefully about the costs and benefits of their proposed actions.
They must also have their benefit-cost analyses reviewed by the White House Office of Information and Regulatory Affairs, also known as “OIRA.” Today, OIRA review follows a process similar to one first adopted under President Reagan and continued under every subsequent president. This review process has had its advocates and detractors, but no one seriously disputes that what Reagan established has firmly entrenched the look-before-you-leap principle in the daily practice of federal rulemaking.
Yet what’s missing in the regulatory process is any comparable institutional commitment to what might be called the look-back-after-you-leap principle.
An agency can spend years working on a prospective regulatory analysis to meet OIRA’s approval. But once it adopts a new regulation, no government-wide institution demands that the agency do any comparably rigorous retrospective analysis to see how that rule’s working in practice. Overall, the process is lot like a consumer spending months to research the purchase of a new automobile, only to buy it and then never bring it in for a tune-up.
Fortunately, some political leaders have started to recognize the need for taking better stock of how regulations are actually working in practice. Bill Clinton tried to encourage agencies to review their rules “periodically,” and he tasked Al Gore with overseeing a one-time government-wide review of existing regulations. George W. Bush’s OIRA Administrator undertook a similar review process, and two years ago President Obama announced yet another similar undertaking. Under the Obama Administration’s regulatory “lookback,” dozens of agencies have scoured their rule-books to identify outmoded and counterproductive rules that need to be modified or eliminated.
All this is good. But so far these efforts to promote retrospective evaluation have tended to resemble the state of prospective analysis prior to the Reagan administration: ad hoc, unsystematic, and fleeting. Reagan’s predecessors – Nixon, Ford, and Carter – had tried to spur regulatory agencies to do more looking before they leaped, but it took Reagan’s executive order to make prospective analysis a regular institutional fixture in regulatory policymaking.
Today, new steps are needed to make systematic retrospective evaluation – the looking back after the leap – a more routine institutional fixture in regulatory policymaking. Agencies need to deploy more rigorous research tools and build regular attention to evaluation into their ongoing practices.
President Obama recognizes the need to embed retrospective review more firmly into governmental practice. Last year, he issued an executive order calling on agencies “to institutionalize regular assessment of significant regulations.” His first OIRA Administrator, Cass Sunstein, proclaimed that the Administration’s lookback would not be a “one-time endeavor” but instead would help sustain “a regulatory culture of regular evaluation.”
How to achieve this goal? In order to maintain momentum and further institutionalize retrospective review, President Obama’s next OIRA Administrator would do well, at a minimum, to take the following three steps:
(1) Issue government-wide retrospective evaluation guidelines. To move retrospective review beyond an overreliance on impressions and anecdotes, OIRA should formulate specific guidelines for agencies to follow when conducting retrospective evaluation. These guidelines should aim to promote the use of sophisticated evaluation research techniques and should mirror the level of technical detail found in Circular A-4, the government’s manual for prospective regulatory impact analysis.
(2) Require the creation of evaluations plans. OIRA should also require federal agencies to include in each prospective regulatory analysis a plan for the subsequent retrospective evaluation of the proposed rule. Such plans should specify metrics to be used in a retrospective evaluation, discuss what types of data would need to be collected, and specify a time frame for subsequent evaluation. These plans would be non-binding, but they would help prompt agencies early in rule development to focus on evaluation needs. They might even help stimulate independent evaluation research by other governmental entities and research organizations.
(3) Send evaluation prompt letters to federal agencies. OIRA should initiate a process for sending evaluation “prompts” to federal agencies. These letters would identify specific existing rules that merit in-depth, scientific review. Characteristics of rules deserving prompt letters could be: Close calls – those rules for which the estimated costs in the original prospective analysis were nearly the same as the estimated benefits; Rules with high uncertainty – those rules where the cost or benefit estimations in a prospective analysis exhibited high levels of uncertainty; and Rules raising common issues – those rules that rely, for example, on assumptions common to other rules.
Taken together, these three actions – evaluation guidelines, evaluation plans, and evaluation prompts – would move the ball forward.
They represent feasible steps to take to institutionalize the current Administration’s lookback initiative and gain a better understanding of what is working well with regulation and what needs fixing. Moreover, these three steps at least have the advantage of requiring no new legislation to implement. They could be initiated soon after President Obama’s next OIRA Administrator is confirmed.
Of course, these three steps will not immediately solve all that ails the nation’s regulatory system; after all, even with a thirty-year head start OIRA review has not eliminated all problems. However, agencies definitely need to know more about what problems actually exist if they are going to stand a chance of fixing them. And yet at present, the government allocates most of its analytic rigor to the making of new rules and pays far too little attention after the fact to what can be learned about whether those rules make any difference, for good or for ill.
Federal regulators do need to look before they leap – but they also need to undertake more rigorous and regular looks backward, after they have leaped, to determine what they have accomplished and how they can do better in the future.
