New OECD report encourages governments to learn better ways to make regulations.
Regulation is a fixture of modern market economies around the world, but according to a new report by the international Organization for Economic Co-operation and Development (OECD), there remain many challenges and opportunities to improve the processes by which governments promulgate new regulations. By offering a unified framework for evaluating process requirements, the OECD report aims to help countries around the world learn what helps – and what doesn’t help – when it comes to making new regulations.
The processes by which governments design regulations have become the object of serious reform efforts in recent decades. For example, in the United States, starting in the 1980s, the White House’s Office of Management and Budget has required agencies to prepare regulatory impact analyses (RIAs) when they want to issue certain new regulations. These RIAs are designed to ensure that, to the extent possible, significant new regulations do not impose costs that exceed their benefits.
Throughout the world, governments have adopted similar procedures for the production of RIAs. They also have adopted and follow a variety of procedural requirements calling for transparency and public participation in the regulatory process. Regulators use these and other process tools – or “regulatory policy,” in the OECD’s parlance – to improve their regulations. Indeed, Chapter 3 of the OECD’s recent report contains a useful survey of the OECD member countries’ regulatory policy practices, including RIA and consultation practices on draft regulations, ex post analysis of existing regulations, and administrative simplification and burden reduction programs.
Despite all of this innovation in regulatory policy, the OECD report says that governments need to do more to evaluate which of these procedures advance policy goals. The report offers a framework for governments to determine whether specific processes have led to specific improvements in terms of, for example, any reductions in regulatory compliance costs or increases in the benefits of regulations. Establishing such linkages is vital if governments are to learn the best ways to make regulations.
Cary Coglianese, Professor of Law at the University of Pennsylvania Law School and a contributing expert to the OECD’s project, explains that careful research is needed to pinpoint the effects of specific regulatory procedures. He notes that changes in regulatory outcomes could be explained not solely by changes in procedures, but by “confounders,” or other factors, such as the substantive challenges confronting regulatory decision makers.
The OECD report explains that governments need to use careful statistical controls to eliminate hasty conclusions about the effects of regulatory policy interventions. To conduct these kinds of careful studies, governments also need to collect appropriate data on regulatory processes and their outcomes.
To make progress evaluating regulatory policy reform, the OECD report develops a flexible framework for evidence-based evaluation. The framework breaks the regulatory process into concrete sequences, with each sequence forming part of an “‘input-process-output-outcome’ logic.”
For instance, just because a regulatory policy is “on the books” does not mean that many resources are being committed to it. Thus, researchers need to collect data on the “inputs” (e.g., the budget for such a program) to see if there is any chance for the policy to thrive. Likewise, research on what specific information is typically included in RIAs advances understanding of the “outputs” of regulatory policy. Finally, researchers should focus their attention on measuring “outcomes,” both in terms of procedural and substantive goals. With more data points at each step of the process, it is possible to begin assessing whether regulatory policy improves regulation.
The organizational schema of the OECD’s framework ultimately allows countries to do a better job of collecting the information that is a prerequisite for any serious attempt to answer the causal question of regulatory policy’s effect on regulatory quality. Of course, as the report notes, simply measuring the inputs, processes, outputs, and outcomes of each stage of the regulatory process does not by itself eliminate the knotty problem of causation – that is, tracing any improvements in strategic objectives back to regulatory policy interventions. But it does make limited inference more defensible and may ultimately pave the way for carefully designed, quasi-experimental research that can deliver robust causal attributions.
In general, the OECD suggests that the flexibility of its framework is its strength: it can be applied in many different countries as well as within each country either to its entire regulatory system or to specific regulated sectors.
The OECD report indicates that its framework has already been applied in two pilot studies, one in Canada and one in the Netherlands. These two pilot studies reveal how countries can adapt the framework to their particular context. In Canada’s pilot study, for instance, evaluators found it useful to add more specific stages in between the basic stages identified in the OECD’s general framework.
This need for adaptation of the framework to particular contexts leads the OECD to conclude that “a larger number of case studies will be necessary to provide countries with a practical understanding of the application of the Framework.”
The implicit message of the OECD’s report is that regulatory policy evaluation should be an ongoing, iterative process. Although it may at times be frustrating that process reforms lack any proven connection with improvements in regulatory objectives, scholars and regulators can collaborate to develop the needed capacity for better performance evaluation.
Indeed, the OECD report notes that, “while regulatory policy evaluation is unlikely to be fully complete” anytime soon, “initial concrete steps are necessary.” That pragmatic message, as well as the concrete but flexible evaluation framework offered, makes the OECD report important reading for anyone around the world interested in improving how governments design new regulations.
