Agency sees increased effectiveness after incorporating feedback from courts and academics.
Two sets of events have sparked scholarly controversy over the proper role of economic analysis in federal financial regulation. First, a trio of court decisions between 2005 and 2011 struck down major U.S. Securities and Exchange Commission (SEC) regulations on the basis of faulty economic analysis. Second, leading lawmakers have sought to require financial regulators to conduct benefit-cost analysis similar to the regulatory impact analysis conducted by executive branch agencies when proposing new major regulations.
Most of the academic controversy has focused specifically on the role of benefit-cost calculations. Columbia Law School Professor Jeffrey Gordon, for example, argues that financial regulators cannot predict the results of financial regulations because individuals change their behavior in response to regulations in unpredictable ways. Harvard Law School Professor John Coates espouses “conceptual” economic analysis but claims that any attempts to quantify benefits and costs are mere guesswork. Chicago Booth School of Business’s John Cochrane, although optimistic that benefit-cost logic provides the right way of framing the policy discussion, warns that responses to economic incentives that would be considered “indirect” effects of other types of regulation actually create the major benefits and costs associated with financial regulation.
Others, however, offer a more sanguine view of the role of economic analysis in the financial regulatory arena. For one, Chicago Law School Professor Eric Posner and Microsoft Research’s Glenn Weyl suggest that economic analysis of financial regulation actually should be easier than it is within those regulatory fields in which it is already employed (such as safety and health regulation): as they explain, the relevant valuations are already expressed in monetary terms and the actors are motivated by money. And even when some information is missing, Harvard Law School Professor Cass Sunstein argues, agencies feasibly can still make an effort to quantify benefits and costs, identify ranges of outcomes, acknowledge uncertainties, and employ “breakeven analysis”—a method of comparing benefits and costs in the face of information gaps that might otherwise make conducting cost-benefit analysis impossible for financial regulators.
In 2012, the SEC’s Office of General Counsel and the Commission’s then-Division of Risk, Strategy, and Financial Innovation issued guidance for economic analysis of regulations in response to several court decisions and other external criticism concerning the SEC’s economic analysis. The court decisions—especially the D.C. Circuit’s 2011 decision in Business Roundtable v. SEC—took a much harder look at the Commission’s analysis than most legal commentators had expected. The upshot was that these decisions have created a quasi-natural experiment, providing researchers with ready-made case studies of the effects of judicial review on the quality and claimed use of economic analysis in regulations.
The SEC’s 2012 guidance provides that a complete economic analysis should include an assessment of the agency’s need for the regulation, an articulation of the baseline against which the effects of the regulation would be measured, alternatives to the proposed regulation, an evaluation of the economic impact of the proposed regulation, and reasonable alternatives based on the regulation’s benefits and costs. These five factors, widely accepted as major elements of regulatory impact analysis, involve more than just the calculation of benefits and costs of a prospective regulation.
All five of these factors are evaluated in the Regulatory Report Card project of the Mercatus Center at George Mason University. The Report Card’s evaluation rubric can be applied to assess the quality of the SEC’s economic analysis before and after the 2012 guidance was issued.
As a preliminary matter, the Report Card includes two kinds of criteria for assessing the quality of the SEC’s economic analysis for a given regulation. Some criteria address the degree to which the analysis reflects a theoretical understanding and empirical evidence of cause-and-effect relationships. These criteria focus on, for example, how well the analysis identifies the nature and cause of a systemic problem that the regulation is intended to solve, the adequacy of the analysis’s assessment of alternative approaches, and whether the analysis identifies intended outcomes and shows how the proposed regulation would achieve these outcomes. Other criteria largely address the analysis’s quantification of benefits and costs, including how well the analysis addresses the baseline, the adequacy of the analysis’s identification of benefits and costs, and whether the analysis includes an assessment of uncertainties about the size of the problem, benefits, and costs.
Because 130 economically significant, non-budget regulations proposed by executive branch agencies were evaluated for the Report Card project, it is also possible to compare the quality of the SEC’s analysis with the quality of analysis performed by these agencies.
The chart below compares the average quality of analysis for four groups of regulations, based on the five criteria articulated in the SEC guidance:
- Seven major SEC regulations finalized prior to the 2012 guidance
- Seven major SEC regulations finalized after the 2012 guidance
- Nine economically significant regulations proposed by executive branch financial regulators between 2008 and 2013
- 130 economically significant regulations proposed by all executive branch regulators between 2008 and 2013.
Source: Data used in chart are from Jerry Ellig, “Improvements in SEC Economic Analysis Since Business Roundtable: A Structured Assessment,” Working Paper, Mercatus Center at George Mason University (Dec. 2016), p. 25.
A score of one point on a criterion indicates that the analysis included some assertions on the criterion without providing much evidence for those assertions. A score of five points indicates a reasonably thorough analysis reflecting potential best practices.
The post-guidance scores for SEC regulations increased significantly with respect to all five of the guidance’s criteria, relative to the pre-guidance scores. The SEC’s post-guidance scores are much closer to those of executive branch regulators—whether one considers the executive branch financial regulators exclusively, or all executive branch regulators.
Econometric analysis, which statistically controls for other factors that might affect the results, confirms these results. The difference between SEC pre- and post-guidance analyses is statistically significant, after controlling for other factors that might explain the difference. In terms of the criteria measuring the analysis’s identification of the problem motivating the regulation, the baseline, and the costs, the SEC post-guidance analyses score similarly to the average executive branch analysis. The SEC analyses still score below the executive branch with respect to the analysis of alternatives and benefits.
Whether one considers SEC economic analyses or regulatory impact analyses produced by executive branch agencies, there is still significant room for improvement. As the chart shows, for most criteria, the average score is slightly below three points. A score of three points indicates that the analysis includes a conceptual explanation and empirical evidence on some major points, but the fact remains that it is not a comprehensive analysis.
Despite this drawback, the study contains good news regardless of whether one favors “conceptual” economic analysis or extensive quantification of benefits and costs. It also suggests that judicial review of agency economic analysis can create powerful incentives for agencies to improve.
This essay draws on a Mercatus Center working paper produced by Ellig, Improvements in SEC Economic Analysis since Business Roundtable: A Structured Assessment. The paper examines recent improvement in the Securities and Exchange Commission’s economic analysis of regulations, in light of several court decisions that had remanded major regulations due to inadequate economic analysis.
