New study suggests agencies primarily listen to industry insiders in finalizing rules.
Americans submit over one million comments each year on federal agencies’ proposed rules, but a new study suggests agencies mainly listen to industry insiders.
A new research paper by Andrei Kirilenko, Shawn Mankad, and George Michailidis analyzed recent rulemakings by the U.S. Commodity Futures Trading Commission (CFTC). Based on this analysis, the study found that professionals in a proposed rule’s affected industry have the only statistically significant impact on the content of a final rule.
The general public’s comments, however, do affect whether the proposed rule is actually promulgated. The study found that if there is a greater consensus among public comments, then there is a greater chance that the rule will be finalized. But if there is greater variance in the comments, the regulator is significantly less likely to promulgate the rule.
In arriving at these conclusions, the authors devised an algorithm, termed “RegRank,” that measures the degree to which regulatory agencies adjust their final rules in response to public comments. RegRank searches an agency’s proposed rules, comments, and final rules and identifies topical themes (for example, the term “swap” in financial regulation) that appear across stages of the regulatory process.
Excluding frequently used words such as days of the week, the algorithm assigns each popular term’s appearance a value. The value is based on the word’s “tone,” or whether it is pro-regulation, neutral, or anti-regulation. In assigning the value, RegRank uses a dictionary of nearly 10,000 words associated with a score of -1 (negative), 0 (neutral), or 1 (positive). The dictionary was constructed using terms found in customer product reviews on the Internet and social media websites.
After the algorithm has identified the themes within the available documents and assigned each a value, it computes the average value for the documents at each stage of the process. This value, the RegRank, is then analyzed to see whether the final rule moved toward or away from the sentiment of public comments.
The algorithm also categorizes each comment based on the commenter’s self-identified occupation and organization. This enables researchers not only to determine if regulators listened to the public, but to whom they listened.
As an example, the authors analyzed the CFTC’s proposed rules, comments, and final rules between January 2010 and September 2013. The CFTC regulates derivatives markets and during that period issued 104 proposed rules, received approximately 60,000 public comments, and promulgated 67 final rules.
The RegRank algorithm found that final rules tended to move in the direction of public comments. For example, one CFTC proposed rule would have changed the definition of “swap” in certain financial regulations. In the initial agency proposal, the term was valued by the algorithm with a RegRank of approximately 0.25, meaning that the term was viewed as slightly positive, or supportive of the proposed regulatory change. Public comments, however, had a less favorable view of the term and produced a RegRank of approximately 0. The final rule’s RegRank was approximately 0.05, which represented a statistically significant movement toward the direction of the public comments’ sentiments.
The swap rule, however, only received seventeen public comments. The number of comments per CFTC proposed rule varied drastically during the studied period. The three most popular proposals, which proposed regulations on derivative position limits, speculative position limits, and retail foreign exchange transactions, received 58% of all comments. Only 22 other rules received more than 100 comments.
Forty-three percent of the comments submitted on the 25 most popular rules came from the general public. This included self-identified radiologists, electricians, chaplains, and recruiters, for example. The other commenters were corporate executives, those associated with the financial industry, such as traders, asset managers, and brokers, or legal counsel. Omitting the most popular proposed rules, this latter category submitted nearly all of the public comments.
After separating general public commenters from those associated with the regulated industry, the authors found that financial professionals’ comments were the only ones that had a statistically significant predictive effect on the RegRank value of the promulgated rule. Other occupations had no discernable influence.
While the general public may not affect the final content of a regulation, the authors found that the public did have a substantial impact on whether a rule would be promulgated at all. If the comments submitted represented a greater consensus, then the rule was more likely to be finalized. If the comments were more varied, however, it was likely the rule would never become finalized. Those regulations where only industry insiders commented had an even lower probability of being promulgated.
Kirilenko, Mankad, and Michailidis only used RegRank on a small sample from a single regulator, but they claim the algorithm could be used to measure the responsiveness of other agencies’ rulemakings and to determine to whom agencies are most likely to listen.
