The CFTC and other regulators should evaluate with caution proposals to automate financial transactions.
The crypto firm FTX recently applied to the Commodity Futures Trading Commission (CFTC) for authorization to clear margined products for retail investors in a “non-intermediated” fashion. The proposal is complicated and raises many concerns about investor protection and financial stability, but FTX’s proposal also raises a more fundamental question that has increasing relevance to regulators everywhere.
How should regulators supervise the software that run automated systems?
Software has been part of many industries’ business models for a long time, but critical regulated activities can now be carried out without human intervention. Just as courts are trying to figure out how to apportion liability for automated decisions with humans increasingly “out of the loop,” regulators must grapple with a new reality about their role: instead of just supervising humans, they are increasingly supervisors of software.
To provide some more context, the CFTC oversees the process of derivatives trading and clearing. The more analog version of derivatives clearing—the version the CFTC is used to overseeing—manages risk by having layers of intermediaries that each perform a risk management function. Brokers sit between investors and a clearinghouse, and both brokers and the clearinghouse make regular assessments of the collateral needed to support trading positions, requesting more margin when needed.
The human relationships involved allow some exercise of discretion. In March 2020, for example, Citi reportedly suffered a technical glitch that prevented it from posting the necessary margin on time, but ICE clearinghouse extended a little grace and refrained from liquidating Citi’s position.
The recent FTX proposal would depart from this model, eliminating brokers with their discretionary margin calls and replacing them with software. The software would assess margin requirements every second of every day based on its real-time interpretation of market events. Without exercising discretion or grace, the software would speedily liquidate any investor who is not in compliance—regardless of the consequences for the individual investor or for financial markets more broadly.
If the proposal is approved, a lot will be riding on FTX’s software. The software will need to perform the functions that FTX has said it will perform, and the software should also meet minimum reliability and cybersecurity standards.
But who is going to set the minimum standards, and who is going to check compliance with them? Who is going to verify that the software code as written matches the proposal? Like many industry regulators, the CFTC does not have a large number of software engineers on staff. So what is an agency to do?
Sometimes it will be appropriate for regulators simply to say “no” to automation. Due to the complexity of software code, an automated system can never be fail-safe. And if automation only makes an activity marginally more efficient than the non-automated alternative, then the benefits will not be worth the risks and the regulator should insist on requiring a “human in the loop.”
If automation is judged desirable, though, then a multi-pronged course of action is needed. If the automated system constitutes critical financial infrastructure, the software involved should be designed in accordance with best practice standards for software used in safety-critical settings such as in aviation and nuclear power plants. Although the harm that financial firms can cause is sometimes minimized as being “only” financial or economic, economic harms can be severe and even translate into physical harm. Just consider, for example, the suicide hotline numbers posted on crypto reddits as crypto assets have failed.
Software that is used to automate financial infrastructure should therefore be considered safety-critical. Decisions made throughout the programming process, such as the choice of code libraries or diagnostic tests that will be run, should follow much more stringent standards than equivalent decisions made in connection with the development of a less critical system such as a social media app.
Unfortunately, the CFTC—like many regulatory agencies—does not currently have the capacity to assess compliance with these kinds of standards, or even to check whether regulated firms are misrepresenting what their software does. Regulatory agencies can and should try to build their in-house technological capacity by hiring more software engineers, but competition for these personnel can be fierce and government salaries are rarely competitive.
Ideally, the U.S. Congress would raise agency budgets commensurate with the increased resources needed to oversee automated systems. But it may be more realistic to concentrate this expertise in “hub” agencies. The U.S. Department of Treasury’s Office of Financial Research, for instance, could serve as a hub of interdisciplinary expertise for financial regulatory agencies. Alternatively, Congress could resurrect the U.S. Office of Technology Assessment to serve as a more general government hub.
Until this software supervisory expertise is developed within the government, allowing a regulated entity to fully automate a critical activity will necessarily entail the regulatory agency abdicating some authority over that activity.
To be clear, even with the necessary expertise, there will be limits on what software standards can achieve. Stringent standards are necessary to help minimize programming errors, but the complexity of the software ensures that it will always be vulnerable to “normal accidents.”
Because something will inevitably go wrong with complex software, it is critical that regulatory agencies also demand some combination of redundancies, frictions, inefficiencies, and backstops so that the public is not entirely dependent on the automated system performing as expected. Just as pilots need to be able to disable autopilot and take control of an aircraft, financial regulators need circuit breakers and other tools to be able to stop automated transactions.
The backstop that FTX has proposed is a $250 million guaranty fund that will be available to absorb losses if necessary. Determining with confidence whether this amount will be adequate to protect the clearinghouse from insolvency may be impossible, given the difficulties in valuing crypto assets and assessing associated risks. But even assuming $250 million is adequate, the guaranty fund will do nothing to protect individual investors who are wrongfully liquidated as a result of software error. It will also do nothing to address the systemic risks that could result if asset prices market-wide are impacted by a technological glitch that forces a mass liquidation of FTX positions, such as in a “flash crash.”
As the CFTC evaluates FTX’s proposal, the agency needs to consider other measures that compensate both for its limited ability to assess the quality of FTX’s technological plumbing, and for the malfunctions that are inevitable even with the highest quality software. So must other regulators contemplating how to supervise other software-automated systems.
