Is Your Phone a Medical Device?

When the U.S. Congress defined the term “medical device” in the Food, Drug, and Cosmetic Act, it mostly had in mind physical hardware products, such as knee replacements, pacemakers, and surgical instruments.

But today, patients and providers often rely on software tools to improve health. Examples include the Apple Watch’s electrocardiogram app and a smart camera that tells ophthalmologists whether a diabetes patient is at risk of blindness.

In a recent article, professor Sara Gerke of Penn State Dickinson Law proposes that Congress broaden the definition of a “medical device” to encompass risky products that rely on artificial intelligence (AI), and that the U.S. Food and Drug Administration (FDA) exercise regulatory oversight over the makers of some of these products.

Admittedly, FDA has adopted a regulation that treats as medical devices any software used for “medical purposes”—disease prevention, treatment, and diagnosis.

But not all software services related to health care serve medical purposes. FDA clarified in 2019 that software tools that only help users maintain “a general state of health or a healthy activity” are not medical devices. A smartphone app that monitors your exercise activity, for example, is currently not considered a medical device. Neither is software intended to reduce an individual’s risk of chronic diseases or conditions, such as an AI service that helps Type 2 diabetes patients eat a balanced diet for their condition.

In her proposal, Gerke calls for Congress to define such “clinical decision” software as medical devices. Doing so would include many risky AI-based health care products that FDA currently does not regulate, Gerke contends.

Gerke offers AI-based mortality prediction models as a telling example. These algorithms analyze a cancer patient’s medical records to predict likelihood of death within the next six months. Gerke argues that, because such algorithms do not directly relate to the prevention, treatment, and diagnosis of a condition, the current statutory definition of a medical device would likely not cover them.

Hospitals increasingly rely on tools such as cancer mortality prediction models in clinical decision-making, which Gerke claims could jeopardize patient safety. Gerke explains that “a model could lead to the cessation of a patient’s treatment if it incorrectly predicts the patient’s early death.”

Her proposed fix is simple: Congress should amend its definition of a medical device to include clinical decision-making tools that are intended for the “prediction or prognosis of disease or other conditions or mortality.”

Gerke also notes that many AI-based tools, including those used in health care, rely on “black box” machine learning models that hide the logic of how they reach their determinations. This opaqueness makes it difficult for providers and patients to review the tool’s recommendations independently.

Gerke first proposes a “gold standard” solution to the challenges that black-box medical algorithms pose: Congress can require the makers of clinical AI to use a “white-box” model—a transparent system that reveals how the clinical algorithms reach their decisions—whenever a white-box system would perform better than a black-box one.

But if companies can demonstrate that a black-box AI system for a particular product would perform better than a white-box one, then FDA should shift its focus to verifying the tool’s safety and effectiveness, argues Gerke. She suggests that FDA can better accomplish this verification if it regulates these black-box systems as medical devices.

Only then can FDA ensure that black-box algorithms in health care are safe and effective through clinical trials, according to Gerke. For this reason, she proposes that FDA adjust its regulation of these black-box products to match the standards it imposes on more traditional medical devices.

But beyond clinical trials, FDA can do more to bring clinical AI tools into compliance with the agency’s medical device rules and standards, Gerke argues.

FDA mostly takes enforcement action against the makers of AI-based medical devices through a discretionary approach that considers the level of risk that a particular device poses, Gerke explains. And in determining what counts as a “risky” AI-based tool, FDA emphasizes whether the tool allows for the user—whether caregiver or patient—to review independently the software’s decisions. If a tool does allow for independent user review of clinical decisions, then FDA usually will not take regulatory action against the tool’s manufacturer, Gerke describes.

Gerke proposes, instead, that FDA focus its regulatory oversight on two types of AI-based medical devices: those that make decisions on “critical or serious” health conditions, irrespective of whether providers or patients can independently review those decisions; and those that make decisions on “non-serious” health conditions, but do not allow for independent review of the software’s decisions by patients or providers.

This shift in focus would likely subject to FDA oversight, for example, mortality-prediction tools even when a physician or patient can independently review the tool’s prognoses before making decisions based on them, Gerke suggests. But lower-risk tools, such as general wellness products like asthma alert mobile apps, might not warrant such oversight under this proposal.

Both Congress and FDA can take decisive action to keep pace with the rising tide of AI-based health care products, Gerke concludes.