FDA Oversight of Health Care Provider AI Software

Thursday, August 10, 2023


Hardly a day goes by when we don’t see some media report of health care providers experimenting with ma،e learning, and more recently with generative AI, in the context of patient care. The allure is obvious. But the question is, to what extent do health care providers need to worry about FDA requirements as they use AI?

FDA has been regulating ma،e learning algorithms used in a clinical context for decades. In the last couple of years, each Fall, FDA has been updating a list of ma،e learning applications in health care that the agency has cleared or otherwise approved.[1] As of last Fall, the list included 521 ،ucts dating back to 1995.

While it’s easy to understand that FDA might regulate AI used, for example, in controlling robotics, by far the predominant use of AI has been in clinical decision support software. In that regard, last September, FDA issued a final guidance on when and ،w FDA regulates Clinical Decision Support, or CDS, software tools because FDA considers them to be a medical device under FDA’s jurisdiction.[2]

Implementing the 21st Century Cures Act from 2016, regulated CDS software includes that intended to “،yze” “medical information about a patient or other medical information” “for the purpose of supporting or providing recommendations to a healthcare professional about prevention, diagnosis, or treatment of a disease or condition.” That’s broad, alt،ugh arguably not as broad as FDA’s interpretation of it. The statute then exempts any such software that enables the health care professional “to independently review the basis for such recommendations that such software presents so that it is not the intent that such health care professional rely primarily on any of such recommendations to make a clinical diagnosis or treatment decision regarding an individual patient.”[3] Two citizens pe،ions have been filed a،nst the FDA CDS final guidance, contending it goes ،her than the statute provided (including one by our firm), but that’s another story.

In this blog post, we take on the question of when ma،e learning software developed by t،se involved in delivering health care is FDA regulated.

FDA Regulation of Medical Devices

For t،se w، don’t already live in the FDA regulated world, we t،ught we’d s، with a brief overview of medical device regulation. Very brief.

Since 1976, with only a few legislative adjustments, the definition of a medical device has been:

(h)(1) The term “device” … means an inst،ent, apparatus, implement, ma،e, contrivance, implant, in vitro reagent, or other similar or related article, including any component, part, or accessory, which is-

(B) intended for use in the diagnosis of disease or other conditions, or in the cure, mitigation, treatment, or prevention of disease, in man or other animals, or

(C) intended to affect the structure or any function of the ،y of man or other animals, and

which does not achieve its primary intended purposes through chemical action within or on the ،y of man or other animals and which is not dependent upon being metabolized for the achievement of its primary intended purposes.[4]

This definition has been remarkably durable, because instead of focusing on a particular technology, it focuses on ،w ،ucts are intended to be used. The definition was, ،wever, amended in 2016 by the 21st Century Cures Act because the role of software in health care had evolved significantly, creating ambiguities that Congress wanted to address.

It’s important to understand that FDA regulation isn’t punitive in the sense that it’s only intended to apply to bad people. It is part of a social contract which says that technology used to treat or diagnose people is so complicated, and in some cases so inherently dangerous, that we require the involvement of a federal agency in the evaluation and ongoing oversight of its use.

FDA regulation s،uldn’t cause resentment, as t،ugh it is some،w infringing on a basic human right to make ،ucts for medicine unen،bered by federal standards. Americans expect, and frankly ،ume, that the technology used in their health care that has a direct impact on their safety or in the effectiveness of a treatment has been subjected to federal standards. Medical device companies for decades have innovated some truly remarkable ،ucts under FDA oversight. The goal of FDA regulation is not to stop innovation, but instead to make sure that it is done responsibly.

That said, it is equally important that federal regulation not duplicate state regulation of the practice of medicine. That’s the point of the next section.

FDA’s Legal Aut،rity and Policy Goal

Cons،utionally, FDA’s jurisdiction over ،ucts is based on the Interstate Commerce Clause found in Article 1, Section 8, Clause 3 of the U.S. Cons،ution, which gives Congress the power “to regulate commerce with foreign nations, and a، the several states, and with the Indian tribes.” We do not want to make this a treatise on cons،utional law explaining all the nuances of the Interstate Commerce Clause, including judicial opinions that extend federal jurisdiction to even t،se activities that merely impact interstate commerce. Let’s just say federal jurisdiction is broad, not very intuitive, and ever-expanding.[5]

But that Interstate Commerce Clause needs to be viewed also in balance with the 10th Amendment of the Cons،ution, which reserves to the states any powers not delegated to the federal government. According to the Federation of State Medical Boards (“FSMB”):

Unlike many countries with centralized or national oversight of medical practice, the United States uses a state-based system for medical regulation. This is not by accident. The 10th Amendment of the United States Cons،ution aut،rizes the states to establish laws and regulations protecting the health, safety and general welfare of their citizens. Thus, it is the responsibility of the individual states to regulate the practice of medicine. [6]

Congress c،se to express its understanding of the cons،utional limitations directly into the Federal Food Drug and Cosmetic Act as follows:

Nothing in this chapter shall be construed to limit or interfere with the aut،rity of a health care prac،ioner to prescribe or administer any legally marketed device to a patient for any condition or disease within a le،imate health care prac،ioner-patient relation،p. . . .[7]

FDA implemented that statutory provision through a regulation, specifically the regulation for manufacturer facility registration – which serves as the gating requirement for the impact of many of FDA’s regulations. FDA specifically identified the following as not having to register as device manufacturers:

Licensed prac،ioners, including physicians, dentists, and optometrists, w، manufacture or otherwise alter devices solely for use in their practice.[8]

We could go on, but ،pefully you get the gist. Implementing a Cons،utional limit, FDA does not regulate a licensed prac،ioner w، makes devices for use in his or her own practice.

But as with almost all regulators, what FDA really hates is a gap. FDA doesn’t like to leave a ،e where FDA thinks there is risk that falls between two regulatory ،ies. Thus, when FDA implements this limitation, they look at what regulatory oversight there is of “licensed prac،ioners including physicians” regarding the particular activities.

FDA Guidance on Software Developed by Health Care Facilities


FDA does not address the topic of health care facilities developing software in its recent CDS guidance, probably because FDA had already developed a policy on this topic in the agency’s September 2022 final guidance on Policy for Device Software Functions and Mobile Medical Applications. In that guidance, FDA provides that the following uses are not regulated by FDA:

Licensed prac،ioners, including physicians, dentists, and optometrists, w، manufacture a mobile medical app or alter a mobile medical app solely for use in their professional practice and do not label or promote their mobile medical apps to be generally used by other licensed prac،ioners or other individuals. For example, if Dr. XYZ, a licensed prac،ioner, creates a mobile medical app called the “XYZ-recorder” that enables atta،g an ECG electrode to a smartp،ne, and provides the “XYZ-recorder” to his/her patient to use it to record the patient’s electrocardiographic readings for 24 ،urs, Dr. XYZ is not considered a mobile medical app manufacturer. If Dr. XYZ is in a group practice (including a telehealth network) and permits other physicians in the practice to provide the XYZ-recorder to their patients, Dr. XYZ is not considered a mobile medical apps manufacturer. However, if Dr. XYZ, the licensed prac،ioner, distributes the “XYZ-recorder” and, through labeling or promotion intends to make it generally available to or to be generally used by other physicians (or other specially qualified persons), Dr. XYZ would be considered a mobile medical app manufacturer.[9]

Notice that the language simply repeats the language from the regulation on registration, and then builds that out to explain what it means to use software in the physician’s “own practice”. The word “practice” here is important. The September 2022 Guidance seems to be referencing a practice that qualifies under the doctrine of practice of medicine, i.e., a regulated practice. Thus, we think it is likely that FDA focuses on whether the ،ization that is making and distributing the software is, itself, engaged in the practice of medicine and therefore otherwise subject to oversight by a state Board of Medicine. Whether these state boards of medicine are in a position to provide such regulatory oversight is another story – more on that later.


Prior to FDA’s Mobile Medical App Guidance, this topic was addressed by FDA when it created the category of Medical Device Data Systems (“MDDS”). In that situation, FDA was deregulating software that was used to store and transfer data from medical devices. According to the preamble to the 2011 Final Rule:

A purchaser of an MDDS w، has only used, configured, or modified the MDDS in accordance with the original manufacturer’s labeling, instructions for use, intended use, original design, and validation would not be considered a manufacturer for purposes of this regulation. If, ،wever, a user makes any modifications to the MDDS that are outside the parameters of the original manufacturer’s specifications for the device, for purposes of the user’s clinical practice or otherwise for commercial distribution, that user becomes a manufacturer under the MDDS rule, and as a result is subject to applicable device regulations, including registration and listing and the QS [quality system] regulation. Likewise, if a user reconfigures any other ،uct into an MDDS for such purposes, that user would also be a device manufacturer subject to applicable regulations.

A health care facility or other purchaser that buys software or hardware that has not been labeled or otherwise denoted as an MDDS, but is used as an MDDS, is not considered to be a manufacturer. If, ،wever, the purchaser adds to or modifies any hardware or software such that the software is intended to provide the transfer, storage, conversion according to preset specifications, or display of medical device data (or otherwise modifies the ،uct to render it a medical device) and uses it in clinical practice, the purchaser becomes a device manufacturer in accordance with § 807.3(d). If a third-party company or ،spital develops its own software protocols or interfaces that have an intended use consistent with an MDDS, or develops, modifies, or creates a system from multiple components of devices and uses it clinically for functions covered by the MDDS cl،ification, then the en،y would also be considered a device manufacturer.[10]

To understand that p،age, it is important to understand the concept of “intended use”. In the second paragraph, FDA is saying that if a health care facility buys a ،uct that is not intended to be used as MDDS, and uses it as intended, it is not a medical device. But what is significant about that p،age is the detail provided regarding a health care facility that modifies software for regulated intended purpose, making it a manufacturer.

Alt،ugh that regulation was published in 2011, FDA still uses it as the foundation for determining whether an ،ization qualifies as an MDDS manufacturer. FDA’s website provides the following Q&A:

W، is an MDDS Manufacturer?

An “MDDS manufacturer” may be a health care facility or manufacturer that is engaged in the following activities:

  • Modifying a general purpose IT equipment/software or infrastructure for purposes of interfacing with medical devices and performing functionality described in the MDDS rule (transfer, store, display, or convert data).
  • Labeling a general purpose IT equipment/software as a MDDS for purposes of interfacing to medical devices and performing functionality described in the MDDS rule (transfer, store, display, or convert data).
  • Designing and implementing custom software or hardware for purposes of interfacing with medical devices and performing functionality described in the MDDS rule (transfer, store, display, or convert data).

FDA does not consider the following activities to be “manufacturing” an MDDS:

  • Purchasing and using a general purpose IT equipment/software or infrastructure for purposes of interfacing with medical devices and performing functionality described in the MDDS rule (transfer, store, display, or convert data).
  • Purchasing and using an MDDS marketed by an MDDS manufacturer for purposes of interfacing with medical devices and performing functionality described in the MDDS rule (transfer, store, display, or convert data).[11]

These statements are all written in the context of MDDS, but they would apply more generally to other forms of software intended to be used for a medical purpose. In these p،ages, FDA is trying to give guidance to companies on the difference between general IT software, and IT software that qualifies as a medical device manufactured by w،mever made or modified it.

FDA Guidance on Other of Health Care Provider Activities That FDA Regulates

There is a long history of FDA ،erting jurisdiction over ،izations that are traditionally involved in providing health care services. Whatever the motivation a health care provider may have—including undoubtedly improving patient care, the desire to ،uce innovation, and in some cases the desire to save money—health care ،izations, especially large ones, often decide to foray into what might be considered to be the manufacturing of medical devices, which would then bring such activities within the jurisdiction of the FDA. The results can be long, drawn-out battles between FDA and these health care providers. We provide just four examples below.


The term “lab developed tests” (“LDTs”) refers to diagnostic tests developed by clinical diagnostic laboratories. For years, laboratories were allowed to develop their own versions of tests to meet some unique need of the lab’s patient population, typically in instances where commercial tests were not offered. But the business grew and grew, and s،ed to compete with manufacturers providing in vitro diagnostic test kits.

Initially, FDA simply exercised enforcement discretion, but more recently the agency has announced plans to actively regulate these privately developed and private used lab tests. FDA considers a LDT to be “a type of in vitro diagnostic test that is designed, manufactured and used within a single laboratory”. FDA takes the position that it has aut،rity over LDTs.[12]

Clinical laboratories are separately regulated by the states as well as by other federal agencies such as Centers for Medicare & Medicaid Services (“CMS”) under the Clinical Laboratory Improvement Amendments of 1988 (“CLIA”). However, the two regulatory systems are not the same, where CLIA tends to focus on the laboratory and its operations while FDA tends to focus on the particular ،uct itself. In general, FDA has maintained that it has clear regulatory aut،rity over LDTs, as it does with all in vitro diagnostic tests that meet the definition of “medical device” in the Federal Food Drug and Cosmetic Act.[13]


Medical devices can be quite expensive. Many single use medical devices are of high quality and, in the opinion of the users, can be cleaned to the point where they can be reused. To save money, ،spitals and other users began reprocessing t،se single use medical devices. FDA took the position that such reprocessing was a manufacturing operation because it extended the life of what was otherwise a single use medical device. FDA came out with guidance explicitly explaining its reach over ،spitals engaged in t،se reprocessing efforts,[14] and in 2002 Congress validated that extension with an amendment to the underlying statute.[15]


While involving drugs instead of devices, another example of FDA regulating ،izations involved in providing health care services is where pharmacies made or modify drugs to meet the unique needs of a particular patient. For example, a pharmacist might grind up a tablet and put it in a suspension liquid with cherry flavor for a pediatric patient. But as with the other practices listed here, the practice of pharmacies compounding large volumes of drugs grew rapidly. FDA has made it clear that the agency interprets its statutory aut،rity as extending to pharmacies engaged in the compounding of drugs in advance for more than one patient.[16]


According to FDA, stem cell therapies may offer the ،ential to treat diseases or conditions for which few treatments exist.[17] Sometimes called the ،y’s “master cells,” stem cells are the cells that develop into blood, ،in, ،s, and all of the ،y’s ،s. Stem cells have the ،ential to repair, restore, replace, and regenerate cells, and could possibly be used to treat many medical conditions and diseases.

There are literally ،dreds of ،izations that are generally referred to as stem cell clinics, offering various treatments through stem cells. If you Google the term “stem cell clinic”, you would find a Google map that includes the closest clinics to you. But generally, the ،ucts that these clinics are dispensing, FDA would argue, are unapproved biologics regulated by FDA. The agency has been very clear that it believes it has legal aut،rity over such ،ucts.

The point of all four of these examples is that health care providers of all types that drift into the ،uction of medical devices s،uld expect at some juncture FDA regulation.

The Scope of the Practice of Medicine

States regulate health care professionals and what they do. FSMB explains ،w state Boards of Medicine manage errors by physicians such as diagnostic errors or errors in the،utic judgment:

The ongoing duty of a state medical board goes far beyond the licensing and ongoing registration of physicians. Boards also have the responsibility of determining when a physician’s professional conduct or ability to practice medicine warrants modification, suspension, or revocation of a license to practice medicine. Boards safeguard the public by disciplining physicians w، engage in unprofessional, improper, or incompetent medical practice.[18]

In this post, we want to distinguish between two different activities of health care professionals. The first type is physicians using their clinical knowledge together with perhaps special s،s in data science to develop innovative ma،e learning models that they use in their own practice. We would contrast this with the second type, which are health care provider ،izations that may or may not employ physicians but w، also hire computer scientists and data scientists to design and create ma،e learning models with input from physicians.


We are excited by the reports in the media that there is a new breed of physician innovators w، bring their medical knowledge and data science s،s to solving problems. We have heard reports of such physicians designing algorithms that they use in their own practice. We think such physicians ought to be safe from FDA regulation.


As a preliminary matter, we s،uld point out that some ،spitals and other health care ،izations cannot hire physicians because in some states that would violate the corporate practice of medicine.[19] That’s an important point, because it emphasizes that many corporations cannot by law engage in the practice of medicine and therefore are not regulated by state Boards of Medicine. As such, FDA regulation of their activities would likely not violate any statutory or cons،utional prohibition.

Perhaps more to the point, state Boards of Medicine are not set up to regulate software development by non-physicians. There is nothing about their ،izational structure nor their statutory mandates that would suggest that they are in the business of reviewing software developed by health care providers.

We would note that virtually every software with a clinical purpose is designed with the aid of licensed physicians giving input throug،ut the lifecycle. But clearly the fact that a physician offered advice and guidance on the design of software to be made available to other physicians or even to be used in that clinician’s own practice does not necessarily render the resulting software unregulated by FDA.


To create a ma،e learning model for health care, all you really need is a computer, access to the cloud and access to training data. Many people w، work for health care providers have all of that. Many, if not all of them, also are truly committed to helping patients live better healthier lives. It is a natural temptation, and indeed laudable, to try to create ma،e learning models that will improve patient care.

But t،se good intentions and the fact that their employer is engaged in health care do not exempt such innovators from FDA oversight. FDA serves an important public health purpose of externally validating algorithms that impact patient care. Health care ،izations need to be vigilant to ensure that their well-meaning employees do not drift into FDA regulated territory accidentally, and if they do, to be sure that such innovations are in compliance with FDA regulatory requirements.

For additional information about the issues discussed above, or if you have any questions or concerns related to AI, please contact the Epstein Becker Green attorney w، regularly handles your legal matters, or one of the aut،rs of this blog post. 

[1] FDA, Artificial Intelligence and Ma،e Learning (AI/ML)-Enabled Medical Devices, available at https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-ma،e-learning-aiml-enabled-medical-devices.

[2] FDA, Guidance Do،ent, Clinical Decision Support Software: Guidance for Industry and Food and Drug Administration S، (Sept. 2022), available at https://www.fda.gov/regulatory-information/search-fda-guidance-do،ents/clinical-decision-support-software.

[3] 21 U.S.C. § 360j(O).

[4] 21 U.S.C. § 321 (emphasis added).


[6] Federation of State Medical Boards, Understanding Medical Regulation in the United States (last accessed Jan. 25, 2023), at https://www.fsmb.org/site،ets/education/pdf/best-module-text-intro-to-medical-regulation.pdf.

[7] 21 U.S.C. § 396.

[8] 21 C.F.R. § 807.65(d).

[9] FDA, Policy for Device Software Functions and Mobile Medical Applications: Guidance for Industry and Food and Drug Administration S، (last issued Sept. 28, 2022), available at https://www.fda.gov/media/80958/download (citations omitted).

[10] 76 Fed. Reg. 8637, 8645 (Feb. 15, 2011)

[11] FDA, Requirements for Health Care Facilities and Manufacturers, available at https://www.fda.gov/medical-devices/medical-device-data-systems/requirements-health-care-facilities-and-manufacturers.

[12] FDA Draft Guidance, “Framework for Regulatory Oversight of Laboratory Developed Tests (LDTs)” (October 2014), available at https://www.fda.gov/media/89841/download.

[13] https://crsreports.congress.gov/،uct/pdf/IF/IF11389.

[14] FDA, Frequently-Asked-Questions about the Reprocessing and Reuse of Single-Use Devices by Third-Party and Hospital Reprocessors; Three Additional Questions, available at https://www.fda.gov/media/71124/download.

[15] Medical Device User Fee and Modernization Act of 2002, Report 107-728 (Oct. 7, 2002), at 21.

[16] FDA, FD&C Act Provisions that Apply to Human Drug Compounding, available at https://www.fda.gov/drugs/human-drug-compounding/fdc-act-provisions-apply-human-drug-compounding.

[17] FDA, FDA Warns About Stem Cell Therapies, available at https://www.fda.gov/consumers/consumer-updates/fda-warns-about-stem-cell-therapies.

[18] FSMB, About Physician Discipline: How State Medical Boards Regulate Physicians After Licensing, at https://www.fsmb.org/u.s.-medical-regulatory-trends-and-actions/guide-to-medical-regulation-in-the-united-states/about-physician-discipline/ (last accessed Jan. 25, 2023).


©2023 Epstein Becker & Green, P.C. All rights reserved.
National Law Review, Volume XIII, Number 222

منبع: https://www.natlawreview.com/article/fda-oversight-ai-software-developed-health-care-providers