AI Compliance in Health Care: FDA, HIPAA, Section 1557, and State Laws

When is AI software a medical device?

AI software is a medical device when it is intended to diagnose, treat, mitigate, cure, or prevent disease and does not fit an exclusion Congress wrote into the law in 2016. Using AI does not make software a device, and avoiding the word does not keep it out.

The exclusion that matters most is for clinical decision support, or CDS. Software is not a device if it meets all four statutory criteria: it does not acquire, process, or analyze a medical image, a signal from an in vitro diagnostic device, or a pattern or signal from a signal acquisition system; it displays, analyzes, or prints medical information; it supports or provides recommendations to a health care professional about prevention, diagnosis, or treatment; and it lets the professional independently review the basis for the recommendation, so the professional does not rely primarily on it (21 U.S.C. 360j(o)(1)(E)).

On January 6, 2026, FDA issued revised final guidance on CDS and on general wellness products, replacing the 2022 and 2019 versions without a prior draft. FDA’s digital health guidance list shows the current CDS guidance dated January 29, 2026. According to law firm summaries:

• FDA will not enforce device requirements for CDS that gives a single recommendation, if it is the only clinically appropriate option and the software meets the other criteria.
• Time-critical decision-making was removed from the third criterion.
• Clear documentation for clinicians on how the software works remains central to the fourth criterion.
• Some non-invasive sensing wellness products may display values, trends, and notifications suggesting follow-up, if claims stay away from specific diseases.

The fourth criterion still decides most cases. If a clinician cannot see why the software recommended what it did, the software is likely a device.

What has FDA said about AI-enabled devices?

FDA’s position on AI-enabled devices sits mainly in three documents: a final guidance on change control plans, a draft guidance on the full product lifecycle, and a public list of authorized devices.

• Predetermined change control plans (final): FDA finalized guidance in December 2024 on PCCPs for AI-enabled device software functions. A PCCP lets a manufacturer get advance authorization for planned model changes, described through a description of modifications, a modification protocol, and an impact assessment. FDA’s digital health guidance list shows an issue date of August 18, 2025 for the current version.
• Lifecycle management (draft): “Artificial Intelligence-Enabled Device Software Functions: Lifecycle Management and Marketing Submission Recommendations” was issued in draft in January 2025 (docket FDA-2024-D-4488). It remained a draft as of this writing.
• AI-enabled device list: FDA maintains a public list of authorized AI-enabled devices. FDA says the list is not comprehensive, and that it will explore ways to tag devices that use foundation models, including large language models.

For a developer, the PCCP is the practical tool. Without one, a retrained model can require a new submission. With one, changes inside the plan’s bounds do not.

What does HIPAA require when an AI vendor sees patient data?

An AI vendor that creates, receives, maintains, or transmits protected health information on behalf of a covered entity is a business associate, and the covered entity needs a business associate agreement with the elements in 45 CFR 164.504(e) before the vendor gets access. HIPAA has no AI-specific rule because the business associate rules already cover the case. Business associates are directly liable for uses and disclosures their contract does not permit.

HHS has said that a cloud provider storing or processing electronic PHI is a business associate even if it holds only encrypted data and has no key. The same logic applies to an AI service that processes clinical notes.

• No PHI goes into a consumer AI tool that does not offer a BAA
• The BAA limits the vendor’s use of PHI, including for training models for other customers
• The minimum necessary standard applies to what you send the model
• Security Rule risk analysis covers the AI system and its data flows
• Subcontractors of the vendor sign their own BAAs

What do health IT certification rules require?

Developers of certified health IT that supply “predictive” decision support interventions must make source attribute information available to users and apply risk management practices, under ONC’s HTI-1 rule. Source attributes include intended use, target population, known risks, training data, and validation. HTI-1, published in January 2024, created this decision support interventions criterion at 45 CFR 170.315(b)(11).

That may change. In December 2025, ASTP/ONC proposed HTI-5 (published in the Federal Register December 29, 2025), which would remove both the source attribute requirement and the predictive DSI risk management requirement. The agency said it had no publicly available evidence that the transparency requirements improved patient care. Comments closed February 27, 2026. As of this writing, HTI-5 was not final.

A hospital buying certified software should not assume the model card information will keep coming. Write it into the contract.

What does Section 1557 require for clinical algorithms?

Section 1557 requires covered entities to make reasonable efforts to identify patient care decision support tools, including AI, whose input variables measure protected characteristics, and to mitigate the discrimination risk from each one. The duty is in 45 CFR 92.210, added by HHS’s 2024 Section 1557 rule, and compliance with its identification and mitigation duties began May 1, 2025.

A covered entity may not discriminate on the basis of race, color, national origin, sex, age, or disability through its use of patient care decision support tools. It must make reasonable efforts to identify tools that use input variables measuring those characteristics, and make reasonable efforts to mitigate the risk of discrimination from each such tool.

The rule reaches flowcharts and clinical calculators as well as machine learning.

Parts of the 2024 rule were vacated by a federal court on gender identity grounds, and HHS published a notice on June 2, 2026 implementing that ruling. That notice did not address 92.210, which remained in the eCFR as of this writing.

• Inventory clinical tools that use race, sex, age, disability, or national origin as inputs
• Document why each variable is used and what the evidence says
• Record the mitigation chosen, such as a race-neutral version or clinician override
• Assign an owner to repeat the review when a tool changes

What do Medicare Advantage plans need to know?

Medicare Advantage plans may use algorithms to help with coverage decisions, but CMS requires each medical necessity decision to rest on the individual patient’s circumstances.

CMS set this out in a February 6, 2024 FAQ on its 2024 coverage criteria and utilization management rule. It said an MA organization may use an algorithm or software to assist coverage determinations, but must make medical necessity decisions based on the individual patient’s circumstances. Basing a decision on a larger data set instead of the patient’s own history, physician recommendation, or clinical notes would not comply.

CMS proposed broader AI guardrails for 2026 but did not finalize them in its April 2025 final rule, and said it may revisit the topic.

In traditional Medicare, CMS’s WISeR model began January 1, 2026 in New Jersey, Ohio, Oklahoma, Texas, Arizona, and Washington. Participating companies use technologies including AI and machine learning in prior authorization review for a set of services. CMS states that licensed clinicians determine all recommendations for non-payment.

Which state laws apply?

California, Texas, Illinois, Nevada, and Colorado have health-specific AI rules, and two of them, Illinois and Nevada, aim specifically at AI in mental health care.

A product that offers emotional support by chatbot should be reviewed against the Illinois and Nevada laws before launch in those states.

What should a health system do first?

Answer four questions for each AI tool. The FDA asks what the software claims to do. HIPAA asks who sees the data. Section 1557 asks whether the tool treats patients differently. Coverage rules ask whether a person made the decision. In practice that means:

• Classify each AI tool: regulated device, non-device CDS, administrative, or patient-facing
• Confirm a BAA is in place for every vendor that touches PHI
• Complete the 92.210 inventory and mitigation record
• Add AI disclosures where California, Texas, or Colorado law requires them
• For payers, document that each denial rests on the individual patient’s record and a clinician’s decision
• Put model documentation requirements in software contracts, independent of HTI-5

See the AI compliance guide for the inventory process and the insurance guide for payer underwriting rules.

This guide is general information as of October 2026, not legal or regulatory advice. Confirm device status with FDA guidance or regulatory counsel.