Model Cards and Nutrition Labels for Health AI, Explained

A health AI model card, sometimes called a nutrition label, is a short standardized document that summarizes what an artificial intelligence model does, the patient population it was built and tested on, how well it performed, and where it should not be used. The Coalition for Health AI (CHAI) has published such a template as a voluntary tool to help hospitals compare products during purchasing. That is a distinct thing from regulatory labeling. When the U.S. Food and Drug Administration authorizes a machine-learning-enabled medical device, the label it clears is legally binding. Both push in the same direction, toward more honest disclosure about how these tools behave, but they carry very different weight.

This article is educational and not medical advice.

What a model card actually contains

The food-label analogy is apt because the goal is the same, to put the important facts in one predictable place so a non-specialist can scan them. According to CHAI, its model card is meant to provide key information to support the evaluation of AI solution performance and safety, and was designed as a starting point for those reviewing AI models during the procurement process.

The template pulls together items a buyer would otherwise have to chase across sales decks and PDFs.

Identity and purpose

Who developed the model, its intended uses, and the specific patient populations it targets. A sepsis-prediction tool validated on adult inpatients is a different product from one aimed at pediatric emergency departments, and the card is supposed to make that scope explicit.

Data and model type

What kind of model it is, what data types it consumes, and the security and compliance accreditations behind it. This is the equivalent of an ingredient list.

Performance and limits

Key performance metrics, maintenance requirements, known risks, out-of-scope uses, known bias, and ethical considerations. The last several matter most and are the easiest to omit in marketing material. A card that names its blind spots is doing the job; one that reads like a brochure is not.

The practical value is comparability. CHAI describes the card as an open, cost-free tool so that a procurement team can line up several vendors and read the same fields in the same order rather than reverse-engineering claims from pitch materials. Participation is voluntary, and a developer can publish a card whether or not it belongs to the coalition.

Where the assurance lab fits

A label is only as good as the testing behind it. CHAI has also advanced a certification framework for independent assurance labs, the organizations that would evaluate a model and populate the metrics a card reports. That framework draws on ISO 17025, the main international standard for testing and calibration laboratories, and was developed with the ANSI National Accreditation Board.

Two design choices are worth flagging. First, the framework calls for mandatory disclosure of conflicts of interest between assurance labs and the developers whose models they test, which is the recurring failure mode in any voluntary evaluation scheme. Second, it draws on FDA thinking about data quality. The ambition is a chain of custody for the numbers: an accredited lab runs the evaluation, discloses its independence, and the result lands on a card in a common format. As of CHAI's 2024 announcement, both the certification framework and the model card were described as forthcoming and subject to stakeholder review, so it is fair to read them as proposals rather than finished, widely adopted standards.

How this differs from FDA labeling

The regulatory track is a separate mechanism with separate force. On June 13, 2024, the FDA, Health Canada, and the United Kingdom's MHRA jointly published "Transparency for Machine Learning-Enabled Medical Devices: Guiding Principles," which builds on the ten good-machine-learning-practice principles the same three agencies identified in 2021. In that framework, transparency describes the degree to which appropriate information about a device, including its intended use, development, and performance, is clearly communicated to the audiences who rely on it.

Three differences are structural.

Legal weight. A CHAI model card is a voluntary disclosure. FDA-cleared labeling is a regulatory instrument. A device that fails to match its cleared label can face enforcement; a vendor whose voluntary card oversells a product faces mainly reputational and contractual consequences.

Scope. FDA oversight attaches to software that meets the definition of a medical device. A large amount of health AI, including administrative triage, ambient documentation, and operational tools, sits outside that boundary and receives no mandatory label at all. Voluntary frameworks are one of the few disclosure mechanisms that reach those products.

What gets guaranteed. Guiding principles are exactly that, principles, not a checklist a manufacturer passes or fails. They shape what agencies expect to see over a device's life cycle. A model card, by contrast, is a fixed document a buyer can read today. The two are complementary: the regulatory principles set direction, and voluntary labels try to make disclosure concrete and comparable before, and often beyond, the point where regulation applies.

How to read one without being misled

A model card is a claim, not a verdict. Three habits help. Check whether the population the card describes matches the patients in front of you, because a strong metric on the wrong population is not reassuring. Look at who generated the performance numbers and whether any independence was disclosed, since a vendor grading its own homework is the weakest form of evidence. And read the out-of-scope and known-bias sections first, because the absence of stated limitations usually signals an incomplete card rather than a flawless model. Treat the label as the beginning of due diligence, not the end of it.