Intermediate

Model Cards

Model cards are standardized documents that accompany trained ML models. They provide essential information about a model's intended use, performance characteristics, limitations, and ethical considerations in a structured, accessible format.

What is a Model Card?

Introduced by Mitchell et al. at Google in 2019, model cards are short documents that accompany trained ML models. They serve as a "nutrition label" for AI, giving stakeholders the information they need to make informed decisions about model adoption and deployment.

✅
Key Insight: A model card should be understandable by both technical and non-technical audiences. Use clear language, include visualizations of performance data, and explicitly state what the model should and should not be used for.

Model Card Sections

Section Content
Model Details Model name, version, type, architecture, training date, developers, license, and contact information.
Intended Use Primary use cases, intended users, out-of-scope uses, and contexts where the model should not be deployed.
Metrics Performance metrics, evaluation methodology, confidence intervals, and disaggregated results across relevant subgroups.
Training Data Dataset description, preprocessing steps, data splits, and any known biases or limitations in the training data.
Ethical Considerations Potential risks, known biases, fairness assessments, privacy implications, and mitigation strategies in place.

Creating an Effective Model Card

  1. Start During Development

    Begin writing your model card alongside model development, not as an afterthought. Document design decisions, training choices, and evaluation results as they happen.

  2. Be Honest About Limitations

    Clearly state what your model cannot do, where it performs poorly, and what biases exist. Transparency about weaknesses builds trust and prevents misuse.

  3. Include Disaggregated Metrics

    Report performance across relevant demographic groups, data slices, and edge cases. Aggregate metrics can hide significant disparities in model behavior.

  4. Update Regularly

    Model cards are living documents. Update them when models are retrained, new evaluations are performed, or new limitations are discovered in production.

Model Card Examples

Hugging Face

Every model on the Hugging Face Hub includes a model card. These have become the de facto standard for open-source ML model documentation.

Google Cloud AI

Google publishes model cards for their commercial AI services, setting a precedent for enterprise AI documentation standards.

OpenAI System Cards

OpenAI publishes system cards for models like GPT-4, covering capabilities, limitations, safety evaluations, and deployment safeguards.

Anthropic Model Cards

Anthropic provides detailed documentation for Claude models, including safety testing results, responsible use guidelines, and known limitations.

💡
Looking Ahead: In the next lesson, we will explore datasheets for datasets — the companion to model cards that documents the data used to train and evaluate AI systems.