Model card vs system card debates often assume one label can contain every fact about AI. It cannot. A model has training, capability, and evaluation evidence. A system combines models with data, prompts, tools, policies, interfaces, users, and human procedures. A service has commercial and operating commitments. Data and evaluations have their own provenance and validity. Documentation works when each artifact has a defined object, audience, owner, evidence source, and update trigger, with links connecting the layers.
The original model cards paper proposed short documents accompanying trained models, including intended uses, evaluation procedures, and performance across relevant groups. The AI FactSheets paper drew on supplier declarations of conformity to cover purpose, performance, safety, security, and provenance for AI services. A system card is widely used for end-to-end safety and deployment reporting, but implementations vary. Treat labels as design patterns, not magic compliance terms.
Set boundaries for model cards, system cards, and FactSheets
A model card describes a model release: provenance, architecture where appropriate, training and evaluation, intended and excluded uses, capabilities, limitations, licenses, and ethical or safety considerations. A system card describes an assembled system and release: components, interaction design, policy, safeguards, red teaming, system evaluations, residual risks, deployment decisions, and monitoring. A FactSheet describes a supplied AI service or product through assertions and evidence that consumers, buyers, assessors, and operators can examine.
These scopes may overlap, especially when one organization releases a model only through one product. Keep object identity explicit anyway. The GPT-4o System Card demonstrates system-level reporting across model data and training, risk identification, external red teaming, evaluations, mitigations, third-party assessments, and societal impacts. It is a useful public example, not a universal template for every system.
| Artifact | Primary object | Primary audience | Core question |
|---|---|---|---|
| Model card | Versioned model | Model users, evaluators, builders | What can this model do, under what evidence and limits? |
| System card | Integrated system or product release | Users, safety reviewers, governance, public | How does the assembled system behave and what mitigations remain? |
| AI FactSheet | Supplied service and provider assertions | Buyers, assessors, operators | What is being declared about purpose, performance, safety, security, and provenance? |
| Data documentation | Dataset or data process | Data stewards, builders, reviewers | Where did data come from and when is it suitable? |
| Evaluation report | Test claim and evidence | Decision makers and reproducibility reviewers | What exactly was tested, how, and with what uncertainty? |
Use model cards for reusable model evidence
Name the exact weights or endpoint release, lineage, modality, training cutoff where known, data description, fine-tuning, context limits, inference settings, intended uses, out-of-scope uses, evaluation datasets, metrics, slices, limitations, security assumptions, license, and maintainer. Include model-level mitigations and their measured effects. Distinguish observed evidence from recommended use and unresolved questions.
Do not put downstream claims in a model card. A base model cannot promise that a bank's loan workflow is fair, a hospital's summarizer is clinically reliable, or an agent's tool calls are authorized. Link known system implementations as separate records. For embedding systems, Edilec's architecture and governance guide illustrates how model choice, source data, retrieval, and operations create distinct system evidence.
Use system cards for integrated behavior and deployment decisions
Document the system boundary, component versions, data flows, prompts, retrieval, tools, policies, user controls, affected groups, intended purpose, foreseeable misuse, red-team scope, system evaluations, human factors, mitigations, residual risks, launch conditions, monitoring, incidents, and rollback. Explain what the system does when uncertain, attacked, denied a tool, or given conflicting instructions. Include environments and product tiers because controls may differ.
A system card should support a release decision, not merely announce one. Link evaluation reports, threat models, impact assessments, policy specifications, and risk acceptances. The NIST Generative AI Profile offers risk-management actions that can inform coverage, but the card should prioritize material risks in its actual context.
Use an AI FactSheet for supplier declarations and buyer decisions
A FactSheet should identify the service, supplier, versions, scope, intended customers, performance assertions, evidence references, safety and security practices, provenance, data handling, support, change policy, incidents, compliance claims, and declaration owner. Phrase claims so they can be tested. 'Enterprise-grade security' is marketing; 'customer prompts are retained for 30 days in named regions under stated access controls' is reviewable.
The original FactSheets proposal emphasizes supplier declarations, but buyers still verify material assertions. A declaration can be wrong, stale, scoped narrowly, or based on tests that do not transfer. Procurement should record accepted claims, open questions, contractual commitments, verification evidence, and conditions. Link to the system card when the service and system are the same, but do not duplicate large sections that will drift.
Connect artifacts through an evidence graph
Give every artifact a stable ID, object ID, version, owner, status, effective date, review date, confidentiality level, and supersedes relationship. A system card references exact model cards, dataset records, evaluation reports, and controls. A customer FactSheet references the service release and selected assurance. An impact assessment references the system and deployment context. Changes should identify dependent artifacts that require review.
| Change | Artifact updated first | Dependent review | Reason |
|---|---|---|---|
| New model weights | Model card | System card, evaluations, FactSheet | Capabilities and limits may shift |
| New retrieval corpus | Data documentation | System card, impact and evaluation | Evidence and privacy context changed |
| Tool permission added | System card and threat model | FactSheet and operating instructions | System gains new effects |
| New customer use | Impact assessment | System card applicability and FactSheet | Context may exceed evidence |
| Vendor contract change | FactSheet or service record | System risk and procurement approval | Operating assumptions changed |
Use the data model documentation guide to define relationships and the compliance-ready delivery guide to make updates part of release evidence. Avoid copying the same paragraph into many artifacts; store a canonical assertion and reference it with the applicable version.
Publish different layers for different audiences
Maintain an internal evidence layer with sensitive architecture, vulnerabilities, test data, incidents, and legal analysis; a controlled customer layer with due-diligence detail under confidentiality; and a public transparency layer that communicates intended use, material limitations, safety work, and contact routes without enabling abuse or exposing protected information. Redaction should have a rationale and owner. Public brevity should not become internal evidence loss.
Use plain language and state uncertainty. Separate facts, provider assertions, test results, assumptions, and risk decisions. The OECD classification framework offers useful dimensions for context, data and input, model, and task and output; these can help readers understand the system without collapsing all documentation into one artifact.
Assign ownership and quality gates
Model engineering owns model facts; product and safety owners own system evidence; service or assurance owners own supplier declarations; data stewards own data documentation; evaluation leads own test reports. Legal, privacy, security, domain, and affected-stakeholder reviewers challenge relevant claims. A publication owner ensures that public wording faithfully reflects approved evidence. No single documentation team should invent technical facts it cannot verify.
At release, test completeness, version consistency, source links, unresolved-risk visibility, confidentiality decisions, and update triggers. Periodically sample claims against source evidence and production behavior. The AI governance operating model can assign these decision rights without creating a documentation bureaucracy.
Write claims that can survive review
Every important statement should identify scope and evidence. Replace 'the model is unbiased' with the evaluated behavior, population, metric, comparison, result, date, and known limitation. Replace 'human in the loop' with who reviews, what information they receive, how much time they have, their authority, and observed override practice. Avoid timeless present tense for results tied to a particular release.
Use a claim taxonomy: measured result, design fact, policy commitment, contractual commitment, professional judgment, known limitation, and unresolved question. Readers can then distinguish what was observed from what is promised. Link measured claims to evaluation IDs and policy claims to controlled specifications. Require legal and communications review for public assertions without allowing either team to soften material limitations beyond recognition.
Test documentation with its audience. Ask a downstream engineer to choose an integration constraint, an operator to identify a stop condition, a buyer to locate data terms, and an affected user to understand recourse. Record whether they find the right answer. Readability is a control property: evidence that cannot guide its intended decision is not made useful merely by being technically present.
Archive superseded artifacts without leaving them discoverable as current guidance. Public pages should name release coverage and link to a version index; internal catalogs should mark status and replacement. When a claim is corrected, record the reason and assess whether customers, operators, or affected users relied on the prior wording. Documentation errors can be incidents when they cause unsafe configuration or misleading assurance, so route material corrections through the incident process.
Key takeaways
- Use model cards for versioned model evidence, system cards for integrated behavior and deployment, and FactSheets for supplier declarations.
- Keep data and evaluation evidence in separately owned artifacts and link exact versions.
- Do not let model evidence imply that a downstream workflow is safe, lawful, or effective.
- Create internal, controlled-customer, and public layers with explicit redaction decisions.
- Trigger dependent reviews when models, data, tools, uses, or contracts change.
Frequently asked questions
Can a small team maintain one combined document?
Yes, if sections identify distinct objects, owners, versions, and triggers. Split it when model reuse, system variants, customer assurance, or update frequency makes one document ambiguous or stale.
Are these legally standardized formats?
Not universally. Model cards and FactSheets originated as reporting proposals, while system-card practice varies. Applicable laws or contracts may require particular technical documentation regardless of the label.
Must every artifact be public?
No. Publish what law, commitments, and transparency goals require, while maintaining deeper controlled evidence. Protect security, privacy, trade secrets, and test integrity through justified access controls.
Conclusion
The useful model card vs system card distinction is about evidence boundaries. Describe the model where model facts belong, the integrated deployment where system behavior belongs, and supplier claims where buyers can evaluate them. Link data, evaluation, impact, and control artifacts through versions. Clear ownership and update triggers make transparency more accurate while reducing duplicated prose and stale claims.