View a markdown version of this page

RAIGT01-BP02 Create a system card that communicates intended usage and limitations - Responsible AI Lens

RAIGT01-BP02 Create a system card that communicates intended usage and limitations

AI system cards are a form of responsible AI documentation that provide stakeholders with a single place to find information on the intended use cases and limitations, responsible AI design choices, and deployment and performance optimization best practices. System cards do not provide guidance on expected performance of the AI system on the specific inputs the deployer may provide; that testing is the responsibility of the deployer.

Level of risk exposed if this best practice is not established: High

Implementation considerations

  1. Identify intended use case(s) to illustrate how users should plan to interact with your system. The use case section gives the reader a tangible example, describing the steps and workflow required end-to-end while calling out limitations in the technology.

  2. Plan a specific set of evaluations for the AI service card. As appropriate, disclose the datasets chosen for the evaluations and how they meet the criteria to support the testing of each Responsible AI dimension. For example, datasets should have appropriate demographic labels for fairness testing, a representative sample of examples from known safety categories, and common as well as uncommon variations in the input examples for robustness testing.

  3. Include performance metrics and success criteria for each use case, with real-world examples demonstrating proper implementation.

  4. Detail system limitations and constraints. Consider financial risk assessment AI where specific market conditions or transaction types might fall outside system capabilities. Document scenarios where system performance may degrade or become unreliable, including environmental factors affecting behavior.

  5. Outline potential failure modes and implementation strategies when appropriate. As an example, describe how a recommendation system might fail during high-traffic periods or with novel user patterns, and provide recommended responses. Include warning signs and blocking strategies for each failure mode.

Resources

Related documents:

Related tools: