RAIDP04-BP02 Periodically evaluate and update datasets in the registry
Schedule regular review cycles that assess whether existing datasets still meet your evolving requirements and quality standards. Increment version numbers and update associated documentation whenever datasets change, maintaining records of what changed and why. Assess whether dataset updates require corresponding system retraining or evaluation re-runs to maintain validity of previous results. Remove or archive outdated dataset versions while preserving the ability to reproduce historical results when needed for auditing or comparison purposes.
Level of risk exposed if this best practice is not established: Medium
Implementation considerations
-
Schedule review processes that automatically flag datasets for evaluation based on age, usage patterns, or changes in your system requirements.
-
Create change management workflows that require documenting the reason for a dataset modification along with version increments.
-
Compare new dataset versions against established quality metrics to catch degradation over time.
-
Design impact assessment procedures that assist you to decide when dataset changes require retraining your models or re-running evaluations.
-
Set up archival processes that move old dataset versions to long-term storage while keeping enough metadata to recreate historical results if needed.
Resources
Related documents:
Related tools:
