View a markdown version of this page

RAIDP03-BP03 Protect the privacy of individuals represented in your datasets - Responsible AI Lens

RAIDP03-BP03 Protect the privacy of individuals represented in your datasets

Translate the guidance of your legal counsel on what constitutes personal information into technical definitions appropriate to your use case. Implement processes to identify and limit personal information in training, evaluation, and auxiliary datasets, using both automated filtering, data obfuscation, and manual review approaches. Validate the effectiveness of your privacy protection mechanisms against your taxonomy of personal information types. Maintain detailed documentation of privacy protection measures and regularly audit datasets so that personal information removal doesn't compromise your ability to measure important system behaviors.

Level of risk exposed if this best practice is not established: High

Implementation considerations

  1. Translate the guidance of your legal counsel into a taxonomy of personal data types. For example, define the string patterns for direct identifiers (like names and addresses), quasi-identifiers (like age and zip code), and other attributes (like health conditions and financial status) relevant to your domain.

  2. Implement multi-layered privacy filtering processes combining automated detection, data obfuscation, and manual review. For instance, use regex patterns and named entity recognition to flag potential personal information, and then apply techniques like tokenization, masking, or synthetic data replacement.

  3. Create test datasets with deliberately inserted personal information to evaluate privacy criteria while preserving data utility.

  4. Balance privacy protection with system and evaluation needs by verifying that your privacy measures don't compromise your system's ability to address your use case or your ability to test release criteria. For instance, verify that anonymization techniques maintain demographic diversity needed for fairness assessments.

  5. Document privacy protection decisions and create audit trails of what information gets filtered, obfuscated, or retained.

Resources

Related documents: