# Redacting or identifying personally identifiable information Redacting transcripts Redaction is used to mask or remove sensitive content, in the form of personally identifiable information (PII), from your transcripts. The types of PII Amazon Transcribe can redact varies between batch and streaming transcriptions. To view the PII list for each transcription method, refer to [Redacting PII in your batch job](pii-redaction-batch.md) and [Redacting or identifying PII in a real-time stream](pii-redaction-stream.md). With streaming transcriptions, you also have the option to flag PII without redacting it; refer to [Example PII identification output](pii-redaction-output.md#pii-redaction-output-id) for an output example. When redaction is enabled, you have the option to generate only a redacted transcript or both a redacted transcript and an unredacted transcript. If you choose to generate only a redacted transcript, note that your media is the only place where the complete conversation is stored. If you delete your original media, there is no record of the unredacted PII. Because of this, it may be prudent to generate an unredacted transcript in addition to a redacted one. To learn more about PII redaction with batch transcriptions, refer to: [Redacting PII in your batch job](pii-redaction-batch.md). To learn more about PII redaction or identification with streaming transcriptions, refer to: [Redacting or identifying PII in a real-time stream](pii-redaction-stream.md). **Important** The redaction feature is designed to identify and remove sensitive data. However, due to the predictive nature of machine learning, Amazon Transcribe may not identify and remove all instances of sensitive data in your transcript. We strongly recommend that you review any redacted output to ensure it meets your needs. The redaction feature does not meet the requirements for de-identification under medical privacy laws, such as the U.S. Health Insurance Portability and Accountability Act of 1996 (HIPAA). For a video walkthrough of redacting and identifying PII, see: [![AWS Videos](http://img.youtube.com/vi/https://www.youtube.com/embed/4H8dQoeLkyM/0.jpg)](http://www.youtube.com/watch?v=https://www.youtube.com/embed/4H8dQoeLkyM) # Redacting PII in your batch job When redacting personally identifiable information (PII) from a transcript during a batch transcription job, Amazon Transcribe replaces each identified instance of PII with `[PII]` in the main text body of your transcript. You can also view the type of PII that is redacted in the word-for-word portion of the transcription output. For an output sample, see [Example redacted output (batch)](pii-redaction-output.md#pii-redaction-output-batch). Redaction with batch transcriptions is available with US English (`en-US`) and US Spanish (`es-US`). Redaction is not compatible with [language identification](lang-id-batch.md). Both redacted and unredacted transcripts are stored in the same output Amazon S3 bucket. Amazon Transcribe stores them in a bucket you specify or in the default Amazon S3 bucket managed by the service. **Types of PII Amazon Transcribe can recognize for batch transcriptions** | PII type | Description | | --- | --- | | ADDRESS | A physical address, such as *100 Main Street, Anytown, USA* or *Suite \$112, Building 123*. An address can include a street, building, location, city, state, country, county, zip, precinct, neighborhood, and more. | | ALL | Redact or identify all PII types listed in this table. | | BANK\$1ACCOUNT\$1NUMBER | A US bank account number. These are typically between 10 - 12 digits long, but Amazon Transcribe also recognizes bank account numbers when only the last 4 digits are present. | | BANK\$1ROUTING | A US bank account routing number. These are typically 9 digits long, but Amazon Transcribe also recognizes routing numbers when only the last 4 digits are present. | | CREDIT\$1DEBIT\$1CVV | A 3-digit card verification code (CVV) that is present on VISA, MasterCard, and Discover credit and debit cards. In American Express credit or debit cards, it is a 4-digit numeric code. | | CREDIT\$1DEBIT\$1EXPIRY | The expiration date for a credit or debit card. This number is usually 4 digits long and formatted as month/year or MM/YY. For example, Amazon Transcribe can recognize expiration dates such as *01/21*, *01/2021*, and *Jan 2021*. | | CREDIT\$1DEBIT\$1NUMBER | The number for a credit or debit card. These numbers can vary from 13 to 16 digits in length, but Amazon Transcribe also recognizes credit or debit card numbers when only the last 4 digits are present. | | EMAIL | An email address, such as *efua.owusu@email.com*. | | NAME | An individual's name. This entity type does not include titles, such as Mr., Mrs., Miss, or Dr. Amazon Transcribe does not apply this entity type to names that are part of organizations or addresses. For example, Amazon Transcribe recognizes the *John Doe Organization* as an organization, and *Jane Doe Street* as an address. | | PHONE | A phone number. This entity type also includes fax and pager numbers. | | PIN | A 4-digit personal identification number (PIN) that allows someone to access their bank account information. | | SSN | A Social Security Number (SSN) is a 9-digit number that is issued to US citizens, permanent residents, and temporary working residents. Amazon Transcribe also recognizes Social Security Numbers when only the last 4 digits are present. | You can start a batch transcription job using the AWS Management Console, AWS CLI, or AWS SDK. ## AWS Management Console 1. Sign in to the [AWS Management Console](https://console.aws.amazon.com/transcribe/). 1. In the navigation pane, choose **Transcription jobs**, then select **Create job** (top right). This will open the **Specify job details** page. 1. After filling in your desired fields on the **Specify job details** page, select **Next** to go to the **Configure job - *optional*** page. Here you'll find the **Content removal** panel with the **PII redaction** toggle. ![\[Amazon Transcribe console screenshot: the 'content removal pane' on the 'configure job' page.\]](http://docs.aws.amazon.com/transcribe/latest/dg/images/content-redact.png) 1. Once you select **PII redaction**, you have the option to select all PII types you want to redact. You can also choose to have an unredacted transcript if you select **Include unredacted transcript in job output** box. ![\[Amazon Transcribe console screenshot: the 'content removal' pane showing PII options.\]](http://docs.aws.amazon.com/transcribe/latest/dg/images/content-redact-select.png) 1. Select **Create job** to run your transcription job. ## AWS CLI This example uses the [start-transcription-job](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/transcribe/start-transcription-job.html) command and `content-redaction` parameter. For more information, see [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartTranscriptionJob.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartTranscriptionJob.html) and [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_ContentRedaction.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_ContentRedaction.html). ``` aws transcribe start-transcription-job \ --region us-west-2 \ --transcription-job-name my-first-transcription-job \ --media MediaFileUri=s3://amzn-s3-demo-bucket/my-input-files/my-media-file.flac \ --output-bucket-name amzn-s3-demo-bucket \ --output-key my-output-files/ \ --language-code en-US \ --content-redaction RedactionType=PII,RedactionOutput=redacted,PiiEntityTypes=NAME,ADDRESS,BANK_ACCOUNT_NUMBER ``` Here's another example using the [start-transcription-job](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/transcribe/start-transcription-job.html) method, and the request body redacts PII for that job. ``` aws transcribe start-transcription-job \ --region us-west-2 \ --cli-input-json file://filepath/my-first-redaction-job.json ``` The file *my-first-redaction-job.json* contains the following request body. ``` { "TranscriptionJobName": "my-first-transcription-job", "Media": { "MediaFileUri": "s3://amzn-s3-demo-bucket/my-input-files/my-media-file.flac" }, "OutputBucketName": "amzn-s3-demo-bucket", "OutputKey": "my-output-files/", "LanguageCode": "en-US", "ContentRedaction": { "RedactionOutput":"redacted", "RedactionType":"PII", "PiiEntityTypes": [ "NAME", "ADDRESS", "BANK_ACCOUNT_NUMBER" ] } } ``` ## AWS SDK for Python (Boto3) This example uses the AWS SDK for Python (Boto3) to redact content using the `ContentRedaction` argument for the [start\$1transcription\$1job](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/transcribe.html#TranscribeService.Client.start_transcription_job) method. For more information, see [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartTranscriptionJob.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartTranscriptionJob.html) and [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_ContentRedaction.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_ContentRedaction.html). For additional examples using the AWS SDKs, including feature-specific, scenario, and cross-service examples, refer to the [Code examples for Amazon Transcribe using AWS SDKs](service_code_examples.md) chapter. ``` from __future__ import print_function import time import boto3 transcribe = boto3.client('transcribe', 'us-west-2') job_name = "my-first-transcription-job" job_uri = "s3://amzn-s3-demo-bucket/my-input-files/my-media-file.flac" transcribe.start_transcription_job( TranscriptionJobName = job_name, Media = { 'MediaFileUri': job_uri }, OutputBucketName = 'amzn-s3-demo-bucket', OutputKey = 'my-output-files/', LanguageCode = 'en-US', ContentRedaction = { 'RedactionOutput':'redacted', 'RedactionType':'PII', 'PiiEntityTypes': [ 'NAME','ADDRESS','BANK_ACCOUNT_NUMBER' ] } ) while True: status = transcribe.get_transcription_job(TranscriptionJobName = job_name) if status['TranscriptionJob']['TranscriptionJobStatus'] in ['COMPLETED', 'FAILED']: break print("Not ready yet...") time.sleep(5) print(status) ``` **Note** PII redaction for batch jobs is only supported in these AWS Regions: Asia Pacific (Hong Kong), Asia Pacific (Mumbai), Asia Pacific (Seoul), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Tokyo), GovCloud (US-West), Canada (Central), EU (Frankfurt), EU (Ireland), EU (London), EU (Paris), Middle East (Bahrain), South America (Sao Paulo), US East (N. Virginia), US East (Ohio), US West (Oregon), and US West (N. California). # Redacting or identifying PII in a real-time stream When redacting personally identifiable information (PII) from a streaming transcription, Amazon Transcribe replaces each identified instance of PII with `[PII]` in your transcript. An additional option available for streaming transcriptions is *PII identification*. When you activate PII Identification, Amazon Transcribe labels the PII in your transcription results under an `Entities` object. For an output sample, see [Example redacted streaming output](pii-redaction-output.md#pii-redaction-output-stream) and [Example PII identification output](pii-redaction-output.md#pii-redaction-output-id). Redaction and identification of PII with streaming transcriptions is available with these English dialects: Scottish (`en-AB`), Australia (`en-AU`), Canada (`en-CA`), British (`en-GB`), Ireland (`en-IE`), India (`en-IN`), New Zealand (`en-NZ`), United States (`en-US`), Wales (`en-WL`), and South Africa (`en-ZA`); Spanish dialects: US (`es-US`), Spain (`es-ES`); French dialects: French (`fr-FR`), Canada (`fr-CA`); Portuguese dialects: Portugal (`pt-PT`), Brazil (`pt-BR`); Italian dialect: Italy (`it-IT`); and German dialects: Germany (`de-DE`), Swiss (`de-CH`). PII identification and redaction for streaming jobs is performed only upon complete transcription of the audio segments. **Types of PII Amazon Transcribe can recognize for streaming transcriptions** | PII type | Description | | --- | --- | | ADDRESS | A physical address, such as *100 Main Street, Anytown, USA* or *Suite \$112, Building 123*. An address can include a street, building, location, city, state, country, county, zip, precinct, neighborhood, and more. | | ALL | Redact or identify all PII types listed in this table. | | BANK\$1ACCOUNT\$1NUMBER | A US bank account number. These are typically between 10 - 12 digits long, but Amazon Transcribe also recognizes bank account numbers when only the last 4 digits are present. | | BANK\$1ROUTING | A US bank account routing number. These are typically 9 digits long, but Amazon Transcribe also recognizes routing numbers when only the last 4 digits are present. | | CREDIT\$1DEBIT\$1CVV | A 3-digit card verification code (CVV) that is present on VISA, MasterCard, and Discover credit and debit cards. In American Express credit or debit cards, it is a 4-digit numeric code. | | CREDIT\$1DEBIT\$1EXPIRY | The expiration date for a credit or debit card. This number is usually 4 digits long and formatted as month/year or MM/YY. For example, Amazon Transcribe can recognize expiration dates such as *01/21*, *01/2021*, and *Jan 2021*. | | CREDIT\$1DEBIT\$1NUMBER | The number for a credit or debit card. These numbers can vary from 13 to 16 digits in length, but Amazon Transcribe also recognizes credit or debit card numbers when only the last 4 digits are present. | | EMAIL | An email address, such as *efua.owusu@email.com*. | | NAME | An individual's name. This entity type does not include titles, such as Mr., Mrs., Miss, or Dr. Amazon Transcribe does not apply this entity type to names that are part of organizations or addresses. For example, Amazon Transcribe recognizes the *John Doe Organization* as an organization, and *Jane Doe Street* as an address. | | PHONE | A phone number. This entity type also includes fax and pager numbers. | | PIN | A 4-digit personal identification number (PIN) that allows someone to access their bank account information. | | SSN | A Social Security Number (SSN) is a 9-digit number that is issued to US citizens, permanent residents, and temporary working residents. Amazon Transcribe also recognizes Social Security Numbers when only the last 4 digits are present. | | AGE | An individual's age, including the quantity and unit of time. For example, in the phrase "I am 40 years old," Amazon Transcribe recognizes "40 years" as an age. | | DATE\$1TIME | A date can include a year, month, day, day of week, or time of day. For example, Amazon Transcribe recognizes "January 19, 2020" or "11 am" as dates. Amazon Transcribe will recognize partial dates, date ranges, and date intervals. It will also recognize decades, such as "the 1990s". | | LICENSE\$1PLATE | A license plate for a vehicle is issued by the state or country where the vehicle is registered. The format for passenger vehicles is typically five to eight digits, consisting of upper-case letters and numbers. The format varies depending on the location of the issuing state or country. | | PASSPORT\$1NUMBER | A unique identifier assigned to an individual's passport. The format typically includes a combination of letters and numbers and varies by country. | | PASSWORD | An alphanumeric string that is used as a password, such as "\$1very20special\$1pass\$1". | | USERNAME | A user name that identifies an account, such as a login name, screen name, nick name, or handle. | | VEHICLE\$1IDENTIFICATION\$1NUMBER | A Vehicle Identification Number (VIN) uniquely identifies a vehicle. VIN content and format are defined in the ISO 3779 specification. Each country has specific codes and formats for VINs. | You can start a streaming transcription using the AWS Management Console, WebSocket, or HTTP/2. ## AWS Management Console 1. Sign into the [AWS Management Console](https://console.aws.amazon.com/transcribe/). 1. In the navigation pane, choose **Real-time transcription**. Scroll down to **Content removal settings** and expand this field if it is minimized. ![\[Amazon Transcribe console screenshot: the 'real-time transcription' page.\]](http://docs.aws.amazon.com/transcribe/latest/dg/images/redaction-stream1.png) 1. Toggle on **PII Identification & redaction**. ![\[Amazon Transcribe console screenshot: the expanded 'content removal settings' panel.\]](http://docs.aws.amazon.com/transcribe/latest/dg/images/redaction-stream2.png) 1. Select **Identification only** or **Identification & redaction**, then select the PII entity types you want to identify or redact in your transcript. ![\[Amazon Transcribe console screenshot: list of PII types that can be selected.\]](http://docs.aws.amazon.com/transcribe/latest/dg/images/redaction-stream3.png) 1. You're now ready to transcribe your stream. Select **Start streaming** and begin speaking. To end your dictation, select **Stop streaming**. ## WebSocket stream This example creates a presigned URL that uses PII redaction (or PII identification) in a WebSocket stream. Line breaks have been added for readability. For more information on using WebSocket streams with Amazon Transcribe, see [Setting up a WebSocket stream](streaming-setting-up.md#streaming-websocket). For more detail on parameters, see [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartStreamTranscription.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartStreamTranscription.html). ``` GET wss://transcribestreaming.us-west-2.amazonaws.com:8443/stream-transcription-websocket? &X-Amz-Algorithm=AWS4-HMAC-SHA256 &X-Amz-Credential=AKIAIOSFODNN7EXAMPLE%2F20220208%2Fus-west-2%2Ftranscribe%2Faws4_request &X-Amz-Date=20220208T235959Z &X-Amz-Expires=300 &X-Amz-Security-Token=security-token &X-Amz-Signature=string &X-Amz-SignedHeaders=content-type%3Bhost%3Bx-amz-date &language-code=en-US &media-encoding=flac &sample-rate=16000 &pii-entity-types=NAME,ADDRESS &content-redaction-type=PII (or &content-identification-type=PII) ``` You cannot use both `content-identification-type` and `content-redaction-type` in the same request. Parameter definitions can be found in the [API Reference](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_Reference.html); parameters common to all AWS API operations are listed in the [Common Parameters](https://docs.aws.amazon.com/transcribe/latest/APIReference/CommonParameters.html) section. ## HTTP/2 stream This example creates an HTTP/2 request with PII identification or PII redaction enabled. For more information on using HTTP/2 streaming with Amazon Transcribe, see [Setting up an HTTP/2 stream](streaming-setting-up.md#streaming-http2). For more detail on parameters and headers specific to Amazon Transcribe, see [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartStreamTranscription.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartStreamTranscription.html). ``` POST /stream-transcription HTTP/2 host: transcribestreaming.us-west-2.amazonaws.com X-Amz-Target: com.amazonaws.transcribe.Transcribe.StartStreamTranscription Content-Type: application/vnd.amazon.eventstream X-Amz-Content-Sha256: string X-Amz-Date: 20220208T235959Z Authorization: AWS4-HMAC-SHA256 Credential=access-key/20220208/us-west-2/transcribe/aws4_request, SignedHeaders=content-type;host;x-amz-content-sha256;x-amz-date;x-amz-target;x-amz-security-token, Signature=string x-amzn-transcribe-language-code: en-US x-amzn-transcribe-media-encoding: flac x-amzn-transcribe-sample-rate: 16000 x-amzn-transcribe-content-identification-type: PII (or x-amzn-transcribe-content-redaction-type: PII) x-amzn-transcribe-pii-entity-types: NAME,ADDRESS transfer-encoding: chunked ``` You cannot use both `content-identification-type` and `content-redaction-type` in the same request. Parameter definitions can be found in the [API Reference](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_Reference.html); parameters common to all AWS API operations are listed in the [Common Parameters](https://docs.aws.amazon.com/transcribe/latest/APIReference/CommonParameters.html) section. **Note** PII redaction for streaming is only supported in these AWS Regions: Asia Pacific (Seoul), Asia Pacific (Sydney), Asia Pacific (Tokyo), Canada (Central), EU (Frankfurt), EU (Ireland), EU (London), US East (N. Virginia), US East (Ohio), and US West (Oregon). # Example PII redaction and identification output Example output The following examples show redacted output from batch and streaming jobs, and PII identification from a streaming job. Transcription jobs using content redaction generate two types of `confidence` values. The Automatic Speech Recognition (ASR) confidence indicates the items that have the `type` of `pronunciation` or `punctuation` is a specific utterance. In the following transcript output, the word `Good` has a `confidence` of `1.0`. This confidence value indicates that Amazon Transcribe is 100 percent confident that the word uttered in this transcript is 'Good'. The `confidence` value for a `[PII]` tag is the confidence that the speech it flagged for redaction is truly PII. In the following transcript output, the `confidence` of `0.9999` indicates that Amazon Transcribe is 99.99 percent confident that the entity it redacted in the transcript is PII. ## Example redacted output (batch) ``` { "jobName": "my-first-transcription-job", "accountId": "111122223333", "isRedacted": true, "results": { "transcripts": [ { "transcript": "Good morning, everybody. My name is [PII], and today I feel like sharing a whole lot of personal information with you. Let's start with my Social Security number [PII]. My credit card number is [PII] and my C V V code is [PII]. I hope that Amazon Transcribe is doing a good job at redacting that personal information away. Let's check." } ], "items": [ { "id": 0, "start_time": "2.86", "end_time": "3.35", "alternatives": [ { "confidence": "1.0", "content": "Good" } ], "type": "pronunciation" }, Items removed for brevity { "id": 8, "start_time": "5.56", "end_time": "6.25", "alternatives": [ { "content": "[PII]", "redactions": [ { "confidence": "0.9999", "type": "NAME", "category": "PII" } ] } ], "type": "pronunciation" }, Items removed for brevity ], }, "status": "COMPLETED" } ``` Here's the unredacted transcript for comparison: ``` { "jobName": "job id", "accountId": "111122223333", "isRedacted": false, "results": { "transcripts": [ { "transcript": "Good morning, everybody. My name is Mike, and today I feel like sharing a whole lot of personal information with you. Let's start with my Social Security number 000000000. My credit card number is 5555555555555555 and my C V V code is 000. I hope that Amazon Transcribe is doing a good job at redacting that personal information away. Let's check." } ], "items": [ { "id": 0, "start_time": "2.86", "end_time": "3.35", "alternatives": [ { "confidence": "1.0", "content": "Good" } ], "type": "pronunciation" }, Items removed for brevity { "id": 8, "start_time": "5.56", "end_time": "6.25", "alternatives": [ { "confidence": "0.9999", "content": "Mike", { ], "type": "pronunciation" }, Items removed for brevity ], }, "status": "COMPLETED" } ``` ## Example redacted streaming output ``` { "TranscriptResultStream": { "TranscriptEvent": { "Transcript": { "Results": [ { "Alternatives": [ { "Transcript": "my name is [NAME]", "Items": [ { "Content": "my", "EndTime": 0.3799375, "StartTime": 0.0299375, "Type": "pronunciation" }, { "Content": "name", "EndTime": 0.5899375, "StartTime": 0.3899375, "Type": "pronunciation" }, { "Content": "is", "EndTime": 0.7899375, "StartTime": 0.5999375, "Type": "pronunciation" }, { "Content": "[NAME]", "EndTime": 1.0199375, "StartTime": 0.7999375, "Type": "pronunciation" } ], "Entities": [ { "Content": "[NAME]", "Category": "PII", "Type": "NAME", "StartTime" : 0.7999375, "EndTime" : 1.0199375, "Confidence": 0.9989 } ] } ], "EndTime": 1.02, "IsPartial": false, "ResultId": "12345a67-8bc9-0de1-2f34-a5b678c90d12", "StartTime": 0.0199375 } ] } } } } ``` ## Example PII identification output PII identification is an additional feature that you can use with your streaming transcription job. The identified PII is listed in each segment's `Entities` section. ``` { "TranscriptResultStream": { "TranscriptEvent": { "Transcript": { "Results": [ { "Alternatives": [ { "Transcript": "my name is mike", "Items": [ { "Content": "my", "EndTime": 0.3799375, "StartTime": 0.0299375, "Type": "pronunciation" }, { "Content": "name", "EndTime": 0.5899375, "StartTime": 0.3899375, "Type": "pronunciation" }, { "Content": "is", "EndTime": 0.7899375, "StartTime": 0.5999375, "Type": "pronunciation" }, { "Content": "mike", "EndTime": 0.9199375, "StartTime": 0.7999375, "Type": "pronunciation" } ], "Entities": [ { "Content": "mike", "Category": "PII", "Type": "NAME", "StartTime" : 0.7999375, "EndTime" : 1.0199375, "Confidence": 0.9989 } ] } ], "EndTime": 1.02, "IsPartial": false, "ResultId": "12345a67-8bc9-0de1-2f34-a5b678c90d12", "StartTime": 0.0199375 } ] } } } } ```