# Enabling speaker partitioning To enable speaker partitioning in Amazon Transcribe Medical, use *speaker diarization*. This enables you to see what the patient said and what the clinician said in the transcription output. When you enable speaker diarization, Amazon Transcribe Medical labels each speaker *utterance* with a unique identifier for each speaker. An *utterance* is a unit of speech that is typically separated from other utterances by silence. In batch transcription, an utterance from the clinician could receive a label of `spk_0` and an utterance the patient could receive a label of `spk_1`. If an utterance from one speaker overlaps with an utterance from another speaker, Amazon Transcribe Medical orders them in the transcription by their start times. Utterances that overlap in the input audio don't overlap in the transcription output. You can enable speaker diarization when you transcribe an audio file using batch transcription job, or in a real-time stream. **Topics** + [ # Enabling speaker partitioning in batch transcriptions ](conversation-diarization-batch-med.md) + [ # Enabling speaker partitioning in real-time streams ](conversation-diarization-streaming-med.md) # Enabling speaker partitioning in batch transcriptions You can enable speaker partitioning in a batch transcription job using either the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html) API or the AWS Management Console. This enables you to partition the text per speaker in a clinician-patient conversation and determine who said what in the transcription output. ## AWS Management Console To use the AWS Management Console to enable speaker diarization in your transcription job, you enable audio identification and then speaker partitioning. 1. Sign in to the [AWS Management Console](https://console.aws.amazon.com/transcribe/). 1. In the navigation pane, under Amazon Transcribe Medical, choose **Transcription jobs**. 1. Choose **Create job**. 1. On the **Specify job details** page, provide information about your transcription job. 1. Choose **Next**. 1. Enable **Audio identification**. 1. For **Audio identification type**, choose **Speaker partitioning**. 1. For **Maximum number of speakers**, enter the maximum number of speakers that you think are speaking in your audio file. 1. Choose **Create**. ## API **To enable speaker partitioning using a batch transcription job (API)** + For the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartMedicalTranscriptionJob.html) API, specify the following. 1. For `MedicalTranscriptionJobName`, specify a name that is unique in your AWS account. 1. For `LanguageCode`, specify the language code that corresponds to the language spoken in the audio file. 1. For the `MediaFileUri` parameter of the `Media` object, specify the name of the audio file that you want to transcribe. 1. For `Specialty`, specify the medical specialty of the clinician speaking in the audio file. 1. For `Type`, specify `CONVERSATION`. 1. For `OutputBucketName`, specify the Amazon S3 bucket to store the transcription results. 1. For the `Settings` object, specify the following. 1. `ShowSpeakerLabels` – `true`. 1. `MaxSpeakerLabels` – An integer between 2 and 10 to indicate the number of speakers that you think are speaking in your audio. The following request uses the AWS SDK for Python (Boto3) to start a batch transcription job of a primary care clinician patient dialogue with speaker partitioning enabled. ``` from __future__ import print_function import time import boto3 transcribe = boto3.client('transcribe', 'us-west-2') job_name = "my-first-transcription-job" job_uri = "s3://amzn-s3-demo-bucket/my-input-files/my-media-file.flac" transcribe.start_medical_transcription_job( MedicalTranscriptionJobName = job_name, Media={ 'MediaFileUri': job_uri }, OutputBucketName = 'amzn-s3-demo-bucket', OutputKey = 'my-output-files/', LanguageCode = 'en-US', Specialty = 'PRIMARYCARE', Type = 'CONVERSATION', OutputBucketName = 'amzn-s3-demo-bucket', Settings = {'ShowSpeakerLabels': True, 'MaxSpeakerLabels': 2 } ) while True: status = transcribe.get_medical_transcription_job(MedicalTranscriptionJobName = job_name) if status['MedicalTranscriptionJob']['TranscriptionJobStatus'] in ['COMPLETED', 'FAILED']: break print("Not ready yet...") time.sleep(5) print(status) ``` The following example code shows the transcription results of a transcription job with speaker partitioning enabled. ``` { "jobName": "job ID", "accountId": "111122223333", "results": { "transcripts": [ { "transcript": "Professional answer." } ], "speaker_labels": { "speakers": 1, "segments": [ { "start_time": "0.000000", "speaker_label": "spk_0", "end_time": "1.430", "items": [ { "start_time": "0.100", "speaker_label": "spk_0", "end_time": "0.690" }, { "start_time": "0.690", "speaker_label": "spk_0", "end_time": "1.210" } ] } ] }, "items": [ { "start_time": "0.100", "end_time": "0.690", "alternatives": [ { "confidence": "0.8162", "content": "Professional" } ], "type": "pronunciation" }, { "start_time": "0.690", "end_time": "1.210", "alternatives": [ { "confidence": "0.9939", "content": "answer" } ], "type": "pronunciation" }, { "alternatives": [ { "content": "." } ], "type": "punctuation" } ] }, "status": "COMPLETED" } ``` ## AWS CLI **To transcribe an audio file of a conversation between a clinician practicing primary care and a patient (AWS CLI)** + Run the following code. ``` aws transcribe start-transcription-job \ --region us-west-2 \ --cli-input-json file://example-start-command.json ``` The following code shows the contents of `example-start-command.json`. ``` { "MedicalTranscriptionJobName": "my-first-med-transcription-job", "Media": { "MediaFileUri": "s3://amzn-s3-demo-bucket/my-input-files/my-audio-file.flac" }, "OutputBucketName": "amzn-s3-demo-bucket", "OutputKey": "my-output-files/", "LanguageCode": "en-US", "Specialty": "PRIMARYCARE", "Type": "CONVERSATION", "Settings":{ "ShowSpeakerLabels": true, "MaxSpeakerLabels": 2 } } ``` # Enabling speaker partitioning in real-time streams Partitioning speakers in real-time streams To partition speakers and label their speech in a real-time stream, use the AWS Management Console or a streaming request. Speaker partitioning works best for between two and five speakers in a stream. Although Amazon Transcribe Medical can partition more than five speakers in a stream, the accuracy of the partitions decrease if you exceed that number. To start an HTTP/2 request, use the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html) API. To start a WebSocket request, use a pre-signed URI. The URI contains the information required to set up bi-directional communication between your application and Amazon Transcribe Medical. ## Enabling speaker partitioning in audio that is spoken into your microphone (AWS Management Console) You can use the AWS Management Console to start a real-time stream of a clinician-patient conversation, or a dictation that is spoken into your microphone in real-time. 1. Sign in to the [AWS Management Console](https://console.aws.amazon.com/transcribe/). 1. In the navigation pane, for Amazon Transcribe Medical choose **Real-time transcription**. 1. For **Audio input type**, choose the type of medical speech that you want to transcribe. 1. For **Additional settings**, choose **Speaker partitioning**. 1. Choose **Start streaming** to start transcribing your real-time audio. 1. Speak into the microphone. ## Enabling speaker partitioning in an HTTP/2 stream To enable speaker partitioning in an HTTP/2 stream of a medical conversation, use the [https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_streaming_StartMedicalStreamTranscription.html) API and specify the following: + For `LanguageCode`, specify the language code that corresponds to the language in the stream. The valid value is `en-US`. + For `MediaSampleHertz`, specify the sample rate of the audio. + For `Specialty`, specify the medical specialty of the provider. + `ShowSpeakerLabel` – `true` For more information on setting up an HTTP/2 stream to transcribe a medical conversation, see [Setting up an HTTP/2 stream](streaming-setting-up.md#streaming-http2). ## Enabling speaker partitioning in a WebSocket request To partition speakers in WebSocket streams with the API, use the following format to create a pre-signed URI to start a WebSocket request and set `show-speaker-label` to `true`. ``` GET wss://transcribestreaming.us-west-2.amazonaws.com:8443/medical-stream-transcription-websocket ?language-code=languageCode &X-Amz-Algorithm=AWS4-HMAC-SHA256 &X-Amz-Credential=AKIAIOSFODNN7EXAMPLE%2F20220208%2Fus-west-2%2Ftranscribe%2Faws4_request &X-Amz-Date=20220208T235959Z &X-Amz-Expires=300 &X-Amz-Security-Token=security-token &X-Amz-Signature=Signature Version 4 signature &X-Amz-SignedHeaders=host &media-encoding=flac &sample-rate=16000 &session-id=sessionId &specialty=medicalSpecialty &type=CONVERSATION &vocabulary-name=vocabularyName &show-speaker-label=boolean ``` The following code shows the truncated example response of a streaming request. ``` { "Transcript": { "Results": [ { "Alternatives": [ { "Items": [ { "Confidence": 0.97, "Content": "From", "EndTime": 18.98, "Speaker": "0", "StartTime": 18.74, "Type": "pronunciation", "VocabularyFilterMatch": false }, { "Confidence": 1, "Content": "the", "EndTime": 19.31, "Speaker": "0", "StartTime": 19, "Type": "pronunciation", "VocabularyFilterMatch": false }, { "Confidence": 1, "Content": "last", "EndTime": 19.86, "Speaker": "0", "StartTime": 19.32, "Type": "pronunciation", "VocabularyFilterMatch": false }, ... { "Confidence": 1, "Content": "chronic", "EndTime": 22.55, "Speaker": "0", "StartTime": 21.97, "Type": "pronunciation", "VocabularyFilterMatch": false }, ... "Confidence": 1, "Content": "fatigue", "EndTime": 24.42, "Speaker": "0", "StartTime": 23.95, "Type": "pronunciation", "VocabularyFilterMatch": false }, { "EndTime": 25.22, "StartTime": 25.22, "Type": "speaker-change", "VocabularyFilterMatch": false }, { "Confidence": 0.99, "Content": "True", "EndTime": 25.63, "Speaker": "1", "StartTime": 25.22, "Type": "pronunciation", "VocabularyFilterMatch": false }, { "Content": ".", "EndTime": 25.63, "StartTime": 25.63, "Type": "punctuation", "VocabularyFilterMatch": false } ], "Transcript": "From the last note she still has mild sleep deprivation and chronic fatigue True." } ], "EndTime": 25.63, "IsPartial": false, "ResultId": "XXXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXX", "StartTime": 18.74 } ] } } ``` Amazon Transcribe Medical breaks your incoming audio stream based on natural speech segments, such as a change in speaker or a pause in the audio. The transcription is returned progressively to your application, with each response containing more transcribed speech until the entire segment is transcribed. The preceding code is a truncated example of a fully-transcribed speech segment. Speaker labels only appear for entirely transcribed segments. The following list shows the organization of the objects and parameters in a streaming transcription output. **`Transcript`** Each speech segment has its own `Transcript` object. **`Results`** Each `Transcript` object has its own `Results` object. This object contains the `isPartial` field. When its value is `false`, the results returned are for an entire speech segment. **`Alternatives`** Each `Results` object has an `Alternatives` object. **`Items`** Each `Alternatives` object has its own `Items` object that contains information about each word and punctuation mark in the transcription output. When you enable speaker partitioning, each word has a `Speaker` label for fully-transcribed speech segments. Amazon Transcribe Medical uses this label to assign a unique integer to each speaker in the stream. The `Type` parameter having a value of `speaker-change` indicates that one person has stopped speaking and that another person is about to begin. **`Transcript`** Each Items object contains a transcribed speech segment as the value of the `Transcript` field. For more information about WebSocket requests, see [Setting up a WebSocket stream](streaming-setting-up.md#streaming-websocket).