Amazon Transcribe performs automatic speech recognition using machine learning, specifically, a neural encoder model combined with a language model. At a high level, the core service works as follows: 1/ Amazon Transcribe receives audio input (along with configuration parameters such as language code and feature selections) via the API or Console; 2/ the service extracts acoustic features from the audio input; 3/ the acoustic features are processed by a neural encoder that generates probabilities over a set of text tokens at each time frame; 4/ a language model is applied to score and rank candidate transcriptions based on these probabilities; 5/ a beam-search decoding algorithm selects the highest-ranking transcription; 6/ additional feature-specific processing is applied as requested (described below); 7/ the final output, including the transcript, word-level timestamps, confidence scores, and any requested feature outputs, is returned to the customer.

Language identification uses a downstream model that consumes high-level representations from the acoustic encoder to produce language-level output probabilities. The model is trained on multilingual data with balanced sampling across languages to ensure all languages are represented during training. Speaker diarization analyzes the audio to differentiate and cluster speakers based on unique voice characteristics, used solely to annotate the transcript with speaker labels and discarded after processing. Speaker labels are assigned within a single audio file or session and are not linked to any speaker identity. Custom vocabularies are implemented through a contextual adapter module that biases the encoder's output probabilities toward recognizing customer-specified terms. When a customer provides a custom vocabulary list, the terms are encoded into embedding vectors, and these vectors are used to boost the probability of the corresponding terms during beam-search decoding. PII identification and redaction is applied as a post-processing step on the transcription output. The service uses pattern matching and contextual analysis to identify PII entities in the transcript text and replaces them with placeholder tags when redaction is enabled. Word-level timing is generated by an alignment model trained on top of the ASR encoder, which produces frame-level token probabilities that are used to estimate the start and end time of each word in the transcript through forced alignment. Confidence scores are generated for each word based on the model's output probabilities and are calibrated during each model update cycle to ensure consistency across model versions.

Amazon Transcribe models are trained and tested on licensed and proprietary datasets, off-the-shelf purchased datasets, and open-source datasets in the public domain. The training data covers a diverse range of acoustic conditions, speaker demographics, languages, and use cases. Training data from licensed and purchased datasets was collected from speakers who consented for their voice recordings to be used for commercial purposes. Training data from open-source datasets was released under a commercial-usage license. Amazon Transcribe may use customer content to develop and improve the service. Customers can opt out of having their content used for service improvement via AWS Organizations or other opt-out mechanisms we may provide. When customers opt out, their content is not used for service improvement. Regardless of opt-out status, customer content is never shared between customers. Amazon Transcribe is available pursuant to the AWS Customer Agreement or other relevant agreements with AWS.

We say that Amazon Transcribe exhibits a particular "behavior" when it generates the same kind of output for the same kinds of audio inputs and configuration parameters (for example, language code, sample rate, speaker diarization settings, custom vocabulary, and PII redaction settings). For a given model architecture, the control levers that we have over the behaviors are primarily a/ the training data corpus (which we curate from a variety of licensed, proprietary, and publicly available sources), b/ the language models and vocabulary that determine how the service ranks candidate transcriptions, c/ the safety filters (such as the default vocabulary filter) applied to post-process outputs, and d/ customer-facing configuration options that allow customers to tailor the service behavior to their use case. Customers can further steer the behavior of Amazon Transcribe to their specific needs through several mechanisms:

These mechanisms serve as compensating controls that customers can use to address known model limitations for their specific use case. The effectiveness of these controls should be evaluated by the customer on their own content.

Intrinsic and confounding variation differ between customer applications. This means that performance will also differ between applications, even if they support the same use case. Consider two applications A and B. Application A enables live captioning for a television news broadcast, with professional presenters speaking clearly into high-quality microphones, scripted speech with domain-specific terminology, and minimal background noise. Application B transcribes customer service calls at a contact center, with callers speaking on mobile phones in noisy environments, unscripted conversational speech with filler words and interruptions, and telephony audio at 8kHz sampling rate. Because A and B have differing kinds of inputs, they will likely have differing error rates, even assuming that each application is deployed perfectly using Amazon Transcribe. In addition, streaming transcription provides real-time results but may have different accuracy characteristics compared to batch transcription, which can process the entire audio file before returning results. Furthermore, performance differs across languages: English (particularly US English) generally delivers the highest accuracy due to the largest and most diverse training data, while lower-resource languages may exhibit higher error rates and greater sensitivity to adverse acoustic conditions. As a result, the overall utility of Amazon Transcribe will depend both on the service and on the workflows it enables. Performance results depend on a variety of factors including Amazon Transcribe itself, the customer workflow, and the evaluation dataset; we recommend that customers test Amazon Transcribe using their own content.

We use multiple datasets, automated evaluations and human inspection to evaluate the performance of Amazon Transcribe models. No single evaluation dataset suffices to completely capture performance. This is because evaluation datasets vary based on use case, intrinsic and confounding variation, the quality of ground truth available, and other factors. Our development testing involves automated testing against publicly available and proprietary datasets, benchmarking against proxies for anticipated customer use cases, evaluation across multiple languages and acoustic conditions, and iterative refinement based on results. Our development process examines Amazon Transcribe's performance using these tests, takes steps to improve the model and the suite of evaluation datasets, and then iterates. In this Service Card, we provide examples of test results to illustrate our methodology.

Automated Evaluations: Automated testing provides consistent comparisons between candidate models by applying standardized metrics across evaluation datasets. The primary metric for transcription accuracy is word error rate (WER), which measures the aggregate of insertion, deletion, and substitution errors divided by the total number of words in the reference transcript. We also measure F1 scores for evaluating the accuracy of specific output elements – such as custom vocabulary terms, named entities, digit transcription, punctuation, and capitalization – which balance the percentage of predicted items that are correct (precision) against the percentage of correct items that are included in the prediction (recall). For features such as speaker diarization, custom vocabulary recognition, punctuation, and capitalization, we use feature-specific metrics including diarization accuracy, custom vocabulary F1 scores, and punctuation and casing F1 scores. Models are evaluated on datasets totaling thousands of hours of audio data, covering multiple languages, acoustic conditions (including telephony at 8kHz and broadband at 16kHz), and speaker demographics. Evaluation datasets include internally commissioned datasets representative of customer use cases, purchased off-the-shelf datasets, and publicly available benchmarking datasets used by the scientific community. All evaluation datasets are versioned, archived, and subject to legal review.

Model Quality Monitoring: Amazon Transcribe employs automated monitoring to track service behavior over time. Canary systems periodically evaluate the service by measuring specific metrics (including WER, latency, and processing speed) against pre-determined reference datasets. These systems automatically trigger alerts when service metrics demonstrate any significant drift or anomalous behavior. Mechanisms are in place to roll back the service to a previously known stable state if needed.

Input Validation: Amazon Transcribe validates the format of the input audio (for example, encoding, sample rate, and channel configuration) before processing it. Non-adherence to prescribed format specifications will result in an error response. All data used in model development is validated and checked for quality and consistency before use in training or evaluation.

Safety is a shared responsibility between AWS and our customers. Our goal for safety is to mitigate key risks of concern to our enterprise customers, and to society more broadly. Amazon Transcribe is an ASR service that converts speech to text verbatim. It is not a generative AI service and does not generate any content that is not spoken in the input audio. This fundamental design characteristic means that the service does not produce hallucinated, fabricated, or novel content. Amazon Transcribe does not use text prompts and is not susceptible to prompt injection attacks.

In a case where a user speaks inappropriate content, the service will transcribe that content verbatim. To help customers manage such scenarios, Amazon Transcribe provides the following safety-relevant features:

Vocabulary Filtering: Customers can define a list of words (e.g., offensive, profane, or otherwise inappropriate words) to mask or remove from transcription results. The default vocabulary filter removes certain sensitive words; customers can supplement this with their own word lists. Note that the default vocabulary filter may not include words from all supported languages; customers should review and supplement the filter for their specific language and use case.

PII Redaction: Customers can automatically identify and redact personally identifiable information from transcripts, including names, addresses, credit card numbers, Social Security numbers, and other PII types. However, PII identification is not guaranteed to detect all PII instances in all contexts, and false positives (non-PII content incorrectly identified as PII) may also occur. Customers operating in regulated environments should not rely solely on automated PII redaction without implementing additional review processes appropriate to their compliance requirements.

Toxicity Detection: Amazon Transcribe Toxicity Detection (available as a separate feature) leverages both audio and text-based cues to identify and classify toxic voice content across seven categories including sexual harassment, hate speech, threats, abuse, profanity, insults, and graphic content. Toxicity Detection is not guaranteed to detect all instances of toxic language in all contexts, and false positives (non-toxic content incorrectly identified as toxic) may also occur. Customers operating in regulated environments should not rely solely on automated toxicity detection without implementing additional review processes appropriate to their compliance requirements. Please see the Service Card for Toxicity Detection for additional useful information.

Customers are responsible for end-to-end testing of their applications on datasets representative of their use cases and any additional safety mitigations, and deciding if test results meet their specific expectations of safety, fairness, and other properties, as well as overall effectiveness.

Amazon Transcribe is designed to work well for speakers across the variety of pronunciations, intonations, vocabularies, and grammatical features that speakers of each supported language may use. We consider speaker communities defined by regions – for example, speakers of different English varieties such as US, British, Australian, Indian, and South African English – and communities defined by multiple dimensions of identity, including accent or ancestry, age, and gender. To measure accuracy across these dimensions, we perform extensive evaluations covering both demographic variation and confounding factors such as acoustic conditions, pronunciation, and speaking rate. For controlled comparison across demographic groups in English, we use evaluation datasets designed to minimize confounding variation, enabling fair comparisons across dimensions such as accent/ancestry, age, and gender.

We find that Amazon Transcribe performs reasonably well across demographic attributes. As an example, on one dataset of read speech with 23 demographic groups, defined by ancestry, gender, and regional dialect (such as Liberian English, New England English), we find the median WER to be 2.4%, and statistically significant differences (p < 0.001) in accuracy between native (WER of 0.9%) and non-native English speakers (WER of 4.8%). Along the gender dimension, the median error rate for females is slightly lower (less than 1 error difference) than that of males, and the error distributions indicate these differences are statistically significant (p < 0.001).

For non-English languages, transcription accuracy varies across languages and locales. This variation is expected and reflects differences in the volume and diversity of training data available for each language, the linguistic characteristics of each language, and the representativeness of available evaluation datasets. In general, languages with larger training datasets and more diverse acoustic coverage perform better than languages with limited training data. We apply quality thresholds as launch criteria for all supported languages, and we continuously work to improve accuracy across all languages by expanding training data diversity and volume. For most non-English evaluation datasets, fine-grained demographic information (such as speaker gender, age, and accent) is not available, which limits the measurement of within-language demographic fairness for those languages. We are investing in expanding the demographic coverage of datasets for non-English languages. Customers should evaluate Amazon Transcribe on their own content for each language and should not assume that performance observed for one language will generalize to another.

Because results depend on Amazon Transcribe, the customer workflow and the evaluation dataset, we recommend that customers additionally test Amazon Transcribe on their own content.

We maximize robustness with a number of techniques, including using large training datasets that capture many kinds of variation across many speakers, and simulating a variety of acoustic conditions during training (such as different noise types, reverberation levels, and telephony codecs). Ideal audio inputs to Amazon Transcribe contain audio with high recording quality, low background noise, and low room reverberation. However, Amazon Transcribe is trained to be resilient even when inputs vary from ideal conditions and can perform well in noisy and multi-speaker settings. We measure model robustness by evaluating performance across a range of acoustic conditions, including clean speech, moderate and high background noise, far-field recording, overlapping speech, and telephony audio. Amazon Transcribe models degrade gracefully under adverse conditions rather than producing catastrophic errors. Additionally, given the architecture of the model (a neural encoder combined with a language model operating on continuous-valued audio inputs) the risk of adversarial attacks such as training data extraction or membership inference is significantly lower compared to text-based generative models. We are not aware of published work demonstrating such attacks on encoder-only ASR architectures of this type.

When Amazon Transcribe transcribes audio, it assigns a confidence score to each word in the transcript, indicating the service's confidence that the word was correctly recognized. Confidence scores are calibrated during each model update cycle to ensure consistency across model versions, which supports the stability of downstream workflows that rely on specific confidence thresholds. Customers can use these confidence scores to identify portions of the transcript that may require additional review, to implement quality thresholds appropriate for their use case, or to trigger human review for low-confidence segments. Additionally, Amazon Transcribe provides word-level timing information, indicating the start and end time of each recognized word in the audio. This "acoustic grounding" can help customers identify challenging portions of the audio where the service may have made errors, and facilitates alignment of transcripts with the source audio for verification purposes. If customers enable alternative transcriptions (available in batch mode), Amazon Transcribe returns alternative versions of the transcript that have lower confidence levels. Customers can explore alternative transcriptions to gain more insight into candidate words and phrases that were generated for each audio input.

Amazon Transcribe processes only audio input data. Audio inputs are never included in the output returned by the service. Inputs and outputs are never shared between customers. Amazon Transcribe may use customer content to develop and improve the service. Customers can opt out of having their content used for service improvement via AWS Organizations or other opt-out mechanisms we may provide. When customers opt out, their content is not used for service development or improvement. Regardless of opt-out status, inputs and outputs are never shared between customers. For more information, see Section 50.3 of the AWS Service Terms and the AWS Data Privacy FAQs. For service-specific privacy and security information, see Amazon Transcribe FAQs and Amazon Transcribe Security.

How Amazon Transcribe processes data. Amazon Transcribe processes audio input directly within the AWS infrastructure in the AWS Region where the customer is using the service. The processing flow differs by mode:

Data access. Only authorized personnel have access to content processed by Amazon Transcribe, in accordance with AWS security policies. AWS implements appropriate technical and physical controls, including encryption, designed to prevent unauthorized access to or disclosure of customer content. For more information, see AWS Data Privacy FAQs.

Amazon Transcribe uses AWS owned encryption by default to protect data at rest. Customers can optionally use customer-managed keys via AWS Key Management Service (AWS KMS) for an additional layer of encryption. This allows customers to control the encryption keys used to protect their transcription output. Amazon Transcribe employs Transport Layer Security (TLS) 1.2 to encrypt data in transit, ensuring secure communication between the service and client applications.

Amazon Transcribe supports AWS PrivateLink for both Batch and Streaming APIs. This allows customers to access Amazon Transcribe privately from their Amazon Virtual Private Cloud (VPC) without using public IPs and avoid traffic traversing the public internet, enhancing network security. Amazon Transcribe integrates with AWS Identity and Access Management (IAM) to enable fine-grained access control of identity-based policies, resource-based policies and service-specific policy condition keys. Amazon Transcribe is integrated with AWS CloudTrail (CloudTrail), which logs all API calls made to the service. Amazon Transcribe integrates with Amazon CloudWatch (CloudWatch) to provide real-time monitoring of transcription jobs.

Amazon Transcribe has undergone application security review. The model is restricted, and its parameters cannot be inferred through external testing. Access controls, encryption, and monitoring mechanisms are implemented to protect against unauthorized access to the service. Customer data is isolated; inputs and outputs are never shared between customers. Amazon Transcribe has achieved ISO/IEC 42001:2023 Artificial Intelligence Management System accredited certification. For a complete list of compliance programs for which Amazon Transcribe is in scope, see AWS Services in Scope by Compliance Program.

Customers operating in regulated industries should review the specific compliance programs relevant to their industry and jurisdiction to confirm that Amazon Transcribe is in scope. Customers who are required to document their use of AI services for audit or regulatory purposes can reference this Service Card, the supporting documentation listed in the Further Information section, and may request additional detail through their AWS account team.

Amazon Transcribe provides information to customers in the following locations: this Service Card, AWS documentation, AWS educational channels (for example, blogs, developer classes), and the AWS Console. We accept feedback through customer support mechanisms such as account managers. Where appropriate for their use case, customers who incorporate Amazon Transcribe in their workflow should consider disclosing their use of ML and ASR technology to end users and other individuals impacted by the application, and customers should give their end users the ability to provide feedback to improve workflows. In their documentation, customers can also reference this Service Card. Amazon Transcribe is a transcription service, not a generative AI system. Its outputs represent the service's best-guess transcription of the audio input provided. Where customers surface transcription outputs to end users (for example, as closed captions or meeting notes), we recommend clearly indicating that the text was generated by automated speech recognition, so that users can assess the output with appropriate expectations.

We have rigorous methodologies to build our AWS AI services responsibly, including a working backwards product development process that incorporates Responsible AI at the design phase, design consultations, and implementation assessments by dedicated Responsible AI science and data experts, routine testing, reviews with customers, best practice development, dissemination, and training.

As with all AWS AI services, numerous stakeholders are involved in Amazon Transcribe's model lifecycle, including security, engineering, data science, product, data, responsible AI, and legal teams. Each model update undergoes quality readiness review before deployment, which includes evaluation of accuracy, fairness, robustness, and other responsible AI dimensions. For details on model lifecycle management, change control, ongoing monitoring, and update communication, see Model Updates under Deployment and Performance Optimization Best Practices below.

Customers who are required to validate AI or ML models as part of their governance processes may find the following information relevant. Amazon Transcribe performs speech-to-text conversion and does not generate novel content. The model assumes that input audio contains naturally occurring human speech in a supported language, recorded at a supported sample rate and format. Customers can use custom vocabularies, PII redaction, vocabulary filtering, confidence score thresholds, and human oversight as compensating controls to address model limitations for their specific use case. Word-level confidence scores are recalibrated during each model update cycle to ensure that scores accurately reflect transcription correctness and that confidence score distributions remain consistent, supporting the stability of downstream workflows that depend on specific thresholds. Amazon Transcribe is monitored by automated canary systems that periodically evaluate accuracy, latency, and other metrics against reference datasets, with alerts triggered on significant drift and rollback mechanisms in place. (See Model Updates and Ongoing Monitoring for details.) Customers should implement their own monitoring appropriate to their use case, including periodic retesting on representative data, tracking of output quality metrics, and review of edge cases. (See Performance Drift under Deployment and Performance Optimization Best Practices.)