# Adding custom synonyms to an index To add custom synonyms to an index, you specify them in a thesaurus file. You can include business-specific or specialized terms in Amazon Kendra using synonyms. Generic English synonyms, such as `leader, head`, are built into Amazon Kendra and should not be included in a thesaurus file, including generic synonyms that use hyphens. Amazon Kendra supports synonyms for all response types, which include `DOCUMENT` response types and `QUESTION_ANSWER` or `ANSWER` response types. Amazon Kendra currently does not support adding synonyms flagged as stopwords. This is to be included in a future release. Amazon Kendra makes correlations between synonyms. For example, using the synonym pair `Dynamo, Amazon DynamoDB`, Amazon Kendra correlates Dynamo with Amazon DynamoDB. The query "What is dynamo?" then returns a document such as "What is Amazon DynamoDB?". With synonyms, Amazon Kendra can more easily pick up the correlation. The thesaurus file is a text file stored in an Amazon S3 bucket. See [Adding a thesaurus to an index](index-synonyms-adding-thesaurus-file.md). The thesaurus file uses the [Solr synonym format](https://lucene.apache.org/solr/guide/6_6/filter-descriptions.html#FilterDescriptions-SynonymGraphFilter). Amazon Kendra has a limit on the number of thesauri per index. See [Quotas](https://docs.aws.amazon.com/kendra/latest/dg/quotas.html). Synonyms can be useful in the following scenarios: + Specialized terms that are not traditional English language synonyms such as `NLP, Natural Language Processing`. + Proper nouns with complex semantic associations. These are nouns that the general public are unlikely to understand, for example, in machine learning, `cost, loss, model performance`. + Different forms of product names, for example, `Elastic Compute Cloud, EC2`. + Domain-specific or business-specific terms, such as product names. For example, `Route53, DNS`. Do not use synonyms in the following scenarios: + Generic English language synonyms such as `leader, head`. These synonyms are not domain-specific,and using synonyms in these scenarios might have unintended effects. + Typographical errors such as `teh => the`. + Morphological variants like the plurals and possessives of nouns, the comparative and superlative form of adjectives, and the past tense, past participle and progressive form of verbs. One example of comparative and superlative adjectives is `good, better, best`. + Unigram (single word) stop words such as `WHO`. Unigram stop words are not allowed in the thesaurus and are excluded from search. For example, `WHO => World Health Organization` is rejected. You can use `W.H.O.` however as a synonym term, and you can use stop words as part of a multi-word synonym. For example, `of` is not allowed but `United States of America` is accepted. Custom synonyms make it easy to improve Amazon Kendra's understanding of your business-specific terminology by expanding your queries to cover your business-specific synonyms. Although synonyms can improve search accuracy, it is important to understand how synonyms affect latency so you can optimize for this. A general rule for synonyms is: the more terms in your query that are matched and expanded with synonyms, the greater potential impact on latency. Other factors that affect latency include the average size of documents indexed, the size of your index, any filtering on search results, and the overall load on your Amazon Kendra index. Queries that don’t match any synonyms are not affected. A general guideline for how synonyms affect latency: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/kendra/latest/dg/index-synonyms.html) \$1*Performance varies based on your specific use of synonyms and configurations on your index. It’s best to test search performance to obtain more accurate benchmarks for your specific use case.* If your thesaurus is large, has a high term expansion ratio, and your latency increase is not within acceptable boundaries, you can try one or both of the following: + Trim your thesaurus to reduce the expansion ratio (number of synonyms per term). + Trim the overall coverage of terms (number of lines in your thesaurus). Alternatively, you can increase the provisioning capacity (virtual storage units) to offset the latency increase. **Topics** + [Creating a thesaurus file](index-synonyms-creating-thesaurus-file.md) + [Adding a thesaurus to an index](index-synonyms-adding-thesaurus-file.md) + [Updating a thesaurus](index-synonyms-update.md) + [Deleting a thesaurus](index-synonyms-delete.md) + [Highlights in search results](index-synonyms-enabling-synonyms-in-results.md) # Creating a thesaurus file An Amazon Kendra thesaurus file is a UTF-8-encoded file containing a list of synonyms in the Solr synonym list format. The thesaurus file must be less than 5 MB. There are two ways to specify synonym mappings: + *Bidirectional synonyms* are specified as a comma-separated list of terms. If your user queries any of the terms, then all the terms in the list are used to search documents, which includes the original queried term. + *Unidirectional synonyms* are specified as terms separated by the symbol "=>" between them to map terms to their synonyms. If your user queries a term on the left of the symbol "=>", then it is mapped to a term on the right to search for documents using the synonym. It is not mapped vice versa, making this unidirectional. The synonyms themselves are case sensitive, but the terms they map to are case insensitive. For example, `ML => Machine Learning` means if your user queries "ML" or "ml" or uses some other case, it will map to "Machine Learning". If you were to map this vice versa, `Machine Learning => ML`, then "Machine Learning" or "machine learning" or some other case would map to "ML". A synonym doesn't search for an exact match on special characters. For example, if you search for "dead-letter-queue", Amazon Kendra can return documents that match "dead letter queue" (no hyphen). If your documents contain hyphens, such as "dead-letter-queue", Amazon Kendra processes the documents during search to remove hyphens. For generic English synonym terms that are built into Amazon Kendra and should not be included in a thesaurus file, Amazon Kendra can search both the hyphen version of the term and the non-hyphen version of the term. For example, if you search "third-party" and "third party", Amazon Kendra returns documents that match either version of those terms. For synonyms that contain stopwords or commonly used words, Amazon Kendra returns documents that match terms including stopwords. For example, you can create a synonym rule to map "on boarding" and "onboarding". You cannot use stopwords alone for synonyms. For example, if you search for "on", Amazon Kendra cannot return all documents that contain "on". Some synonym rules are ignored. For example, `a => b` is a rule, but `a => a` is ignored and doesn't count as a rule. The term count is the number of unique terms in the theaurus file. The below example file includes terms `AWS CodeStar`, `ML`, `Machine Learning`, `autoscaling group`, `ASG`, and more. There is a maximum amount of synonym rules per thesaurus and a maximum amount of synonyms per term. For more information, see [Quotas for Amazon Kendra](quotas.md). The following example shows a thesaurus file with synonym rules. Each line contains a single synonym rule. Blank lines and comments are ignored. ``` # Lines starting with pound are comments and blank lines are ignored. # Synonym relationships can be defined as unidirectional or bidirectional relationships. # Unidirection relationships are represented by any term sequence # on the left hand side (LHS) of "=>" followed by synonyms on the right hand side (RHS) CodeStar => AWS CodeStar # This will map CodeStar to AWS CodeStar, but not vice-versa # To map terms vice versa ML => Machine Learning Machine Learning => ML # Multiple synonym relationships may be defined in one line as well by comma seperation. autoscaling group, ASG => Auto Scaling group, autoscaling # The above is equivalent to: # autoscaling group => Auto Scaling group, autoscaling # ASG => Auto Scaling group, autoscaling # Bi-directional synonyms are comma separated terms with no "=>" DNS, Route53, Route 53 # DNS, Route53, and Route 53 map to one another and are interchangeable at match time # The above is equivalent to: # DNS => Route53, Route 53 # Route53 => DNS, Route 53 # Route 53 => DNS, Route53 # Overlapping LHS terms will be merged Beta => Alpha Beta => Gamma Beta, Delta # is equivalent to: # Beta => Alpha, Gamma, Delta # Delta => Beta # Each line contains a single synonym rule. # Synonym rule count is the total number of lines defining synonym relationships # Term count is the total number of unique terms for all rules. # Comments and blanks lines do not count. ``` # Adding a thesaurus to an index The following procedures show how to add a thesaurus file containing synonyms to an index. It can take up to 30 minutes to see the effects of your updated thesaurus file. For more information about the thesaurus file, see [Creating a thesaurus file](index-synonyms-creating-thesaurus-file.md). ------ #### [ Console ] **To add a thesaurus** 1. In the left navigation pane, under the index where you want to add a list of synonyms, your thesaurus, choose **Synonyms**. 1. On the **Synonym** page, choose **Add Thesaurus**. 1. In **Define thesaurus**, give your thesaurus a name and an optional description. 1. In **Thesaurus settings**, provide the Amazon S3 path to your thesaurus file. The file must be smaller than 5 MB. 1. For **IAM Role**, select a role or select **Create a new role** and specify a role name to create a new role. Amazon Kendra uses this role to access the Amazon S3 resource on your behalf. The IAM role has the prefix "AmazonKendra-". 1. Choose **Save** to save the configuration and add the thesaurus. Once the thesaurus is ingested, it is active and synonyms are highlighted in results. It can take up to 30 minutes to see the effects of your thesaurus file. ------ #### [ CLI ] To add a thesarus to an index with the AWS CLI, call `create-thesaurus`: ``` aws kendra create-thesaurus \ --index-id index-id \ --name "thesaurus-name" \ --description "thesaurus-description" \ --source-s3-path "Bucket=bucket-name,Key=thesaurus/synonyms.txt" \ --role-arn role-arn ``` Call `list-thesauri` to see a list of thesauruses: ``` aws kendra list-thesauri \ --index-id index-id ``` To view details for a thesaurus, call `describe-thesaurus`: ``` aws kendra describe-thesaurus \ --index-id index-id \ --index-id thesaurus-id ``` It can take up to 30 minutes to see the effects of your thesaurus file. ------ #### [ Python ] ``` import boto3 from botocore.exceptions import ClientError import pprint import time kendra = boto3.client("kendra") print("Create a thesaurus") thesaurus_name = "thesaurus-name" thesaurus_description = "thesaurus-description" thesaurus_role_arn = "role-arn" index_id = "index-id" s3_bucket_name = "bucket-name" s3_key = "thesaurus-file" source_s3_path= { 'Bucket': s3_bucket_name, 'Key': s3_key } try: thesaurus_response = kendra.create_thesaurus( Description = thesaurus_description, Name = thesaurus_name, RoleArn = thesaurus_role_arn, IndexId = index_id, SourceS3Path = source_s3_path ) pprint.pprint(thesaurus_response) thesaurus_id = thesaurus_response["Id"] print("Wait for Kendra to create the thesaurus.") while True: # Get thesaurus description thesaurus_description = kendra.describe_thesaurus( Id = thesaurus_id, IndexId = index_id ) # If status is not CREATING quit status = thesaurus_description["Status"] print("Creating thesaurus. Status: " + status) if status != "CREATING": break time.sleep(60) except ClientError as e: print("%s" % e) print("Program ends.") ``` ------ #### [ Java ] ``` package com.amazonaws.kendra; import software.amazon.awssdk.services.kendra.KendraClient; import software.amazon.awssdk.services.kendra.model.CreateThesaurusRequest; import software.amazon.awssdk.services.kendra.model.CreateThesaurusResponse; import software.amazon.awssdk.services.kendra.model.DescribeThesaurusRequest; import software.amazon.awssdk.services.kendra.model.DescribeThesaurusResponse; import software.amazon.awssdk.services.kendra.model.S3Path; import software.amazon.awssdk.services.kendra.model.ThesaurusStatus; public class CreateThesaurusExample { public static void main(String[] args) throws InterruptedException { KendraClient kendra = KendraClient.builder().build(); String thesaurusName = "thesaurus-name"; String thesaurusDescription = "thesaurus-description"; String thesaurusRoleArn = "role-arn"; String s3BucketName = "bucket-name"; String s3Key = "thesaurus-file"; String indexId = "index-id"; System.out.println(String.format("Creating a thesaurus named %s", thesaurusName)); CreateThesaurusRequest createThesaurusRequest = CreateThesaurusRequest .builder() .name(thesaurusName) .indexId(indexId) .description(thesaurusDescription) .roleArn(thesaurusRoleArn) .sourceS3Path(S3Path.builder() .bucket(s3BucketName) .key(s3Key) .build()) .build(); CreateThesaurusResponse createThesaurusResponse = kendra.createThesaurus(createThesaurusRequest); System.out.println(String.format("Thesaurus response %s", createThesaurusResponse)); String thesaurusId = createThesaurusResponse.id(); System.out.println(String.format("Waiting until the thesaurus with ID %s is created.", thesaurusId)); while (true) { DescribeThesaurusRequest describeThesaurusRequest = DescribeThesaurusRequest.builder() .id(thesaurusId) .indexId(indexId) .build(); DescribeThesaurusResponse describeThesaurusResponse = kendra.describeThesaurus(describeThesaurusRequest); ThesaurusStatus status = describeThesaurusResponse.status(); if (status != ThesaurusStatus.CREATING) { break; } TimeUnit.SECONDS.sleep(60); } System.out.println("Thesaurus creation is complete."); } } ``` ------ # Updating a thesaurus You can change the configuration of a thesaurus after it is created. You can change details like thesaurus name and IAM information. You can also change the location of the thesaurus file Amazon S3 path. If you change the path to the thesaurus file, Amazon Kendra replaces the existing thesaurus with the thesaurus specified in the updated path. It can take up to 30 minutes to see the effects of your updated thesaurus file. **Note** If there are validation or syntax errors in the thesaurus file, the previously uploaded thesaurus file is retained. The following procedures show how to modify thesaurus details. ------ #### [ Console ] **To modify thesaurus details** 1. In the left navigation pane, under the index you want to modify, choose **Synonyms**. 1. On the **Synonym** page, select the thesaurus you want to modify and then choose **Edit**. 1. On the **Update thesaurus** page, update the thesaurus details. 1. (Optional) Choose **Change the thesaurus file path** and then specify an Amazon S3 path to the new thesaurus file. Your existing thesaurus file is replaced by the file you specify. If you do not change the path, Amazon Kendra reloads the thesaurus from the existing path. If you select **Keep the current thesaurus file**, Amazon Kendra does not reload the thesaurus file. 1. Choose **Save** to save the configuration. You can also reload the thesaurus from the existing thesaurus path. **To reload a thesaurus from an existing path** 1. In the left navigation pane, under the index you want to modify, choose **Synonyms**. 1. On the **Synonym** page, select the thesaurus you want to reload and then choose **Refresh**. 1. On the **Reload thesaurus file** page, confirm you want to refresh the thesaurus file. ------ #### [ CLI ] To update a thesaurus, call `update-thesaurus`: ``` aws kendra update-thesaurus \ --index-id index-id \ --name "thesaurus-name" \ --description "thesaurus-description" \ --source-s3-path "Bucket=bucket-name,Key=thesaurus/synonyms.txt" \ --role-arn role-arn ``` ------ #### [ Python ] ``` import boto3 from botocore.exceptions import ClientError import pprint import time kendra = boto3.client("kendra") print("Update a thesaurus") thesaurus_name = "thesaurus-name" thesaurus_description = "thesaurus-description" thesaurus_role_arn = "role-arn" thesaurus_id = "thesaurus-id" index_id = "index-id" s3_bucket_name = "bucket-name" s3_key = "thesaurus-file" source_s3_path= { 'Bucket': s3_bucket_name, 'Key': s3_key } try: kendra.update_thesaurus( Id = thesaurus_id, IndexId = index_id, Description = thesaurus_description, Name = thesaurus_name, RoleArn = thesaurus_role_arn, SourceS3Path = source_s3_path ) print("Wait for Kendra to update the thesaurus.") while True: # Get thesaurus description thesaurus_description = kendra.describe_thesaurus( Id = thesaurus_id, IndexId = index_id ) # If status is not UPDATING quit status = thesaurus_description["Status"] print("Updating thesaurus. Status: " + status) if status != "UPDATING": break time.sleep(60) except ClientError as e: print("%s" % e) print("Program ends.") ``` ------ #### [ Java ] ``` package com.amazonaws.kendra; import software.amazon.awssdk.services.kendra.KendraClient; import software.amazon.awssdk.services.kendra.model.UpdateThesaurusRequest; import software.amazon.awssdk.services.kendra.model.DescribeThesaurusRequest; import software.amazon.awssdk.services.kendra.model.DescribeThesaurusResponse; import software.amazon.awssdk.services.kendra.model.S3Path; import software.amazon.awssdk.services.kendra.model.ThesaurusStatus; public class UpdateThesaurusExample { public static void main(String[] args) throws InterruptedException { KendraClient kendra = KendraClient.builder().build(); String thesaurusName = "thesaurus-name"; String thesaurusDescription = "thesaurus-description"; String thesaurusRoleArn = "role-arn"; String s3BucketName = "bucket-name"; String s3Key = "thesaurus-file"; String thesaurusId = "thesaurus-id"; String indexId = "index-id"; UpdateThesaurusRequest updateThesaurusRequest = UpdateThesaurusRequest .builder() .id(thesaurusId) .indexId(indexId) .name(thesaurusName) .description(thesaurusDescription) .roleArn(thesaurusRoleArn) .sourceS3Path(S3Path.builder() .bucket(s3BucketName) .key(s3Key) .build()) .build(); kendra.updateThesaurus(updateThesaurusRequest); System.out.println(String.format("Waiting until the thesaurus with ID %s is updated.", thesaurusId)); // a new source s3 path requires re-consumption by Kendra // and so can take as long as a Create Thesaurus operation while (true) { DescribeThesaurusRequest describeThesaurusRequest = DescribeThesaurusRequest.builder() .id(thesaurusId) .indexId(indexId) .build(); DescribeThesaurusResponse describeThesaurusResponse = kendra.describeThesaurus(describeThesaurusRequest); ThesaurusStatus status = describeThesaurusResponse.status(); if (status != ThesaurusStatus.UPDATING) { break; } TimeUnit.SECONDS.sleep(60); } System.out.println("Thesaurus update is complete."); } } ``` ------ # Deleting a thesaurus The following procedures show how to delete a thesaurus. ------ #### [ Console ] 1. In the left navigation pane, under the index you want to modify, choose **Synonyms**. 1. On the **Synonym** page, select the thesaurus you want to delete. 1. On the **Thesaurus detail** page, choose **Delete** and then confirm to delete. ------ #### [ CLI ] To delete a thesarus to an index with the AWS CLI, call `delete-thesaurus`: ``` aws kendra delete-thesaurus \ --index-id index-id \ --id thesaurus-id ``` ------ #### [ Python ] ``` import boto3 from botocore.exceptions import ClientError kendra = boto3.client("kendra") print("Delete a thesaurus") thesaurus_id = "thesaurus-id" index_id = "index-id" try: kendra.delete_thesaurus( Id = thesaurus_id, IndexId = index_id ) except ClientError as e: print("%s" % e) print("Program ends.") ``` ------ #### [ Java ] ``` package com.amazonaws.kendra; import software.amazon.awssdk.services.kendra.KendraClient; import software.amazon.awssdk.services.kendra.model.DeleteThesaurusRequest; public class DeleteThesaurusExample { public static void main(String[] args) throws InterruptedException { KendraClient kendra = KendraClient.builder().build(); String thesaurusId = "thesaurus-id"; String indexId = "index-id"; DeleteThesaurusRequest updateThesaurusRequest = DeleteThesaurusRequest .builder() .id(thesaurusId) .indexId(indexId) .build(); kendra.deleteThesaurus(updateThesaurusRequest); } } ``` ------ # Highlights in search results Synonym highlighting is on by default. Highlight information is included in Amazon Kendra SDK and CLI query results. If you interact with Amazon Kendra using the SDK or CLI, you determine how to display results. Synonym highlights will have the highlight type `THESAURUS_SYNONYM`. For more information about highlights, see the [Highlight](https://docs.aws.amazon.com/kendra/latest/APIReference/API_Highlight.html) object.