Automatic semantic enrichment for Amazon OpenSearch Service

Amazon OpenSearch Service uses word-to-word matching (lexical search) to find results, similar to other traditional search engines. This approach works well for specific queries like product codes or model numbers, but struggles with abstract searches where understanding user intent becomes crucial. For example, when you search for "shoes for the beach," lexical search matches individual words "shoes," "beach," "for," and "the" in catalog items, potentially missing relevant products like "water-resistant sandals" or "surf footwear" that don't contain the exact search terms.

Automatic Semantic Enrichment solves this limitation by considering both keyword matches and the contextual meaning behind searches. This feature understands search intent and improves search relevance by up to 20%. Enable this feature for text fields in your index to enhance search results.

How it works

The enrichment process analyzes designated text fields and generates semantic embeddings that capture meaning and context. These embeddings help the search engine understand relationships between concepts, synonyms, and related terms even when they don't appear in your search query. For example, if a user searches for "how to treat a headache", a semantic search system might return the following results:

The system understands the underlying intent even when these exact phrases aren't in the original query.

Automatic semantic enrichment offers the following benefits:

Requirements and considerations

Before implementing automatic semantic enrichment, consider the following requirements and limitations:

Pricing

With Automatic Semantic Enrichment, you pay only for the resources your workload consumes. The compute capacity is measured in OpenSearch Compute Units (OCUs). Check the pricing details for your specific Region and pricing illustration on the https://aws.amazon.com/opensearch-service/pricing/OpenSearch Service pricing page.

Index set up example

For a practical example, refer to the blog post https://aws.amazon.com/blogs/big-data/boosting-search-relevance-automatic-semantic-enrichment-in-amazon-opensearch-serverless/ on index setup for product catalog search using automatic semantic enrichment. Although the blog focuses on OpenSearch Serverless, the approach applies to managed clusters as well. Use this example as a starting point, then test with your own workload to validate search relevance improvements.

Configuring permissions for automatic semantic enrichment

Before creating an index with automatic semantic enrichment, you need to configure the required permissions. This section explains the permissions needed for different index operations and how to set them up for both AWS Identity and Access Management (IAM) and fine-grained access control scenarios.

IAM permissions

The following IAM permissions are required for automatic semantic enrichment operations. These permissions vary depending on the specific index operation you want to perform.

CreateIndex API permissions

To create an index with automatic semantic enrichment, you need the following IAM permissions:

UpdateIndex API permissions

To update an existing index with automatic semantic enrichment, you need the following IAM permissions:

GetIndex API permissions

To retrieve information about an index with automatic semantic enrichment, you need the following IAM permissions:

DeleteIndex API permissions

To delete an index with automatic semantic enrichment, you need the following IAM permissions:

Sample IAM policy

The following sample identity-based access policy provides the permissions necessary for a user to manage indexes with automatic semantic enrichment:

{
    "Version": "2012-10-17", 		 	 	 
    "Statement": [
        {
            "Sid": "AllowSemanticEnrichmentIndexOperations",
            "Effect": "Allow",
            "Action": [
                "es:CreateIndex",
                "es:UpdateIndex",
                "es:GetIndex",
                "es:DeleteIndex",
                "es:ESHttpHead",
                "es:ESHttpGet",
                "es:ESHttpPut",
                "es:ESHttpPost",
                "es:ESHttpDelete"
            ],
            "Resource": "arn:aws:es:aws-region:111122223333:domain/domain-name/*"
        }
    ]
}

Replace aws-region, 111122223333, and domain-name with your specific values. You can further restrict access by specifying particular index patterns in the resource ARN.

Fine-grained access control permissions

If your Amazon OpenSearch Service domain has fine-grained access control enabled, you need additional permissions beyond the IAM permissions. The following permissions are required for each index operation.

CreateIndex API permissions

When fine-grained access control is enabled, the following additional permissions are required for creating an index with automatic semantic enrichment:

UpdateIndex API permissions

When fine-grained access control is enabled, the following additional permissions are required for updating an index with automatic semantic enrichment:

GetIndex API permissions

When fine-grained access control is enabled, the following additional permissions are required for retrieving information about an index with automatic semantic enrichment:

DeleteIndex API permissions

When fine-grained access control is enabled, the following additional permission is required for deleting an index with automatic semantic enrichment: