# Working with S3 Vectors and vector buckets Working with S3 Vectors and vector buckets ## What is Amazon S3 Vectors? Amazon S3 Vectors delivers purpose-built, cost-optimized vector storage for AI agents, inference, RAG, and semantic search. S3 Vectors is designed to provide the same elasticity, durability, and availability as Amazon S3 and delivers subsecond latency for infrequent queries and as low as 100 milliseconds for more frequent queries. You get a dedicated set of API operations to store, access, and query vector data without provisioning any infrastructure. S3 Vectors consists of several key components that work together: + **Vector buckets** – A new bucket type that's purpose-built to store and query vectors. + **Vector indexes** – Within a vector bucket, you can organize your vector data within vector indexes. You perform similarity queries on your vector data within vector indexes. + **Vectors **– You store vectors in your vector index. For similarity search and AI applications, vectors are created as vector embeddings which are numerical representations that preserve semantic relationships between content (such as text, images, or audio) so similar items are positioned closer together. S3 Vectors can perform similarity searches based on semantic meaning rather than exact matching through comparing how close vectors are to each other mathematically. When adding vector data to a vector index, you can also attach metadata for future filtering queries based on a set of conditions (for example, timestamps, categories, and user preferences). Writes to S3 Vectors are strongly consistent, which means that you can immediately access the most recently added data. As you write, update, and delete vectors over time, S3 Vectors automatically optimizes the vector data to achieve the best possible price performance for vector storage, even as the data sets scale and evolve. You can control access to your vector data with the existing access control mechanisms of Amazon S3, including bucket and IAM policies. For more information about vector index limits per bucket and vector limits per index, see [Limitations and restrictions](s3-vectors-limitations.md). ## Use cases: Similarity searches across large datasets Similarity searches allow you to find items that are conceptually related to each other based on their vector representations, rather than exact keyword matches. These searches identify content with similar meanings or characteristics, even when the exact words or visual elements differ. Common use cases for similarity search with S3 Vectors include: + **Medical imaging** - Find similarities in millions of medical images to assist with diagnosis and treatment planning + **Copyright infringement** - Identify potentially derivative content across large media libraries + **Image deduplication** - Detect and remove duplicate or near-duplicate images from large image collections + **Video understanding** - Search for specific scenes or content within video assets + **Enterprise document search** - Enable semantic search across corporate documents to find relevant information based on meaning + **Personalization** - Deliver tailored recommendations by finding similar items You should use S3 Vectors if you want to build cost-effective vector search and agentic AI applications with sub-second search times. With vector buckets, you only pay for what you use, and could save costs for uploading, storing, and querying vector embeddings. For more information about pricing, see [Amazon S3 pricing](https://aws.amazon.com/s3/pricing/). ## Features of S3 Vectors ### Purpose-built storage for vectors S3 Vectors is the first purpose-built object storage in the cloud to store and query vectors. Vector buckets are designed to provide cost-effective, elastic, and durable storage for vector data. Vector embeddings are transforming how customers use and retrieve their unstructured data, from detecting similarities across medical images, finding anomalies in thousands of hours of video footage, navigating through large code bases, and identifying the most relevant case law for a given legal matter. These emerging applications combine with embedding models to encode the semantic meaning of data (for example, text, images, video, code) as numerical vector embeddings. Within a vector bucket, you organize your vector data within vector indexes, without provisioning infrastructure. As you write, update, and delete vectors over time, S3 Vectors automatically optimize the vector data to achieve the best possible price performance for vector storage, even as the data sets scale and evolve. For more information about vector index limits per bucket and vector limits per index, see [Limitations and restrictions](s3-vectors-limitations.md). ### Perform similarity queries With S3 Vectors, you can perform queries to find the most similar vectors to a query vector, with sub-second response times for infrequent queries and as low as 100 milliseconds for more frequent queries. S3 Vectors is ideal for workloads where queries are less frequent. ### Metadata filtering You can attach metadata (for example, year, author, genre, and location) as key-value pairs to your vectors. By default, all metadata is filterable unless you explicitly specify it as non-filterable. You can use filterable metadata to filter your query results based on specific attributes, enhancing the relevance of your queries. Vector indexes support string, number, boolean, and list types of metadata. For more information about metadata size limits per vector and filterable metadata size limits per vector, see [Limitations and restrictions](s3-vectors-limitations.md). ### Access management and security You can manage access for resources in vector buckets with IAM and [Service Control Policies](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_scps.html) in AWS Organizations. S3 Vectors uses a different service namespace than Amazon S3: the `s3vectors` namespace. Therefore, you can design policies specifically for the S3 Vectors service and its resources. You can design policies to grant access to individual vector indexes, all vector indexes within a vector bucket, or all vector buckets in an account. All Amazon S3 Block Public Access settings are always enabled for vector buckets and cannot be disabled. ### Integration with AWS services S3 Vectors integrates with other AWS services to enhance your vector processing capabilities: + **[Amazon OpenSearch Service](https://aws.amazon.com/opensearch-service/)** - Optimize vector storage costs while continuing to use OpenSearch API operations. This is ideal for workloads that need advanced search functionality such as hybrid search, aggregations, advanced filtering, and faceted search. You can also export a snapshot of an S3 vector index to Amazon OpenSearch Serverless for high QPS and low latency vector search. + **[Amazon Bedrock Knowledge Bases](https://aws.amazon.com/bedrock/knowledge-bases/)** - Select a vector index in S3 Vectors as your vector store to save on storage costs for retrieval augmented generation (RAG) applications. + **[Amazon Bedrock in SageMaker Unified Studio](https://aws.amazon.com/bedrock/unifiedstudio/)** - Develop and test knowledge bases using S3 Vectors as your vector store. # Tutorial: Getting started with S3 Vectors In this tutorial, you create an S3 vector bucket and a vector index in an AWS Region in the Amazon S3 console. Next, you use an Amazon Bedrock embedding model to generate vector embeddings of your data and store them in your vector index to perform semantic searches. To get started, sign up for an AWS account if you don't already have one. For more information about setting up with an account, see [Getting started with Amazon S3](GetStartedWithS3.md). **Topics** + [ ## Step 1: Create a vector bucket with the console ](#s3-vectors-create-bucket-tutorial) + [ ## Step 2: Create a vector index in a vector bucket with the console ](#s3-vectors-create-index-tutorial) + [ ## Step 3: Insert vectors into a vector index with the SDK for Python (Boto3) ](#s3-vectors-insert-vectors-tutorial) + [ ## Step 4. Query vectors in a vector index with the SDK for Python (Boto3) ](#s3-vectors-query-vectors-tutorial) + [ ## (Optional) Automate vector embedding creation with the S3 Vectors Embed CLI ](#s3-vectors-embed-CLI-tutorial) + [ ## (Optional) Integrate S3 Vectors with Amazon Bedrock Knowledge Bases ](#s3-vectors-bedrock-kb-tutorial) + [ ## (Optional) Integrate S3 Vectors with Amazon OpenSearch ](#s3-vectors-opensearch-tutorial) ## Step 1: Create a vector bucket with the console S3 vector buckets are a type of Amazon S3 bucket that's purpose-built to store and query vectors. You use dedicated API operations to write and query vector data. You can control access to your vector data with access control mechanisms such as IAM identity-based policies and resource-based policies. In this step, you use the Amazon S3 console to create your first vector bucket. For other ways to create a vector bucket, see [Creating a vector bucket](s3-vectors-buckets-create.md). ### To create a vector bucket 1. Sign in to the AWS Management Console and open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/). 1. In the left navigation pane, choose **Vector buckets**. 1. On the **Vector buckets** page, choose **Create vector bucket**. 1. On the **Create vector bucket** page, for the **Vector bucket name** field, enter a name. In this tutorial, we use *media-embeddings* as the vector bucket name. Vector bucket names must be 3 to 63 characters and unique within an AWS Region. Valid characters are lowercase letters (a-z), numbers (0-9), and hyphens (-). **Note** The vector bucket name can't be changed after the bucket is created. 1. Under **Encryption**, choose **Specify encryption type**. You have the option to specify encryption type as **Server-side encryption with AWS Key Management Service keys (SSE-KMS)** or the default **Server-side encryption with Amazon S3 managed keys (SSE-S3)**. In this tutorial, we choose **Server-side encryption with Amazon S3 managed keys (SSE-S3)**. For more information about setting encryption configuration for vector buckets, see [Data protection and encryption in S3 Vectors](s3-vectors-data-encryption.md). **Note** The encryption type can't be changed after the vector bucket is created. 1. Under **Tags (Optional)**, you can add tags as key-value pairs to help organize and specify permissions for your vector buckets. Enter a **Key** and a **Value**. To add another tag, choose **Add Tag**. You can enter up to 50 tags for a vector bucket. For more information, see [Using tags with S3 vector buckets](s3-vectors-tags.md). 1. Choose **Create vector bucket**. ## Step 2: Create a vector index in a vector bucket with the console **Note** Choose your vector index configuration parameters carefully. After you create a vector index, you can't update the vector index name, dimension, distance metric, or non-filterable metadata keys. To change any of these values, you must create a new vector index. In this step, you create a vector index in your vector bucket. The vector index is used to store and query your vector embeddings. For more information about vector indexes, see [Vector indexes](s3-vectors-indexes.md). ### To create a vector index 1. Navigate to the newly created vector bucket in the **Vector buckets** list on the Amazon S3 console. 1. Choose **Create vector index**. 1. On the **Create vector index** page, enter a name in the **Vector index name** field. In this tutorial, we use *movies* as the vector index name. Vector index names must be 3 to 63 characters and unique within this vector bucket. Valid characters are lowercase letters (a-z), numbers (0-9), hyphens (-), and dots (.). The vector index name can't be changed after index creation. 1. Under **Dimension**, specify a numeric value between 1 and 4096 that determines how many numbers will be in each vector that's generated by your vector embedding model. Embedding models are specialized machine learning (ML) models that convert data (such as text or images) into numerical vectors. Embedding models typically produce outputs between 500-2000 dimensions, with each dimension being a floating-point number. In this tutorial, we'll use the Titan Text v2 embedding model from Amazon Bedrock. This embedding model uses 1024-dimensional vectors as the default, so we'll set dimension for the vector index to `1024`. The dimension value can't be changed after index creation. Carefully specify the dimension value based on your embedding model's recommendations. 1. Under **Distance metric**, choose either **Cosine** (which measures angular similarity) or **Euclidean** (which measures straight-line distance) as the distance metric to define how similarity between vectors is calculated during queries. In this tutorial, we choose **Cosine**. The distance metric can't be changed after index creation. Carefully choose the distance metric based on your embedding model's recommendations. 1. Under **Additional settings (optional)**, you can specify non-filterable metadata keys to store supplementary information alongside your vector embeddings. **Note** You can attach filterable metadata as key-value pairs to each vector when you insert vector data after index creation. By default, all metadata keys that are attached to vectors are filterable and can be used as filters in a similarity query. Only metadata keys that are specified as non-filterable during vector index creation are excluded from filtering. Each non-filterable metadata key must be between 1 to 63 characters and unique within the vector index. These keys are useful for storing reference information that you don't need to filter on during similarity queries. For example, when working with text embeddings, you might want to preserve the original text chunks for reference purposes. In this tutorial, we add a non-filterable metadata key that's named `source_text` to store the text data that we want to create vector embeddings for. You can't update non-filterable metadata keys after index creation. Plan your metadata structure carefully. 1. Under **Encryption**, choose **Specify encryption type**. You have the option to use the bucket level encryption settings or override the encryption settings for the vector index. If you override the bucket-level settings, you have the option to specify encryption type for the vector index as **Server-side encryption with AWS Key Management Service keys (SSE-KMS)** or the **Server-side encryption with Amazon S3 managed keys (SSE-S3)**. In this tutorial, we choose to **Use bucket settings for encryption**. For more information about setting encryption configuration for vector buckets and indexes, see [Data protection and encryption in S3 Vectors](s3-vectors-data-encryption.md). 1. Under **Tags (Optional)**, you can add tags as key-value pairs to help track and organize vector index costs using AWS Billing and Cost Management. Enter a **Key** and a **Value**. To add another tag, choose **Add Tag**. You can enter up to 50 tags for a vector index. For more information, see [Using tags with S3 vector buckets](s3-vectors-tags.md). 1. Choose **Create vector index**. Verify that your new vector index appears inside the bucket. ## Step 3: Insert vectors into a vector index with the SDK for Python (Boto3) To insert, list, and query vectors, use the AWS SDKs, AWS CLI, or Amazon S3 REST API. This step is to insert vector embeddings to your vector index with the [PutVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_PutVectors.html) API operation. You must first generate vector embeddings by using the embedding model of your choice. If you use Amazon Bedrock, generate embeddings by using the [InvokeModel](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html) API operation to specify your preferred embedding model. Alternatively, to generate and insert vector embeddings with a single command, see [(Optional) Automate vector embedding creation with the S3 Vectors Embed CLI](#s3-vectors-embed-CLI-tutorial). The following example code generates 1024-dimensional vector embeddings with the Titan Text Embeddings V2 model from Amazon Bedrock by using the AWS SDK for Python (Boto3), and stores them in your vector index by using the `PutVectors` API. Along with each vector, we attach key-value pairs as filterable metadata. Additionally, we use a non-filterable metadata key that's named *source\$1text* to preserve the original text from which each vector is derived. To maximize request throughput and optimize for speed and efficiency, we recommend that you insert and delete vectors in batches. For more information, see [S3 Vectors best practices](s3-vectors-best-practices.md). ``` # Populate a vector index with embeddings from Amazon Titan Text Embeddings V2. import boto3 import json # Create Bedrock Runtime and S3 Vectors clients in the AWS Region of your choice. bedrock = boto3.client("bedrock-runtime", region_name="us-west-2") s3vectors = boto3.client("s3vectors", region_name="us-west-2") # Texts to convert to embeddings. texts = [ "Star Wars: A farm boy joins rebels to fight an evil empire in space", "Jurassic Park: Scientists create dinosaurs in a theme park that goes wrong", "Finding Nemo: A father fish searches the ocean to find his lost son" ] # Generate vector embeddings. embeddings = [] for text in texts: response = bedrock.invoke_model( modelId="amazon.titan-embed-text-v2:0", body=json.dumps({"inputText": text}) ) # Extract embedding from response. response_body = json.loads(response["body"].read()) embeddings.append(response_body["embedding"]) # Write embeddings into vector index with metadata. s3vectors.put_vectors( vectorBucketName="media-embeddings", indexName="movies", vectors=[ { "key": "Star Wars", "data": {"float32": embeddings[0]}, "metadata": {"source_text": texts[0], "genre":"scifi"} }, { "key": "Jurassic Park", "data": {"float32": embeddings[1]}, "metadata": {"source_text": texts[1], "genre":"scifi"} }, { "key": "Finding Nemo", "data": {"float32": embeddings[2]}, "metadata": {"source_text": texts[2], "genre":"family"} } ] ) ``` ## Step 4. Query vectors in a vector index with the SDK for Python (Boto3) After you store your vectors embeddings in the vector index, you can perform a similarity search by using the [QueryVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_QueryVectors.html) API operation. You must first generate your query vector embeddings by using the same embedding model that's used during insertion in [Step 3: Insert vectors into a vector index with the SDK for Python (Boto3)](#s3-vectors-insert-vectors-tutorial). In this tutorial, use the Titan Text Embeddings V2 model from Amazon Bedrock through the SDK for Python (Boto3). You can perform a similarity search to return the closest matching vectors. In a similarity search, you can optionally filter by using metadata keys to narrow results based on specific criteria while maintaining semantic relevance. ``` # Query a vector index with an embedding from Amazon Titan Text Embeddings V2. import boto3 import json # Create Bedrock Runtime and S3 Vectors clients in the AWS Region of your choice. bedrock = boto3.client("bedrock-runtime", region_name="us-west-2") s3vectors = boto3.client("s3vectors", region_name="us-west-2") # Query text to convert to an embedding. input_text = "adventures in space" # Generate the vector embedding. response = bedrock.invoke_model( modelId="amazon.titan-embed-text-v2:0", body=json.dumps({"inputText": input_text}) ) # Extract embedding from response. model_response = json.loads(response["body"].read()) embedding = model_response["embedding"] # Query vector index. response = s3vectors.query_vectors( vectorBucketName="media-embeddings", indexName="movies", queryVector={"float32": embedding}, topK=3, returnDistance=True, returnMetadata=True ) print(json.dumps(response["vectors"], indent=2)) # Query vector index with a metadata filter. response = s3vectors.query_vectors( vectorBucketName="media-embeddings", indexName="movies", queryVector={"float32": embedding}, topK=3, filter={"genre": "scifi"}, returnDistance=True, returnMetadata=True ) print(json.dumps(response["vectors"], indent=2)) ``` ## (Optional) Automate vector embedding creation with the S3 Vectors Embed CLI Amazon S3 Vectors Embed CLI is a standalone command line tool that simplifies the process of working with vector embeddings in S3 Vectors. With single commands, you can create vector embeddings for your data by using Amazon Bedrock, and store and query them in your S3 vector index. The tool supports the following commands: + `s3vectors-embed put`: Generate vector embeddings of text, file content, or S3 objects, and store them as vectors in an S3 vector index. + `s3vectors-embed query`: Embed a query input and search for similar vectors in an S3 vector index. For more information about the commands and usage, see the [Amazon S3 Vectors Embed CLI GitHub repository](https://github.com/awslabs/s3vectors-embed-cli). ## (Optional) Integrate S3 Vectors with Amazon Bedrock Knowledge Bases Amazon Bedrock Knowledge Bases integrates with S3 Vectors for cost effective and long-term storage of vector datasets. ### Prerequisites Follow the [prerequisites](https://docs.aws.amazon.com/bedrock/latest/userguide/getting-started-console.html) to ensure you have necessary permissions for creating an Amazon Bedrock knowledge base. + Appropriate IAM permissions for both S3 Vectors and Amazon Bedrock services. For more information about IAM permissions for S3 Vectors, see [Identity and Access management in S3 Vectors](s3-vectors-access-management.md). + Your source documents prepared for ingestion into the knowledge base. + An understanding of your embedding model requirements. ### Create an Amazon Bedrock knowledge base with S3 Vectors in the console Amazon Bedrock Knowledge Bases provides a fully managed end-to-end RAG workflow. When you create a knowledge base with S3 Vectors, Amazon Bedrock automatically fetches data from your S3 data source, converts content into text blocks, generates embeddings, and stores them in your vector index. You can then query the knowledge base and generate responses based on chunks retrieved from your source data. **To create an Amazon Bedrock knowledge base with S3 Vectors in the console** 1. Sign in to the AWS Management Console using an [IAM role with Amazon Bedrock permissions](https://docs.aws.amazon.com/bedrock/latest/userguide/getting-started.html), and open the Amazon Bedrock console at [https://console.aws.amazon.com/bedrock/](https://console.aws.amazon.com/bedrock/). 1. In the left navigation pane, choose **Knowledge Bases**. 1. In the **Knowledge Bases** section, choose **Create**. In the dropdown list, choose **Knowledge Base with vector store**. 1. Keep the **Knowledge Base name**, **Knowledge Base description**, and **IAM permissions** at their default values in this tutorial. 1. For **Choose a data source type**, choose the data source to connect your knowledge base to. In this tutorial, choose **Amazon S3**. 1. (Optionally) Provide any **Tags** or **Application logs**. 1. Choose **Next**. 1. On the **Configure data source** page, keep the **Data source name**, **Data source location**, **Parsing strategy**, and **Chunking strategy** at the default settings in this tutorial. 1. In the **S3 URI**, enter the S3 general purpose bucket that contains your source files, or choose **Browse S3** and choose the S3 general purpose bucket. 1. For encrypted S3 data, select **Add customer-managed KMS key for S3 data** and specify your customer-managed KMS key. 1. Choose **Next** to proceed to the next step that specifies the **Embeddings model** and **Vector store** options. 1. In the **Embeddings model** section, choose **Select model** and choose an embedding model compatible for use with S3 Vectors. You must use floating-point embeddings. Binary type isn't supported. 1. (Optional) Expand the **Additional configurations** section to see the following configuration options (not all models support all configurations): + **Embeddings type** – Whether to convert the data to floating-point (float32) vector embeddings (more precise, but more costly) or binary vector embeddings (less precise, but less costly). To integrate with S3 Vectors, you must choose **Floating-point vector embeddings**. + **Vector dimensions** – Choose the embedding models recommended dimension size. 1. In the **Vector store** section, choose the recommended method **Quick create a new vector store** for automatic setup of a new vector bucket, or choose **Use an existing vector store** to use an existing vector bucket if available. For information about using an existing vector store flow, see [Prerequisites for using a vector store you created for a knowledge base](https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base-setup.html) in the *Amazon Bedrock User Guide*. 1. For **Additional configurations** – By default, your vector bucket uses server-side encryption with Amazon S3 managed keys (SSE-S3). You can choose to use your own KMS key for server-side encryption with AWS Key Management Service keys (SSE-KMS) for enhanced control over encryption settings. 1. Choose **Next** to review the Knowledge Base details and then choose **Create Knowledge Base**. **Note** The time that it takes to create a knowledge base depends on your specific configurations. When the creation of a knowledge base completes, the status of the knowledge base changes to state it is either ready or available. Once your knowledge base is ready and available, sync your data source for the first time or whenever you want to keep your content up to date. To sync your data source, choose your knowledge base in the console and choose **Sync** within the data source overview section. ## (Optional) Integrate S3 Vectors with Amazon OpenSearch [Amazon OpenSearch Service](https://aws.amazon.com/opensearch-service/) is a fully managed service that simplifies the deployment, scaling, and operation of OpenSearch in the AWS Cloud. There are two integrations between S3 Vectors and OpenSearch. One is to export vector data from S3 Vectors to OpenSearch Serverless for high-performance search capabilities. The other uses S3 Vectors as a cost-effective storage engine within OpenSearch while maintaining access to OpenSearch functionality. For more information, see [Using S3 Vectors with OpenSearch Service](s3-vectors-opensearch.md). # Vector buckets Vector buckets Vector buckets are a type of Amazon S3 bucket designed specifically for storing and querying vector data. Vector buckets use dedicated APIs to manage vector data efficiently and reduce costs of upload, storing, and querying vector embeddings. Vector buckets provide the foundation for organizing your vector data into indexes, enabling you to perform similarity searches across large datasets while benefiting from the availability, durability, scalability, and cost-effectiveness of Amazon S3. Vector buckets are optimized for long-term vector storage with sub-second search times. You can perform similarity queries on your vector data and optionally attach metadata to filter queries based on specific conditions such as dates, categories, or user preferences. Each vector bucket has a unique Amazon Resource Name (ARN) and resource policy attached to it. The ARNs of vector buckets follow the following format: ``` arn:aws:s3vectors:Region:OwnerAccountID:bucket/bucket-name ``` Within a vector bucket, you create vector indexes to store and query your data. Each vector bucket exists within a specific AWS Region and you can create multiple vector indexes inside a vector bucket. Vector buckets support security and access control mechanisms, including IAM identity-based policies and bucket policies. You can use bucket policies to grant or restrict access to specific indexes within your vector bucket. Key characteristics of vector buckets: + Purpose-built for vector storage and similarity search operations. + Strongly consistent writes ensure that the vector data is immediately accessible. + Automatic optimization of vector data for best price-performance as datasets scale. For more information about vector index limits per bucket and other limitations, see [Limitations and restrictions](s3-vectors-limitations.md). **Topics** + [ # Vector bucket naming rules ](s3-vectors-buckets-naming.md) + [ # Creating a vector bucket ](s3-vectors-buckets-create.md) + [ # Listing vector buckets ](s3-vectors-buckets-list.md) + [ # Viewing vector bucket attributes ](s3-vectors-buckets-details.md) + [ # Deleting an empty vector bucket ](s3-vectors-buckets-delete.md) + [ # Managing vector bucket policies ](s3-vectors-bucket-policy.md) + [ # Using tags with S3 vector buckets ](s3-vectors-tags.md) # Vector bucket naming rules Vector bucket names must follow specific naming conventions to ensure uniqueness within an AWS Region. Amazon S3 enforces the following bucket naming requirements, and you can't create a vector bucket if these rules aren't followed. Additionally, there are best practices that, while not enforced, help prevent conflicts when working with vector buckets programmatically or through the console. ## Vector bucket naming requirements When creating vector buckets, you must follow these requirements: + Vector bucket names must be unique in the same AWS account for each AWS Region. + Vector bucket names must be between 3 and 63 characters long. + Vector bucket names can consist only of lowercase letters (a-z), numbers (0-9), and hyphens (-). + Vector bucket names must begin and end with a letter or number. ## Best practices for naming We recommend following these best practices when naming your vector buckets: + Use descriptive names that reflect the purpose of your vector data (for example, product-recommendations, document-embeddings). + Avoid using sensitive information in bucket names as they may appear in logs and URLs. + Keep names concise but meaningful for easier management and identification. These naming conventions ensure that your vector buckets can be reliably accessed through the AWS Management Console, Amazon S3 REST API, the AWS CLI, and AWS SDKs. # Creating a vector bucket Creating a vector bucket You can create a vector bucket using the S3 console or the AWS CLI. All data stored in vector buckets are always encrypted at rest. By default, vector buckets use SSE-S3 to encrypt vector data. You can choose to configure buckets to use server-side encryption with AWS Key Management Service (AWS KMS) keys (SSE-KMS) instead. The bucket encryption settings can’t be changed after a vector bucket is created, so it's important to choose the appropriate encryption method based on your security requirements and compliance needs. For more information about security in vector buckets, see [Data protection and encryption in S3 Vectors](s3-vectors-data-encryption.md). ## Using the S3 console 1. Sign in to the AWS Management Console and open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/). 1. In the navigation pane, choose **Vector buckets**. 1. Choose **Create vector bucket**. 1. For **Vector bucket name**, enter a name for your bucket. The bucket name must follow the naming rules: + Bucket name must be between 3 and 63 characters long. + Bucket name can only include lowercase letters, numbers, and hyphens. + Bucket name must be unique within your AWS account for an AWS Region. For more information about vector bucket naming rules, see [Vector bucket naming rules](s3-vectors-buckets-naming.md). **Important** You can't change the vector bucket name after you create the bucket. 1. For **Encryption**, choose one of the following options: + **Don't specify encryption type** – Amazon S3 automatically applies server-side encryption with Amazon S3 managed keys (SSE-S3) as the base level of encryption for new vectors. Choose this option for the simplest setup with no additional configuration. + **Specify encryption type** – Choose a specific encryption method: + **Server-side encryption with Amazon S3 managed keys (SSE-S3)** – Explicitly choose to use SSE-S3. Amazon S3 encrypts your vector data as it writes it to storage and decrypts it when you access it. AWS manages all encryption keys automatically. + **Server-side encryption with AWS Key Management Service keys (SSE-KMS)** – Uses customer managed keys (CMKs) in AWS KMS, giving you more control over your encryption keys, key rotation, and access policies. If you select SSE-KMS, you have additional options: + **Choose from your AWS KMS keys** – Select an existing customer managed key from your account. + **Enter AWS KMS key ARN** – Specify the full ARN of a KMS key (required format). + **Create a KMS key** – Opens the AWS KMS console to create a new customer managed key. KMS key requirements: + The KMS key must be in the same Region as the vector bucket. + You must specify the full KMS key ARN (key IDs and aliases aren't supported). + You must grant the S3 Vectors service principal (`indexing.s3vectors.amazonaws.com`) the `kms:Decrypt` permission to use the key. For more information about an example AWS KMS key policy, see [Data protection and encryption in S3 Vectors](s3-vectors-data-encryption.md). For detailed information about encryption options and KMS key setup, see [Using SSE-KMS encryption](s3-vectors-data-encryption.md#s3-vectors-sse-kms-encryption). **Important** Encryption settings can't be changed after the vector bucket is created. Choose carefully based on your long-term security and compliance requirements. 1. Under **Tags (Optional)**, you can add tags as key-value pairs to help track and organize vector index costs using AWS Billing and Cost Management. Enter a **Key** and a **Value**. To add another tag, choose **Add Tag**. You can enter up to 50 tags for a vector index. For more information, see [Using tags with S3 vector buckets](s3-vectors-tags.md). 1. Choose **Create vector bucket**. After creation, you'll see a confirmation message. The new vector bucket appears in your vector buckets list and is ready for creating vector indexes within the bucket. ## Using the AWS CLI You can create a vector bucket with SSE-S3 encryption using the following command. To use this example, replace the *user input placeholders* with your own information. ``` aws s3vectors create-vector-bucket \ --vector-bucket-name "amzn-s3-demo-vector-bucket" ``` To create a vector bucket with SSE-KMS encryption using a customer managed KMS key: ``` aws s3vectors create-vector-bucket \ --vector-bucket-name "amzn-s3-demo-vector-bucket" \ --encryption-configuration '{"sseType": "aws:kms", "kmsKeyArn": "arn:aws:kms:us-east-1:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"}' ``` ## Using the AWS SDKs ------ #### [ SDK for Python ] ``` import boto3 # Create a S3 Vectors client in the AWS Region of your choice. s3vectors = boto3.client("s3vectors", region_name="us-west-2") #Create a vector bucket s3vectors.create_vector_bucket(vectorBucketName="media-embeddings") ``` ------ # Listing vector buckets Listing vector buckets You can view all your vector buckets using the Amazon S3 console, AWS CLI, or AWS SDKs. The listing operations support prefix-based filtering to help you find specific buckets when you have many vector buckets in your account. For more information about `ListVectorBuckets`, prefix limits, and response limits, see [ListVectorBuckets](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_ListVectorBuckets.html) in the *Amazon S3 API Reference*. ## Prefix search capability Prefix search allows you to list buckets that start with a specific prefix, making it easier to organize and find related vector buckets. This is particularly useful when you use naming conventions that group related buckets together: + **Environment-based**: `production-vectors-`, `staging-vectors-`, `dev-vectors-` + **Use case-based**: `ml-model-vectors-`, `document-search-`, `image-similarity-` + **Team-based**: `data-science-vectors-`, `ml-platform-vectors-` ## Using the S3 console To list vector buckets 1. Sign in to the AWS Management Console and open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/). 1. In the navigation pane, choose **Vector buckets**. The console displays a list of all your vector buckets with the following information: + **Name** – The unique name of the vector bucket + **Creation date** – When the bucket was created + **Amazon Resource Name (ARN)** – The full ARN for programmatic access To filter the list: + To find a bucket based on the start of the bucket name, enter a vector bucket name or prefix in the search box above the bucket list. + Use prefixes to find groups of related buckets (for example, type "prod-" to find all production buckets) The list updates in real-time as you type ## Using the AWS CLI ``` aws s3vectors list-vector-buckets ``` ## Using the AWS SDKs ------ #### [ SDK for Python ] ``` import boto3 # Create a S3 Vectors client in the AWS Region of your choice. s3vectors = boto3.client("s3vectors", region_name="us-west-2") #List vector buckets response = s3vectors.list_vector_buckets() buckets = response["vectorBuckets"] print(buckets) ``` ------ # Viewing vector bucket attributes Viewing vector bucket attributes You can view detailed information about a vector bucket, including its properties, encryption settings and creation details using the Amazon S3 REST API, AWS SDKs, S3 Console, or the AWS Command Line Interface (AWS CLI). For more information about `GetVectorBucket`, see [GetVectorBucket](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_GetVectorBucket.html) in the *Amazon S3 API Reference*. ## Using the S3 console 1. Sign in to the console and open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/). 1. In the navigation pane, choose **Vector buckets**. 1. The console displays a list of all your vector buckets. Find a bucket based on the start of the bucket name, enter a vector bucket name or prefix in the search box above the bucket list. Once you locate your vector bucket, you can view detailed information about it, including its encryption settings, tags, and creation details in the **Properties** tab. ## Using the AWS CLI ``` aws s3vectors get-vector-bucket --vector-bucket-name "amzn-s3-demo-vector-bucket" ``` # Deleting an empty vector bucket Deleting a vector bucket You can delete a vector bucket when you no longer need it. To delete a vector bucket, you must first delete all the vector indexes within the bucket. When you delete a vector index, all vector data within it is deleted. Use the Amazon S3 REST API, AWS SDKs, S3 Console, or the AWS Command Line Interface (AWS CLI) to delete a vector bucket. Before you can delete a vector bucket, you must: + Delete all vector indexes in the bucket. + Ensure no operations are in progress on the bucket or its indexes. **Important** Bucket deletion is permanent and can't be undone. All data and configuration associated with the bucket is permanently lost. The bucket name becomes available for reuse after deletion. Any applications or scripts referencing the bucket will receive errors after deletion. ## Using the S3 console 1. Sign in to the console and open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/). 1. In the navigation pane, choose **Vector buckets**. 1. The console displays a list of all your vector buckets. Find a bucket based on the start of the bucket name, enter a vector bucket name or prefix in the search box above the bucket list. Once you locate and choose your vector bucket, select the **Delete** option. 1. To confirm this deletion, type **delete** and then select **Delete vector bucket**. ## Using the AWS CLI First, check that no vector indexes exist in the bucket. For more information about how to verify the bucket is empty, see [Listing vector indexes](s3-vectors-index-list.md). If indexes exist, you must delete all vectors from each index and then delete the indexes. For more information about how to verify the bucket is empty, see [Listing vectors](s3-vectors-list.md), [Deleting vectors from a vector index](s3-vectors-delete.md), and [Deleting a vector index](s3-vectors-index-delete.md). To delete the empty vector bucket, use the following example command and replace the *user input placeholders* with your own information. ``` aws s3vectors delete-vector-bucket \ --vector-bucket-name "amzn-s3-demo-vector-bucket" ``` ## Using the AWS SDKs ------ #### [ SDK for Python ] ``` import boto3 # Create a S3 Vectors client in the AWS Region of your choice. s3vectors = boto3.client("s3vectors", region_name="us-west-2") #Delete a vector bucket response = s3vectors.delete_vector_bucket(vectorBucketName="media-embeddings") ``` ------ # Managing vector bucket policies Managing vector bucket policies Vector bucket policies are resource-based policies that you attach directly to vector buckets to control access to the bucket and its contents. You can add, view, edit, delete vector bucket policies by using the Amazon S3 REST API, AWS SDKs, S3 Console, or the AWS Command Line Interface (AWS CLI). Bucket policies for vector buckets can grant permissions to principals from other AWS accounts, making them useful for cross-account access scenarios. ## Policy management operations + [PutVectorBucketPolicy](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_PutVectorBucketPolicy.html) – Add or update a bucket policy. + [GetVectorBucketPolicy](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_GetVectorBucketPolicy.html) – Retrieve the current bucket policy. + [DeleteVectorBucketPolicy](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_DeleteVectorBucketPolicy.html) – Remove the bucket policy. ## Adding a vector bucket policy ### Using the S3 console 1. Open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/). 1. In the left navigation pane, choose **Amazon S3**. 1. Choose **Vector buckets** and select the vector bucket name that you want to add a policy to. 1. Choose the **Permissions** tab. 1. Under **Vector bucket policy**, choose **Edit**. 1. In the policy editor, enter your policy JSON. 1. (Optional) Choose **Policy examples** to see sample policies that you can adapt to your needs. 1. After entering your policy, choose **Save changes**. ### Using the AWS CLI To add or update a bucket policy, use the following example command and replace the *user input placeholders* with your own information. ``` aws s3vectors put-vector-bucket-policy \ --vector-bucket-name "amzn-s3-demo-vector-bucket" \ --policy '{"Version": "2012-10-17", "Statement":[{"Effect":"Allow","Principal":{"AWS":"arn:aws:iam::111122223333:root"},"Action":"s3vectors:*","Resource":"arn:aws:s3vectors:aws-region:111122223333:bucket/amzn-s3-demo-vector-bucket"}]}' ``` ## Viewing a vector bucket policy ### Using the S3 console 1. Open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/). 1. In the left navigation pane, choose **Amazon S3**. 1. Choose **Vector buckets** and select the vector bucket name that you want to view the policy for. 1. Choose the **Permissions** tab. ### Using the AWS CLI To retrieve a bucket policy, use the following example command and replace the *user input placeholders* with your own information. ``` aws s3vectors get-vector-bucket-policy \ --vector-bucket-name "amzn-s3-demo-vector-bucket" ``` ## Deleting a vector bucket policy ### Using the S3 console 1. Open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/). 1. In the left navigation pane, choose **Amazon S3**. 1. Choose **Vector buckets** and select the vector bucket name that you want to delete the policy for. 1. Choose the **Permissions** tab. 1. Under the **Vector bucket policy**, choose **Delete**. ### Using the AWS CLI To delete a bucket policy, use the following example command and replace the *user input placeholders* with your own information. ``` aws s3vectors delete-vector-bucket-policy \ --vector-bucket-name "amzn-s3-demo-vector-bucket" ``` For detailed information about creating and managing bucket policies, including policy examples and best practices, see [S3 Vectors resource-based policy examples](s3-vectors-resource-based-policies.md). # Using tags with S3 vector buckets Tagging vector buckets An AWS tag is a key-value pair that holds metadata about resources, in this case Amazon S3 vector buckets. You can tag S3 vector buckets when you create them or manage tags on existing vector buckets. For general information about tags, see [Tagging for cost allocation or attribute-based access control (ABAC)](tagging.md). **Note** There is no additional charge for using tags on vector buckets beyond the standard S3 API request rates. For more information, see [Amazon S3 pricing](https://aws.amazon.com/s3/pricing/). ## Common ways to use tags with vector buckets Use tags on your S3 vector buckets for: + **Attribute-based access control (ABAC)** – Scale access permissions and grant access to S3 vector buckets based on their tags. For more information, see [Using tags for attribute-based access control (ABAC)](tagging.md#using-tags-for-abac). ### ABAC for S3 vector buckets Amazon S3 vector buckets support attribute-based access control (ABAC) using tags. Use tag-based condition keys in your AWS organizations, IAM, and S3 vector bucket policies. For enterprises, ABAC inAmazon S3 supports authorization across multiple AWS accounts. In your IAM policies, you can control access to S3 vector buckets based on the vector bucket's tags by using the following global condition keys: `aws:ResourceTag/key-name` Use this key to compare the tag key-value pair that you specify in the policy with the key-value pair attached to the resource. For example, you could require that access to a resource is allowed only if the resource has the attached tag key `Dept` with the value `Marketing`. For more information, see [Controlling access to AWS resources](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_tags.html#access_tags_control-resources). `aws:RequestTag/key-name` Use this key to compare the tag key-value pair that was passed in the request with the tag pair that you specify in the policy. For example, you could check whether the request includes the tag key `Dept` and that it has the value `Accounting`. For more information, see [Controlling access during AWS requests](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_tags.html#access_tags_control-requests). You can use this condition key to restrict which tag key-value pairs can be passed during the `TagResource` and `CreateVectorBucket` API operations. `aws:TagKeys` Use this key to compare the tag keys in a request with the keys that you specify in the policy. We recommend that when you use policies to control access using tags, use the `aws:TagKeys` condition key to define what tag keys are allowed. For example policies and more information, see [Controlling access based on tag keys](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_tags.html#access_tags_control-tag-keys). You can create an S3 vector bucket with tags. To allow tagging during the `CreateVectorBucket` API operation, you must create a policy that includes both the `s3vectors:TagResource` and `s3vectors:CreateVectorBucket` actions. You can then use the `aws:TagKeys` condition key to enforce using specific tags in the `CreateVectorBucket` request. `s3vectors:VectorBucketTag/tag-key` Use this condition key to grant permissions to specific data in vector buckets using tags. This condition key acts on the tags assigned to the vector bucket for all S3 Vectors actions. Even when you create a index with tags, this condition key acts on the tags applied to the vector bucket that contains that index. For example, you could require that access to a bucket is allowed only if the bucket has the attached tag key `Dept` with the value `Marketing`. When accessing indexes, this condition references tags associated with the vector bucket containing that index, while the `aws:ResourceTag/tag-key` will reference the tags of the index itself. ### Example ABAC policies for vector buckets See the following example ABAC policies for Amazon S3 vector buckets. #### 1.1 - IAM policy to create or modify vector buckets with specific tags In this IAM policy, users or roles with this policy can only create S3 vector buckets if they tag the vector bucket with the tag key `project` and tag value `Trinity` in the vector bucket creation request. They can also add or modify tags on existing S3 vector buckets as long as the `TagResource` request includes the tag key-value pair `project:Trinity`. This policy does not grant read, write, or delete permissions on the vector buckets or its objects. ``` { "Version": "2012-10-17", "Statement": [ { "Sid": "CreateVectorBucketWithTags", "Effect": "Allow", "Action": [ "s3vectors:CreateVectorBucket", "s3vectors:TagResource" ], "Resource": "*", "Condition": { "StringEquals": { "aws:RequestTag/project": [ "Trinity" ] } } } ] } ``` #### 1.2 - Vector bucket policy to restrict operations on the vector bucket using tags In this vector bucket policy, IAM principals (users and roles) can perform operations using the `PutVectorBucketPolicy` action on the vector bucket only if the value of the vector bucket's `project` tag matches the value of the principal's `project` tag. ``` { "Version": "2012-10-17", "Statement": [ { "Sid": "AllowObjectOperations", "Effect": "Allow", "Principal": { "AWS": "111122223333" }, "Action": "s3vectors:PutVectorBucketPolicy", "Resource": "arn:aws::s3vectors:us-west-2:111122223333:bucket/amzn-s3-demo-vector-bucket", "Condition": { "StringEquals": { "aws:ResourceTag/project": "${aws:PrincipalTag/project}" } } } ] } ``` #### 1.3 - IAM policy to modify tags on existing resources maintaining tagging governance In this IAM policy, IAM principals (users or roles) can modify tags on a vector bucket only if the value of the vector bucket's `project` tag matches the value of the principal's `project` tag. Only the four tags `project`, `environment`, `owner`, and `cost-center` specified in the `aws:TagKeys` condition keys are permitted for these vector buckets. This helps enforce tag governance, prevents unauthorized tag modifications, and keeps the tagging schema consistent across your vector buckets. ``` { "Version": "2012-10-17", "Statement": [ { "Sid": "EnforceTaggingRulesOnModification", "Effect": "Allow", "Action": [ "s3vectors:TagResource" ], "Resource": "arn:aws::s3vectors:us-west-2:111122223333:bucket/*", "Condition": { "StringEquals": { "aws:ResourceTag/project": "${aws:PrincipalTag/project}" }, "ForAllValues:StringEquals": { "aws:TagKeys": [ "project", "environment", "owner", "cost-center" ] } } } ] } ``` #### 1.4 - Using the `s3vectors:VectorBucketTag` condition key In this IAM policy, the condition statement allows access to the vector bucket's and vector index's operations only if the vector bucket has the tag key `Environment` and tag value `Production`. ``` { "Version": "2012-10-17", "Statement": [ { "Sid": "AllowAccessToTaggedBucket", "Effect": "Allow", "Action": "*", "Resource": "arn:aws::s3vectors:us-west-2:111122223333:bucket/*", "Condition": { "StringEquals": { "s3vectors:VectorBucketTag/Environment": "Production" } } } ] } ``` # Managing tags for vector buckets You can add or manage tags for S3 vector buckets using the Amazon S3 Console, the AWS Command Line Interface (AWS CLI), the AWS SDKs, or using the S3 APIs: [TagResource](https://docs.aws.amazon.com/), [UntagResource](https://docs.aws.amazon.com/), and [ListTagsForResource](https://docs.aws.amazon.com/). For more information, see: **Topics** # Creating vector buckets with tags You can tag Amazon S3 vector buckets when you create them. There is no additional charge for using tags on vector buckets beyond the standard S3 API request rates. For more information, see [Amazon S3 pricing](https://docs.aws.amazon.com/s3/pricing/). For more information about tagging vector buckets, see [Using tags with S3 vector buckets](s3-vectors-tags.md). ## Permissions To create a vector bucket with tags, you must have the following permissions: + `s3vectors:CreateVectorBucket` + `s3vectors:TagResource` ## Troubleshooting errors If you encounter an error when attempting to create a vector bucket with tags, you can do the following: + Verify that you have the required [Permissions](#bucket-tags-permissions) to create the vector bucket and add a tag to it. + Check your IAM user policy for any attribute-based access control (ABAC) conditions. You may be required to label your vector buckets only with specific tag keys and values. For more information, see [Using tags for attribute-based access control (ABAC)](tagging.md#using-tags-for-abac). ## Steps You can create a vector bucket with tags applied by using the Amazon S3 console, the AWS Command Line Interface (AWS CLI), the Amazon S3 REST API, and AWS SDKs. ### Using the S3 console **To create a vector bucket with tags using the Amazon S3 console** 1. Sign in to the AWS Management Console and open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/). 1. In the left navigation pane, choose **vector buckets**. 1. Choose **create vector bucket** to create a new vector bucket. 1. Create a vector bucket as you normally would; see [Creating a vector bucket](s3-vectors-buckets-create.md). 1. On the **Create vector bucket** page, **Tags** is an option when creating a new vector bucket. 1. Enter a name for the vector bucket. 1. Choose **Add new Tag** to open the Tags editor and enter a tag key-value pair. The tag key is required, but the value is optional. 1. To add another tag, select **Add new Tag** again. You can enter up to 50 tag key-value pairs. 1. After you complete specifying the options for your new vector bucket, choose **Create vector bucket**. ### Using the REST API For information about the Amazon S3 REST API support for creating a vector bucket with tags, see the following section in the *Amazon S3 Vectors API Reference*: [CreateVectorBucket](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_CreateVectorBucket.html) ### Using the AWS CLI To install the AWS CLI, see [Installing the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) in the *AWS Command Line Interface User Guide*. The following CLI example shows you how to create a vector bucket with tags by using the AWS CLI. To use the command replace the *user input placeholders* with your own information. When you create a vector bucket you must provide configuration details and use the following naming convention: `example-vector-bucket` ``` aws s3vector create-vector-bucket --vector-bucket-name acc-bucket \ --tags Department=Accounting,Stage=Prod ``` # Adding a tag to a vector bucket You can add tags to Amazon S3 vector buckets and modify these tags. There is no additional charge for using tags on vector buckets beyond the standard S3 API request rates. For more information, see [Amazon S3 pricing](https://docs.aws.amazon.com/s3/pricing/). For more information about tagging vector buckets, see [Using tags with S3 vector buckets](s3-vectors-tags.md). ## Permissions To add a tag to a vector bucket, you must have the following permission: + `s3vectors:TagResource` ## Troubleshooting errors If you encounter an error when attempting to add a tag to a vector bucket, you can do the following: + Verify that you have the required [Permissions](#add-bucket-tag-permissions) to add a tag to a vector bucket. + If you attempted to add a tag key that starts with the AWS reserved prefix `aws:`, change the tag key and try again. ## Steps You can add tags to vector buckets by using the Amazon S3 console, the AWS Command Line Interface (AWS CLI), the Amazon S3 REST API, and AWSSDKs. ### Using the S3 console **To add tags to a vector bucket using the Amazon S3 console** 1. Sign in to the AWS Management Console and open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/). 1. In the left navigation pane, choose **vector buckets**. 1. Choose the vector bucket name. 1. Choose the **Properties** tab. 1. Scroll to the **Tags** section and choose **Add new Tag**. 1. This opens the **Add Tags** page. You can enter up to 50 tag key value pairs. 1. If you add a new tag with the same key name as an existing tag, the value of the new tag overrides the value of the existing tag. 1. You can also edit the values of existing tags on this page. 1. After you have added the tag(s), choose **Save changes**. ### Using the REST API For information about the Amazon S3 REST API support for adding tags to a vector bucket, see the following section in the *Amazon S3 Vectors API Reference*: [TagResource](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_TagResource.html) ### Using the AWS CLI To install the AWS CLI, see [Installing the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) in the *AWS Command Line Interface User Guide*. The following CLI example shows you how to add tags to a vector bucket by using the AWS CLI. To use the command replace the *user input placeholders* with your own information. ``` aws s3vectors tag-resource \ --resource-arn arn:aws:s3vectors:us-east-1:012345678900:bucket/acc-bucket \ --tags Stage=Prod,CostCenter=Marketing ``` # Viewing vector bucket tags You can view or list tags applied to Amazon S3 vector buckets. For more information about tagging vector buckets, see [Using tags with S3 vector buckets](s3-vectors-tags.md). ## Permissions To view tags applied to a vector bucket, you must have the following permission: + `s3vectors:ListTagsForResource` ## Troubleshooting errors If you encounter an error when attempting to list or view the tags of a vector bucket, you can do the following: + Verify that you have the required [Permissions](#view-bucket-tag-permissions) to view or list the tags of the vector bucket. ## Steps You can view tags applied to vector buckets by using the Amazon S3 console, the AWS Command Line Interface (AWS CLI), the Amazon S3 REST API, and AWS SDKs. ### Using the S3 console **To view tags applied to a vector bucket using the Amazon S3 console** 1. Sign in to the AWS Management Console and open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/). 1. In the left navigation pane, choose **vector buckets**. 1. Choose the vector bucket name. 1. Choose the **Properties** tab. 1. Scroll to the **Tags** section to view all of the tags applied to the vector bucket. 1. The **Tags** section shows the User-defined tags by default. You can select the AWS-generated tags tab to view tags applied to your vector bucket by AWS services. ### Using the REST API For information about the Amazon S3 REST API support for viewing the tags applied to a vector bucket, see the following section in the Amazon Simple Vectors API Reference: [ListTagsforResource](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_ListTagsForResource.html) ### Using the AWS CLI To install the AWS CLI, see [Installing the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) in the *AWS Command Line Interface User Guide*. The following CLI example shows you how to view tags applied to a vector bucket. To use the command replace the *user input placeholders* with your own information. ``` aws s3vectors list-tags-for-resource \ --resource-arn arn:aws:s3vectors:us-east-1:012345678900:bucket/acc-bucket ``` # Deleting a tag from a vector bucket You can remove tags from S3 vector buckets. An AWS tag is a key-value pair that holds metadata about resources, in this case Amazon S3 vector buckets. For more information about tagging vector buckets, see [Using tags with S3 vector buckets](s3-vectors-tags.md). **Note** If you delete a tag and later learn that it was being used to track costs or for access control, you can add the tag back to the vector bucket. ## Permissions To delete a tag from a vector bucket, you must have the following permission: + `s3vectors:UntagResource` ## Troubleshooting errors If you encounter an error when attempting to delete a tag from a vector bucket, you can do the following: + Verify that you have the required [Permissions](#delete-bucket-tag-permissions) to delete a tag from a vector bucket. ## Steps You can delete tags from vector buckets by using the Amazon S3 console, the AWS Command Line Interface (AWS CLI), the Amazon S3 REST API, and AWS SDKs. ### Using the S3 console **To delete tags from a vector bucket using the Amazon S3 console** 1. Sign in to the AWS Management Console and open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/). 1. In the left navigation pane, choose **vector buckets**. 1. Choose the vector bucket name. 1. Choose the **Properties** tab. 1. Scroll to the **Tags** section and select the checkbox next to the tag or tags that you would like to delete. 1. Choose **Delete**. 1. The **Delete user-defined tags** pop-up appears and asks you to confirm the deletion of the tag or tags you selected. 1. Choose **Delete** to confirm. ### Using the REST API For information about the Amazon S3 REST API support for deleting tags from a vector bucket, see the following section in the *Amazon S3 Vectors API Reference*: [UntagResource](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_UntagResource.html) ### Using the AWS CLI To install the AWS CLI, see [Installing the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) in the *AWS Command Line Interface User Guide*. The following CLI example shows you how to delete tags from a vector bucket by using the AWS CLI. To use the command replace the *user input placeholders* with your own information. ``` aws s3vectors untag-resource \ --resource-arn arn:aws:s3vectors:us-east-1:012345678900:bucket/acc-bucket \ --tag-keys CostCenter Department ``` # Vector indexes **Note** Choose your vector index configuration parameters carefully. After you create a vector index, you can't update the vector index name, dimension, distance metric, or non-filterable metadata keys. To change any of these values, you must create a new vector index. Vector indexes are resources within vector buckets that store and organize vector data for efficient similarity search operations. When you create a vector index, you specify the distance metric (`Cosine` or `Euclidean`), the number of dimensions that a vector should have, and optionally a list of metadata fields that you want to exclude from filtering during similarity queries. For more information about vector index limits per bucket, vector limits per index, and dimension limits per vector, see [Limitations and restrictions](s3-vectors-limitations.md). Each vector index has a unique Amazon Resource Name (ARN). The ARNs of vector indexes follow the following format: ``` arn:aws:s3vectors:region:account-id:bucket/bucket-name/index/index-name ``` ## Vector index naming requirements + Vector index names must be unique within the vector bucket. + Vector index names must be between 3 and 63 characters long. + Valid characters are lowercase letters (a-z), numbers (0-9), hyphens (-), and dots (.). + Vector index names must begin and end with a letter or number. ## Dimension requirements A dimension is the number of values in a vector. All vectors added to the index must have exactly this number of values. + A dimension must be an integer between 1 and 4096. + A larger dimension requires more storage space. ## Distance metric options Distance metric specifies how similarity between vectors is calculated. When creating vector embeddings, choose your embedding model's recommended distance metric for more accurate results. + **Cosine** – Measures the cosine of the angle between vectors. Best for normalized vectors and when direction matters more than magnitude. + **Euclidean** – Measures the straight-line distance between vectors. Best when both direction and magnitude are important. ## Non-filterable metadata keys Metadata keys allow you to attach additional information to your vectors as key-value pairs during storage and retrieval. By default, all metadata is filterable, so you can use it to filter query results. However, you can designate specific metadata keys as non-filterable when you want to store information with vectors without using it for filtering. Unlike default metadata keys, these keys can't be used as query filters. Non-filterable metadata keys can be retrieved but can't be searched, queried, or filtered. You can only access it after finding the index. Non-filterable metadata keys allow you to enrich vectors with additional context that you want to retrieve with search results but don't need for filtering. A common example of a non-filterable metadata key is when you embed text into vectors and want to include the original text itself as non-filterable metadata. This allows you to return the source text alongside vector search results without increasing your filterable metadata size limits. Other examples include storing creation timestamps, source URLs, or descriptive information purely for reference. Non-filterable metadata keys can be accessed when retrieving vectors but, unlike default metadata keys, these keys can't be used as query filters. Requirements for non-filterable metadata keys are as follows. + Non-filterable metadata keys must be unique within the vector index. + Non-filterable metadata keys must be 1 to 63 characters long. + Non-filterable metadata keys can't be modified after the vector index is created. + S3 Vectors support up to 10 non-filterable metadata keys per index. For more information about non-filterable metadata keys, see [Non-filterable metadata](s3-vectors-metadata-filtering.md#s3-vectors-metadata-filtering-non-filterable). **Topics** + [ ## Vector index naming requirements ](#s3-vectors-indexes-naming) + [ ## Dimension requirements ](#s3-vectors-indexes-dimensions) + [ ## Distance metric options ](#s3-vectors-indexes-distance-metrics) + [ ## Non-filterable metadata keys ](#s3-vectors-indexes-metadata) + [ # Creating a vector index in a vector bucket ](s3-vectors-create-index.md) + [ # Listing vector indexes ](s3-vectors-index-list.md) + [ # Deleting a vector index ](s3-vectors-index-delete.md) + [ # Using tags with S3 vector indexes ](vector-index-tagging.md) # Creating a vector index in a vector bucket **Note** Choose your vector index configuration parameters carefully. After you create a vector index, you can't update the vector index name, dimension, distance metric, or non-filterable metadata keys. To change any of these values, you must create a new vector index. A vector index is a resource within a vector bucket that stores and organizes vector data for efficient similarity search. When you create a vector index, you define the characteristics that all vectors in that index must share, such as the dimension, the distance metric used for similarity calculations, and optionally non-filterable metadata keys. You can also optionally configure dedicated encryption settings and tags for the vector index at the time of index creation. For more information about vector index naming requirements, dimension requirements, distance metric options, and non-filterable metadata keys, see [Limitations and restrictions](s3-vectors-limitations.md). For more information about setting encryption configuration for vector indexes, see [Data protection and encryption in S3 Vectors](s3-vectors-data-encryption.md). For more information about setting tags, see [Using tags with S3 vector buckets](s3-vectors-tags.md). Vector indexes must be created within an existing vector bucket and require specific configuration parameters that can't be modified after creation. ## Using the S3 console **To create a vector index** 1. Open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/). 1. In the navigation pane, choose **Vector buckets**. 1. In the list of vector buckets, choose the name of the bucket where you want to create a vector index. 1. Choose **Create vector index**. 1. For **Vector index name**, enter a name for your vector index. Vector index names must be unique within the vector bucket. Index name must be between 3 and 63 characters. Valid characters are lowercase letters (a-z), numbers (0-9), hyphens (-), and dots (.). For more information about the vector index naming requirements, see [Limitations and restrictions](s3-vectors-limitations.md). 1. For **Dimension**, enter the number of values in each vector. **Note** The value for **Dimension** determines how many numerical values each vector will contain. All vectors added to this index must have exactly this number of values. Dimension must be between 1 and 4096. A larger dimension requires more storage space. Choose based on your embedding model's output dimensions. For more information about the dimension requirements, see [Limitations and restrictions](s3-vectors-limitations.md). 1. For **Distance metric**, choose one of the following options: + **Cosine** – Measures the cosine of the angle between vectors. Best for normalized vectors and when direction matters more than magnitude + **Euclidean** – Measures the straight-line distance between vectors. Best when both direction and magnitude are important. 1. (Optional) Under **Non-filterable metadata**, configure metadata keys that will be stored but not used for filtering: To add non-filterable metadata keys: 1. Choose **Add key**. 1. Enter a key name (1-63 characters and unique within this vector index). 1. Repeat to add additional keys (maximum 10 keys). **Note** You can attach filterable metadata as key-value pairs to each vector when you insert vector data after you create a vector index. By default, all metadata keys that are attached to vectors are filterable and can be used as filters in a similarity query. Only metadata keys that are specified as non-filterable during vector index creation are excluded from filtering. For more information about metadata size limits per vector, including both total and filterable metadata constraints, see [Limitations and restrictions](s3-vectors-limitations.md). 1. Review your configuration carefully. **Note** These settings can't be changed after creation. 1. Under **Encryption**, choose **Specify encryption type**. You have the option to **Use bucket settings for encryption** or override the encryption settings for the vector index. If you override the bucket-level settings, you have the option to specify encryption type for the vector index as **Server-side encryption with AWS Key Management Service keys (SSE-KMS)** or the **Server-side encryption with Amazon S3 managed keys (SSE-S3)**. For more information about setting encryption configuration for vector indexes, see [Data protection and encryption in S3 Vectors](s3-vectors-data-encryption.md). 1. Under **Tags (Optional)**, you can add tags as key-value pairs to help track and organize vector index costs using AWS Billing and Cost Management. Enter a **Key** and a **Value**. To add another tag, choose **Add Tag**. You can enter up to 50 tags for a vector index. For more information, see [Using tags with S3 vector buckets](s3-vectors-tags.md). 1. Choose **Create vector index**. ## Using the AWS CLI To create a vector index in a vector bucket, use the following example commands and replace the `user input placeholders` with your own information. **Example 1: Creating a vector index with non-filterable metadata keys** ``` aws s3vectors create-index \ --vector-bucket-name "amzn-s3-demo-vector-bucket" \ --index-name "idx" \ --data-type "float32" \ --dimension 1 \ --distance-metric "cosine" \ --metadata-configuration '{"nonFilterableMetadataKeys":["nonFilterableKey1"]}' ``` **Example 2: Creating a vector index without non-filterable metadata keys** ``` aws s3vectors create-index \ --vector-bucket-name "amzn-s3-demo-vector-bucket" \ --index-name "idx2" \ --data-type "float32" \ --dimension 4096 \ --distance-metric "euclidean" ``` In addition, all metadata (both filterable and non-filterable) is retrieved the same way by using the `GetVectors`, `ListVectors`, or `QueryVectors` API operations. The following CLI command shows how to retrieve vectors with metadata (including non-filterable metadata). Example request: ``` aws s3vectors get-vectors \ --vector-bucket-name "amzn-s3-demo-vector-bucket" \ --index-name "idx" \ --keys '["vec1", "vec3"]' \ --return-data \ --return-metadata \ ``` Example response: ``` { "vectors": [ { "key": "vec1", "data": { "float32": [ 0.10000000149011612, 0.20000000298023224, 0.30000001192092896, 0.4000000059604645, 0.5 ] }, "metadata": { "category": "test", "text": "First vector" } }, { "key": "vec3", "data": { "float32": [ 0.6000000238418579, 0.699999988079071, 0.800000011920929, 0.8999999761581421, 1.0 ] }, "metadata": { "text": "Third vector", "category": "test" } } ] } ``` The response will include all metadata associated with the vector, regardless of whether it was specified as filterable or non-filterable during index creation. ## Using the AWS SDKs ------ #### [ SDK for Python ] ``` import boto3 # Create a S3 Vectors client in the AWS Region of your choice. s3vectors = boto3.client("s3vectors", region_name="us-west-2") #Create a vector index "movies" in the vector bucket "media-embeddings" without non-filterable metadata keys s3vectors.create_index( vectorBucketName="media-embeddings", indexName="movies", dimension=3, distanceMetric="cosine", dataType = "float32" ) #Create a vector index "movies" in the vector bucket "media-embeddings" with non-filterable metadata keys s3vectors.create_index( vectorBucketName="media-embeddings", indexName="movies", dimension=3, distanceMetric="cosine", dataType = "float32", metadataConfiguration= {"nonFilterableMetadataKeys": ["nonFilterableMetadataKey1"]} ) ``` ------ # Listing vector indexes You can view all vector indexes within a vector bucket. The listing operation supports prefix-based filtering to help you find specific indexes when you have many indexes in a bucket. For more information about `ListIndexes`, prefix limits, and response limits, see [ListIndexes](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_ListIndexes.html) in the Amazon Simple Storage Service API Reference. ## Prefix search capability Prefix search allows you to list indexes that start with a specific prefix, making it easier to organize and find related vector indexes. This is particularly useful when you use naming conventions that group related indexes together: + **By data type:** `text-embeddings-`, `image-features-`, `audio-vectors-` + **By model:** `model1-embeddings-`, `model2-vectors-`, `custom-model-` + **By use case:** `search-index-`, `recommendation-`, `similarity-` + **By environment:** `prod-vectors-`, `staging-vectors-`, `dev-vectors-` ### Using the S3 console **To list vector indexes** 1. Sign in to the AWS Management Console and open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/). 1. In the left navigation pane, choose **Vector buckets**. 1. In the list of vector buckets, choose the name of the bucket containing the indexes you want to view. 1. The console displays a comprehensive list of all vector indexes in the bucket, including: + **Name** – The name for each index. + **Create date** – When the index was created. + **Amazon Resource Name (ARN)** – Full ARN for each index. **To filter the list** 1. Enter an index name or prefix in the search box above the index list. Use prefixes to find groups of related indexes. 1. The list updates in real-time as you type. ### Using the AWS CLI Use the following example commands and replace the *user input placeholders* with your own information. **To list indexes with a specific prefix in a vector bucket** Example request: ``` aws s3vectors list-indexes \ --vector-bucket-name "amzn-s3-demo-bucket" \ --prefix "idx" \ --max-results 1 ``` Example response: ``` { "nextToken": "lObb29ZkzxMGtBXs97Rkbs26xdtKemu4brsnq2jX8DCocADkILv5cRphemXS3PXXFnQBihQBmESgEeKaGA", "indexes": [ { "vectorBucketName": "amzn-s3-demo-bucket", "indexName": "idx", "indexArn": "arn:aws:s3vectors:aws-region:111122223333:bucket/amzn-s3-demo-vector-bucket/index/idx", "creationTime": "2025-06-12T15:50:23+00:00" } ] } ``` **To list indexes with pagination** Example request: ``` aws s3vectors list-indexes \ --vector-bucket-name "amzn-s3-demo-bucket" \ --prefix "idx" \ --next-token "lObb29ZkzxMGtBXs97Rkbs26xdtKemu4brsnq2jX8DCocADkILv5cRphemXS3PXXFnQBihQBmESgEeKaGA" ``` Example response: ``` { "indexes": [ { "vectorBucketName": "amzn-s3-demo-bucket", "indexName": "idx2", "indexArn": "arn:aws:s3vectors:aws-region:111122223333:bucket/amzn-s3-demo-vector-bucket/index/idx2", "creationTime": "2025-06-12T15:45:37+00:00" } ] } ``` ### Using the AWS SDKs ------ #### [ SDK for Python ] ``` import boto3 # Create a S3 Vectors client in the AWS Region of your choice. s3vectors = boto3.client("s3vectors", region_name="us-west-2") #List vector indexes in your vector bucket response = s3vectors.list_indexes(vectorBucketName="media-embeddings") indexes = response["indexes"] print(indexes) ``` ------ # Deleting a vector index You can delete a vector index when you no longer need it. This operation permanently removes the index and all vectors that are stored within it. **Important** When you delete a vector index, you need to know the following: You can delete vector indexes even when the indexes contain vectors. All vectors stored in the index are permanently deleted All metadata associated with those vectors is permanently lost The operation can't be undone or reversed Any ongoing operations on the index will fail immediately Applications querying the index will receive errors The index name becomes available for reuse within the bucket ## Using the AWS CLI Before you delete a vector index, verify the vector index. For more information about how to check the index details, see [GetIndex](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetIndex.html) in the *Amazon S3 API Reference*. For more information about how to list vectors inside the index to see what will be deleted, see [Listing vector indexes](s3-vectors-index-list.md). To delete a vector index, use the following example commands. Replace the *user input placeholders* with your own information. ``` aws s3vectors delete-index --vector-bucket-name "amzn-s3-demo-vector-bucket" \ --index-name "idx2" ``` For more information about how to verify whether the index is deleted, see [Listing vector indexes](s3-vectors-index-list.md). ## Using the AWS SDKs ------ #### [ SDK for Python ] ``` import boto3 # Create a S3 Vectors client in the AWS Region of your choice. s3vectors = boto3.client("s3vectors", region_name="us-west-2") #Delete a vector index response = s3vectors.delete_index( vectorBucketName="media-embeddings", indexName="movies") ``` ------ # Using tags with S3 vector indexes Tagging vector indexes An AWS tag is a key-value pair that holds metadata about resources, in this case Amazon S3 vector indexes. You can tag S3 vector indexes when you create them or manage tags on existing vector indexes. For general information about tags, see [Tagging for cost allocation or attribute-based access control (ABAC)](tagging.md). **Note** There is no additional charge for using tags on vector indexes beyond the standard S3 API request rates. For more information, see [Amazon S3 pricing](https://aws.amazon.com/s3/pricing/). ## Common ways to use tags with vector indexes Use tags on your S3 vector indexes for: + **Cost allocation** – Track storage costs by vector index tag in AWS Billing and Cost Management. For more information, see [Using tags for cost allocation](tagging.md#using-tags-for-cost-allocation). + **Attribute-based access control (ABAC)** – Scale access permissions and grant access to S3 vector indexes based on their tags. For more information, see [Using tags for attribute-based access control (ABAC)](tagging.md#using-tags-for-abac). **Note** You can use the same tags for both cost allocation and access control. ### ABAC for S3 vector indexes Amazon S3 vector indexes support attribute-based access control (ABAC) using tags. Use tag-based condition keys in your AWS organizations, IAM, and S3 vector index policies. For enterprises, ABAC inAmazon S3 supports authorization across multiple AWS accounts. In your IAM policies, you can control access to S3 vector indexes based on the vector index's tags by using the following global condition keys: `aws:ResourceTag/key-name` Use this key to compare the tag key-value pair that you specify in the policy with the key-value pair attached to the resource. For example, you could require that access to a resource is allowed only if the resource has the attached tag key `Dept` with the value `Marketing`. For more information, see [Controlling access to AWS resources](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_tags.html#access_tags_control-resources). `aws:RequestTag/key-name` Use this key to compare the tag key-value pair that was passed in the request with the tag pair that you specify in the policy. For example, you could check whether the request includes the tag key `Dept` and that it has the value `Accounting`. For more information, see [Controlling access during AWS requests](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_tags.html#access_tags_control-requests). You can use this condition key to restrict which tag key-value pairs can be passed during the `TagResource` and `CreateIndex` API operations. `aws:TagKeys` Use this key to compare the tag keys in a request with the keys that you specify in the policy. We recommend that when you use policies to control access using tags, use the `aws:TagKeys` condition key to define what tag keys are allowed. For example policies and more information, see [Controlling access based on tag keys](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_tags.html#access_tags_control-tag-keys). You can create an S3 vector index with tags. To allow tagging during the `CreateVectorBucket` API operation, you must create a policy that includes both the `s3vectors:TagResource` and `s3vectors:CreateVectorBucket` actions. You can then use the `aws:TagKeys` condition key to enforce using specific tags in the `CreateVectorBucket` request. ### Example ABAC policies for vector indexes See the following example ABAC policies for Amazon S3 vector indexes. #### 1.1 - IAM policy to create or modify vector indexes with specific tags In this IAM policy, users or roles with this policy can only create S3 vector indexes if they tag the vector index with the tag key `project` and tag value `Trinity` in the vector index creation request. They can also add or modify tags on existing S3 vector indexes as long as the `TagResource` request includes the tag key-value pair `project:Trinity`. This policy does not grant read, write, or delete permissions on the vector indexes or its objects. ``` { "Version": "2012-10-17", "Statement": [ { "Sid": "CreateVectorIndexWithTags", "Effect": "Allow", "Action": [ "s3vectors:CreateIndex", "s3vectors:TagResource" ], "Resource": "*", "Condition": { "StringEquals": { "aws:RequestTag/project": [ "Trinity" ] } } } ] } ``` #### 1.2 - IAM policy to modify tags on existing resources maintaining tagging governance In this IAM policy, IAM principals (users or roles) can modify tags on a vector index only if the value of the vector index's `project` tag matches the value of the principal's `project` tag. Only the four tags `project`, `environment`, `owner`, and `cost-center` specified in the `aws:TagKeys` condition keys are permitted for these vector indexes. This helps enforce tag governance, prevents unauthorized tag modifications, and keeps the tagging schema consistent across your vector indexes. ``` { "Version": "2012-10-17", "Statement": [ { "Sid": "EnforceTaggingRulesOnModification", "Effect": "Allow", "Action": [ "s3vectors:TagResource" ], "Resource": "arn:aws::s3vectors:us-west-2:111122223333:bucket/*", "Condition": { "StringEquals": { "aws:ResourceTag/project": "${aws:PrincipalTag/project}" }, "ForAllValues:StringEquals": { "aws:TagKeys": [ "project", "environment", "owner", "cost-center" ] } } } ] } ``` # Managing tags for vector indexes You can add or manage tags for S3 vector indexes using the Amazon S3 Console, the AWS Command Line Interface (AWS CLI), the AWS SDKs, or using the S3 APIs: [TagResource](https://docs.aws.amazon.com/), [UntagResource](https://docs.aws.amazon.com/), and [ListTagsForResource](https://docs.aws.amazon.com/). For more information, see: **Topics** + [ # Creating vector indexes with tags ](creating-vector-indexes-with-tags.md) + [ # Adding a tag to a vector index ](adding-tag-vector-index.md) + [ # Viewing vector index tags ](viewing-vector-index-tags.md) + [ # Deleting a tag from a vector index ](deleting-tag-vector-index.md) # Creating vector indexes with tags You can tag Amazon S3 vector indexes when you create them. There is no additional charge for using tags on vector indexes beyond the standard S3 API request rates. For more information, see [Amazon S3 pricing](https://docs.aws.amazon.com/s3/pricing/). For more information about tagging vector indexes, see [Using tags with S3 vector indexes](vector-index-tagging.md). ## Permissions To create a vector index with tags, you must have the following permissions: + `s3vectors:CreateIndex` + `s3vectors:TagResource` ## Troubleshooting errors If you encounter an error when attempting to create a vector index with tags, you can do the following: + Verify that you have the required [Permissions](#index-tags-permissions) to create the vector index and add a tag to it. + Check your IAM user policy for any attribute-based access control (ABAC) conditions. You may be required to label your vector indexes only with specific tag keys and values. For more information, see [Using tags for attribute-based access control (ABAC)](tagging.md#using-tags-for-abac). ## Steps You can create a vector index with tags applied by using the Amazon S3 console, the AWS Command Line Interface (AWS CLI), the Amazon S3 REST API, and AWS SDKs. ### Using the S3 console **To create a vector index with tags using the Amazon S3 console** 1. Sign in to the AWS Management Console and open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/). 1. In the left navigation pane, choose **vector indexes**. 1. Choose **create vector index** to create a new vector index. 1. Create a vector index as you normally would; see [Creating a vector index in a vector bucket](s3-vectors-create-index.md). 1. On the **Create vector index** page, **Tags** is an option when creating a new vector index. 1. Enter a name for the vector index. 1. Choose **Add new Tag** to open the Tags editor and enter a tag key-value pair. The tag key is required, but the value is optional. 1. To add another tag, select **Add new Tag** again. You can enter up to 50 tag key-value pairs. 1. After you complete specifying the options for your new vector index, choose **Create vector index**. ### Using the REST API For information about the Amazon S3 REST API support for creating a vector index with tags, see the following section in the *Amazon S3 Vectors API Reference*: [CreateIndex](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_CreateIndex.html) ### Using the AWS CLI To install the AWS CLI, see [Installing the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) in the *AWS Command Line Interface User Guide*. The following CLI example shows you how to create a vector index with tags by using the AWS CLI. To use the command replace the *user input placeholders* with your own information. When you create a vector index you must provide configuration details and use the following naming convention: `example-vector-index` ``` aws s3vectors create-index --vector-bucket-name acc-bucket --data-type "float32" \ --index-name accounts-index --dimension 1024 --distance-metric euclidean \ --tags Department=Accounting,Stage=Prod ``` # Adding a tag to a vector index You can add tags to Amazon S3 vector indexes and modify these tags. There is no additional charge for using tags on vector indexes beyond the standard S3 API request rates. For more information, see [Amazon S3 pricing](https://docs.aws.amazon.com/s3/pricing/). For more information about tagging vector indexes, see [Using tags with S3 vector indexes](vector-index-tagging.md). ## Permissions To add a tag to a vector index, you must have the following permission: + `s3vectors:TagResource` ## Troubleshooting errors If you encounter an error when attempting to add a tag to a vector index, you can do the following: + Verify that you have the required [Permissions](#add-index-tag-permissions) to add a tag to a vector index. + If you attempted to add a tag key that starts with the AWS reserved prefix `aws:`, change the tag key and try again. ## Steps You can add tags to vector indexes by using the Amazon S3 console, the AWS Command Line Interface (AWS CLI), the Amazon S3 REST API, and AWSSDKs. ### Using the S3 console **To add tags to a vector index using the Amazon S3 console** 1. Sign in to the AWS Management Console and open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/). 1. In the left navigation pane, choose **vector indexes**. 1. Choose the vector index name. 1. Choose the **Properties** tab. 1. Scroll to the **Tags** section and choose **Add new Tag**. 1. This opens the **Add Tags** page. You can enter up to 50 tag key value pairs. 1. If you add a new tag with the same key name as an existing tag, the value of the new tag overrides the value of the existing tag. 1. You can also edit the values of existing tags on this page. 1. After you have added the tag(s), choose **Save changes**. ### Using the REST API For information about the Amazon S3 REST API support for adding tags to a vector index, see the following section in the *Amazon S3 Vectors API Reference*: [TagResource](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_TagResource.html) ### Using the AWS CLI To install the AWS CLI, see [Installing the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) in the *AWS Command Line Interface User Guide*. The following CLI example shows you how to add tags to a vector index by using the AWS CLI. To use the command replace the *user input placeholders* with your own information. ``` aws s3vectors tag-resource \ --resource-arn arn:aws:s3vectors:us-east-1:012345678900:bucket/acc-bucket/index/accounts-index \ --tags Stage=Prod,CostCenter=Marketing ``` # Viewing vector index tags You can view or list tags applied to Amazon S3 vector indexes. For more information about tagging vector indexes, see [Using tags with S3 vector indexes](vector-index-tagging.md). ## Permissions To view tags applied to a vector index, you must have the following permission: + `s3vectors:ListTagsForResource` ## Troubleshooting errors If you encounter an error when attempting to list or view the tags of a vector index, you can do the following: + Verify that you have the required [Permissions](#view-index-tag-permissions) to view or list the tags of the vector index. ## Steps You can view tags applied to vector indexes by using the Amazon S3 console, the AWS Command Line Interface (AWS CLI), the Amazon S3 REST API, and AWS SDKs. ### Using the S3 console **To view tags applied to a vector index using the Amazon S3 console** 1. Sign in to the AWS Management Console and open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/). 1. In the left navigation pane, choose **vector indexes**. 1. Choose the vector index name. 1. Choose the **Properties** tab. 1. Scroll to the **Tags** section to view all of the tags applied to the vector index. 1. The **Tags** section shows the User-defined tags by default. You can select the AWS-generated tags tab to view tags applied to your vector index by AWS services. ### Using the REST API For information about the Amazon S3 REST API support for viewing the tags applied to a vector index, see the following section in the Amazon Simple Vectors API Reference: [ListTagsforResource](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_ListTagsForResource.html) ### Using the AWS CLI To install the AWS CLI, see [Installing the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) in the *AWS Command Line Interface User Guide*. The following CLI example shows you how to view tags applied to a vector index. To use the command replace the *user input placeholders* with your own information. ``` aws s3vectors list-tags-for-resource \ --resource-arn arn:aws:s3vectors:us-east-1:012345678900:bucket/acc-bucket/index/accounts-index ``` # Deleting a tag from a vector index You can remove tags from S3 vector indexes. An AWS tag is a key-value pair that holds metadata about resources, in this case Amazon S3 vector indexes. For more information about tagging vector indexes, see [Using tags with S3 vector indexes](vector-index-tagging.md). **Note** If you delete a tag and later learn that it was being used to track costs or for access control, you can add the tag back to the vector index. ## Permissions To delete a tag from a vector index, you must have the following permission: + `s3vectors:UntagResource` ## Troubleshooting errors If you encounter an error when attempting to delete a tag from a vector index, you can do the following: + Verify that you have the required [Permissions](#delete-index-tag-permissions) to delete a tag from a vector index. ## Steps You can delete tags from vector indexes by using the Amazon S3 console, the AWS Command Line Interface (AWS CLI), the Amazon S3 REST API, and AWS SDKs. ### Using the S3 console **To delete tags from a vector index using the Amazon S3 console** 1. Sign in to the AWS Management Console and open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/). 1. In the left navigation pane, choose **vector indexes**. 1. Choose the vector index name. 1. Choose the **Properties** tab. 1. Scroll to the **Tags** section and select the checkbox next to the tag or tags that you would like to delete. 1. Choose **Delete**. 1. The **Delete user-defined tags** pop-up appears and asks you to confirm the deletion of the tag or tags you selected. 1. Choose **Delete** to confirm. ### Using the REST API For information about the Amazon S3 REST API support for deleting tags from a vector index, see the following section in the *Amazon S3 Vectors API Reference*: [UntagResource](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_UntagResource.html) ### Using the AWS CLI To install the AWS CLI, see [Installing the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) in the *AWS Command Line Interface User Guide*. The following CLI example shows you how to delete tags from a vector index by using the AWS CLI. To use the command replace the *user input placeholders* with your own information. ``` aws s3vectors untag-resource \ --resource-arn arn:aws:s3vectors:us-east-1:012345678900:bucket/acc-bucket/index/accounts-index \ --tag-keys CostCenter Department ``` # Vectors Each vector consists of a key, which uniquely identifies each vector in a vector index. Additionally, you can attach metadata (for example, year, author, genre, location) as key value pairs to each vector. Vector data operations include inserting, listing, querying, and deleting vectors. To generate new vector embeddings of your unstructured data, you can use the [InvokeModel](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html) API operation from Amazon Bedrock to specify the model ID of the embedding model that you want to use. Additionally, the open-source Amazon S3 Vectors Embed CLI tool provides a simplified way to generate embeddings and perform semantic searches from the command line. For more information about this open source tool that automates both vector embedding generation with Amazon Bedrock foundation models and semantic search operations within your S3 vector indexes, see [Creating vector embeddings and performing semantic searches with `s3vectors-embed-cli`](s3-vectors-cli.md). ## Vector concepts **Vector keys**: Each vector is identified by a unique vector key within the index. Vector keys can be up to 1,024 characters long and must be unique within the vector index. Keys are case-sensitive and can contain any UTF-8 characters. **Vector dimension**: A dimension is the number of values in a vector. Larger dimensions require more storage space. All vectors in an index must have the same number of dimensions, which is specified when you create the index. A dimension must be an integer between 1 and 4096. **Metadata**: You can attach metadata to vectors as key-value pairs to provide additional context and enable filtering during queries. Metadata includes both filterable and non-filterable metadata keys. Filterable metadata is used for query filtering. Non-filterable metadata keys are specified during a vector index creation and provides additional context but can’t be used for filtering. Metadata supports string, number, and boolean types. For more information about filterable and non-filterable metadata, see [Metadata filtering](s3-vectors-metadata-filtering.md). For more information about metadata limits, including size limits per vector and maximum metadata keys per vector, see [Limitations and restrictions](s3-vectors-limitations.md). **Topics** + [ ## Vector concepts ](#s3-vectors-concepts) + [ # Inserting vectors into a vector index ](s3-vectors-index-create.md) + [ # Listing vectors ](s3-vectors-list.md) + [ # Querying vectors ](s3-vectors-query.md) + [ # Deleting vectors from a vector index ](s3-vectors-delete.md) + [ # Metadata filtering ](s3-vectors-metadata-filtering.md) # Inserting vectors into a vector index Inserting vectors into a vector index You can add vectors to a vector index with the [PutVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_PutVectors.html) API operation. Each vector consists of a key, which uniquely identifies each vector in a vector index. If you put a vector with a key that already exists in the index, it will overwrite the existing vector completely, which makes the previous vector no longer searchable. To maximize write throughput and optimize for costs, it's recommended that you insert vectors in large batches, up to the maximum batch size for `PutVectors`. However, for workloads that need to use smaller batches - such as when live, incoming vector data must become immediately searchable - you can achieve higher write throughput by using a higher number of concurrent `PutVectors` requests, up to the maximum allowed requests per second limit. For more information about the maximum batch size for `PutVectors`, which is the limit of vectors per `PutVectors` API call, and the maximum requests and vectors per second limit, see [Limitations and restrictions](s3-vectors-limitations.md). Additionally, you can attach metadata (for example, year, author, genre, location) as key-value pairs to each vector. By default, all metadata keys that are attached to vectors are filterable and can be used as filters in a similarity query. Only metadata keys that are specified as non-filterable during vector index creation are excluded from filtering. S3 vector indexes support string, number, boolean, and list types of metadata. For more information about the total metadata size limit per vector and the filterable metadata size limit per vector, see [Limitations and restrictions](s3-vectors-limitations.md). If the metadata size exceeds these limits, the `PutVectors` API operation will return a `400 Bad Request` error. Before adding vector data to your vector index with the `PutVectors` API operation, you need to convert your raw data into vector embeddings, which are numerical representations of your content as arrays of floating-point numbers. The vector embeddings capture the semantic meaning of your content, enabling similarity searches once they're stored in your vector index through the `PutVectors` operation. You can generate vector embeddings using various methods depending on your data type and use case. These methods include using machine learning frameworks, specialized embedding libraries, or AWS services such as Amazon Bedrock. For example, if you're using Amazon Bedrock, you can generate embeddings with the [InvokeModel](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html) API operation and your preferred embedding model. Additionally, Amazon Bedrock Knowledge Bases provides a fully managed end-to-end RAG workflow where Amazon Bedrock automatically fetches data from your S3 data source, converts content into text blocks, generates embeddings, and stores them in your vector index. You can then query the knowledge base and generate responses based on chunks retrieved from your source data. Furthermore, the open-source Amazon S3 Vectors Embed CLI tool provides a simplified way to generate embeddings and perform semantic searches from the command line. For more information about this open source tool that automates both vector embedding generation with Amazon Bedrock foundation models and semantic search operations within your S3 vector indexes, see [Creating vector embeddings and performing semantic searches with `s3vectors-embed-cli`](s3-vectors-cli.md). **Note** When inserting vector data into your vector index, you must provide the vector data as `float32` (32-bit floating point) values. If you pass higher-precision values to an AWS SDK, S3 Vectors converts the values to 32-bit floating point before storing them, and [GetVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_GetVectors.html), [ListVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_ListVectors.html), and [QueryVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_QueryVectors.html) operations return the `float32` values. Different AWS SDKs may have different default numeric types, so ensure your vectors are properly formatted as `float32` values regardless of which SDK you're using. For example, in Python, use `numpy.float32` or explicitly cast your values. ## Using the AWS SDKs ------ #### [ SDK for Python ] ``` # Populate a vector index with embeddings from Amazon Titan Text Embeddings V2. import boto3 import json # Create Bedrock Runtime and S3 Vectors clients in the AWS Region of your choice. bedrock = boto3.client("bedrock-runtime", region_name="us-west-2") s3vectors = boto3.client("s3vectors", region_name="us-west-2") # Texts to convert to embeddings. texts = [ "Star Wars: A farm boy joins rebels to fight an evil empire in space", "Jurassic Park: Scientists create dinosaurs in a theme park that goes wrong", "Finding Nemo: A father fish searches the ocean to find his lost son" ] # Generate vector embeddings. embeddings = [] for text in texts: response = bedrock.invoke_model( modelId="amazon.titan-embed-text-v2:0", body=json.dumps({"inputText": text}) ) # Extract embedding from response. response_body = json.loads(response["body"].read()) embeddings.append(response_body["embedding"]) # Write embeddings into vector index with metadata. s3vectors.put_vectors( vectorBucketName="media-embeddings", indexName="movies", vectors=[ { "key": "Star Wars", "data": {"float32": embeddings[0]}, "metadata": {"source_text": texts[0], "genre":"scifi"} }, { "key": "Jurassic Park", "data": {"float32": embeddings[1]}, "metadata": {"source_text": texts[1], "genre":"scifi"} }, { "key": "Finding Nemo", "data": {"float32": embeddings[2]}, "metadata": {"source_text": texts[2], "genre":"family"} } ] ) ``` ------ # Listing vectors Listing vectors You can list vectors in a vector index with the [ListVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_ListVectors.html) API operation. For more information about the maximum number of vectors that can be returned per page, see [Limitations and restrictions](s3-vectors-limitations.md). The response includes a pagination token when results are truncated. For more information about the response elements of `ListVectors`, see [ListVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_ListVectors) in the *Amazon S3 API Reference*. You can also use `ListVectors` to export vector data from a specified vector index. `ListVectors` is strongly consistent. After a WRITE operation, you can immediately list vectors with all changes reflected. ## Using the AWS CLI To list vectors, use the following example commands. Replace the *user input placeholders* with your own information. The `segment-count` and `segment-index` parameters allow you to partition your listing operations across multiple parallel requests. When you specify a `segment-count` value (such as `2`), you divide the index into that many segments. The `segment-index` parameter (starting from 0) determines which segment to list. This approach helps improve performance when listing large vector indexes by enabling parallel processing. For more information about `segment-count` and `segment-index`, see [ListVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_ListVectors) in the *Amazon S3 API Reference*. **To list all vectors in an index** Example request: ``` aws s3vectors list-vectors \ --vector-bucket-name "amzn-s3-demo-vector-bucket" \ --index-name "idx" \ --segment-count 2 \ --segment-index 0 \ --return-data \ --return-metadata ``` Example response: ``` { "vectors": [ { "key": "vec3", "data": { "float32": [0.4000000059604645] }, "metadata": { "nonFilterableKey": "val4", "filterableKey": "val2" } } ] } ``` **To list vectors with pagination** Example request: ``` aws s3vectors list-vectors \ --vector-bucket-name "amzn-s3-demo-vector-bucket" \ --index-name "idx" \ --segment-count 2 \ --segment-index 0 \ --return-data \ --return-metadata \ --next-token "zWfh7e57H2jBfBtRRmC7OfMwl209G9dg3j2qM6kM4t0rps6ClYzJykgMOil9eGqU5nhf_gTq53IfoUdTnsg" ``` Example response: ``` { "vectors": [ { "key": "vec1", "data": { "float32": [0.5] }, "metadata": { "nonFilterableKey": "val2", "filterableKey": "val1" } } ] } ``` ## Using the AWS SDKs ------ #### [ SDK for Python ] Example: List vectors in a vector index ``` import boto3 # Create a S3 Vectors client in the AWS Region of your choice. s3vectors = boto3.client("s3vectors", region_name="us-west-2") #List vectors in your vector index response = s3vectors.list_vectors( vectorBucketName="media-embeddings", indexName="movies", maxResults = 600, returnData = True, returnMetadata = True ) vectors = response["vectors"] print(vectors) ``` Example: List all vectors in a vector index in parallel ``` import boto3 # Create a S3 Vectors client in the AWS Region of your choice. s3vectors = boto3.client("s3vectors", region_name="us-west-2") #List vectors in the 1st half of vectors in the index. response = s3vectors.list_vectors( vectorBucketName="media-embeddings", indexName="movies", segmentCount=2, segmentIndex=1, maxResults = 600, returnData = True, returnMetadata = True ) vectors = response["vectors"] #List vectors starting from the 2nd half of vectors in the index. # This can be ran in parallel with the first `list_vectors` call. response = s3vectors.list_vectors( vectorBucketName="media-embeddings", indexName="movies", segmentCount=2, segmentIndex=1, maxResults = 600, returnData = True, returnMetadata = True ) vectors = response["vectors"] print(vectors) ``` ------ # Querying vectors Querying vectors You can run a similarity query with the [QueryVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_QueryVectors.html) API operation, where you specify the query vector, the number of relevant results to return (the top K nearest neighbors), and the index ARN. Additionally, you can use metadata filters in a query, to search only the vectors that match the filter. If you make a request to filter on a non-filterable metadata field, the request will return a `400 Bad Request` error. For more information about metadata filtering, see [Metadata filtering](s3-vectors-metadata-filtering.md). In the response, the vector keys are returned by default. You can optionally include the distance and metadata in the response. When generating the query vector, you should use the same vector embedding model that was used to generate the initial vectors that are stored in the vector index. For example, if you use the Amazon Titan Text Embeddings V2 model in Amazon Bedrock to generate vector embeddings of your documents, use the same embedding model to convert a question to a query vector. Additionally, Amazon Bedrock Knowledge Bases provides a fully managed end-to-end RAG workflow where Amazon Bedrock automatically fetches data from your S3 data source, converts content into text blocks, generates embeddings, and stores them in your vector index. You can then query the knowledge base and generate responses based on chunks retrieved from your source data. For more information about how to query vectors from an Amazon Bedrock knowledge base in the console, see [(Optional) Integrate S3 Vectors with Amazon Bedrock Knowledge Bases](s3-vectors-getting-started.md#s3-vectors-bedrock-kb-tutorial). Furthermore, the open-source Amazon S3 Vectors Embed CLI tool provides a simplified way to perform semantic searches from the command line. This open source tool streamlines the query process by handling both the vector embedding generation with Amazon Bedrock foundation models and executing semantic search operations against your S3 vector indexes. For more information about using this tool for querying your vector data, see [Creating vector embeddings and performing semantic searches with `s3vectors-embed-cli`](s3-vectors-cli.md). S3 Vectors delivers sub-second response times for cold queries, leveraging Amazon S3 elastic throughput to efficiently search across millions of vectors. This makes it highly cost-effective for workloads with infrequent queries. For warm queries, S3 Vectors can deliver response times as low as 100ms, benefiting workloads with repeated or frequent query patterns. For performing similarity queries for your vector embeddings, several factors can affect average recall performance, including the vector embedding model, the size of the vector dataset (the number of vectors and dimensions), and the distribution of queries. S3 Vectors delivers 90%\$1 average recall for most datasets. Average recall measures the quality of query results. A 90% average recall means that the response contains 90% of the actual closest vectors (ground truth) that are stored in the vector index relative to the query vector. However, because actual performance may vary depending on your specific use cases, we recommend conducting your own tests with representative data and queries to validate that S3 Vectors meet your recall requirements. ## Using the AWS SDKs ------ #### [ SDK for Python ] ``` # Query a vector index with an embedding from Amazon Titan Text Embeddings V2. import boto3 import json # Create Bedrock Runtime and S3 Vectors clients in the AWS Region of your choice. bedrock = boto3.client("bedrock-runtime", region_name="us-west-2") s3vectors = boto3.client("s3vectors", region_name="us-west-2") # Query text to convert to an embedding. input_text = "adventures in space" # Generate the vector embedding. response = bedrock.invoke_model( modelId="amazon.titan-embed-text-v2:0", body=json.dumps({"inputText": input_text}) ) # Extract embedding from response. model_response = json.loads(response["body"].read()) embedding = model_response["embedding"] # Query vector index. response = s3vectors.query_vectors( vectorBucketName="media-embeddings", indexName="movies", queryVector={"float32": embedding}, topK=3, returnDistance=True, returnMetadata=True ) print(json.dumps(response["vectors"], indent=2)) # Query vector index with a metadata filter. response = s3vectors.query_vectors( vectorBucketName="media-embeddings", indexName="movies", queryVector={"float32": embedding}, topK=3, filter={"genre": "scifi"}, returnDistance=True, returnMetadata=True ) print(json.dumps(response["vectors"], indent=2)) ``` ------ # Deleting vectors from a vector index You can delete specific vectors from a vector index by specifying their vector keys using the [DeleteVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_DeleteVectors.html) API. This operation is useful for removing outdated or incorrect data while preserving the rest of your vector data. ## Using the AWS CLI To delete vectors, use the following example commands. Replace the *user input placeholders* with your own information. ``` aws s3vectors delete-vectors \ --vector-bucket-name "amzn-s3-demo-vector-bucket" \ --index-name "idx" \ --keys '["vec2","vec3"]' ``` ## Using the AWS SDKs ------ #### [ SDK for Python ] ``` import boto3 # Create a S3 Vectors client in the AWS Region of your choice. s3vectors = boto3.client("s3vectors", region_name="us-west-2") #Delete vectors in a vector index response = s3vectors.delete_vectors( vectorBucketName="media-embeddings", indexName="movies", keys=["Star Wars", "Finding Nemo"]) ``` ------ # Metadata filtering Metadata filtering allows you to filter query results based on specific attributes attached to your vectors. You can use metadata filters with query operations to find vectors that match both similarity criteria and specific metadata conditions. S3 Vectors supports two types of metadata: filterable metadata and non-filterable metadata. The key difference is that filterable metadata can be used in query filters but has stricter size limitations, while non-filterable metadata can't be used in filters but can store larger amounts of data within its size limits. For more information about metadata limits, including size limits per vector and maximum metadata keys per vector, see [Limitations and restrictions](s3-vectors-limitations.md). S3 Vectors performs vector search and filter evaluation in tandem. S3 Vectors searches through candidate vectors in the index to find the top K similar vectors while simultaneously validating if each candidate vector matches your metadata filter conditions. For example, if you search for similar movie embeddings and you filter by genre='mystery', S3 Vectors only returns similar movie embeddings where the genre metadata matches 'mystery'. In contrast to applying the metadata filter after the vector search, this filtering approach is more likely to find matching results. Note: queries with filters may return fewer than top K results when the vector index contains very few matching results. **Topics** + [ ## Filterable metadata ](#s3-vectors-metadata-filtering-filterable) + [ ## Examples of valid filterable metadata ](#s3-vectors-metadata-filtering-examples) + [ ## Non-filterable metadata ](#s3-vectors-metadata-filtering-non-filterable) ## Filterable metadata Filterable metadata allows you to filter query results based on specific metadata values. By default, all metadata fields are filterable in a similarity query unless explicitly specified as non-filterable during vector index creation. S3 Vectors supports string, number, boolean, and list types of metadata with a size limit per vector. The metadata type is ideal for attributes that you want to filter on, such as categories, timestamps, or status values. If the metadata size exceeds the supported limits, the [PutVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_PutVectors.html) API operation will return a `400 Bad Request` error. For more information about the filterable metadata size limit per vector, see [Limitations and restrictions](s3-vectors-limitations.md). The following operations can be used with filterable metadata. | Operator | Valid Input Types | Description | | --- | --- | --- | | \$1eq | String, Number, Boolean | Exact match comparison for single values.When comparing with an array metadata value, returns true if the input value matches any element in the array. For example, `{"category": {"$eq": "documentary"}}` would match a vector with metadata `"category": ["documentary", "romance"]`. | | \$1ne | String, Number, Boolean | Not equal comparison | | \$1gt | Number | Greater than comparison | | \$1gte | Number | Greater than or equal comparison | | \$1lt | Number | Less than comparison | | \$1lte | Number | Less than or equal comparison | | \$1in | Non-empty array of primitives | Match any value in array | | \$1nin | Non-empty array of primitives | Match none of the values in array | | \$1exists | Boolean | Check if field exists | | \$1and | Non-empty array of filters | Logical AND of multiple conditions | | \$1or | Non-empty array of filters | Logical OR of multiple conditions | ## Examples of valid filterable metadata **Simple equality** ``` {"genre": "documentary"} ``` This filter matches vectors where the genre metadata key equals "documentary". When you don't specify an operator, S3 Vectors automatically uses the \$1eq operator. **Explicit equality** ``` // Example: Exact match {"genre": {"$eq": "documentary"}} ``` ``` // Example: Not equal to {"genre": {"$ne": "drama"}} ``` **Numeric comparison** ``` {"year": {"$gt": 2019}} ``` ``` {"year": {"$gte": 2020}} ``` ``` {"year": {"$lt": 2020}} ``` ``` {"year": {"$lte": 2020}} ``` **Array operations** ``` {"genre": {"$in": ["comedy", "documentary"]}} ``` ``` {"genre": {"$nin": ["comedy", "documentary"]}} ``` **Existence check** ``` {"genre": {"$exists": true}} ``` The `$exists` filter matches vectors that have a "genre" metadata key, regardless of the value that's stored for that metadata key. **Logical operations** ``` {"$and": [{"genre": {"$eq": "drama"}}, {"year": {"$gte": 2020}}]} ``` ``` {"$or": [{"genre": {"$eq": "drama"}}, {"year": {"$gte": 2020}}]} ``` **Price range (Multiple conditions on the same field)** ``` {"price": {"$gte": 10, "$lte": 50}} ``` For more information about how to query vectors with metadata filtering, see [Metadata filtering](s3-vectors.md#s3-vectors-filtering-metadata). ## Non-filterable metadata Non-filterable metadata can't be used in query filters but can store larger amounts of contextual data than filterable metadata. It's ideal for storing large text chunks, detailed descriptions, or other contextual information that doesn't need to be searchable but can be returned with query results. For example, you might store full document text, image descriptions, or detailed product specifications as non-filterable metadata. Non-filterable metadata keys must be explicitly configured during vector index creation. Once a metadata key is designated as non-filterable during index creation, it can't be changed to filterable later. You can configure multiple metadata keys as non-filterable per vector index, with each metadata key name limited to 63 characters. For more information about the maximum number of non-filterable metadata keys that's allowed per vector index, see [Limitations and restrictions](s3-vectors-limitations.md). While you can't filter on non-filterable metadata, you can retrieve it alongside query results using the `return-metadata` parameter. You can use non-filterable metadata for some use cases as follows. + Use it to provide context for your application without parsing separate data sources. + Store larger text chunks that would exceed the filterable metadata size limits. + Include it in vector exports by using the [ListVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_ListVectors.html) API operation. For more information about configuring non-filterable metadata, see [Creating a vector index in a vector bucket](s3-vectors-create-index.md). # Limitations and restrictions Limitations and restrictions Amazon S3 Vectors has certain limitations and restrictions that you should be aware of when planning your vector storage and search applications. + Vector buckets per AWS Region in an account: 10,000 + Vector indexes per vector bucket: 10,000 + Vectors per vector index: Up to 2 billion + Dimension value per vector: 1 to 4,096 + Total metadata per vector: Up to 40 KB (filterable \$1 non-filterable) + Total metadata keys per vector: Up to 50 + Filterable metadata per vector: Up to 2 KB + Non-filterable metadata keys per vector index: Up to 10 + Combined PutVectors and DeleteVectors requests per second per vector index: Up to 1,000 + Combined vectors inserted and deleted per second per vector index: Up to 2,500 + Request payload size: Up to 20 MiB + Vectors per [PutVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_PutVectors.html) API call: Up to 500 + Vectors per [DeleteVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_DeleteVectors.html) API call: Up to 500 + Vectors per [GetVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_GetVectors.html) API call: Up to 100 + Top-K results per [QueryVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_QueryVectors.html) request: Up to 100 + Vectors listed per page in a [ListVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_ListVectors.html) response: Up to 1,000 + Vector buckets listed per page in a [ListVectorBuckets](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_ListVectorBuckets.html) response: Up to 500. + Vector indexes listed per page in a [ListIndexes](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_ListIndexes.html) response: Up to 500. + Segment count for parallel listing in a ListVectors API call: Up to 16 # S3 Vectors best practices S3 Vectors best practices Amazon S3 Vectors delivers purpose-built, cost-optimized vector storage for use by AI-enabled applications and semantic search of your content stored in Amazon S3. Designed to provide S3 level elasticity and durability for storing vector datasets with sub-second query performance for cold queries and as low as 100 milliseconds for warm queries, S3 Vectors is ideal for applications that need to build and grow vector indexes. With S3 Vectors you can use a dedicated set of API operations to store, access, and perform similarity queries on vector data without provisioning any infrastructure. For more information, see [Working with S3 Vectors and vector buckets](s3-vectors.md). To ensure the maximum benefit from S3 Vectors, we recommend that you perform the following best practices. **Inserting and deleting vectors** Your application can achieve up to one thousand [PutVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_PutVectors.html) or [DeleteVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_GetVectors.html) requests per second per vector index, or can insert or delete up to two thousand five hundred vectors per second per vector index — whichever limit is reached first. If you exceed the request rates, you might receive a `429 TooManyRequestsException` error. To optimize costs, we recommend inserting and deleting vectors in large batches, up to the maximum batch size of 500 vectors per API request. If your workload requires smaller batches, you can send concurrent requests up to the 1,000 requests per second limit. To reach maximum throughput of 2,500 vectors per second, you can send 5 batches per second with 500 vectors each, or 1,000 batches per second with an average of 2.5 vectors each. **Accessing and querying vectors in an S3 vector index** Your application can achieve hundreds of [QueryVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_QueryVectors.html), [GetVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_GetVectors.html), or [ListVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_ListVectors.html) requests per second per S3 vector index. If you exceed the request rates, you might receive a `429 TooManyRequestsException` error. We recommend you use a retry mechanism and configure your application to send fewer requests. **Scaling across vector indexes** To improve query performance per vector index, consider configuring your application to divide vectors across multiple vector indexes when possible. For example, if you have multi-tenant workloads and your application queries each tenant independently, consider storing each tenant's vectors in a separate vector index. For more information, see [Vector indexes](s3-vectors-indexes.md). **Implementing multi-tenancy with separate vector indexes** You can achieve multi-tenancy by organizing your vector data using a single vector index for each tenant. You can use IAM and bucket policies to restrict each tenant's access to only their designated vector index. This approach helps maintain data isolation and simplifies management by removing the need to create separate buckets for each tenant. For more information, see [Identity and Access management in S3 Vectors](s3-vectors-access-management.md). **Configuring non-filterable metadata fields for vector indexes** When creating a vector index, configure metadata fields that don't require filtering as non-filterable metadata keys. For example, store text chunks for vector embeddings as non-filterable metadata fields when you need them only for reference. For more information, see [Non-filterable metadata](s3-vectors-metadata-filtering.md#s3-vectors-metadata-filtering-non-filterable). # Creating vector embeddings and performing semantic searches with `s3vectors-embed-cli` Creating and searching vector embeddings with `s3vectors-embed-cli` Amazon S3 Vectors provides a standalone open-source tool called Amazon S3 Vectors Embed CLI (`s3vectors-embed-cli`) that simplifies working with vector data by combining embedding generation with vector operations in single commands. This CLI tool helps you get started with S3 Vectors quickly by automating vector embedding generation with Amazon Bedrock foundation models and semantic search operations within your vector indexes. The S3 Vectors Embed CLI provides two main commands for integrating Amazon Bedrock embedding models with S3 Vectors operations: + `s3vectors-embed put`: Generate and insert vector embeddings into an vector index. Specifically, convert text and images into vector embeddings through Amazon Bedrock models and automatically store them in your vector index. + `s3vectors-embed query`: Generate vector embeddings from your query input through Amazon Bedrock models and query vectors in your vector index. The Amazon S3 Vectors Embed CLI is available in the [Amazon Web Services - Labs GitHub repository](https://github.com/awslabs). For detailed installation instructions, command parameters, examples, and best practices, see the [Amazon S3 Vectors Embed CLI GitHub repository](https://github.com/awslabs/s3vectors-embed-cli). For the lower-level S3 Vectors API operations that provide more control and customization, see [Amazon S3 Vectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_Operations_Amazon_S3_Vectors.html) in the *Amazon Simple Storage Service API Reference*. # Using S3 Vectors with other AWS services S3 Vectors integrates with other AWS services to enhance your vector processing capabilities and provide comprehensive solutions for AI and machine learning workloads. These integrations allow you to leverage the cost-effective storage of S3 Vectors alongside the specialized capabilities of other AWS services. ## Available integrations S3 Vectors provides native integrations with the following AWS services: + [Amazon OpenSearch Service](https://aws.amazon.com/opensearch-service/) - You can export a snapshot of a vector index to Amazon OpenSearch Service for high queries per second (QPS) and low latency vector search. Additionally, Amazon OpenSearch Service adds Amazon S3 Vectors as a new low-cost engine for customers who want to optimize cost while continuing to use Amazon OpenSearch Service API operations for advanced search functionality including hybrid search, aggregations, advanced filtering, faceted search, and more. + [Amazon Bedrock Knowledge Bases](https://aws.amazon.com/bedrock/knowledge-bases/) - Use S3 Vectors as your vector store for Retrieval Augmented Generation (RAG) applications, reducing storage costs while maintaining query performance for knowledge base operations. You can access this integration through the Amazon Bedrock console or [Amazon SageMaker AI Unified Studio](https://aws.amazon.com/sagemaker/unified-studio/). ## Integration benefits These integrations provide several key advantages: + **Cost optimization**: Store large vector datasets cost-effectively in S3 Vectors while using specialized services for specific workloads, such as using Amazon OpenSearch for advanced search functionality. + **Performance flexibility**: Choose the right integration for your performance requirements: S3 Vectors for lower throughput storage and sporadic querying, and other services for high-throughput, low-latency operations. + **Workflow integration**: Seamlessly incorporate vector operations into existing AWS based AI and ML pipelines. + **Simplified management**: Reduce operational complexity by using managed integrations rather than building custom solutions. **Topics** + [ ## Available integrations ](#s3-vectors-integration-available) + [ ## Integration benefits ](#s3-vectors-integration-benefits) + [ # Using S3 Vectors with OpenSearch Service ](s3-vectors-opensearch.md) + [ # Using S3 Vectors with Amazon Bedrock Knowledge Bases ](s3-vectors-bedrock-kb.md) # Using S3 Vectors with OpenSearch Service Amazon S3 Vectors integrates with OpenSearch to provide flexible vector storage and search capabilities. This integration allows you to optimize costs by storing vectors in S3 Vectors while leveraging the advanced search features of OpenSearch. There are two integrations between S3 Vectors and OpenSearch. One is to export vector data from S3 Vectors to OpenSearch Serverless for high-performance search capabilities. The other uses S3 Vectors as a cost-effective storage engine within OpenSearch while maintaining access to OpenSearch functionality. ## Export to OpenSearch Serverless You can export vector indexes from S3 Vectors to OpenSearch Serverless collections for high-performance search operations, hybrid search, aggregations, advanced filtering, and faceted search. When you export, the data is copied to OpenSearch Serverless while remaining in S3 Vectors, which means you'll be paying for both services during this time. ### When to use this integration Consider exporting to OpenSearch Serverless from S3 Vectors when you need the following: + **Hybrid search capabilities** that combine vector similarity with keyword search. + **High query throughput** for demanding workloads. + **Low-latency responses** for real-time applications that require millisecond response times. + **Advanced analytics** that include aggregations, faceted search, and complex filtering. ### Prerequisites Before using S3 Vectors with OpenSearch, ensure you have the following: + An existing S3 vector bucket with vector indexes containing your data. + Appropriate IAM permissions for both S3 Vectors and OpenSearch Service. + An understanding of your performance requirements to choose the appropriate integration method. ### Getting started **Using the AWS Management Console** **To export vector data to OpenSearch** 1. Sign in to the AWS Management Console and open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/). 1. In the navigation pane, choose **Vector buckets**. 1. In the list of vector buckets, choose the name of the bucket containing the vector data that you want to export. 1. For **Vector indexes**, choose the radio button next to the vector index that you want to export. 1. Choose **Advanced search export**, then choose **Export to OpenSearch**. **Important** **Point-in-time export**: The export captures data up to the initiation of the export. If you make updates to your vector data during the ingestion to OpenSearch, not all updates will be reflected in OpenSearch. **One-time operation**: This is a one-time export and will not stay in sync with your S3 Vectors data. You must manually re-export to capture any subsequent changes. Then, to configure and manage the integration of S3 Vectors with Amazon OpenSearch Service, you'll work primarily through the OpenSearch console. **To view exports to OpenSearch** 1. Sign in to the AWS Management Console and open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/). 1. In the navigation pane, choose **Vector buckets**. 1. In the list of vector buckets, choose the name of the bucket containing the vector data that you want to export. 1. For **Vector indexes**, choose **Advanced search export**, then choose **View exports to OpenSearch**. Then, you'll view exports to OpenSearch through the OpenSearch console. For more information about creating and managing OpenSearch Serverless collections, see [Creating and managing Amazon OpenSearch Service Serverless collections](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/serverless-collections.html) in the *Amazon OpenSearch Service Developer Guide*. ## OpenSearch with S3 Vectors engine You can use S3 Vectors as the underlying storage engine for [Amazon OpenSearch Managed clusters](https://aws.amazon.com/opensearch-service/features/managed/), providing cost-optimized vector storage while maintaining OpenSearch functionality. ### When to use this integration Consider using OpenSearch with the S3 Vectors engine when you need the following: + **Hybrid search capabilities** that combine vector similarity with keyword search. + **Lower query throughput** that may have less frequent or sporadic usage patterns. + **Higher latency tolerance** for applications that can accept longer response times in exchange for cost savings. + **Advanced analytics** that include aggregations, faceted search, and complex filtering. + **Existing OpenSearch workflows** that you want to extend with cost-effective vector storage. ### Prerequisites Before using OpenSearch with S3 Vectors engine, ensure you have: + An existing OpenSearch managed domain. For more information, see [Creating and managing Amazon OpenSearch Service domains](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/createupdatedomains.html) in the *Amazon OpenSearch Service Developer Guide*. + An understanding of your performance requirements to choose the appropriate integration method. ### Getting Started To use OpenSearch with S3 Vectors engine, set your engine to `S3_Vectors` during index creation in OpenSearch. For more information about the template that you use when creating an index in OpenSearch Service, including where to specify the engine type, see [Methods and engines](https://docs.opensearch.org/docs/latest/field-types/supported-field-types/knn-methods-engines/). For more information about the integration between OpenSearch and S3 Vectors engine, see [Advanced search capabilities with an S3 Vectors engine](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/s3-vector-opensearch-integration-engine.html) in the *Amazon OpenSearch Service Developer Guide*. # Using S3 Vectors with Amazon Bedrock Knowledge Bases S3 Vectors integrates with [Amazon Bedrock Knowledge Bases](https://aws.amazon.com/bedrock/knowledge-bases/) and [Amazon SageMaker AI Unified Studio](https://aws.amazon.com/sagemaker/unified-studio/) to simplify and reduce the cost of vector storage for retrieval augmented generation (RAG) applications. For more information about high-level CLI commands that integrate Amazon Bedrock embedding models with S3 Vectors operations, see . **Topics** + [ ## Integration overview ](#s3-vectors-bedrock-kb-overview) + [ ## When to use this integration ](#s3-vectors-bedrock-kb-when) + [ ## Supported embedding models ](#s3-vectors-bedrock-kb-models) + [ ## Prerequisites and permissions ](#s3-vectors-bedrock-kb-prereq) + [ ## Creating a knowledge base with S3 Vectors ](#s3-vectors-bedrock-kb-create) + [ ## Managing and querying your knowledge base ](#s3-vectors-bedrock-kb-manage) + [ ## Limitations ](#s3-vectors-bedrock-kb-limits) ## Integration overview When creating a knowledge base in Amazon Bedrock, you can select S3 Vectors as your vector store. This integration provides the following: + **Cost savings** for RAG applications with large vector datasets. + **Seamless integration** with the fully managed RAG workflow of Amazon Bedrock. + **Automatic vector management** handled by the Amazon Bedrock service. + **Sub-second cold query latency and as low as 100 millisecond warm query latency** for knowledge base retrieval operations. Amazon Bedrock Knowledge Bases provides a fully managed end-to-end RAG workflow. When you create a knowledge base with S3 Vectors, Amazon Bedrock automatically fetches data from your S3 data source, converts content into text blocks, generates embeddings, and stores them in your vector index. You can then query the knowledge base and generate responses based on chunks retrieved from your source data. ## When to use this integration Consider using S3 Vectors with Amazon Bedrock Knowledge Bases when you need the following: + **Cost-effective vector storage** for large datasets where sub-second query latency meets your application requirements. + **Text and image-based document retrieval** for use cases like searching through manuals, policies, and visual content. + **RAG applications** that prioritize storage cost optimization over ultra-low latency responses. + **Managed vector operations** without needing to learn S3 Vectors API operations directly - you can continue using familiar Amazon Bedrock interfaces. + **Long-term vector storage** with the durability and scalability of Amazon S3 This integration is ideal for organizations building RAG applications that need to search through and extract insights from written content and images, where the cost benefits of S3 Vectors align with acceptable query performance requirements. ## Supported embedding models Refer to [supported models for vector embeddings](https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base-supported.html#knowledge-base-supported-embeddings) in the *Amazon Bedrock User Guide*. ## Prerequisites and permissions Before creating a knowledge base with S3 Vectors, ensure you have the following: + Appropriate IAM permissions for both S3 Vectors and Amazon Bedrock services. For more information about IAM permissions for S3 Vectors, see [Identity and Access management in S3 Vectors](s3-vectors-access-management.md). For more information about IAM permissions for your Amazon Bedrock Knowledge Bases service role to access S3 Vectors, see [Permissions to access your vector store in Amazon S3 Vectors](https://docs.aws.amazon.com/bedrock/latest/userguide/kb-permissions.html#kb-permissions-s3vectors) in the *Amazon Bedrock User Guide*. + Your source documents prepared for ingestion into the knowledge base. + An understanding of your embedding model requirements. When setting up security configurations, you can choose an IAM role that provides Amazon Bedrock permission to access required AWS services. You can let Amazon Bedrock create the service role or use your own custom role. If you use a custom role, configure a vector bucket policy that restricts access to the vector bucket and vector index to the custom role. For detailed information about required permissions and IAM roles, see [Create a service role for Amazon Bedrock Knowledge Bases](https://docs.aws.amazon.com/bedrock/latest/userguide/kb-permissions.html) in the *Amazon Bedrock User Guide*. The service role must also have permissions for S3 Vectors and AWS KMS API operations. ## Creating a knowledge base with S3 Vectors You can create a knowledge base that uses S3 Vectors through two methods. ### Method one: Using the Amazon Bedrock console When creating a knowledge base in the Amazon Bedrock console, you can choose "S3 vector bucket" as your vector store option. You have two setup options: + **Quick create a new vector store** - Amazon Bedrock creates an S3 vector bucket and vector index and configures them with the required settings for you. By default, the vector bucket is encrypted using server-side encryption with Amazon S3 managed keys (SSE-S3). You can optionally encrypt the bucket using AWS KMS. For more information about **Quick create a new vector store** in the console, see [Create a knowledge base by connecting to a data source in Amazon Bedrock Knowledge Bases](https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base-create.html) in the *Amazon Bedrock User Guide*. + **Choose a vector store you have created** - Choose an existing S3 vector bucket and vector index from your account that you've previously created. For more information about creating an S3 vector bucket and vector index in the Amazon Bedrock Knowledge Bases console, see the S3 Vectors tab in [Prerequisites for using a vector store you created for a knowledge base](https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base-setup.html) in the *Amazon Bedrock User Guide*. For detailed step-by-step instructions, see [Create a knowledge base by connecting to a data source in Amazon Bedrock Knowledge Bases](https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base-create.html) in the *Amazon Bedrock User Guide*. ### Method two: Using Amazon SageMaker Unified Studio You can also create and manage knowledge bases with S3 Vectors through Amazon Bedrock in [Amazon SageMaker AI Unified Studio](https://aws.amazon.com/sagemaker/unified-studio/). This provides a unified development environment for building and testing AI applications that use knowledge bases. [Amazon Bedrock in SageMaker AI Unified Studio](https://aws.amazon.com/bedrock/unifiedstudio/) is designed for users who need integrated notebook capabilities and work across multiple AWS ML and analytics services. You can quickly create an S3 vector bucket and configure it as the vector store for your knowledge bases when you build generative AI applications. For information about using S3 Vectors with Amazon Bedrock in SageMaker AI Unified Studio, see [Add a data source to your Amazon Bedrock app](https://docs.aws.amazon.com/sagemaker-unified-studio/latest/userguide/data-sources.html) in the *SageMaker AI Unified Studio User Guide*. ## Managing and querying your knowledge base ### Data synchronization and management Amazon Bedrock Knowledge Bases offers ingestion job operations to keep your data source and vector embeddings synchronized. When you sync your data source, Amazon Bedrock scans each document and verifies whether it has been indexed into the vector store. You can also directly index documents into the vector store using the [IngestKnowledgeBaseDocuments](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent_IngestKnowledgeBaseDocuments.html) operation. Best practice is to create a separate vector store for each knowledge base to ensure data synchronization. When you delete a knowledge base or data source resource, Amazon Bedrock offers two data deletion policies: `Delete` (default) and `Retain`. If you choose the `Delete` policy, vectors in the vector index and vector bucket are automatically deleted. ### Querying and retrieval After your knowledge base is set up, you can do the following: + **Retrieve chunks** from your source data using the [Retrieve](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent-runtime_Retrieve.html) API operation. + **Generate responses** based on retrieved chunks using the [RetrieveAndGenerate](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent-runtime_RetrieveAndGenerate.html) API operation. + **Test queries** directly in the Amazon Bedrock console. Responses are returned with citations to the original source data. ## Limitations When using S3 Vectors with Amazon Bedrock Knowledge Bases, you should know the following limitations: + **Semantic search only**: S3 Vectors supports semantic search but not hybrid search capabilities. + **S3 Vectors size limits**: Each vector has a total metadata size limit and a size limit for filterable metadata, which may limit custom metadata and filtering options. When using S3 Vectors as your vector store with Amazon Bedrock Knowledge Bases, you can attach up to 1KB of custom metadata and 35 metadata keys per vector. + **Chunking strategy constraints**: When using very high token counts with hierarchical chunking in Amazon Bedrock Knowledge Bases, you can exceed the maximum metadata size limits as parent-child chunk relationships and hierarchical context are stored as non-filterable metadata in S3 Vectors. For more information about metadata size limits per vector, see [Limitations and restrictions](s3-vectors-limitations.md). + **Floating-point vectors only**: Binary vector embeddings aren't supported. For comprehensive guidance on working with Amazon Bedrock Knowledge Bases, see [Retrieve data and generate AI responses with Amazon Bedrock Knowledge Bases](https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base.html) in the *Amazon Bedrock User Guide*. # AWS Regions, endpoints, and quotas for S3 Vectors AWS Regions, endpoints, and quotas for S3 Vectors The following sections include the supported AWS Regions and service quotas for Amazon S3 Vectors. ## S3 Vectors AWS Regions and endpoints Amazon S3 Vectors is currently available in the following AWS Regions. To connect programmatically to an AWS service, you use an endpoint. For more information about AWS service endpoints, see [AWS service endpoints](https://docs.aws.amazon.com/general/latest/gr/rande.html) in the AWS General Reference. For more Amazon S3 endpoint information, see [Amazon S3 endpoints and quotas](https://docs.aws.amazon.com/general/latest/gr/s3.html) in the AWS General Reference. S3 Vectors endpoints are dual-stack endpoints that support requests over IPv6 and IPv4. S3 Vectors supports private connectivity with AWS PrivateLink interface endpoints. For more information, see [VPC endpoints for S3 Vectors](s3-vectors-privatelink.md). **S3 Vectors AWS Regions and endpoints** | Region Name | Region | Endpoint | Protocol | Signature Version(s) Support | | --- | --- | --- | --- | --- | | Africa (Cape Town) | af-south-1 | s3vectors.af-south-1.api.aws | HTTPS | 4 | | Asia Pacific (Hong Kong) | ap-east-1 | s3vectors.ap-east-1.api.aws | HTTPS | 4 | | Asia Pacific (Hyderabad) | ap-south-2 | s3vectors.ap-south-2.api.aws | HTTPS | 4 | | Asia Pacific (Jakarta) | ap-southeast-3 | s3vectors.ap-southeast-3.api.aws | HTTPS | 4 | | Asia Pacific (Malaysia) | ap-southeast-5 | s3vectors.ap-southeast-5.api.aws | HTTPS | 4 | | Asia Pacific (Melbourne) | ap-southeast-4 | s3vectors.ap-southeast-4.api.aws | HTTPS | 4 | | Asia Pacific (Mumbai) | ap-south-1 | s3vectors.ap-south-1.api.aws | HTTPS | 4 | | Asia Pacific (New Zealand) | ap-southeast-6 | s3vectors.ap-southeast-6.api.aws | HTTPS | 4 | | Asia Pacific (Osaka) | ap-northeast-3 | s3vectors.ap-northeast-3.api.aws | HTTPS | 4 | | Asia Pacific (Seoul) | ap-northeast-2 | s3vectors.ap-northeast-2.api.aws | HTTPS | 4 | | Asia Pacific (Singapore) | ap-southeast-1 | s3vectors.ap-southeast-1.api.aws | HTTPS | 4 | | Asia Pacific (Sydney) | ap-southeast-2 | s3vectors.ap-southeast-2.api.aws | HTTPS | 4 | | Asia Pacific (Taipei) | ap-east-2 | s3vectors.ap-east-2.api.aws | HTTPS | 4 | | Asia Pacific (Thailand) | ap-southeast-7 | s3vectors.ap-southeast-7.api.aws | HTTPS | 4 | | Asia Pacific (Tokyo) | ap-northeast-1 | s3vectors.ap-northeast-1.api.aws | HTTPS | 4 | | Canada (Central) | ca-central-1 | s3vectors.ca-central-1.api.aws | HTTPS | 4 | | Canada West (Calgary) | ca-west-1 | s3vectors.ca-west-1.api.aws | HTTPS | 4 | | Europe (Frankfurt) | eu-central-1 | s3vectors.eu-central-1.api.aws | HTTPS | 4 | | Europe (Ireland) | eu-west-1 | s3vectors.eu-west-1.api.aws | HTTPS | 4 | | Europe (London) | eu-west-2 | s3vectors.eu-west-2.api.aws | HTTPS | 4 | | Europe (Milan) | eu-south-1 | s3vectors.eu-south-1.api.aws | HTTPS | 4 | | Europe (Paris) | eu-west-3 | s3vectors.eu-west-3.api.aws | HTTPS | 4 | | Europe (Spain) | eu-south-2 | s3vectors.eu-south-2.api.aws | HTTPS | 4 | | Europe (Stockholm) | eu-north-1 | s3vectors.eu-north-1.api.aws | HTTPS | 4 | | Europe (Zurich) | eu-central-2 | s3vectors.eu-central-2.api.aws | HTTPS | 4 | | Mexico (Central) | mx-central-1 | s3vectors.mx-central-1.api.aws | HTTPS | 4 | | South America (Sao Paulo) | sa-east-1 | s3vectors.sa-east-1.api.aws | HTTPS | 4 | | US East (N. Virginia) | us-east-1 | s3vectors.us-east-1.api.aws | HTTPS | 4 | | US East (Ohio) | us-east-2 | s3vectors.us-east-2.api.aws | HTTPS | 4 | | US West (N. California) | us-west-1 | s3vectors.us-west-1.api.aws | HTTPS | 4 | | US West (Oregon) | us-west-2 | s3vectors.us-west-2.api.aws | HTTPS | 4 | ## S3 Vectors service quotas Quotas, also referred to as limits, are the maximum number of service resources or operations for your AWS account. The limits are the service quotas for S3 Vectors resources. For detailed limits that apply to S3 Vectors, see [Limitations and restrictions](s3-vectors-limitations.md). Unless otherwise noted, each quota is per AWS Region. You can request increases for some quotas, and other quotas can't be increased. For more information about requesting a quota increase, see [Requesting a quota increase](https://docs.aws.amazon.com/servicequotas/latest/userguide/request-quota-increase.html) in the *Service Quotas User Guide*. For more Amazon S3 quota information, see [Amazon S3 endpoints and quotas](https://docs.aws.amazon.com/general/latest/gr/s3.html) in the *AWS General Reference*. # Security in S3 Vectors Amazon S3 Vectors offers highly durable vector storage. Data written to S3 Vectors is stored on S3, which is designed for 99.999999999% (11 nines) of data durability. S3 Vectors implements a security approach that includes encryption at rest and secure data transmission. Whether you're storing millions of vectors for machine learning applications or building semantic search systems, S3 Vectors provides the security controls necessary to meet your organization's compliance and data protection requirements. **Topics** + [ # Identity and Access management in S3 Vectors ](s3-vectors-access-management.md) + [ # Data protection and encryption in S3 Vectors ](s3-vectors-data-encryption.md) + [ # VPC endpoints for S3 Vectors ](s3-vectors-privatelink.md) # Identity and Access management in S3 Vectors Identity and Access management in S3 Vectors Access management in S3 Vectors follows AWS security best practices, providing multiple layers of control to ensure that only authorized users and applications can access your vector data. The service integrates with IAM and supports both identity-based and resource-based policies, giving you flexibility in how you structure and manage permissions across your organization. ## Authenticating and authorizing requests S3 Vectors uses AWS standard authentication and authorization mechanisms to secure access to vector buckets and their contents. Every request to S3 Vectors must be authenticated using valid AWS credentials, and the service evaluates permissions based on the combination of identity-based policies, resource-based policies, and any applicable service control policies. The authentication process begins when a client makes a request to S3 Vectors using AWS credentials (access keys, temporary credentials from AWS STS, or IAM roles). The service validates these credentials and then evaluates the permissions associated with the authenticated identity against the requested action and target resource. This evaluation process considers multiple policy types and applies the principle of least privilege to determine whether the request should be allowed or denied. Authorization in S3 Vectors operates at multiple levels of granularity. You can control access at the vector bucket level, individual vector index level, or even specific operations within an index. This hierarchical permission model allows you to implement sophisticated access control schemes that align with your organizational structure and data governance requirements. ## Resource types defined for vector buckets S3 Vectors defines specific resource types that can be referenced in IAM policies and resource-based policies. Understanding these resource types is essential for creating effective access control policies that provide the right level of access to the right users and applications. The following table describes the resource types available in S3 Vectors. **Resource types available in S3 Vectors** | Resource type | ARN format | Description | | --- | --- | --- | | VectorBucket | arn:aws:s3vectors:region:123456789012:bucket/bucket-name | Represents a vector bucket and is used for bucket-level operations such as creating, deleting, or configuring the bucket | | Index | arn:aws:s3vectors:region:123456789012:bucket/bucket-name/index/index-name | Represents a vector index within a bucket and is used for index-specific operations such as querying vectors or managing index contents | ## Policy actions for vector buckets S3 Vectors provides a comprehensive set of policy actions that correspond to the various operations you can perform on vector buckets and indexes. These actions are designed to provide fine-grained control over who can perform specific operations, allowing you to implement the principle of least privilege effectively. The following table lists all available policy actions for S3 Vectors resources. **Policy actions for S3 Vectors resources** | Resource type | API operations | Policy actions | Description of policy actions | Access level | Condition keys | | --- | --- | --- | --- | --- | --- | | Account | [ListVectorBuckets](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_ListVectorBuckets.html) | s3vectors:ListVectorBuckets | Grants permission to list all vector buckets in the account and region | List | | | VectorBucket | [CreateVectorBucket](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_CreateVectorBucket.html) | s3vectors:CreateVectorBucket | Grants permission to create a new vector bucket with specified configuration | Write | s3vectors:sseType, s3vectors:kmsKeyArn | | VectorBucket | [GetVectorBucket](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_GetVectorBucket.html) | s3vectors:GetVectorBucket | Grants permission to retrieve vector bucket attributes and configuration | Read | | | VectorBucket | [DeleteVectorBucket](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_DeleteVectorBucket.html) | s3vectors:DeleteVectorBucket | Grants permission to delete an empty vector bucket | Write | | | VectorBucket | [ListIndexes](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_ListIndexes.html) | s3vectors:ListIndexes | Grants permission to list all indexes within a vector bucket | List | | | VectorBucket | [PutVectorBucketPolicy](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_PutVectorBucketPolicy.html) | s3vectors:PutVectorBucketPolicy | Grants permission to apply or update a resource-based policy on a vector bucket | Permissions management | | | VectorBucket | [GetVectorBucketPolicy](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_GetVectorBucketPolicy.html) | s3vectors:GetVectorBucketPolicy | Grants permission to retrieve the resource-based policy attached to a vector bucket | Read | | | VectorBucket | [DeleteVectorBucketPolicy](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_DeleteVectorBucketPolicy.html) | s3vectors:DeleteVectorBucketPolicy | Grants permission to remove the resource-based policy from a vector bucket | Permissions management | | | Index | [CreateIndex](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_CreateIndex.html) | s3vectors:CreateIndex | Grants permission to create a new vector index with specified dimensions and metadata configuration | Write | | | Index | [GetIndex](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_GetIndex.html) | s3vectors:GetIndex | Grants permission to retrieve vector index attributes and configuration | Read | | | Index | [DeleteIndex](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_DeleteIndex.html) | s3vectors:DeleteIndex | Grants permission to delete a vector index and all its contents | Write | | | Index | [QueryVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_QueryVectors.html) | (Required) s3vectors:QueryVectors | Grants permission to perform similarity queries against vectors in an index. **With `s3vectors:QueryVectors` only**, you can retrieve vector keys of approximate nearest neighbors and their computed distances from the query vector. This permission is sufficient only when you don't set any metadata filters and don't request metadata (by keeping the `returnMetadata` parameter set to false or not specified). | Read | | | | | (Conditionally required): s3vectors:GetVectors | Required if you set metadata filters, set `returnMetadata` to true in your request. **With both `s3vectors:QueryVectors` and `s3vectors:GetVectors`**, you can filter results by using metadata criteria and retrieve vector keys along with their associated data, metadata, and computed distances from the query vector. | Read | | | Index | [PutVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_PutVectors.html) | s3vectors:PutVectors | Grants permission to add or update vectors in an index | Write | | | Index | [GetVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_GetVectors.html) | s3vectors:GetVectors | Grants permission to retrieve specific vectors and their metadata by vector key | Read | | | Index | [ListVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_ListVectors.html) | (Required) s3vectors:ListVectors | Grants permission to list vector keys in an index. **With `s3vectors:ListVectors` only**, you can list vector keys when both `returnData` and `returnMetadata` parameters are false or not specified. | Read | | | | | (Conditionally required): s3vectors:GetVectors | Required if you set either `returnData` or `returnMetadata` parameter to true in your request. **With both `s3vectors:ListVectors` and `s3vectors:GetVectors`**, you can retrieve vector keys along with their associated data and metadata by setting `returnData` and `returnMetadata` to true. | Read | | | Index | [DeleteVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_DeleteVectors.html) | s3vectors:DeleteVectors | Grants permission to delete specific vectors from an index | Write | | These actions can be combined in various ways to create policies that match your specific access requirements. For example, you might create a read-only policy that includes `s3vectors:GetVectorBucket`, `s3vectors:ListIndexes`, `s3vectors:QueryVectors`, and `s3vectors:GetVectors` actions, or a policy that includes query and vector retrieval permissions but excludes administrative actions like creating or deleting indexes. ## Condition keys for vector buckets **Condition keys for vector buckets** | | Condition keys | Description | Type | | --- | --- | --- | --- | | 1 | s3vectors:sseType | Filters access by server-side encryption type Valid values: AES256 \$1 aws:kms | String | | 2 | s3vectors:kmsKeyArn | Filters access by the AWS AWS KMS key ARN for the key used to encrypt a vector bucket | ARN | # S3 Vectors identity-based policy examples S3 Vectors identity-based policy examples IAM identity-based policies are JSON documents that you attach to IAM users, groups, or roles to define what actions they can perform on S3 Vectors resources. These policies are evaluated in the context of the identity making the request and provide a centralized way to manage permissions across your AWS environment. Identity-based policies provide a clear audit trail of who has what permissions and can be easily modified as your access requirements evolve. When designing identity-based policies for S3 Vectors, consider the different types of users and applications that will interact with your vector data. Common patterns include data scientists who need to query vectors, data engineers who need to load and manage vector data, administrators who need full control over bucket configuration, and applications that need specific read or write access to particular vector indexes. ## Example policies ### Administrative access policy This policy provides full administrative access to S3 Vectors resources, suitable for platform administrators or DevOps teams: ``` { "Version": "2012-10-17", "Statement": [ { "Sid": "AllowAdministrativeAccess", "Effect": "Allow", "Action": [ "s3vectors:CreateVectorBucket", "s3vectors:PutVectorBucketPolicy", "s3vectors:DeleteVectorBucket", "s3vectors:DeleteVectorBucketPolicy", "s3vectors:GetVectorBucket", "s3vectors:GetVectorBucketPolicy", "s3vectors:ListVectorBuckets", "s3vectors:CreateIndex", "s3vectors:DeleteIndex", "s3vectors:GetIndex", "s3vectors:ListIndexes", "s3vectors:DeleteVectors", "s3vectors:GetVectors", "s3vectors:ListVectors", "s3vectors:PutVectors", "s3vectors:QueryVectors" ], "Resource": "*" } ] } ``` ### Application-specific access policy This policy is designed for applications that need to perform specific operations on designated vector indexes: ``` { "Version": "2012-10-17", "Statement": [ { "Sid": "AllowApplicationVectorAccess", "Effect": "Allow", "Action": [ "s3vectors:QueryVectors", "s3vectors:GetVectors", "s3vectors:PutVectors", "s3vectors:ListVectors" ], "Resource": [ "arn:aws:s3vectors:aws-region:123456789012:bucket/amzn-s3-demo-vector-bucket/index/product-recommendations", "arn:aws:s3vectors:aws-region:123456789012:bucket/amzn-s3-demo-vector-bucket/index/content-similarity" ] }, { "Sid": "AllowGetIndex", "Effect": "Allow", "Action": "s3vectors:GetIndex", "Resource": "arn:aws:s3vectors:aws-region:123456789012:bucket/amzn-s3-demo-vector-bucket/index/*" }, { "Sid": "AllowIndexInspection", "Effect": "Allow", "Action": "s3vectors:ListIndexes", "Resource": "arn:aws:s3vectors:aws-region:123456789012:bucket/amzn-s3-demo-vector-bucket" } ] } ``` # S3 Vectors resource-based policy examples S3 Vectors resource-based policy examples Resource-based policies are attached to a resource. You can create resource-based policies for vector buckets. Resource-based policies for S3 Vectors use the standard AWS policy format in JSON that you attach directly to vector buckets to control access to the bucket and its contents. Unlike identity-based policies that are attached to users, groups, or roles, resource-based policies are attached to the resource itself (the vector bucket) and can grant permissions to principals from other AWS accounts. This makes them ideal for scenarios where you need to share vector data across organizational boundaries or implement fine-grained access controls based on the specific resource being accessed. Resource-based policies are evaluated in combination with identity-based policies, and the effective permissions are determined by the union of all applicable policies. This means that a principal needs permission from both the identity-based policy (attached to their user/role) and the resource-based policy (attached to the bucket) to perform an action, unless the resource-based policy explicitly grants the permission. ## Example 1: Cross-account access policy This policy demonstrates how to grant specific permissions to users from different AWS accounts: ``` { "Version": "2012-10-17", "Statement": [ { "Sid": "CrossAccountBucketAccess", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam:123456789012:role/Admin" }, "Action": [ "s3vectors:CreateIndex", "s3vectors:ListIndexes", "s3vectors:QueryVectors", "s3vectors:PutVectors", "s3vectors:DeleteIndex" ], "Resource": [ "arn:aws:s3vectors:aws-region:111122223333:bucket/amzn-s3-demo-vector-bucket/*", "arn:aws:s3vectors:aws-region:111122223333:bucket/amzn-s3-demo-vector-bucket" ] } ] } ``` ## Example 2: Deny vector index level actions This policy demonstrates how to deny specific vector index level actions to an IAM role: ``` { "Version": "2012-10-17", "Statement": [ { "Sid": "DenyIndexLevelActions", "Effect": "Deny", "Principal": { "AWS": "arn:aws:iam:123456789012:role/External-Role-Name" }, "Action": [ "s3vectors:QueryVectors", "s3vectors:PutVectors", "s3vectors:DeleteIndex", "s3vectors:GetVectors", "s3vectors:GetIndex", "s3vectors:DeleteVectors", "s3vectors:CreateIndex", "s3vectors:ListVectors" ], "Resource": "arn:aws:s3vectors:aws-region:111122223333:bucket/amzn-s3-demo-vector-bucket/*" } ] } ``` ## Example 3: Deny modification operations at both vector index and bucket levels This policy demonstrates how to deny modification requests for both vector index and bucket-level actions by specifying multiple resources: ``` { "Version": "2012-10-17", "Statement": [ { "Sid": "DenyModificationActionsAtBucketandIndexLevels", "Effect": "Deny", "Principal": { "AWS": "arn:aws:iam:123456789012:role/External-Role-Name" }, "Action": [ "s3vectors:CreateVectorBucket", "s3vectors:DeleteVectorBucket", "s3vectors:PutVectorBucketPolicy", "s3vectors:DeleteVectorBucketPolicy", "s3vectors:CreateIndex", "s3vectors:DeleteIndex", "s3vectors:PutVectors", "s3vectors:DeleteVectors" ], "Resource": [ "arn:aws:s3vectors:aws-region:111122223333:bucket/amzn-s3-demo-vector-bucket/*", "arn:aws:s3vectors:aws-region:111122223333:bucket/amzn-s3-demo-vector-bucket" ] } ] } ``` # Data protection and encryption in S3 Vectors Data protection and encryption in S3 Vectors Amazon S3 Vectors provides 99.999999999% (11 9s) of durability for your vector data, which ensures exceptional reliability for your vector storage needs. This durability is backed by the proven infrastructure of Amazon S3, which is designed to maintain data integrity and availability even in the face of hardware failures or other disruptions. Data protection in S3 Vectors encompasses multiple layers of security controls designed to protect your vector data both at rest and in transit. By default, all new vectors in Amazon S3 Vectors vector buckets use server-side encryption with Amazon S3 managed keys (SSE-S3). When you create a vector bucket with SSE-S3 encryption, all subsequent operations on the bucket automatically use encryption. S3 Vectors also integrates with AWS Key Management Service (KMS) to provide flexible encryption key management options, allowing you to choose customer-managed keys for permission control and auditability. When creating a vector index within a vector bucket, you can optionally override the vector bucket level encryption settings and provide an encryption configuration (SSE-S3 or KMS) at the vector index level. If no specific encryption is specified upon vector index creation, the index will inherit the encryption configuration from the vector bucket it belongs to. ## Setting server-side encryption behavior for Amazon S3 vector buckets and indexes Setting bucket encryption Encryption configuration in S3 Vectors is a fundamental security setting to specify when you create a vector bucket. This design ensures that all vector data stored in the bucket is encrypted from the moment of creation. By default, the encryption configuration applies to all vectors, vector indexes, and metadata within the bucket, providing consistent protection across your entire vector dataset in a vector bucket. You can also optionally override the vector bucket level encryption settings and provide a dedicated encryption configuration (SSE-S3 or AWS KMS) at the vector index level. **Important** Encryption settings for a vector bucket can't be changed after the vector bucket is created. You must carefully consider your encryption requirements during the bucket creation process, including compliance requirements, key management preferences, and integration with existing security infrastructure. When you set the SSE-S3 or SSE-KMS encryption type at the vector bucket level, by default it applies to all vector indexes and vectors within the bucket. The encryption configuration applies to not only the vector data itself but also all associated metadata. You can also optionally override the vector bucket level encryption settings and provide a dedicated encryption configuration (SSE-S3 or KMS) at the vector index level. Encryption settings for a vector index can't be changed after the vector index is created. ### Using SSE-S3 encryption Server-side encryption with Amazon S3 managed keys (SSE-S3) provides a simple and effective encryption solution for vector buckets where AWS manages all aspects of the encryption process. This encryption method uses `AES-256` encryption and is designed to provide strong security with minimal operational overhead, providing organizations with robust encryption without the complexity of the needs to manage encryption keys. With SSE-S3, Amazon S3 handles the generation, rotation, and management of encryption keys automatically. SSE-S3 provides strong security with no additional configuration or ongoing management requirements. The encryption and decryption processes are handled automatically by the service, and there are no additional charges for using SSE-S3 encryption beyond the standard S3 Vectors pricing. ### Using SSE-KMS encryption Server-side encryption with AWS Key Management Service keys (SSE-KMS) provides enhanced control over encryption keys and enables detailed audit logging of key usage. This encryption method is ideal for organizations with strict compliance requirements, those that need to implement custom key rotation policies, or environments where detailed audit trails of data access are required. SSE-KMS allows you to use customer managed keys (CMKs) for encrypting your vector data. Customer managed keys provide the highest level of control, allowing you to define key policies, enable or disable keys, and monitor key usage through AWS CloudTrail. This level of control makes SSE-KMS particularly suitable for regulated industries or organizations with specific data governance requirements. When using SSE-KMS with customer managed keys, you have complete control over who can use the keys to encrypt and decrypt data. You can create detailed key policies that specify which users, roles, or services can access the keys. #### Important considerations for SSE-KMS + **KMS key format requirements:** S3 Vectors requires that you specify KMS keys using the full Amazon Resource Name (ARN) format. Key IDs or key aliases aren't supported. + **Service principal permissions:** When you use customer managed keys with S3 Vectors, you must explicitly grant permissions to the S3 Vectors service principal to use your KMS key. This requirement ensures that the service can encrypt and decrypt your data on your behalf. The service principal that requires access is `indexing.s3vectors.amazonaws.com`. **Example: KMS key policy for S3 Vectors** To use a customer managed KMS key with S3 Vectors, you must update your key policy to include permissions for the S3 Vectors service principal. Here's a comprehensive key policy example. ``` { "Version": "2012-10-17", "Statement": [ { "Sid": "AllowS3VectorsServicePrincipal", "Effect": "Allow", "Principal": { "Service": "indexing.s3vectors.amazonaws.com" }, "Action": "kms:Decrypt", "Resource": "*", "Condition": { "ArnLike": { "aws:SourceArn": "arn:aws:s3vectors:aws-region:123456789012:bucket/*" }, "StringEquals": { "aws:SourceAccount": "123456789012" }, "ForAnyValue:StringEquals": { "kms:EncryptionContextKeys": ["aws:s3vectors:arn", "aws:s3vectors:resource-id"] } } }, { "Sid": "AllowApplicationAccess", "Effect": "Allow", "Principal": { "AWS": [ "arn:aws:iam:123456789012:role/VectorApplicationRole", "arn:aws:iam:123456789012:user/DataScientist" ] }, "Action": [ "kms:Decrypt", "kms:GenerateDataKey" ], "Resource": "*", "Condition": { "StringEquals": { "kms:ViaService": "s3vectors.aws-region.amazonaws.com" }, "ForAnyValue:StringEquals": { "kms:EncryptionContextKeys": ["aws:s3vectors:arn", "aws:s3vectors:resource-id"] } } } ] } ``` + **Required KMS permissions:** + S3 Vectors service principal permission: + `kms:Decrypt` – Required by the S3 Vectors service principal (`indexing.s3vectors.amazonaws.com`) on your customer managed key to maintain and optimize the index in background operations + IAM principal permissions: + `kms:Decrypt` – Required for all vector-level operations ([PutVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_PutVectors.html), [GetVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_GetVectors.html), [QueryVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_QueryVectors.html), [DeleteVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_DeleteVectors.html), [ListVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_ListVectors.html)) + `kms:GenerateDataKey` – Required to create a vector bucket by using the customer managed key + **Cross-account access considerations:** When implementing cross-account access patterns with SSE-KMS, you must ensure that the KMS key policy allows access from the appropriate principals in other accounts. The key ARN format becomes particularly important in cross-account scenarios, as it provides an unambiguous reference to the key regardless of the account context from which it's being accessed. # Setting encryption in S3 Vectors Setting encryption in S3 Vectors This topic explains how to set the encryption configuration for your S3 vector buckets and indexes. Before you begin, make sure you have the following: + Appropriate permissions to view bucket and index properties. ## Using the S3 console **To configure encryption for a vector bucket** 1. Open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/). 1. In the navigation pane, choose **Vector buckets**. 1. Choose **Create vector bucket**. 1. For **Bucket name**, enter a name for your bucket. The bucket name must: + Be unique within your account for this AWS Region + Be between 3 and 63 characters long + Consist only of lowercase letters, numbers, and hyphens (-) 1. For **Encryption**, choose + **Specify encryption type** – Choose a specific encryption method: + **Server-side encryption with Amazon S3 managed keys (SSE-S3)** – With SSE-S3, Amazon S3 handles the generation, rotation, and management of encryption keys automatically. + **Server-side encryption with AWS Key Management Service keys (SSE-KMS)** – Similar to SSE-S3, but uses customer managed keys (CMKs) in AWS KMS, giving you more control over your keys. For more information about customer managed keys, see [Customer managed keys](https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html#customer-cmk) in the *AWS Key Management Service Developer Guide*. If you select this option, under **AWS KMS key**, choose one of the following options: + **Choose from your AWS KMS keys** – Select an existing KMS key from the dropdown list + **Enter AWS KMS key ARN** – Enter the Amazon Resource Name (ARN) of a KMS key + **Create a KMS key** – Create a new customer managed key in the AWS KMS console. For more information, see [Creating symmetric customer managed keys](https://docs.aws.amazon.com/kms/latest/developerguide/create-keys.html#create-symmetric-cmk) in the *AWS Key Management Service Developer Guide*. **Note** The following requirements apply to the KMS key: AWS KMS key ID must not be empty Your KMS key must be in the same Region where this bucket is being created AWS KMS key ARN must start with "arn:aws:kms:" **Important** Encryption settings can't be changed after the vector bucket is created. 1. If you chose **Enter AWS KMS key ARN**, enter the ARN in the text field provided. 1. If you chose **Create a KMS key**, the console opens the AWS KMS console in a new tab. For instructions on creating a KMS key, see [Creating symmetric customer managed keys](https://docs.aws.amazon.com/kms/latest/developerguide/create-keys.html#create-symmetric-cmk) in the *AWS Key Management Service Developer Guide*. 1. Choose **Create vector bucket**. **Important** When using KMS encryption, ensure that the IAM principals that need to access objects in the bucket have the necessary KMS permission (kms:Decrypt) for the selected KMS key. **To configure encryption for a vector index** 1. Open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/). 1. In the navigation pane, choose **Vector buckets**. 1. In the list of vector buckets, choose the name of the bucket where you want to create a vector index. 1. Choose **Create vector index**. 1. For **Vector index name**, enter a name for your vector index. Vector index names must be unique within the vector bucket. Index name must be between 3 and 63 characters. Valid characters are lowercase letters (a-z), numbers (0-9), hyphens (-), and dots (.). For more information about the vector index naming requirements, see [Vector bucket naming rules](s3-vectors-buckets-naming.md). 1. For **Dimension**, enter the number of values in each vector. **Note** The value for **Dimension** determines how many numerical values each vector will contain. All vectors added to this index must have exactly this number of values. Dimension must be between 1 and 4096. A larger dimension requires more storage space. Choose based on your embedding model's output dimensions. For more information about the dimension requirements, see [Limitations and restrictions](s3-vectors-limitations.md). 1. For **Distance metric**, choose one of the following options: + **Cosine** – Measures the cosine of the angle between vectors. Best for normalized vectors and when direction matters more than magnitude + **Euclidean** – Measures the straight-line distance between vectors. Best when both direction and magnitude are important. 1. (Optional) Under **Non-filterable metadata**, configure metadata keys that will be stored but not used for filtering: To add non-filterable metadata keys: 1. Choose **Add key**. 1. Enter a key name (1-63 characters and unique within this vector index). 1. Repeat to add additional keys (maximum 10 keys). **Note** You can attach filterable metadata as key-value pairs to each vector when you insert vector data after you create a vector index. By default, all metadata keys that are attached to vectors are filterable and can be used as filters in a similarity query. Only metadata keys that are specified as non-filterable during vector index creation are excluded from filtering. For more information about metadata size limits per vector, including both total and filterable metadata constraints, see [Limitations and restrictions](s3-vectors-limitations.md). 1. For **Encryption**, choose **Specify encryption type** and then choose one of the following options: + **Use bucket settings for encryption** – Amazon S3 applies the vector bucket encryption settings to encrypt vector data in the vector index. + **Override bucket settings for encryption** – Specify a specific encryption type for the vector index: + **Server-side encryption with Amazon S3 managed keys (SSE-S3)** – With SSE-S3, Amazon S3 handles the generation, rotation, and management of encryption keys automatically. + **Server-side encryption with AWS Key Management Service keys (SSE-KMS)** – Similar to SSE-S3, but uses customer managed keys (CMKs) in AWS KMS, giving you more control over your keys. For more information about customer managed keys, see [Customer managed keys](https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html#customer-cmk) in the *AWS Key Management Service Developer Guide*. If you select this option, under **AWS KMS key**, choose one of the following options: + **Choose from your AWS KMS keys** – Select an existing KMS key from the dropdown list + **Enter AWS KMS key ARN** – Enter the Amazon Resource Name (ARN) of a KMS key + **Create a KMS key** – Create a new customer managed key in the AWS KMS console. For more information, see [Creating symmetric customer managed keys](https://docs.aws.amazon.com/kms/latest/developerguide/create-keys.html#create-symmetric-cmk) in the *AWS Key Management Service Developer Guide*. **Note** The following requirements apply to the KMS key: AWS KMS key ID must not be empty. Your KMS key must be in the same Region where this bucket is being created. AWS KMS key ARN must start with "arn:aws:kms:" **Important** Encryption settings can't be changed after the vector index is created. If you chose **Enter AWS KMS key ARN**, enter the ARN in the text field provided. If you chose **Create a KMS key**, the console opens the AWS KMS console in a new tab. For instructions on creating a KMS key, see *Creating symmetric customer managed keys* in the *AWS Key Management Service Developer Guide*. **Important** When using KMS encryption, ensure that the IAM principals that need to access objects in the bucket have the necessary KMS permission (kms:Decrypt) for the selected KMS key. 1. Under **Tags (Optional)**, you can add tags as key-value pairs to help track and organize vector index costs using AWS Billing and Cost Management. Enter a **Key** and a **Value**. To add another tag, choose **Add Tag**. You can enter up to 50 tags for a vector index. For more information, see [Using tags with S3 vector buckets](s3-vectors-tags.md). 1. Review your configuration carefully. **Note** These settings can't be changed after creation. 1. Choose **Create vector index**. ## Using the AWS CLI The following example shows how to create a vector bucket with the SSE-S3 encryption configuration by using the AWS CLI. To use this example, replace the *user input placeholders* with your own information. ``` aws s3vectors create-vector-bucket \ --vector-bucket-name "amzn-s3-demo-vector-bucket" \ --encryption-configuration '{"sseType": "AES256"}' ``` The following examples shows how to create a vector bucket that uses the SSE-KMS encryption configuration with a customer managed key. To use this example, replace the *user input placeholders* with your own information. ``` aws s3vectors create-vector-bucket \ --vector-bucket-name "amzn-s3-demo-vector-bucket" \ --encryption-configuration '{"sseType": "aws:kms", "kmsKeyArn": "arn:aws:kms:us-east-1:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"}' ``` The following example shows how to create a vector index with the SSE-S3 encryption configuration by using the AWS CLI. To use this example, replace the *user input placeholders* with your own information. ``` aws s3vectors create-index \ --vector-bucket-name "amzn-s3-demo-vector-bucket" \ --index-name "amzn-s3-demo-vector-index" \ --encryption-configuration '{"sseType": "AES256"}' ``` The following examples shows how to create a vector index that uses the SSE-KMS encryption configuration with a customer managed key. To use this example, replace the *user input placeholders* with your own information. ``` aws s3vectors create-index \ --vector-bucket-name "amzn-s3-demo-vector-bucket" \ --index-name "amzn-s3-demo-vector-index" \ --encryption-configuration '{"sseType": "aws:kms", "kmsKeyArn": "arn:aws:kms:us-east-1:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890abc"}' ``` The following examples shows will create a vector index using the encryption settings of the vector bucket. To use this example, replace the *user input placeholders* with your own information. ``` aws s3vectors create-index \ --vector-bucket-name "amzn-s3-demo-vector-bucket" \ --index-name "amzn-s3-demo-vector-index" \ ``` # Viewing encryption configuration in S3 Vectors Viewing encryption configuration in S3 Vectors After creating your vector bucket, you can verify the encryption configuration using the console. Alternatively, you can use the GetVectorBucket and GetIndex API operations via the AWS REST API, AWS CLI, or AWS SDKs. ## Using the AWS CLI Use the `get-vector-bucket` command to retrieve detailed bucket information, including encryption configuration. To use this example, replace the *user input placeholders* with your own information. ``` aws s3vectors get-vector-bucket \ --vector-bucket-name amzn-s3-demo-vector-bucket ``` Use the `get-index` command to retrieve detailed vector index information, including encryption configuration. To use this example, replace the *user input placeholders* with your own information. ``` aws s3vectors get-index \ --vector-bucket-name amzn-s3-demo-vector-bucket --index-name amzn-s3-demo-vector-index ``` # VPC endpoints for S3 Vectors To access S3 Vectors from your virtual private cloud (VPC), Amazon S3 supports interface VPC endpoints by using AWS PrivateLink (PrivateLink). PrivateLink provides private connectivity between your VPC and S3 Vectors without requiring an internet gateway or NAT device. Interface endpoints are represented by one ore more elastic network interfaces (ENIs) that are assigned private IP addresses from subnets in your VPC. Requests to S3 Vectors over interface endpoints stay on the AWS network. You can also access interface endpoints in your VPC from on-premises applications through AWS Direct Connect or AWS Virtual Private Network (AWS VPN). For more information about how to connect your VPC with your on-premises network, see the *[AWS Direct Connect User Guide](https://docs.aws.amazon.com/directconnect/latest/UserGuide/Welcome.html)* and the *[AWS Site-to-Site VPN User Guide](https://docs.aws.amazon.com/vpn/latest/s2svpn/VPC_VPN.html)*. For general information about interface endpoints, see [Access an AWS service using an interface VPC endpoint](https://docs.aws.amazon.com/vpc/latest/privatelink/create-interface-endpoint.html) in the *AWS PrivateLink Guide*. ## Benefits of using PrivateLink with S3 Vectors Using PrivateLink with S3 Vectors provides several security and operational benefits: + **Enhanced security**: Traffic between your VPC and S3 Vectors remains within the AWS network and doesn't traverse the internet. + **Simplified network architecture**: Access S3 Vectors without configuring internet gateways, NAT devices, or VPN connections. + **Granular access control**: Use VPC endpoint policies to control which vector buckets and vector indexes can be accessed through the endpoint. + **Compliance support**: Meet regulatory requirements that mandate private network connectivity for sensitive data. ## VPC endpoint DNS names and resolution When you create a VPC endpoint, S3 Vectors generates two types of endpoint-specific DNS names: Regional and Zonal. The Regional and Zonal DNS names of interface VPC endpoints for S3 Vectors are as follows: + **Regional DNS name**: `vpce-1a2b3c4d-5e6f.s3vectors.region.vpce.amazonaws.com` - The regional VPC endpoint DNS name. Always resolve to private IP addresses. + **Zonal DNS name**: `vpce-1a2b3c4d-5e6f-availability_zone_code.s3vectors.region.vpce.amazonaws.com` - Zone-specific VPC endpoint DNS names. Always resolve to private IP addresses. You can also use the DNS name of the public endpoint `s3vectors.region.api.aws` as the private DNS name of the endpoint service if you have private DNS enabled for the VPC endpoint. ## IP addressing for interface endpoints S3 Vectors regional, zonal, and private DNS endpoints support IPv4, IPv6, and dualstack IP types for AWS PrivateLink. For more information, see [IP address types](https://docs.aws.amazon.com/vpc/latest/privatelink/privatelink-access-aws-services.html#aws-service-ip-address-type) and [DNS record IP type for AWS services](https://docs.aws.amazon.com/vpc/latest/privatelink/privatelink-access-aws-services.html#aws-services-dns-record-ip-type) in the *AWS PrivateLink Guide*. The following are some things you should know before trying to access S3 Vectors vector indexes and vector buckets over IPv6 in your VPC: + The client you use to access vectors and your S3 Vectors client must both have dual-stack enabled. + If your VPC security group doesn't have IPv6 set up, you'll need to configure a rule to allow IPv6 traffic. For more information, see [Step 3: Update your security group rules](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-migrate-ipv6-add.html#vpc-migrate-ipv6-sg-rules) in the *VPC User Guide* and [Configure security group rules](https://docs.aws.amazon.com/ec2/latest/userguide/working-with-security-groups.html#adding-security-group-rule) in the *Amazon EC2 User Guide*. + If your VPC doesn't have IPv6 CIDRs assigned, you will need to manually add an IPv6 CIDR block to your VPC. For more information, see [Add IPv6 support for your VPC](https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints-access.html#vpc-endpoint-ipv6) in the *AWS PrivateLink Guide*. + If you use IP address filtering IAM policies, they must be updated to handle IPv6 addresses. For more information about managing access permissions with IAM, see [Identity and Access management in S3 Vectors](s3-vectors-access-management.md). ## Creating a VPC interface endpoint for S3 Vectors You can create a VPC interface endpoint for S3 Vectors using the VPC console, AWS CLI, AWS SDKs, or AWS API. ### Using the S3 console 1. Open the VPC console at [https://console.aws.amazon.com/vpc/](https://console.aws.amazon.com/vpc/). 1. In the navigation pane, choose **Endpoints**. 1. Choose **Create endpoint**. 1. For **Service category**, choose **AWS services**. 1. For **Services**, search for `s3vectors` and select `com.amazonaws.region.s3vectors`. 1. For **VPC**, select the VPC where you want to create the endpoint. 1. (Optional) Under **Additional settings**, for **Enable DNS name**, choose whether to enable the private DNS feature. When enabled, requests that use the public service endpoint (`s3vectors.region.api.aws`), such as requests made through AWS SDKs, resolve to your VPC endpoint instead of the public endpoint. 1. For **Subnets**, select the subnets where you want to create the endpoint network interfaces. 1. For **IP address type**, choose the IP address type for the endpoint: + **IPv4**: Assign IPv4 addresses to the endpoint network interfaces. This option is supported only if all selected subnets have IPv4 address ranges. + **IPv6**: Assign IPv6 addresses to the endpoint network interfaces. This option is supported only if all selected subnets are IPv6 only subnets. + **Dualstack**: Assign both IPv4 and IPv6 addresses to the endpoint network interfaces. This option is supported only if all selected subnets have both IPv4 and IPv6 address ranges. 1. For **Security groups**, select the security groups to associate with the endpoint network interfaces. 1. (Optional) For **Policy**, you can attach a VPC endpoint policy to control access to S3 Vectors through the endpoint. To allow all operations by all principals on all S3 Vectors resources over the interface endpoint, choose **Full access**. To restrict access, choose **Custom** and enter a policy. For more information, see [Control access to VPC endpoints using endpoint policies](https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints-access.html) in the AWS PrivateLink Guide. If you don't attach a policy, the default policy allows full access. 1. Choose **Create endpoint**. ### Using the AWS CLI To create a new VPC endpoint that returns both IPv4 and IPv6 for S3 Vectors, use the following example CLI command. For more information, see [create-vpc-endpoint](https://docs.aws.amazon.com/cli/latest/reference/ec2/create-vpc-endpoint.html). ``` aws ec2 create-vpc-endpoint \ --vpc-id vpc-12345678 \ --service-name com.amazonaws.region.s3vectors \ --vpc-endpoint-type Interface \ --subnet-ids subnet-12345678 subnet-87654321 \ --security-group-ids sg-12345678 \ --ip-address-type dualstack \ --private-dns-enabled ``` The `--private-dns-enabled` parameter enables the private DNS feature. When enabled, requests to `s3vectors.region.api.aws` will route through your VPC endpoint. For more information about creating VPC endpoints, see [Create a VPC endpoint](https://docs.aws.amazon.com/vpc/latest/privatelink/create-interface-endpoint.html#create-interface-endpoint-aws) in the *VPC User Guide*. ## VPC endpoint policies for S3 Vectors Similar to resource-based policies, you can attach an endpoint policy to your VPC endpoint to control access to vector indexes and vector buckets. For more information about the endpoint policies, see [Control access to VPC endpoints using endpoint policies](https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints-access.html) in the AWS PrivateLink Guide. ### Example VPC endpoint policies The following example VPC endpoint policy allows access to all S3 Vectors operations for all principals: ``` { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": "*", "Action": [ "s3vectors:*" ], "Resource": "*" } ] } ``` The following example VPC endpoint policy restricts access to a specific vector bucket: ``` { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": "*", "Action": [ "s3vectors:GetVectorBucket", "s3vectors:ListIndexes", "s3vectors:GetIndex", "s3vectors:QueryVectors", "s3vectors:GetVectors" ], "Resource": [ "arn:aws:s3vectors:us-west-2:111122223333:bucket/amzn-s3-demo-vector-bucket", "arn:aws:s3vectors:us-west-2:111122223333:bucket/amzn-s3-demo-vector-bucket/*" ] } ] } ``` The following example VPC endpoint policy allows access only during business hours using the `aws:CurrentTime` condition key: ``` { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": "*", "Action": "s3vectors:*", "Resource": "*", "Condition": { "DateGreaterThan": { "aws:CurrentTime": "08:00Z" }, "DateLessThan": { "aws:CurrentTime": "18:00Z" } } } ] } ``` ## Configuring S3 Vectors clients for VPC endpoints When using VPC endpoints with S3 Vectors, you can configure your S3 Vectors clients to use either the service DNS name or the VPC endpoint DNS name. ### Using the AWS SDKs ------ #### [ SDK for Python ] The following example shows how to configure the S3 Vectors client in SDK for Python (Boto3) to use a VPC endpoint: ``` import boto3 # Using service DNS name (requires private DNS feature enabled on VPC endpoint) s3vectors_client = boto3.client( 's3vectors', region_name='us-west-2', endpoint_url='https://s3vectors.us-west-2.api.aws' ) # Using VPC endpoint DNS name s3vectors_client = boto3.client( 's3vectors', region_name='us-west-2', endpoint_url='https://vpce-12345678.s3vectors.us-west-2.vpce.amazonaws.com' ) ``` ------ ## Troubleshooting VPC endpoints If you're experiencing issues with your interface VPC endpoint, consider the following troubleshooting steps: + **DNS resolution**: Verify that DNS queries for the endpoint resolve to private IP addresses within your VPC CIDR range when using private DNS. + **Security groups**: Ensure that the security group associated with the VPC endpoint allows inbound HTTPS traffic (port 443) from your VPC resources. + **Route tables**: Verify that your subnet route tables don't have conflicting routes that might redirect traffic away from the VPC endpoint. + **VPC endpoint policy**: Check that your VPC endpoint policy allows the necessary S3 Vectors actions and resources. + **Client configuration**: If the private DNS feature is disabled, configure your S3 Vectors client to use the VPC endpoint DNS name instead of the service DNS name. ## Monitoring VPC endpoint usage You can monitor your S3 Vectors VPC endpoint usage through CloudTrail [NetworkActivity](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/logging-network-events-with-cloudtrail.html) events logs. For more information about S3 Vectors logging, see [Logging with AWS CloudTrail for S3 Vectors](s3-vectors-logging.md). # Logging with AWS CloudTrail for S3 Vectors Amazon S3 Vectors is integrated with AWS CloudTrail, a service that provides a record of actions that are taken by a user, role, or an AWS service. CloudTrail captures all API calls for S3 Vectors as events. Using the information that's collected by CloudTrail, you can determine the request that was made to S3 Vectors, the IP address from which the request was made, when it was made, and additional details. When a supported event activity occurs in S3 Vectors, that activity is recorded in a CloudTrail event. You can use CloudTrail trail to log management events and data events for S3 Vectors. To learn more about CloudTrail, see the [CloudTrail User Guide](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/). ## S3 Vectors information in CloudTrail CloudTrail is enabled on your AWS account when you create the account. When activity occurs in S3 Vectors, that activity is recorded in a CloudTrail event along with other AWS service events in Event history. You can view, search, and download recent events in your AWS account. For more information, see [Viewing Events with CloudTrail Event History](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/view-cloudtrail-events.html). For an ongoing record of events in your AWS account, including events for S3 Vectors, create a trail. A trail enables CloudTrail to deliver log files to an Amazon S3 bucket. By default, when you create a trail in the console, the trail applies to all AWS Regions. The trail logs events from all Regions in the AWS partition and delivers the log files to the Amazon S3 bucket that you specify. Additionally, you can configure other AWS services to further analyze and act upon the event data collected in CloudTrail logs. For more information, see [Overview for Creating a Trail](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html), [Configuring Amazon SNS Notifications for CloudTrail](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/getting-notifications-top-level.html), [Receiving CloudTrail Log Files from Multiple Regions](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/receive-cloudtrail-log-files-from-multiple-regions.html), and [Receiving CloudTrail Log Files from Multiple Accounts](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-receive-logs-from-multiple-accounts.html) in the *CloudTrail User Guide*. All S3 Vectors API actions are logged by CloudTrail and are documented in the [Amazon S3 Vectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_Operations_Amazon_S3_Vectors.html) API Reference. For example, calls to the [CreateVectorBucket](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_CreateVectorBucket.html), [CreateIndex](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_CreateIndex.html), and [QueryVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_QueryVectors.html) actions generate entries in the CloudTrail log files. Every event or log entry contains information about who generated the request. The identity information helps you determine the following: + Whether the request was made with root or IAM user credentials. + Whether the request was made with temporary security credentials for a role or federated user. + Whether the request was made by another AWS service. For more information, see the [CloudTrail userIdentity Element](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-event-reference-user-identity.html) in the *CloudTrail User Guide*. ## CloudTrail management events for S3 Vectors Management events provide information about management operations that are performed on resources in your AWS account. These are also known as control plane operations. By default, CloudTrail logs management events. For S3 Vectors, CloudTrail logs the following management events: + [CreateVectorBucket](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_CreateVectorBucket.html) + [DeleteVectorBucket](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_DeleteVectorBucket.html) + [GetVectorBucket](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_GetVectorBucket.html) + [ListVectorBuckets](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_ListVectorBuckets.html) + [PutVectorBucketPolicy](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_PutVectorBucketPolicy.html) + [GetVectorBucketPolicy](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_GetVectorBucketPolicy.html) + [DeleteVectorBucketPolicy](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_DeleteVectorBucketPolicy.html) + [CreateIndex](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_CreateIndex.html) + [DeleteIndex](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_DeleteIndex.html) + [GetIndex](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_GetIndex.html) + [ListIndexes](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_ListIndexes.html) The `eventSource` for S3 Vectors management events and data events is `s3vectors.amazonaws.com`. For more information about management events, see [Logging management events](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/logging-management-events-with-cloudtrail.html) in the *CloudTrail User Guide*. ## CloudTrail data events for S3 Vectors Data events provide information about the resource operations performed on or in a resource. These are also known as data plane operations. By default, CloudTrail doesn't log data events. However, you can configure trails to log data events for S3 Vectors resources. When you configure your trail to log data events, you can specify the S3 Vectors resource type. S3 Vectors supports the following resource types for data events: + `AWS::S3Vectors::VectorBucket` - Logs data events for all vector indexes in the specified vector buckets + `AWS::S3Vectors::Index` - Logs data events for specific vector indexes For S3 Vectors, CloudTrail logs the following data events: Vector data operations: + [PutVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_PutVectors.html) - Logs when vectors are added to a vector index + [GetVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_GetVectors.html) - Logs when vectors are retrieved from a vector index + [DeleteVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_DeleteVectors.html) - Logs when vectors are deleted from a vector index + [ListVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_ListVectors.html) - Logs when vectors in a vector index are listed + [QueryVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_QueryVectors.html) - Logs when similarity queries are performed on a vector index The `eventSource` for S3 Vectors data events is `s3vectors.amazonaws.com`. ## Enabling data event logging for S3 Vectors You can enable data event logging for S3 Vectors resources when you create or update a CloudTrail trail. You can specify logging for all vector buckets and vector indexes in your account, or you can specify individual vector buckets or vector indexes. For detailed steps about creating a trail, see [Creating a trail](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-a-trail-using-the-console-first-time.html) in the *CloudTrail User Guide*. To enable data event logging for all S3 Vectors resources: + When creating or updating a trail, choose **Data events**. + For **Resource type**, choose `AWS::S3Vectors::VectorBucket`. + For **Resource ARN**, enter `arn:aws:s3vectors:_:_:bucket/*` to log events for all vector buckets, or specify individual vector bucket ARNs (for example, `arn:aws:s3vectors:us-east-1:123456789012:bucket/amzn-s3-demo-vector-bucket`). To enable data event logging for specific vector indexes: + When creating or updating a trail, choose **Data events**. + For **Resource type**, choose `AWS::S3Vectors::Index`. + For **Resource ARN**, enter the ARN of the specific vector index, such as: `arn:aws:s3vectors:us-east-1:123456789012:bucket/amzn-s3-demo-vector-bucket/index/my-index`. For more information about data events, see [Logging data events](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/logging-data-events-with-cloudtrail.html) in the *CloudTrail User Guide*. **Topics** + [ ## S3 Vectors information in CloudTrail ](#s3-vectors-logging-information) + [ ## CloudTrail management events for S3 Vectors ](#s3-vectors-logging-management-events) + [ ## CloudTrail data events for S3 Vectors ](#s3-vectors-logging-data-events) + [ ## Enabling data event logging for S3 Vectors ](#s3-vectors-logging-enabling-data-events) + [ # CloudTrail log file example for S3 Vectors ](s3-vectors-cloudtrail-log-example.md) # CloudTrail log file example for S3 Vectors The following examples show CloudTrail log entries for S3 Vectors data events. Data events are logged when you perform operations on vector data within your vector indexes. ## Example: CloudTrail log file for `GetVectors` data event ``` { "eventVersion": "1.11", "userIdentity": { "type": "IAMUser", "principalId": "123456789012", "arn": "arn:aws:iam::123456789012:user/myUserName", "accountId": "123456789012", "accessKeyId": "AKIAIOSFODNN7EXAMPLE", "userName": "myUserName" }, "eventTime": "2024-11-22T17:12:25Z", "eventSource": "s3vectors.amazonaws.com", "eventName": "GetVectors", "awsRegion": "us-east-1", "sourceIPAddress": "192.0.2.0", "userAgent": "[aws-cli/2.18.5]", "requestParameters": { "vectorBucketName": "amzn-s3-demo-vector-bucket", "returnMetadata": "false", "indexName": "111aa1111-22bb-33cc-44dd-5555eee66ffff", "returnData": "false" }, "responseElements": null, "additionalEventData": { "SignatureVersion": "SigV4", "cipherSuite": "ECDHE-RSA-AES128-GCM-SHA256" }, "requestID": "07D681123BD12AED", "eventID": "f2b287f3-0df1-1234-a2f4-c4bdfed47657", "readOnly": true, "resources": [{ "accountId": "123456789012", "type": "AWS::S3Vectors::VectorBucket", "ARN": "arn:aws:s3vectors:us-east-1:123456789012:bucket/amzn-s3-demo-vector-bucket" }, { "accountId": "123456789012", "type": "AWS::S3Vectors::Index", "ARN": "arn:aws:s3vectors:us-east-1:123456789012:bucket/amzn-s3-demo-vector-bucket/index/111aa1111-22bb-33cc-44dd-5555eee66ffff" }], "eventType": "AwsApiCall", "managementEvent": false, "recipientAccountId": "444455556666", "eventCategory": "Data", "tlsDetails": { "tlsVersion": "TLSv1.2", "cipherSuite": "ECDHE-RSA-AES128-GCM-SHA256", "clientProvidedHostHeader": "client-host" } } ``` ## Understanding S3 Vectors log file entries CloudTrail log files contain one or more log entries. An event represents a single request from any source and includes information about the requested action, the date and time of the action, request parameters, and so on. CloudTrail log files aren't an ordered stack trace of the public API calls, so they don't appear in any specific order. S3 Vectors CloudTrail log entries contain the following key elements: + **eventSource** - Always `s3vectors.amazonaws.com` for S3 Vectors events. + **eventName** - The S3 Vectors API operation that was performed. + **eventCategory** - Either `Management` for control plane operations or `Data` for data plane operations. + **readOnly**: `true` for read operations (for examples, [GetVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_GetVectors.html), [QueryVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_QueryVectors.html), [ListVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_ListVectors.html)) and `false` for write operations ([PutVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_PutVectors.html), [DeleteVectors](https://docs.aws.amazon.com/AmazonS3/latest/API/API_S3VectorBuckets_DeleteVectors.html)). + **resources** - The S3 Vectors resources involved in the operation, including vector buckets and vector indexes. + **requestParameters** - The parameters that were sent with the request. + **responseElements** - The response elements returned by the S3 Vectors service.