Build a RAG application using Amazon Bedrock Knowledge Bases

Create a local directory and download sample PDFs.

$ mkdir -p ~/kb-pdfs && cd ~/kb-pdfs
curl -sSL -o aws-overview.pdf https://d1.awsstatic.com/whitepapers/aws-overview.pdf
curl -sSL -o wellarchitected-framework.pdf https://docs.aws.amazon.com/pdfs/wellarchitected/latest/framework/wellarchitected-framework.pdf
curl -sSL -o s3-userguide.pdf https://docs.aws.amazon.com/pdfs/AmazonS3/latest/userguide/s3-userguide.pdf

Upload each file to the access point. Replace access-point-alias with your access point alias (for example, my-kb-ap-a1b2c3d4e5f6g7h8i9j0kl1mnop2uuse1a-ext-s3alias). You can find the alias in the Amazon FSx console under Attached Amazon S3 access points for your volume, or by running aws fsx describe-s3-access-point-attachments.

$ for f in *.pdf; do
    aws s3 cp "$f" "s3://access-point-alias/$f"
done

Verify the files landed on the volume.

$ aws s3 ls s3://access-point-alias/

Create an Amazon S3 vector bucket. Vector bucket names follow the same global-uniqueness rules as standard Amazon S3 buckets. This tutorial uses fsxn-kb-vectors; replace with a unique name.

$ aws s3vectors create-vector-bucket --vector-bucket-name fsxn-kb-vectors

Create a vector index in the bucket. The index dimension must match the output dimension of the embedding model; Titan Text Embeddings v2 outputs 1024 dimensions. The nonFilterableMetadataKeys setting marks the Bedrock metadata fields as non-filterable, which keeps them out of the 2 KB per-vector filterable metadata limit.

$ aws s3vectors create-index --vector-bucket-name fsxn-kb-vectors \
    --index-name bedrock-kb-index \
    --dimension 1024 --distance-metric cosine --data-type float32 \
    --metadata-configuration '{"nonFilterableMetadataKeys":["AMAZON_BEDROCK_METADATA","AMAZON_BEDROCK_TEXT"]}'

Note the indexArn in the response; you use it in Step 4: Create the knowledge base and data source.

Open the Amazon Bedrock console at https://console.aws.amazon.com/bedrock/.

In the left navigation pane, choose Knowledge bases, and then choose Create knowledge base.

Under Knowledge base details, enter a name (for example, fsxn-kb) and description.

Under IAM permissions, choose Create and use a new service role.

For Data source, choose Amazon S3, and then choose Next.

Enter a data source name (for example, fsxn-s3ap-source).

For S3 URI, enter s3:// followed by your access point alias, for example s3://my-kb-ap-a1b2c3d4e5f6g7h8i9j0kl1mnop2uuse1a-ext-s3alias. The console does not distinguish between a bucket name and an access point alias in this field; the access point alias is accepted as-is.

Choose Next.

Under Embeddings model, choose Titan Text Embeddings v2.

Under Vector database, choose Quick create a new vector store and select Amazon S3 Vectors. Choose Next.

Review the configuration and choose Create knowledge base. The knowledge base can take several minutes to create.

Create the knowledge base. Replace the placeholders with your values. The indexArn is the ARN of the Amazon S3 vector index you created in Step 2.

$ aws bedrock-agent create-knowledge-base --name fsxn-kb \
    --role-arn arn:aws:iam::account-id:role/fsxn-kb-role \
    --knowledge-base-configuration '{
        "type":"VECTOR",
        "vectorKnowledgeBaseConfiguration":{
            "embeddingModelArn":"arn:aws:bedrock:region::foundation-model/amazon.titan-embed-text-v2:0"
        }
    }' \
    --storage-configuration '{
        "type":"S3_VECTORS",
        "s3VectorsConfiguration":{
            "indexArn":"index-arn"
        }
    }'

Note the knowledgeBaseId in the response.

Create the data source. Pass the Amazon S3 access point alias as the bucket name in the bucketArn field, using the form arn:aws:s3:::access-point-alias.

$ aws bedrock-agent create-data-source \
    --knowledge-base-id knowledge-base-id \
    --name fsxn-s3ap-source \
    --data-source-configuration '{
        "type":"S3",
        "s3Configuration":{"bucketArn":"arn:aws:s3:::access-point-alias"}
    }'

Note the dataSourceId in the response.

In the Amazon Bedrock console, open your knowledge base.

In the Data source section, select your data source, and then choose Sync.

Wait for Sync status to show Ready.

Start the ingestion job.

$ aws bedrock-agent start-ingestion-job \
    --knowledge-base-id knowledge-base-id \
    --data-source-id data-source-id

Note the ingestionJobId in the response.

Poll the job until it completes.

$ aws bedrock-agent get-ingestion-job \
    --knowledge-base-id knowledge-base-id \
    --data-source-id data-source-id \
    --ingestion-job-id ingestion-job-id

The status field transitions from IN_PROGRESS to COMPLETE. The statistics field shows how many documents were scanned and indexed.

In the Amazon Bedrock console, open your knowledge base.

Choose Test knowledge base.

Under Generate responses, select a text generation model (for example, Nova Lite).

Enter a question such as What are the pillars of the AWS Well-Architected Framework? and choose Run. The answer displays with citation references that link to the source documents in the Amazon S3 access point.