S3¶

Client¶

class S3.Client¶

A low-level client representing Amazon Simple Storage Service (S3)

Note

For information about using the Amazon S3 API—including authentication, signing requests, code examples, and error handling—see the Amazon S3 Developer Guide.

Welcome to the Amazon S3 API Reference. This guide explains the Amazon Simple Storage Service (Amazon S3) application programming interface (API).

You can use any toolkit that supports HTTP to use the REST API. You can even use a browser to fetch objects, as long as they are anonymously readable.

The REST API uses the standard HTTP headers and status codes, so that standard browsers and toolkits work as expected. In some areas, we have added functionality to HTTP (for example, we added headers to support access control). In these cases, we have done our best to add the new functionality in a way that matched the style of standard HTTP usage.

The current version of the Amazon S3 API is 2006-03-01.

Amazon S3 supports the REST API.

Note

Support for SOAP over HTTP is deprecated, but it is still available over HTTPS. However, new Amazon S3 features will not be supported for SOAP. We recommend that you use either this REST API or the Amazon Web Services SDKs.

import boto3

client = boto3.client('s3')

These are the available methods:

Paginators¶

Paginators are available on a client instance via the get_paginator method. For more detailed instructions and examples on the usage of paginators, see the paginators user guide.

The available paginators are:

Waiters¶

Waiters are available on a client instance via the get_waiter method. For more detailed instructions and examples on the usage or waiters, see the waiters user guide.

The available waiters are:

Resources¶

Resources are available in boto3 via the resource method. For more detailed instructions and examples on the usage of resources, see the resources user guide.

The available resources are:

Examples¶

List objects in an Amazon S3 bucket¶

The following example shows how to use an Amazon S3 bucket resource to list the objects in the bucket.

import boto3

s3 = boto3.resource('s3')
bucket = s3.Bucket('amzn-s3-demo-bucket')
for obj in bucket.objects.all():
    print(obj.key)

List top-level common prefixes in Amazon S3 bucket¶

This example shows how to list all of the top-level common prefixes in an Amazon S3 bucket:

import boto3

client = boto3.client('s3')
paginator = client.get_paginator('list_objects')
result = paginator.paginate(Bucket='amzn-s3-demo-bucket', Delimiter='/')
for prefix in result.search('CommonPrefixes'):
    print(prefix.get('Prefix'))

Restore Glacier objects in an Amazon S3 bucket¶

The following example shows how to initiate restoration of glacier objects in an Amazon S3 bucket, determine if a restoration is on-going, and determine if a restoration is finished.

import boto3

s3 = boto3.resource('s3')
bucket = s3.Bucket('amzn-s3-demo-bucket')
for obj_sum in bucket.objects.all():
    obj = s3.Object(obj_sum.bucket_name, obj_sum.key)
    if obj.storage_class == 'GLACIER':
        # Try to restore the object if the storage class is glacier and
        # the object does not have a completed or ongoing restoration
        # request.
        if obj.restore is None:
            print('Submitting restoration request: %s' % obj.key)
            obj.restore_object(RestoreRequest={'Days': 1})
        # Print out objects whose restoration is on-going
        elif 'ongoing-request="true"' in obj.restore:
            print('Restoration in-progress: %s' % obj.key)
        # Print out objects whose restoration is complete
        elif 'ongoing-request="false"' in obj.restore:
            print('Restoration complete: %s' % obj.key)

Uploading/downloading files using SSE KMS¶

This example shows how to use SSE-KMS to upload objects using server side encryption with a key managed by KMS.

We can either use the default KMS master key, or create a custom key in AWS and use it to encrypt the object by passing in its key id.

With KMS, nothing else needs to be provided for getting the object; S3 already knows how to decrypt the object.

import boto3
import os

BUCKET = 'amzn-s3-demo-bucket'
s3 = boto3.client('s3')
keyid = '<the key id>'

print("Uploading S3 object with SSE-KMS")
s3.put_object(Bucket=BUCKET,
              Key='encrypt-key',
              Body=b'foobar',
              ServerSideEncryption='aws:kms',
              # Optional: SSEKMSKeyId
              SSEKMSKeyId=keyid)
print("Done")

# Getting the object:
print("Getting S3 object...")
response = s3.get_object(Bucket=BUCKET,
                         Key='encrypt-key')
print("Done, response body:")
print(response['Body'].read())

Uploading/downloading files using SSE Customer Keys¶

This example shows how to use SSE-C to upload objects using server side encryption with a customer provided key.

First, we’ll need a 32 byte key. For this example, we’ll randomly generate a key but you can use any 32 byte key you want. Remember, you must the same key to download the object. If you lose the encryption key, you lose the object.

Also note how we don’t have to provide the SSECustomerKeyMD5. Boto3 will automatically compute this value for us.

import boto3
import os

BUCKET = 'amzn-s3-demo-bucket'
KEY = os.urandom(32)
s3 = boto3.client('s3')

print("Uploading S3 object with SSE-C")
s3.put_object(Bucket=BUCKET,
              Key='encrypt-key',
              Body=b'foobar',
              SSECustomerKey=KEY,
              SSECustomerAlgorithm='AES256')
print("Done")

# Getting the object:
print("Getting S3 object...")
# Note how we're using the same ``KEY`` we
# created earlier.
response = s3.get_object(Bucket=BUCKET,
                         Key='encrypt-key',
                         SSECustomerKey=KEY,
                         SSECustomerAlgorithm='AES256')
print("Done, response body:")
print(response['Body'].read())

Downloading a specific version of an S3 object¶

This example shows how to download a specific version of an S3 object.

import boto3
s3 = boto3.client('s3')

s3.download_file(
    "amzn-s3-demo-bucket", "key-name", "tmp.txt",
    ExtraArgs={"VersionId": "my-version-id"}
)

Filter objects by last modified time using JMESPath¶

This example shows how to filter objects by last modified time using JMESPath.

import boto3
s3 = boto3.client("s3")

s3_paginator = s3.get_paginator('list_objects_v2')
s3_iterator = s3_paginator.paginate(Bucket='amzn-s3-demo-bucket')

filtered_iterator = s3_iterator.search(
    "Contents[?to_string(LastModified)>='\"2022-01-05 08:05:37+00:00\"'].Key"
)

for key_data in filtered_iterator:
    print(key_data)

Client Context Parameters¶

Client context parameters are configurable on a client instance via the client_context_params parameter in the Config object. For more detailed instructions and examples on the exact usage of context params see the configuration guide.

The available s3 client context params are:

disable_s3_express_session_auth (boolean) - Disables this client’s usage of Session Auth for S3Express
buckets and reverts to using conventional SigV4 for those.