AWS Snowball Edge is no longer available to new customers. New customers should explore [AWS DataSync](https://aws.amazon.com/datasync/) for online transfers, [AWS Data Transfer Terminal](https://aws.amazon.com/data-transfer-terminal/) for secure physical transfers, or AWS Partner solutions. For edge computing, explore [AWS Outposts](https://aws.amazon.com/outposts/). # Using Amazon S3 compatible storage on Snowball Edge Amazon S3 compatible storage on Snowball Edge delivers secure object storage with increased resiliency, scale, and an expanded Amazon S3 API feature-set to rugged, mobile edge, and disconnected environments. Using Amazon S3 compatible storage on Snowball Edge, you can store data and run highly available applications on Snowball Edge for edge computing. You can create Amazon S3 buckets on the Snowball Edge devices to store and retrieve objects on premises for applications that require local data access, local data processing, and data residency. Amazon S3 compatible storage on Snowball Edge provides a new storage class, `SNOW`, which uses the Amazon S3 APIs, and is designed to store data durably and redundantly across multiple Snowball Edge devices. You can use the same APIs and features on Snowball Edge buckets that you do on Amazon S3, including bucket lifecycle policies, encryption, and tagging. When the device or devices are returned to AWS, all data created or stored in Amazon S3 compatible storage on Snowball Edge is erased. For more information, see [Local Compute and Storage Only Jobs](https://docs.aws.amazon.com/snowball/latest/developer-guide/computetype.html). You can deploy Amazon S3 compatible storage on Snowball Edge in standalone configuration or in cluster configuration. In standalone configuration, you can provision S3 capacity on the device and the balance is available as block storage. In cluster configuration, all data disk capacity is used for S3 storage. A cluster may consist of a minimum of 3 devices up to a maximum of 16 devices. Depending on the size of cluster, S3 service is designed to sustain device fault tolerance of 1 or 2 devices. With AWS DataSync, you can transfer objects between Amazon S3 compatible storage on Snowball Edge on a Snowball Edge device and AWS storage services. For more information, see [Configuring transfers with S3 compatible storage on Snowball Edge](https://docs.aws.amazon.com/datasync/latest/userguide/s3-compatible-storage-snow.html) in the AWS DataSync User Guide. Following is the Amazon S3 compatible storage on Snowball Edge storage capacity and block storage capacity for a standalone device using Amazon S3 compatible storage on Snowball Edge. For fault tolerance and storage capacity of clusters, see [this table](ClusterOverview.md#cluster-table). ------ #### [ Snowball Edge Compute Optimized with NVMe storage ] **Storage capacity of Amazon S3 compatible storage on Snowball Edge and block storage of Snowball Edge Compute Optimized (Compute Optimized with AMD EPYC Gen2 and NVMe) devices** | Amazon S3 compatible storage on Snowball Edge storage capacity (in TB) | Block storage capacity (in TB) | | --- | --- | | 3 | 17.5 | | 5.5 | 14.5 | | 10.5 | 8.5 | | 12 | 6.5 | | 13 | 5.5 | | 16.5 | 1.5 | ------ #### [ Snowball Edge storage optimized 210 TB ] **Storage capacity of Amazon S3 compatible storage on Snowball Edge and block storage of Snowball Edge storage optimized 210 TB devices** | Amazon S3 compatible storage on Snowball Edge storage capacity (in TB) | Block storage capacity (in TB) | | --- | --- | | 20 | 206 | | 40 | 182 | | 60 | 158 | | 80 | 134 | | 100 | 110 | | 120 | 86 | | 140 | 62 | | 160 | 38 | | 180 | 14 | | 190 | 2 | ------ Amazon S3 compatible storage on Snowball Edge specifications: + The maximum number of Snowball Edge buckets is 100 per device or per cluster. + The S3 on Snowball Edge bucket owner account owns all objects in the bucket. + Only the S3 on Snowball Edge bucket owner account can perform operations on the bucket. + Object size limitations are consistent with those in Amazon S3. + All objects stored on S3 on Snowball Edge have SNOW as the storage class. + By default, all objects stored in the SNOW storage class are stored using server-side encryption with Amazon S3 managed encryption keys (SSE-S3). You can also explicitly choose to store objects by using server-side encryption with customer-provided encryption keys (SSE-C). + If there is not enough space to store an object on your Snowball Edge, the API returns an insufficient capacity exception (ICE). **Topics** + [Order Amazon S3 compatible storage on Snowball Edge](s3-edge-snow-order-device.md) + [Setting up and starting Amazon S3 compatible storage on Snowball Edge](s3-edge-snow-setting-up.md) + [Working with S3 buckets with Amazon S3 compatible storage on Snowball Edge](working-s3-snow-buckets.md) + [Determining whether you can access an Amazon S3 compatible storage on Snowball Edge bucket on a Snowball Edge](working-s3-snow-buckets-determine-bucket-access-s3-snow.md) + [Retrieving a list of buckets or regional buckets in Amazon S3 compatible storage on Snowball Edge on a Snowball Edge](working-s3-snow-buckets-list-buckets-s3-snow.md) + [Getting a bucket with Amazon S3 compatible storage on Snowball Edge on a Snowball Edge](working-s3-snow-buckets-get-bucket-s3-snow.md) + [Creating an S3 bucket in Amazon S3 compatible storage on Snowball Edge on a Snowball Edge](working-s3-snow-buckets-creating-s3-snow-bucket.md) + [Deleting a bucket in Amazon S3 compatible storage on Snowball Edge on a Snowball Edge](working-s3-snow-buckets-delete-bucket-s3-snow.md) + [Creating and managing an object lifecycle configuration using the AWS CLI](working-s3-snow-buckets-lifecycle-s3-snow.md) + [Copying an object to an Amazon S3 compatible storage on Snowball Edge bucket on a Snowball Edge](objects-copy-s3-snow.md) + [Listing objects in a bucket in Amazon S3 compatible storage on Snowball Edge on a Snowball Edge](objects-list-s3-snow.md) + [Getting an object from a bucket in Amazon S3 compatible storage on Snowball Edge on a Snowball Edge](objects-get-s3-snow.md) + [Deleting objects in buckets in Amazon S3 compatible storage on Snowball Edge](objects-delete-s3-snow.md) + [Supported REST API actions for Amazon S3 compatible storage on Snowball Edge](s3-snow-api.md) + [Using Amazon S3 compatible storage on Snowball Edge with a cluster of Snow devices](ClusterOverview.md) + [Configuring Amazon S3 compatible storage on Snowball Edge event notifications](s3-snow-event-notifications.md) + [Configuring local SMTP notifications on Snowball Edge](s3-snow-smtp-notifications.md) # Order Amazon S3 compatible storage on Snowball Edge Ordering a device for Amazon S3 compatible storage on Snowball Edge is very similar to the process for ordering a Snowball Edge. To order, see [Creating a job to order a Snowball Edge device](create-job-common.md) in this guide and keep these items in mind during the ordering process: + For **Choose a job type**, choose **Local compute and storage only**. + Under **Snow devices**, choose **Snowball Edge Compute Optimized** + Under **Select the storage type**, select **Amazon S3 compatible storage on Snowball Edge**. + For a standalone device, under **Storage capacity**, choose **Single device** and then select your desired storage amount. + For a cluster, under **Storage capacity** select **Cluster** and then select your desired storage capacity and fault tolerance. # Setting up and starting Amazon S3 compatible storage on Snowball Edge Install and configure software tools from AWS to your local environment to interact with the Snowball Edge device or cluster of devices and Amazon S3 compatible storage on Snowball Edge. Then, use these tools to set up the Snowball Edge device or cluster and start Amazon S3 compatible storage on Snowball Edge. ## Prerequisites Amazon S3 compatible storage on Snowball Edge requires you to have the Snowball Edge Client and the AWS CLI installed to your local environment. You can also use SDK for .NET and AWS Tools for Windows PowerShell to work with Amazon S3 compatible storage on Snowball Edge. AWS recommends using the following versions of these tools: + **Snowball Edge Client** – Use the latest version. For more information, see [Downloading and installing the Snowball Edge Client](https://docs.aws.amazon.com//snowball/latest/developer-guide/using-client-commands.html#download-the-client) in this guide. + **AWS CLI** – Version 2.11.15 or newer. For more information, see [Installing, updating, and uninstalling the AWS CLI](https://docs.aws.amazon.com/cli/v1/userguide/cli-chap-install.html) in the AWS Command Line Interface User Guide. + **SDK for .NET** – AWSSDK.S3Control 3.7.304.8 or newer. For more information, see [AWS SDK for .NET](https://docs.aws.amazon.com/sdk-for-net). + **AWS Tools for Windows PowerShell** – Version 4.1.476 or newer. For more information, see [AWS Tools for PowerShell User Guide](https://docs.aws.amazon.com/powershell/latest/userguide/). ## Setting up your local environment This section describes how to set up and configure the Snowball Edge Client and your local environment for use with Amazon S3 compatible storage on Snowball Edge. 1. Download and install the Snowball Edge Client. For more information, see [Downloading and installing the Snowball Edge Client](https://docs.aws.amazon.com/snowball/latest/developer-guide/using-client-commands.html#download-the-client). 1. Configure a profile for the Snowball Edge Client. For more information, see [Configuring a profile for the Snowball Edge Client](https://docs.aws.amazon.com/snowball/latest/developer-guide/using-client-commands.html#client-configuration). 1. If you are using SDK for .NET, set the `clientConfig.AuthenticationRegion` parameter value as follows: ``` clientConfig.AuthenticationRegion = "snow" ``` ### Setting up your Snowball Edge device #### Setting up IAM on the Snowball Edge AWS Identity and Access Management (IAM) helps you to enable granular access to AWS resources that run on your Snowball Edge devices. You use IAM to control who is authenticated (signed in) and authorized (has permissions) to use resources. IAM is supported locally on the Snowball Edge. You can use the local IAM service to create roles and attach IAM policies to them. You can use these policies to allow the access necessary to perform assigned tasks. The following example allows full access to the Amazon S3 API: ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": "s3:*", "Resource": "*" } ] } ``` ------ For more IAM policy examples, see the [AWS Snowball Edge Developer Guide](https://docs.aws.amazon.com/snowball/latest/developer-guide/using-local-iam.html#policy-examples). ## Starting the Amazon S3 compatible storage on Snowball Edge service Use the following instructions to start the Amazon S3 compatible storage on Snowball Edge service on a Snowball Edge device or cluster. If you prefer a more user-friendly experience, you can start the Amazon S3 compatible storage on Snowball Edge service for a standalone device or cluster of devices using AWS OpsHub. See [Set up Amazon S3 compatible storage on Snowball Edge with AWS OpsHub](s3-edge-snow-opshub.md). 1. Unlock your Snowball Edge device or cluster of devices by running the following command: + For a single device: ``` snowballEdge unlock-device --endpoint https://snow-device-ip ``` + For a cluster: ``` snowballEdge unlock-cluster ``` 1. Run the following command and make sure that the Snowball Edge device or cluster of devices are unlocked: + For a single device: ``` snowballEdge describe-device --endpoint https://snow-device-ip ``` + For a cluster: ``` snowballEdge describe-cluster --device-ip-addresses [snow-device-1-ip] [snow-device-2-ip] / [snow-device-3-ip] [snow-device-4-ip] [snow-device-5-ip] / [snow-device-6-ip] ``` 1. For each device (whether you have one or a cluster), to start Amazon S3 compatible storage on Snowball Edge, do the following: 1. Fetch the device's `PhysicalNetworkInterfaceId` by running the following `describe-device` command: ``` snowballEdge describe-device --endpoint https://snow-device-ip ``` 1. Run the following `create-virtual-network-interface` command twice to create the virtual network interfaces (VNIs) for the `s3control` (for bucket operations) and `s3api` (for object operations) endpoints. ``` snowballEdge create-virtual-network-interface --ip-address-assignment dhcp --manifest-file manifest --physical-network-interface-id "PhysicalNetworkInterfaceId" --unlock-code unlockcode --endpoint https://snow-device-ip ``` The command returns a JSON structure that includes the IP address. Make a note of that IP address. For details about these commands, see [Setting up a Virtual Network Interface (VNI) on a Snowball Edge](https://docs.aws.amazon.com/snowball/latest/developer-guide/network-config-ec2.html#setup-vni). **Note** Starting Amazon S3 compatible storage on Snowball Edge consumes device resources. 1. Start the Amazon S3 compatible storage on Snowball Edge service by running the following `start-service` command. which includes the IP addresses of your devices and the Amazon Resource Names (ARNs) of the VNIs that you created for the `s3control` and `s3api` endpoints: To start the service on a single device: ``` snowballEdge start-service --service-id s3-snow --device-ip-addresses snow-device-1-ip --virtual-network-interface-arns vni-arn-1 vni-arn-2 ``` To start the service on a cluster: ``` snowballEdge start-service --service-id s3-snow --device-ip-addresses snow-device-1-ip snow-device-2-ip snow-device-3-ip --virtual-network-interface-arns vni-arn-1 vni-arn-2 vni-arn-3 vni-arn-4 vni-arn-5 vni-arn-6 ``` For `--virtual-network-interface-arns`, include ARNs for all the VNIs that you created in the previous step. Separate each ARN using a space. 1. Run the following `describe-service` command for a single device: ``` snowballEdge describe-service --service-id s3-snow ``` Wait until service status is `Active`. Run the following `describe-service` command for a cluster: ``` snowballEdge describe-service --service-id s3-snow \ --device-ip-addresses snow-device-1-ip snow-device-2-ip snow-device-3-ip ``` ## Viewing information about Amazon S3 compatible storage on Snowball Edge endpoints When the Amazon S3 compatible storage on Snowball Edge service is running, you can use the `describe-service` Snowball Edge Client command to view the IP addresses associated with the s3control and s3api endpoints. ``` snowballEdge describe-service --service-id s3-snow --endpoint https://snow-device-ip-address --profile profile-name ``` **Example output of `describe-service` command** In this example, the IP address of the s3control endpoint is 192.168.1.222 and the IP address of the s3api endpoint is 192.168.1.152. ``` { "ServiceId": "s3-snow", "Autostart": true, "Status": { "State": "ACTIVATING", "Details": "Attaching storage" }, "ServiceCapacities": [ { "Name": "S3 Storage", "Unit": "Byte", "Used": 148599705600, "Available": 19351400294400 } ], "Endpoints": [ { "Protocol": "https", "Port": 443, "Host": "192.168.1.222", "CertificateAssociation": { "CertificateArn": "arn:aws:snowball-device:::certificate/30c563f1124707705117f57f6c3accd42a4528ed6dba1e35c1822a391a717199d8c49973d3c0283494d987463e826f2c" }, "Description": "s3-snow bucket API endpoint (for s3control SDK)", "DeviceId": "JID-beta-207429000001-23-12-28-03-51-11", "Status": { "State": "ACTIVE" } }, { "Protocol": "https", "Port": 443, "Host": "192.168.1.152", "CertificateAssociation": { "CertificateArn": "arn:aws:snowball-device:::certificate/30c563f1124707705117f57f6c3accd42a4528ed6dba1e35c1822a391a717199d8c49973d3c0283494d987463e826f2c" }, "Description": "s3-snow object & bucket API endpoint (for s3api SDK)", "DeviceId": "JID-beta-207429000001-23-12-28-03-51-11", "Status": { "State": "ACTIVATING" } } ] } ``` # Working with S3 buckets with Amazon S3 compatible storage on Snowball Edge With Amazon S3 compatible storage on Snowball Edge, you can create Amazon S3 buckets on your Snowball Edge devices to store and retrieve objects on premises for applications that require local data access, local data processing, and data residency. Amazon S3 compatible storage on Snowball Edge provides a new storage class, `SNOW`, which uses the Amazon S3 APIs, and is designed to store data durably and redundantly across multiple Snowball Edge devices. You can use the same APIs and features on Snowball Edge buckets that you do on Amazon S3, including bucket lifecycle policies, encryption, and tagging. You can use Amazon S3 compatible storage on Snowball Edge using the AWS Command Line Interface (AWS CLI) or programatically through the AWS Java SDK. With the AWS CLI, you can set up an s3api or s3control endpoint and interact with it through commands. We recommend using the s3api endpoint because the same endpoint can be used for bucket and object operations. **Note** The s3api endpoint is available for version 8004 and newer of the Snowball Edge software. To find the version of the Snowball Edge software installed on a device, use the `snowballEdge check-for-updates` command. To update a Snowball Edge device, see [Updating software on Snowball Edge devices](https://docs.aws.amazon.com/snowball/latest/developer-guide/updating-device.html). ## Using the AWS CLI Follow these instructions to work with Amazon S3 buckets on your device using the AWS CLI. **To set up the AWS CLI** 1. Create a profile for object endpoints in `~/.aws/config`. ``` [profile your-profile] aws_access_key_id = your-access-id aws_secret_access_key = your-access-key region = snow ca_bundle = dev/apps/ca-certs/your-ca_bundle ``` 1. Obtain a certificate from your device. For information, see the *[Snowball Edge Developer Guide](https://docs.aws.amazon.com/snowball/latest/developer-guide/using-client-commands.html#snowball-edge-certificates-cli)*. 1. If you installed the SDK in a virtual environment, activate it using the following command: ``` source your-virtual-environment-name/bin/activate ``` After you set up your operations, you can use the s3api SDK or s3control SDK to access S3 buckets on Snowball Edge with the AWS CLI. **Example of accessing S3 bucket using the s3api SDK** ``` aws s3api list-buckets --endpoint-url https://s3api-endpoint-ip --profile your-profile ``` **Example of accessing S3 buckets using the s3control SDK** ``` aws s3control list-regional-buckets --account-id bucket-owner --endpoint-url https://s3ctrlapi-endpoint-ip --profile your-profile ``` **Example of accessing S3 objects using the s3api SDK** ``` aws s3api list-objects-v2 --endpoint-url https://s3api-endpoint-ip --profile your-profile ``` ## Using the Java SDK Use the following example to work with Amazon S3 buckets and objects using the Java SDK. ``` import software.amazon.awssdk.services.s3.S3Client; import software.amazon.awssdk.auth.credentials.AwsBasicCredentials; import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider; import software.amazon.awssdk.http.SdkHttpClient;bg import software.amazon.awssdk.http.apache.ApacheHttpClient; import software.amazon.awssdk.regions.Region; import java.net.URI; AwsBasicCredentials creds = AwsBasicCredentials.create(accessKey, secretKey); // set creds by getting Access Key and Secret Key from snowball edge SdkHttpClient httpClient = ApacheHttpClient.builder().tlsTrustManagersProvider(trustManagersProvider).build(); // set trust managers provider with client certificate from snowball edge String s3SnowEndpoint = "10.0.0.0"; // set s3-snow object api endpoint from describe service S3Client s3Client = S3Client.builder().httpClient(httpClient).region(Region.of("snow")).endpointOverride(new URI(s3SnowEndpoint)).credentialsProvider(StaticCredentialsProvider.create(creds)).build(); ``` ## Bucket ARN format You can use the Amazon Resource Name (ARN) format listed here to identify an Amazon S3 bucket on a Snowball Edge device: ``` arn:partition:s3:snow:account-id:device/device-id/bucket/bucket-name ``` Where *partition* is the partition of the Region where you ordered your Snowball Edge device. *device-id* is the job\$1id if the device is a standalone Snowball Edge device, or the *cluster\$1id* if you have a Snowball Edge cluster. ## Bucket location format The bucket location format specifies the Snowball Edge device where the bucket will be created. The bucket location has the following format: ``` /device-id/bucket/bucket-name ``` For more information, see [create-bucket](https://awscli.amazonaws.com/v2/documentation/api/2.0.34/reference/s3api/create-bucket.html) in the AWS CLI Command Reference. # Determining whether you can access an Amazon S3 compatible storage on Snowball Edge bucket on a Snowball Edge The following example uses the `head-bucket` command to determine if an Amazon S3 bucket exists and you have permissions to access it using the AWS CLI. To use this command, replace each user input placeholder with your own information. ``` aws s3api head-bucket --bucket sample-bucket --endpoint-url https://s3api-endpoint-ip --profile your-profile ``` # Retrieving a list of buckets or regional buckets in Amazon S3 compatible storage on Snowball Edge on a Snowball Edge Use the `list-regional-buckets` or `list-buckets` to list Amazon S3 compatible storage on Snowball Edge buckets using the AWS CLI. **Example of retrieving a list of buckets or regional buckets with AWS CLI** ``` aws s3api list-buckets --endpoint-url https://s3api-endpoint-ip --profile your-profile ``` For more information about the `list-buckets` command, see [list-buckets](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3api/list-buckets.html) in the AWS CLI Command Reference ``` aws s3control list-regional-buckets --account-id 123456789012 --endpoint-url https://s3ctrlapi-endpoint-ip --profile your-profiles ``` For more information about the `list-regional-buckets` command, see [list-regional-buckets](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3control/list-regional-buckets.html) in the AWS CLI Command Reference. The following SDK for Java example gets a list of buckets on Snowball Edge devices. For more information, see [ListBuckets](https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListBuckets.html) in the Amazon Simple Storage Service API Reference. ``` import com.amazonaws.services.s3.model.*; public void listBuckets() { ListBucketsRequest reqListBuckets = new ListBucketsRequest() .withAccountId(AccountId) ListBucketsResult respListBuckets = s3APIClient.RegionalBuckets(reqListBuckets); System.out.printf("ListBuckets Response: %s%n", respListBuckets.toString()); } ``` The following PowerShell example gets a list of buckets on Snowball Edge devices. ``` Get-S3CRegionalBucketList -AccountId 012345678910 -Endpoint "https://snowball_ip" -Region snow ``` The following .NET example gets a list of buckets on Snowball Edge devices. ``` using Amazon.S3Control; using Amazon.S3Control.Model; namespace SnowTest; internal class Program { static async Task Main(string[] args) { var config = new AmazonS3ControlConfig { ServiceURL = "https://snowball_ip", AuthenticationRegion = "snow" // Note that this is not RegionEndpoint }; var client = new AmazonS3ControlClient(config); var response = await client.ListRegionalBucketsAsync(new ListRegionalBucketsRequest() { AccountId = "012345678910" }); } } ``` # Getting a bucket with Amazon S3 compatible storage on Snowball Edge on a Snowball Edge The following example gets an Amazon S3 compatible storage on Snowball Edge bucket using the AWS CLI. To use this command, replace each user input placeholder with your own information. ``` aws s3control get-bucket --account-id 123456789012 --bucket amzn-s3-demo-bucket --endpoint-url https://s3ctrlapi-endpoint-ip --profile your-profile ``` For more information about this command, see [get-bucket](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3control/get-bucket.html) in the AWS CLI Command Reference. The following Amazon S3 compatible storage on Snowball Edge example gets a bucket using the SDK for Java. For more information, see [GetBucket](https://docs.aws.amazon.com/AmazonS3/latest/API/API_control_GetBucket.html) in the [Amazon Simple Storage Service API Reference](https://docs.aws.amazon.com/AmazonS3/latest/API/). ``` import com.amazonaws.services.s3control.model.*; public void getBucket(String bucketName) { GetBucketRequest reqGetBucket = new GetBucketRequest() .withBucket(bucketName) .withAccountId(AccountId); GetBucketResult respGetBucket = s3ControlClient.getBucket(reqGetBucket); System.out.printf("GetBucket Response: %s%n", respGetBucket.toString()); } ``` # Creating an S3 bucket in Amazon S3 compatible storage on Snowball Edge on a Snowball Edge You can create Amazon S3 buckets on your Snowball Edge device to store and retrieve objects at the edge for applications that require local data access, local data processing, and data residency. Amazon S3 compatible storage on Snowball Edge provides a new storage class, `SNOW`, which uses Amazon S3 and is designed to store data durably and redundantly across multiple devices . You can use the same APIs and features as you do on Amazon S3 buckets, including bucket lifecycle policies, encryption, and tagging. The following example creates an Amazon S3 bucket for a Snowball Edge device using the AWS CLI. To run this command, replace the user input placeholders with your own information. **Example of creating an S3 bucket** ``` aws s3api create-bucket --bucket your-snow-bucket --endpoint-url https://s3api-endpoint-ip --profile your-profile ``` ``` aws s3control create-bucket --bucket your-snow-bucket --endpoint-url https://s3ctrlapi-endpoint-ip --profile your-profile ``` # Deleting a bucket in Amazon S3 compatible storage on Snowball Edge on a Snowball Edge You can use the s3api SDK or s3control SDK to delete a bucket in Amazon S3 compatible storage on Snowball Edge. **Important** The AWS account that creates the bucket owns it and is the only one that can delete it. Snowball Edge buckets must be empty before they can be deleted. You cannot recover a bucket after it has been deleted. The following examples delete an Amazon S3 compatible storage on Snowball Edge bucket using the AWS CLI. To use this command, replace each user input placeholder with your own information. **Example of deleting a bucket** ``` aws s3api delete-bucket --bucket amzn-s3-demo-bucket --endpoint-url https://s3api-endpoint-ip --profile your-profile ``` For more information about this command, see [delete-bucket](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3api/delete-bucket.html) in the AWS CLI Command Reference. ``` aws s3control delete-bucket --account-id 123456789012 --bucket amzn-s3-demo-bucket --endpoint-url https://s3ctrlapi-endpoint-ip --profile your-profile ``` For more information about this command, see [delete-bucket](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3control/delete-bucket.html) in the AWS CLI Command Reference. # Creating and managing an object lifecycle configuration using the AWS CLI You can use Amazon S3 Lifecycle to optimize storage capacity for Amazon S3 compatible storage on Snowball Edge. You can create lifecycle rules to expire objects as they age or are replaced by newer versions. You can create, enable, disable, or delete a lifecycle rule. For more information about Amazon S3 Lifecycle, see [Managing your storage lifecycle](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html). **Note** The AWS account that creates the bucket owns it and is the only one that can create, enable, disable, or delete a lifecycle rule. To create and manage a lifecycle configuration for an Amazon S3 compatible storage on Snowball Edge bucket using the AWS Command Line Interface (AWS CLI), see the following examples. ## PUT a lifecycle configuration on a Snowball Edge bucket The following AWS CLI example puts a lifecycle configuration policy on a Snowball Edge bucket. This policy specifies that all objects that have the flagged prefix (*myprefix*) and tags expire after 10 days. To use this example, replace each user input placeholder with your own information. First, save the lifecycle configuration policy to a JSON file. For this example, the file is named **lifecycle-example.json**. ``` { "Rules": [{ "ID": "id-1", "Filter": { "And": { "Prefix": "myprefix", "Tags": [{ "Value": "mytagvalue1", "Key": "mytagkey1" }, { "Value": "mytagvalue2", "Key": "mytagkey2" } ] } }, "Status": "Enabled", "Expiration": { "Days": 10 } }] } ``` After you save the file, submit the JSON file as part of the `put-bucket-lifecycle-configuration` command. To use this command, replace each user input placeholder with your own information. **Example of `put-bucket-lifecycle` command** ``` aws s3api put-bucket-lifecycle-configuration --bucket example-snow-bucket \\ --lifecycle-configuration file://lifecycle-example.json --endpoint-url https://s3api-endpoint-ip --profile your-profile ``` For more information about this command, see [put-bucket-lifecycle-configuration](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3api/put-bucket-lifecycle-configuration.html) in the AWS CLI Command Reference. ``` aws s3control put-bucket-lifecycle-configuration --bucket example-snow-bucket \\ --lifecycle-configuration file://lifecycle-example.json \\ --endpoint-url https://s3ctrlapi-endpoint-ip --profile your-profile ``` For more information about this command, see [put-bucket-lifecycle-configuration](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3control/put-bucket-lifecycle-configuration.html) in the AWS CLI Command Reference. # Copying an object to an Amazon S3 compatible storage on Snowball Edge bucket on a Snowball Edge The following example uploads a file named *sample-object.xml* to an Amazon S3 compatible storage on Snowball Edge bucket that you have write permissions for using the AWS CLI. To use this command, replace each user input placeholder with your own information. ``` aws s3api put-object --bucket sample-bucket --key sample-object.xml --body sample-object.xml --endpoint-url s3api-endpoint-ip --profile your-profile ``` The following Amazon S3 compatible storage on Snowball Edge example copies an object into a new object in the same bucket using the SDK for Java. To use this command, replace each user input placeholder with your own information. ``` import com.amazonaws.AmazonServiceException; import com.amazonaws.SdkClientException; import com.amazonaws.services.s3.AmazonS3; import com.amazonaws.services.s3.AmazonS3ClientBuilder; import com.amazonaws.services.s3.model.CopyObjectRequest; add : import java.io.IOException; public class CopyObject { public static void main(String[] args) { String bucketName = "*** Bucket name ***"; String sourceKey = "*** Source object key ***"; String destinationKey = "*** Destination object key ***"; try { // This code expects that you have AWS credentials set up per: // https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/setup-credentials.html AmazonS3 s3Client = AmazonS3ClientBuilder.standard() .enableUseArnRegion() .build(); // Copy the object into a new object in the same bucket. CopyObjectRequest copyObjectRequest = new CopyObjectRequest(sourceKey, destinationKey); s3Client.copyObject(copyObjectRequest); CopyObjectRequest copyObjectRequest = CopyObjectRequest.builder() .sourceKey(sourceKey) .destinationKey(destKey) .build(); } catch (AmazonServiceException e) { // The call was transmitted successfully, but Amazon S3 couldn't process // it, so it returned an error response. e.printStackTrace(); } catch (SdkClientException e) { // Amazon S3 couldn't be contacted for a response, or the client // couldn't parse the response from Amazon S3. e.printStackTrace(); } } } ``` # Listing objects in a bucket in Amazon S3 compatible storage on Snowball Edge on a Snowball Edge The following example lists objects in an Amazon S3 compatible storage on Snowball Edge bucket using the AWS CLI. The SDK command is `s3-snow:ListObjectsV2`. To use this command, replace each user input placeholder with your own information. ``` aws s3api list-objects-v2 --bucket sample-bucket --endpoint-url s3api-endpoint-ip --profile your-profile ``` For more information about this command, see [list-objects-v2](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3api/list-objects-v2.html) in the *AWS CLI Command Reference*. The following Amazon S3 compatible storage on Snowball Edge example lists objects in a bucket using the SDK for Java. To use this command, replace each user input placeholder with your own information. This example uses [ListObjectsV2](https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html), which is the latest revision of the ListObjects API operation. We recommend that you use this revised API operation for application development. For backward compatibility, Amazon S3 continues to support the prior version of this API operation. ``` import com.amazonaws.AmazonServiceException; import com.amazonaws.SdkClientException; import com.amazonaws.services.s3.AmazonS3; import com.amazonaws.services.s3.AmazonS3ClientBuilder; import com.amazonaws.services.s3.model.ListObjectsV2Request; import com.amazonaws.services.s3.model.ListObjectsV2Result; import com.amazonaws.services.s3.model.S3ObjectSummary; public class ListObjectsV2 { public static void main(String[] args) { String bucketName = "*** Bucket name ***"; try { // This code expects that you have AWS credentials set up per: // https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/setup-credentials.html AmazonS3 s3Client = AmazonS3ClientBuilder.standard() .enableUseArnRegion() .build(); System.out.println("Listing objects"); // maxKeys is set to 2 to demonstrate the use of // ListObjectsV2Result.getNextContinuationToken() ListObjectsV2Request req = new ListObjectsV2Request().withBucketName(bucketName).withMaxKeys(2); ListObjectsV2Result result; do { result = s3Client.listObjectsV2(req); for (S3ObjectSummary objectSummary : result.getObjectSummaries()) { System.out.printf(" - %s (size: %d)\n", objectSummary.getKey(), objectSummary.getSize()); } // If there are more than maxKeys keys in the bucket, get a continuation token // and list the next objects. String token = result.getNextContinuationToken(); System.out.println("Next Continuation Token: " + token); req.setContinuationToken(token); } while (result.isTruncated()); } catch (AmazonServiceException e) { // The call was transmitted successfully, but Amazon S3 couldn't process // it, so it returned an error response. e.printStackTrace(); } catch (SdkClientException e) { // Amazon S3 couldn't be contacted for a response, or the client // couldn't parse the response from Amazon S3. e.printStackTrace(); } } } ``` # Getting an object from a bucket in Amazon S3 compatible storage on Snowball Edge on a Snowball Edge The following example gets an object named *sample-object.xml* from an Amazon S3 compatible storage on Snowball Edge bucket using the AWS CLI. The SDK command is `s3-snow:GetObject`. To use this command, replace each user input placeholder with your own information. ``` aws s3api get-object --bucket sample-bucket --key sample-object.xml --endpoint-url s3api-endpoint-ip --profile your-profile ``` For more information about this command, see [get-object](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3api/get-object.html) in the *AWS CLI Command Reference*. The following Amazon S3 compatible storage on Snowball Edge example gets an object using the SDK for Java. To use this command, replace each user input placeholder with your own information. For more information, see [GetObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html) in the [Amazon Simple Storage Service API Reference](https://docs.aws.amazon.com/AmazonS3/latest/API/). ``` import com.amazonaws.AmazonServiceException; import com.amazonaws.SdkClientException; import com.amazonaws.services.s3.AmazonS3; import com.amazonaws.services.s3.AmazonS3ClientBuilder; import com.amazonaws.services.s3.model.GetObjectRequest; import com.amazonaws.services.s3.model.ResponseHeaderOverrides; import com.amazonaws.services.s3.model.S3Object; import java.io.BufferedReader; import java.io.IOException; import java.io.InputStream; import java.io.InputStreamReader; public class GetObject { public static void main(String[] args) throws IOException { String bucketName = "*** Bucket name ***"; String key = "*** Object key ***"; S3Object fullObject = null, objectPortion = null, headerOverrideObject = null; try { // This code expects that you have AWS credentials set up per: // https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/setup-credentials.html AmazonS3 s3Client = AmazonS3ClientBuilder.standard() .enableUseArnRegion() .build(); GetObjectRequest getObjectRequest = GetObjectRequest.builder() .bucket(bucketName) .key(key) .build()); s3Client.getObject(getObjectRequest); } catch (AmazonServiceException e) { // The call was transmitted successfully, but Amazon S3 couldn't process // it, so it returned an error response. e.printStackTrace(); } catch (SdkClientException e) { // Amazon S3 couldn't be contacted for a response, or the client // couldn't parse the response from Amazon S3. e.printStackTrace(); } finally { // To ensure that the network connection doesn't remain open, close any open input streams. if (fullObject != null) { fullObject.close(); } if (objectPortion != null) { objectPortion.close(); } if (headerOverrideObject != null) { headerOverrideObject.close(); } } } private static void displayTextInputStream(InputStream input) throws IOException { // Read the text input stream one line at a time and display each line. BufferedReader reader = new BufferedReader(new InputStreamReader(input)); String line = null; while ((line = reader.readLine()) != null) { System.out.println(line); } System.out.println(); } } ``` # Deleting objects in buckets in Amazon S3 compatible storage on Snowball Edge You can delete one or more objects from an Amazon S3 compatible storage on Snowball Edge bucket. The following example deletes an object named *sample-object.xml* using the AWS CLI. To use this command, replace each user input placeholder with your own information. ``` aws s3api delete-object --bucket sample-bucket --key key --endpoint-url s3api-endpoint-ip --profile your-profile ``` For more information about this command, see [delete-object](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3control/delete-object.html) in the *AWS CLI Command Reference*. The following Amazon S3 compatible storage on Snowball Edge example deletes an object in a bucket using the SDK for Java. To use this example, specify the key name for the object that you want to delete. For more information, see [DeleteObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObject.html) in the Amazon Simple Storage Service API Reference. ``` import com.amazonaws.AmazonServiceException; import com.amazonaws.SdkClientException; import com.amazonaws.services.s3.AmazonS3; import com.amazonaws.services.s3.AmazonS3ClientBuilder; import com.amazonaws.services.s3.model.DeleteObjectRequest; public class DeleteObject { public static void main(String[] args) { String bucketName = "*** Bucket name ***"; String keyName = "*** key name ****"; try { // This code expects that you have AWS credentials set up per: // https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/setup-credentials.html AmazonS3 s3Client = AmazonS3ClientBuilder.standard() .enableUseArnRegion() .build(); DeleteObjectRequest deleteObjectRequest = DeleteObjectRequest.builder() .bucket(bucketName) .key(keyName) .build())); s3Client.deleteObject(deleteObjectRequest); } catch (AmazonServiceException e) { // The call was transmitted successfully, but Amazon S3 couldn't process // it, so it returned an error response. e.printStackTrace(); } catch (SdkClientException e) { // Amazon S3 couldn't be contacted for a response, or the client // couldn't parse the response from Amazon S3. e.printStackTrace(); } } } ``` # Supported REST API actions for Amazon S3 compatible storage on Snowball Edge The following lists show the API operations that are supported by Amazon S3 compatible storage on Snowball Edge, including links to the related operations for Amazon S3 in AWS Regions. Supported bucket API operations for the s3api endpoint: + [CreateBucket](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3api/create-bucket.html) + [DeleteBucket](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3api/delete-bucket.html) + [DeleteBucketLifecycle](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3api/delete-bucket-lifecycle.html) + [GetBucketLifecycleConfiguration](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3api/get-bucket-lifecycle-configuration.html) + [ListBuckets]( https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3api/list-buckets.html) + [PutBucketLifecycleConfiguration](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3api/put-bucket-lifecycle-configuration.html) Supported bucket API operations for the s3control endpoint: + [CreateBucket](https://docs.aws.amazon.com/AmazonS3/latest/API/API_CreateBucket.html) + [DeleteBucket](https://docs.aws.amazon.com/AmazonS3/latest/API/API_control_DeleteBucket.html) + [DeleteBucketLifecycle](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketLifecycle.html) + [GetBucket](https://docs.aws.amazon.com/AmazonS3/latest/API/API_control_GetBucket.html) + [GetBucketLifecycleConfiguration](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketLifecycleConfiguration.html) + [ListBuckets](https://docs.aws.amazon.com/AmazonS3/latest/API/API_control_ListRegionalBuckets.html) + [PutBucketLifecycleConfiguration](https://docs.aws.amazon.com/AmazonS3/latest/API/API_control_PutBucketLifecycleConfiguration.html) Supported object API operations: + [AbortMultipartUpload](https://docs.aws.amazon.com/AmazonS3/latest/API/API_AbortMultipartUpload.html) + [CompleteMultipartUpload](https://docs.aws.amazon.com/AmazonS3/latest/API/API_CompleteMultipartUpload.html) + [CopyObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_CopyObject.html) + [CreateMultipartUpload](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3control/create-bucket.html) + [DeleteObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObject.html) + [DeleteObjects](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObjects.html) + [DeleteObjectTagging](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObjectTagging.html) + [GetObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html) + [GetObjectTagging](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectTagging.html) + [HeadBucket](https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadBucket.html) + [HeadObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadObject.html) + [ListMultipartUploads](https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListMultipartUploads.html) + [ListObjects](https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjects.html) + [ListObjectsV2](https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html) + [ListParts](https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListParts.html) + [PutObject](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html) + [PutObjectTagging](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectTagging.html) + [UploadPart](https://docs.aws.amazon.com/AmazonS3/latest/API/API_UploadPart.html) + [UploadPartCopy](https://docs.aws.amazon.com/AmazonS3/latest/API/API_UploadPartCopy.html) # Using Amazon S3 compatible storage on Snowball Edge with a cluster of Snow devices A cluster is a collection of three or more Snowball Edge devices used as a single logical unit for local storage and compute purposes. A cluster offers two primary benefits over a standalone Snowball Edge device for local storage and computing: + **Increased durability** – The S3 data stored in a cluster of Snowball Edge devices enjoys increased data durability over a single device. In addition, the data on the cluster remains safe and viable, despite possible hardware outages affecting the cluster. Clusters can withstand the loss of one device in clusters of 3 and 4 devices and up to two devices in clusters of 5 to 16 devices before the data is in danger. You can replace unhealthy nodes to maintain the durability and safety of data stored in the cluster. + **Increased storage** – With Snowball Edge storage optimized devices, you can create a single, 16 node cluster with up to 2.6 PB of usable S3-compatible storage capacity. With Snowball Edge compute optimized devices, you can create a single, 16 node cluster of up to 501 TB of usable S3-compatible storage capacity. A cluster of Snowball Edge devices is made of leaderless nodes. Any node can write data to and read data from the entire cluster, and all nodes are capable of performing the behind-the-scenes management of the cluster. Keep the following considerations in mind when planning to use a cluster of Snowball Edge devices: + We recommend that you provide a redundant power source for all devices in the cluster to reduce potential performance and stability issues for the cluster. + As with standalone local storage and compute jobs, the data stored in a cluster can't be imported into Amazon S3 without ordering additional devices as a part of separate import jobs. If you order additional devices as import jobs, you can transfer the data from the cluster to the import job devices. + To get data onto a cluster from Amazon S3, use the Amazon S3 API to create Amazon S3 buckets on the cluster to store and retrieve objects from S3. Also, you can use AWS DataSync to transfer objects between AWS storage services and Amazon S3 compatible storage on Snowball Edge on a Snowball Edge device. For more information, see [Configuring transfers with S3 compatible storage on Snowball Edge](https://docs.aws.amazon.com/datasync/latest/userguide/s3-compatible-storage-snow.html). + You can create a job to order a cluster of devices from the AWS Snow Family Management Console, the AWS CLI, or one of the AWS SDKs. For more information, see [Getting started with Snowball Edge](getting-started.md). + Each device in the cluster has a node ID. A *node ID* is a unique identifier for each device in the cluster, like a job ID for a standalone device. You can get node IDs from the AWS Snow Family Management Console, the AWS CLI, the AWS SDKs, and the Snowball Edge client. The Snowball Edge client commands `describe-device` and `describe-cluster` return node IDs with other information about devices or the cluster. + The lifespan of a cluster is limited by the security certificate granted to the cluster devices when the cluster is provisioned. By default, Snowball Edge devices can be used for up to 360 days before they need to be returned. At the end of that time, the devices stop responding to read/write requests. If you need to keep one or more devices for longer than 360 days, contact AWS Support. + When AWS receives a returned device that was part of a cluster, we perform a complete erasure of the device. This erasure follows the National Institute of Standards and Technology (NIST) 800-88 standards. **Amazon S3 compatible storage on Snowball Edge cluster fault tolerance and storage capacity** | Cluster size | Fault tolerance | Storage capacity of Snowball Edge Compute Optimized (Compute Optimized with AMD EPYC Gen2 and NVMe) devices (in TB) | Storage capacity of Snowball Edge storage optimized 210 TB devices (in TB) | | --- | --- | --- | --- | | 3 | Loss of up to 1 node | 38 | 438 | | 4 | Loss of up to 1 node | 57 | 657 | | 5 | Loss of up to 2 nodes | 57 | 657 | | 6 | Loss of up to 2 nodes | 76 | 904 | | 7 | Loss of up to 2 nodes | 95 | 1096 | | 8 | Loss of up to 2 nodes | 114 | 1315 | | 9 | Loss of up to 2 nodes | 133 | 1534 | | 10 | Loss of up to 2 nodes | 152 | 1754 | | 11 | Loss of up to 2 nodes | 165 | 1970 | | 12 | Loss of up to 2 nodes | 171 | 1973 | | 13 | Loss of up to 2 nodes | 190 | 2192 | | 14 | Loss of up to 2 nodes | 209 | 2411 | | 15 | Loss of up to 2 nodes | 225 | 2625 | | 16 | Loss of up to 2 nodes | 228 | 2631 | After you unlock a cluster, you're ready to store and access data on that cluster. You can use the Amazon S3 compatible endpoint to read from and write data to a cluster. To read from or write data to a cluster, you must have a read/write quorum with no more than the allowed number of unavailable nodes in your cluster of devices. ## Snowball Edge cluster quorums A *quorum* represents the minimum number of Snowball Edge devices in a cluster that must be communicating with each other to maintain a read/write quorum. When all devices in a cluster are healthy, you have a *read/write quorum* for your cluster. If one or two of those devices goes offline, you reduce the operational capacity of the cluster. However, you can still read and write to the cluster. With all but one or two devices the cluster operating, the cluster still has a read/write quorum. The number of nodes that can go offline before the operational capacity of the cluster is affected is found in [this table](#cluster-table). Quorom may be lost if a cluster loses more than the number of devices indicated in [this table](#cluster-table). When a quorom is lost, the cluster is offline and the data in the cluster is unavailable. You might be able fix this, or the data might be permanently lost, depending on the severity of the event. If it is a temporary external power event, and you can power the Snowball Edge devices back on and unlock all the nodes in the cluster, your data is available again. **Important** If a minimum quorum of healthy nodes doesn't exist, contact AWS Support. You can use the `describe-cluster` command to view the lock state and network reachability of each node. Ensuring that the devices in your cluster are healthy and connected is an administrative responsibility that you take on when you using cluster storage. For more information, see [Getting device status](https://docs.aws.amazon.com/snowball/latest/developer-guide/using-client-commands.html#client-status). If you determine one or more nodes are unhealthy, you can replace nodes in the cluster to maintain quorom and the health and stability of your data. For more information, see [Replacing a node in a cluster](replacement.md). # Reconnecting an unavailable cluster node A *node*, or device within a cluster, can become temporarily unavailable due to an issue like power or network loss without damaging the data on the node. When this happens, it affects the status of your cluster. A node's network reachability and lock status is reported in the Snowball Edge client by using the `snowballEdge describe-cluster` command. We recommend that you physically position your cluster so you have access to the front, back, and top of all nodes. This way, you can access power and network cables on the back, shipping labels on the top for node IDs, and LCD screens on the front of the devices for the IP addresses and other administrative information. When you detect that a node is unavailable, we recommend that you try one of the following procedures, depending on the scenario that caused the node to become unavailable. **To reconnect an unavailable node** 1. Ensure that the node is powered on. 1. Ensure that the node is connected to the same internal network that the rest of the cluster is connected to. 1. If you need to power up the node, wait up to 20 minutes for it to finish. 1. Run the `snowballEdge unlock-cluster` command or the `snowballEdge associate-device` command. For an example, see [Unlocking Snowball Edge devices](https://docs.aws.amazon.com/snowball/latest/developer-guide/using-client-commands.html#setting-up-client). **To reconnect an unavailable node that lost network connectivity, but didn't lose power** 1. Ensure that the node is connected to the same internal network that the rest of the cluster is on. 1. Run the `snowballEdge describe-device` command to see when the previously unavailable node is added back to the cluster. For an example, see [Getting Device Status](https://docs.aws.amazon.com/snowball/latest/developer-guide/using-client-commands.html#client-status). After you perform the preceding procedures, your nodes should be working normally. You should also have a read/write quorum. If that's not the case, then one or more of your nodes might have a more serious issue and might need to be removed from the cluster. # Replacing a node in a cluster To replace a node, you first need to order a replacement. You can order a replacement node from the console, the AWS CLI, or one of the AWS SDKs. If you're ordering a replacement node from the console, you can order replacements for any job that hasn't been canceled or completed. Then, you diassociate the unhealthy node from the cluster, connect the replacement node to your network and unlock the cluster including the replacement node, associate the replacement node with the cluster, and restart the Amazon S3 compatible storage on Snowball Edge service. **To order a replacement node from the console** 1. Sign in to the [AWS Snow Family Management Console](https://console.aws.amazon.com/snowfamily/home). 1. Find and choose a job for a node that belongs to the cluster that you created from the Job dashboard. 1. For **Actions**, choose **Replace node**. Doing this opens the final step of the job creation wizard, with all settings identical to how the cluster was originally created. 1. Choose **Create job**. Your replacement Snowball Edge is now on its way to you. Use the following procedure to remove the unhealthy node from the cluster. **To remove a node from a cluster** 1. Power off the node to be removed. For more information, see [Powering off the Snowball Edge](https://docs.aws.amazon.com/snowball/latest/developer-guide/turnitoff.html). 1. Use the `describe-cluster` command to ensure the unhealthy node unreachable. This is indicated by the value of `UNREACHABLE` for the `State` name of the `NetworkReachability` object. ``` snowballEdge describe-cluster --manifest-file path/to/manifest/file.bin --unlock-code unlock-code --endpoint https://ip-address-of-device-in-cluster ``` **Example of `describe-cluster` output** ``` { "ClusterId": "CID12345678-1234-1234-1234-123456789012", "Devices": [ { "DeviceId": "JID12345678-1234-1234-1234-123456789012", "UnlockStatus": { "State": "UNLOCKED" }, "ActiveNetworkInterface": { "IpAddress": "10.0.0.0" }, "ClusterAssociation": { "ClusterId": "CID12345678-1234-1234-1234-123456789012", "State": "ASSOCIATED" }, "NetworkReachability": { "State": "REACHABLE" }, "Tags": [] }, { "DeviceId": "JID12345678-1234-1234-1234-123456789013", "UnlockStatus": { "State": "UNLOCKED" }, "ActiveNetworkInterface": { "IpAddress": "10.0.0.1" }, "ClusterAssociation": { "ClusterId": "CID12345678-1234-1234-1234-123456789012", "State": "ASSOCIATED" }, "NetworkReachability": { "State": "REACHABLE" }, "Tags": [] }, { "DeviceId": "JID12345678-1234-1234-1234-123456789014", "ClusterAssociation": { "ClusterId": "CID12345678-1234-1234-1234-123456789012", "State": "ASSOCIATED" }, "NetworkReachability": { "State": "UNREACHABLE" } } ] } ``` 1. Use the `describe-service` command to ensure the status of the `s3-snow` service is `DEGRADED`. ``` snowballEdge describe-service --service-id s3-snow --device-ip-addresses snow-device-1-address snow-device-2-address --manifest-file path/to/manifest/file.bin --unlock-code unlock-code --endpoint https://snow-device-ip-address ``` **Example of output of `describe-service` command** ``` { "ServiceId": "s3-snow", "Autostart": true, "Status": { "State": "DEGRADED" }, "ServiceCapacities": [ { "Name": "S3 Storage", "Unit": "Byte", "Used": 38768180432, "Available": 82961231819568 } ], "Endpoints": [ { "Protocol": "https", "Port": 443, "Host": "10.0.0.10", "CertificateAssociation": { "CertificateArn": "arn:aws:snowball-device:::certificate/7Rg2lP9tQaHnW4sC6xUzF1vGyD3jB5kN8MwEiYpT" }, "Description" : "s3-snow bucket API endpoint (for s3control SDK)", "DeviceId": "JID-beta-207012320001-24-02-05-17-17-26", "Status": { "State": "ACTIVE" } }, { "Protocol": "https", "Port": 443, "Host": "10.0.0.11", "CertificateAssociation": { "CertificateArn": "arn:aws:snowball-device:::certificate/7Rg2lP9tQaHnW4sC6xUzF1vGyD3jB5kN8MwEiYpT" }, "Description": "Description" : "s3-snow object & bucket API endpoint (for s3api SDK)", "DeviceId": "JID-beta-207012320001-24-02-05-17-17-26", "Status": { "State": "ACTIVE" } }, { "Protocol": "https", "Port": 443, "Host": "10.0.0.12", "CertificateAssociation": { "CertificateArn": "arn:aws:snowball-device:::certificate/7Rg2lP9tQaHnW4sC6xUzF1vGyD3jB5kN8MwEiYpT" }, "Description": "Description" : "s3-snow bucket API endpoint (for s3control SDK)", "DeviceId": "JID-beta-207012240003-24-02-05-17-17-27", "Status": { "State": "ACTIVE" } }, { "Protocol": "https", "Port": 443, "Host": "10.0.0.13", "CertificateAssociation": { "CertificateArn": "arn:aws:snowball-device:::certificate/7Rg2lP9tQaHnW4sC6xUzF1vGyD3jB5kN8MwEiYpT" }, "Description": "Description" : "s3-snow object & bucket API endpoint (for s3api SDK)", "DeviceId": "JID-beta-207012320001-24-02-05-17-17-27", "Status": { "State": "ACTIVE" } } ] } ``` 1. Use the `disassociate-device` command to disassociate and remove the unhealthy node from the cluster. ``` snowballEdge disassociate-device --device-id device-id --manifest-file path/to/manifest/file.bin --unlock-code unlock-code --endpoint https://ip-address-of-unhealthy-device ``` **Example output of `disassociate-device` command** ``` Disassociating your Snowball Edge device from the cluster. Your Snowball Edge device will be disassociated from the cluster when it is in the "DISASSOCIATED" state. You can use the describe-cluster command to determine the state of your cluster. ``` 1. Use the `describe-cluster` command again to ensure the unhealthy node is disassociated from the cluster. ``` snowballEdge describe-cluster --manifest-file path/to/manifest/file.bin --unlock-code unlock-code --endpoint https:ip-address-of-healthy-device ``` **Example of `describe-cluster` command showing node is disassociated** ``` { "ClusterId": "CID12345678-1234-1234-1234-123456789012", "Devices": [ { "DeviceId": "JID12345678-1234-1234-1234-123456789012", "UnlockStatus": { "State": "UNLOCKED" }, "ActiveNetworkInterface": { "IpAddress": "10.0.0.0" }, "ClusterAssociation": { "ClusterId": "CID12345678-1234-1234-1234-123456789012", "State": "ASSOCIATED" }, "NetworkReachability": { "State": "REACHABLE" }, "Tags": [] }, { "DeviceId": "JID12345678-1234-1234-1234-123456789013", "UnlockStatus": { "State": "UNLOCKED" }, "ActiveNetworkInterface": { "IpAddress": "10.0.0.1" }, "ClusterAssociation": { "ClusterId": "CID12345678-1234-1234-1234-123456789012", "State": "ASSOCIATED" }, "NetworkReachability": { "State": "REACHABLE" }, "Tags": [] }, { "DeviceId": "JID12345678-1234-1234-1234-123456789014", "ClusterAssociation": { "ClusterId": "CID12345678-1234-1234-1234-123456789012", "State": "DISASSOCIATED" } } ] } ``` 1. Power off and return the unhealthy device to AWS. For more information, see [Powering off the Snowball Edge](https://docs.aws.amazon.com/snowball/latest/developer-guide/turnitoff.html) and [Returning the Snowball Edge Device](https://docs.aws.amazon.com/snowball/latest/developer-guide/return-device.html). When the replacement device arrives, use the following procedure to add it to the cluster. **To add a replacement device** 1. Position the replacement device for the cluster such that you have access to the front, back, and top of all devices. 1. Power up the node and ensure that the node is connected to the same internal network as the rest of the cluster. For more information, see [Connecting to Your Local Network](https://docs.aws.amazon.com/snowball/latest/developer-guide/getting-started-connect.html). 1. Use the `unlock-cluster` command and include the IP address of the new node. ``` snowballEdge unlock-cluster --manifest-file path/to/manifest/file.bin --unlock-code unlock-code --endpoint https://ip-address-of-cluster-device --device-ip-addresses node-1-ip-address node-2-ip-address new-node-ip-address ``` The state of the new node will be `DEGRADED` until you associate it with the cluster in the next step. 1. Use the `associate-device` command to associate the replacement node with the cluster. ``` snowballEdge associate-device --device-ip-address new-node-ip-address ``` **Example of `associate-device` command output** ``` Associating your Snowball Edge device with the cluster. Your Snowball Edge device will be associated with the cluster when it is in the ASSOCIATED state. You can use the describe-device command to determine the state of your devices. ``` 1. Use the `describe-cluster` command to ensure the new node is associated with the cluster. ``` snowballEdge describe-cluster --manifest-file path/to/manifest/file.bin --unlock-code unlock-code --endpoint https://node-ip-address ``` **Example of `describe-cluster` command output** ``` { "ClusterId": "CID12345678-1234-1234-1234-123456789012", "Devices": [ { "DeviceId": "JID12345678-1234-1234-1234-123456789012", "UnlockStatus": { "State": "UNLOCKED" }, "ActiveNetworkInterface": { "IpAddress": "10.0.0.0" }, "ClusterAssociation": { "ClusterId": "CID12345678-1234-1234-1234-123456789012", "State": "ASSOCIATED" }, "NetworkReachability": { "State": "REACHABLE" }, "Tags": [] }, { "DeviceId": "JID-CID12345678-1234-1234-1234-123456789013", "UnlockStatus": { "State": "UNLOCKED" }, "ActiveNetworkInterface": { "IpAddress": "10.0.0.1" }, "ClusterAssociation": { "ClusterId": "CID12345678-1234-1234-1234-123456789012", "State": "ASSOCIATED" }, "NetworkReachability": { "State": "REACHABLE" }, "Tags": [] }, { "DeviceId": "JID-CID12345678-1234-1234-1234-123456789015", "UnlockStatus": { "State": "UNLOCKED" }, "ActiveNetworkInterface": { "IpAddress": "10.0.0.2" }, "ClusterAssociation": { "ClusterId": "CID12345678-1234-1234-1234-123456789012", "State": "ASSOCIATED" }, "NetworkReachability": { "State": "REACHABLE" }, "Tags": [] } } ] } ``` 1. On the new node, create two virtual network interfaces (VNIs). For more information, see [Starting the Amazon S3 compatible storage on Snowball Edge service](s3-edge-snow-setting-up.md#setting-up-s3-on-snow-cluster) 1. Use the `stop-service` command to stop the s3-snow service. ``` snowballEdge stop-service --service-id s3-snow --device-ip-addresses cluster-device-1-ip-address cluster-device-2-ip-address cluster-device-3-ip-address --manifest-file path/to/manifest/file.bin --unlock-code unlock-code --endpoint https://snow-device-ip-address ``` **Example of `stop-service` command output** ``` Stopping the AWS service on your Snowball Edge. You can determine the status of the AWS service using the describe-service command. ``` 1. Use the `start-service` command to start the s3-snow service after adding the new node to the cluster. ``` snowballEdge start-service --service-id s3-snow --device-ip-addresses cluster-device-1-ip-address cluster-device-2-ip-address cluster-device-3-ip-address --virtual-network-interface-arns "device-1-vni-ip-address-a" "device-1-vni-ip-address-b" "device-2-vni-ip-address-a" "device-2-vni-ip-address-b" "device-3-vni-ip-address-a" "device-3-vni-ip-address-b" --manifest-file path/to/manifest/file.bin --unlock-code unlock-code --endpoint https://snow-device-ip-address ``` **Example of `start-service` command output** ``` Starting the AWS service on your Snowball Edge. You can determine the status of the AWS service using the describe-service command. ``` 1. Use the `describe-service` command to ensure the s3-snow service started. ``` snowballEdge describe-service --service-id s3-snow --device-ip-addresses snow-device-1-address snow-device-2-address snow-device-3-address --manifest-file path/to/manifest/file.bin --unlock-code unlock-code --endpoint https://snow-device-ip-address ``` **Example of `descibe-service` command output** ``` { "ServiceId": "s3-snow", "Autostart": true, "Status": { "State": "ACTIVE" }, "ServiceCapacities": [{ "Name": "S3 Storage", "Unit": "Byte", "Used": 38768180432, "Available": 82961231819568 }], "Endpoints": [{ "Protocol": "https", "Port": 443, "Host": "10.0.0.10", "CertificateAssociation": { "CertificateArn": "arn:aws:snowball-device:::certificate/7Rg2lP9tQaHnW4sC6xUzF1vGyD3jB5kN8MwEiYpT" }, "Description": "s3-snow bucket API endpoint (for s3control SDK)", "DeviceId": "JID12345678-1234-1234-1234-123456789012", "Status": { "State": "ACTIVE" } }, { "Protocol": "https", "Port": 443, "Host": "10.0.0.11", "CertificateAssociation": { "CertificateArn": "arn:aws:snowball-device:::certificate/7Rg2lP9tQaHnW4sC6xUzF1vGyD3jB5kN8MwEiYpT" }, "Description": "s3-snow object & bucket API endpoint (for s3api SDK)", "DeviceId": "JID12345678-1234-1234-1234-123456789013", "Status": { "State": "ACTIVE" } }, { "Protocol": "https", "Port": 443, "Host": "10.0.0.12", "CertificateAssociation": { "CertificateArn": "arn:aws:snowball-device:::certificate/7Rg2lP9tQaHnW4sC6xUzF1vGyD3jB5kN8MwEiYpT" }, "Description": "s3-snow bucket API endpoint (for s3control SDK)", "DeviceId": "JID12345678-1234-1234-1234-123456789015", "Status": { "State": "ACTIVE" } }, { "Protocol": "https", "Port": 443, "Host": "10.0.0.13", "CertificateAssociation": { "CertificateArn": "arn:aws:snowball-device:::certificate/7Rg2lP9tQaHnW4sC6xUzF1vGyD3jB5kN8MwEiYpT" }, "Description": "s3-snow object & bucket API endpoint (for s3api SDK)", "DeviceId": "JID-beta-207012320001-24-02-05-17-17-27", "Status": { "State": "ACTIVE" } }, { "Protocol": "https", "Port": 443, "Host": "10.0.0.14", "CertificateAssociation": { "CertificateArn": "arn:aws:snowball-device:::certificate/7Rg2lP9tQaHnW4sC6xUzF1vGyD3jB5kN8MwEiYpT" }, "Description": "s3-snow bucket API endpoint (for s3control SDK)", "DeviceId": "JID-beta-207012240003-24-02-05-17-17-28", "Status": { "State": "ACTIVE" } }, { "Protocol": "https", "Port": 443, "Host": "10.0.0.15", "CertificateAssociation": { "CertificateArn": "arn:aws:snowball-device:::certificate/7Rg2lP9tQaHnW4sC6xUzF1vGyD3jB5kN8MwEiYpT" }, "Description": "s3-snow object & bucket API endpoint (for s3api SDK), "DeviceId": "JID-beta-207012320001-24-02-05-17-17-28", "Status": { "State": "ACTIVE" } } }] } ``` # Configuring Amazon S3 compatible storage on Snowball Edge event notifications Amazon S3 compatible storage on Snowball Edge supports Amazon S3 event notifications for object API calls based on the Message Queuing Telemetry Transport (MQTT) protocol. You can use Amazon S3 compatible storage on Snowball Edge to receive notifications when certain events happen in your S3 bucket. To enable notifications, add a notification configuration that identifies the events that you want the service to publish. Amazon S3 compatible storage on Snowball Edge supports the following notification types: + New object created events + Object removal events + Object tagging events **Configure Amazon S3 event notifications** 1. Before you begin, you must have MQTT infrastructure in your network. 1. In your Snowball Edge client, run the `snowballEdge configure` command to set up the Snowball Edge device. When prompted, enter the following information: + The path to your manifest file. + The device's unlock code. + The device's endpoint (for example, *https://10.0.0.1*). 1. Run the following `put-notification-configuration` command to send notifications to an external broker. ``` snowballEdge put-notification-configuration --broker-endpoint ssl://mqtt-broker-ip-address:8883 --enabled true --service-id s3-snow --ca-certificate file:path-to-mqtt-broker-ca-cert ``` 1. Run the following `get-notification-configuration` command to verify that everything is set up correctly: ``` snowballEdge get-notification-configuration --service-id s3-snow ``` This returns the broker endpoint and enabled field. After you configure the entire cluster to send notifications to the MQTT broker in the network, every object API call will result in an event notification. **Note** You need to subscribe to the topic s3SnowEvents/*Device ID* (or *Cluster Id* if it is a cluster)/bucketName. You can also use wildcards, for example topic name can be *\$1* or *s3SnowEvents/\$1*. The following is an example Amazon S3 compatible storage on Snowball Edge event log: ``` { "eventDetails": { "additionalEventData": { "AuthenticationMethod": "AuthHeader", "CipherSuite": "ECDHE-RSA-AES128-GCM-SHA256", "SignatureVersion": "SigV4", "bytesTransferredIn": 1205, "bytesTransferredOut": 0, "x-amz-id-2": "uLdTfvdGTKlX6TBgCZtDd9Beef8wzUurA+Wpht7rKtfdaNsnxeLILg==" }, "eventName": "PutObject", "eventTime": "2023-01-30T14:13:24.772Z", "requestAuthLatencyMillis": 40, "requestBandwidthKBs": 35, "requestID": "140CD93455CB62B4", "requestLatencyMillis": 77, "requestLockLatencyNanos": 1169953, "requestParameters": { "Content-Length": "1205", "Content-MD5": "GZdTUOhYHvHgQgmaw2gl4w==", "Host": "10.0.2.251", "bucketName": "bucket", "key": "file-key" }, "requestTTFBLatencyMillis": 77, "responseElements": { "ETag": ""19975350e8581ef1e042099ac36825e3"", "Server": "AmazonS3", "x-amz-id-2": "uLdTfvdGTKlX6TBgCZtDd9Beef8wzUurA+Wpht7rKtfdaNsnxeLILg==", "x-amz-request-id": "140CD93455CB62B4" }, "responseStatusCode": 200, "sourceIPAddress": "172.31.37.21", "userAgent": "aws-cli/1.27.23 Python/3.7.16 Linux/4.14.301-224.520.amzn2.x86_64 botocore/1.29.23", "userIdentity": { "identityType": "IAMUser", "principalId": "531520547609", "arn": "arn:aws:iam::531520547609:root", "userName": "root" } } } ``` For more information about Amazon S3 event notifications, see [Amazon S3 Event Notifications](https://docs.aws.amazon.com/AmazonS3/latest/userguide/EventNotifications.html). # Configuring local SMTP notifications on Snowball Edge You can set up local notifications for your Snowball Edge devices with Simple Mail Transfer Protocol (SMTP). The local notifications send emails to configured servers when the service state (Active, Degraded, Inactive) changes, or if you cross capacity utilization thresholds of 80%, 90%, or 100%. Before you begin, confirm that: + You have access to the latest Snowball Edge client. + Your device is unlocked and ready to use. + Your device can connect to the internet (if using Amazon Simple Email Service or external SMTP server) or to a local SMTP server. ## Configuring the Snowball Edge for local notifications Set up a Snowball Edge to send email notifications. **To configure the device for SMTP notifications** 1. Run the following command to add an SMTP configuration to your device: ``` # If you don't specify a port, port 587 is the default. SMTP_ENDPOINT=your-local-smtp-server-endpoint:port # For multiple email recipients, separate with commas RECIPIENTS_LIST=your-email-address snowballEdge put-notification-configuration \ --service-id local-monitoring \ --enabled true \ --type smtp \ --broker-endpoint "$SMTP_ENDPOINT" \ --sender example-sender@domain.com \ --recipients "$RECIPIENTS_LIST" ``` You receive a test email from **example-sender@domain.com** if you're successful. 1. Test the configuration by running the following `get-notification-configuration` command: ``` snowballEdge get-notification-configuration \ --service-id local-monitoring ``` The response doesn't include a password or certificate, even if you provide them.