Snapshot management - Research and Engineering Studio
Create a snapshotApply a snapshot

Snapshot management

Snapshot management simplifies the process of saving and migrating data between environments, ensuring consistency and accuracy. With snapshots, you can save your environment state and migrate data into a new environment with the same state.

Snapshot management page

From the Snapshot management page, you can:

  1. View all created snapshots and their status.

  2. Create a snapshot. Before you can create a snapshot, you will need to create a bucket with the appropriate permissions.

  3. View all applied snapshots and their status.

  4. Apply a snapshot.

Create a snapshot

Before you can create a snapshot, you must provide an Amazon S3 bucket with the necessary permissions. For information on creating a bucket, see Creating a bucket. We recommend enabling bucket versioning and server access logging. These settings can be enabled from the bucket's Properties tab after provisioning.

Note

This Amazon S3 bucket's lifecycle will not be managed within the product. You will need to manage the bucket lifecycle from the console.

To add permissions to the bucket:

  1. Choose the bucket you created from the Buckets list.

  2. Choose the Permissions tab.

  3. Under Bucket policy, choose Edit.

  4. Add the following statement to the bucket policy. Replace these values with your own:

    • AWS_ACCOUNT_ID

    • RES_ENVIRONMENT_NAME

    • AWS_REGION

    • S3_BUCKET_NAME

    Important

    There are limited version strings supported by AWS. For more information, see https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_version.html

    JSON
    {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Export-Snapshot-Policy",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::111122223333:role/{RES_ENVIRONMENT_NAME}-cluster-manager-role-{AWS_REGION}}"
            },
            "Action": [
                "s3:GetObject",
                "s3:ListBucket",
                "s3:AbortMultipartUpload",
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": [
                "arn:aws:s3:::{S3_BUCKET_NAME}",
                "arn:aws:s3:::{S3_BUCKET_NAME}/*"
            ]
        },
        {
            "Sid": "AllowSSLRequestsOnly",
            "Action": "s3:*",
            "Effect": "Deny",
            "Resource": [
                "arn:aws:s3:::{S3_BUCKET_NAME}",
                "arn:aws:s3:::{S3_BUCKET_NAME}/*"
            ],
            "Condition": {
                "Bool": {
                    "aws:SecureTransport": "false"
                }
            },
            "Principal": "*"
        }
    ]
}
To create the snapshot:

  1. Choose Create Snapshot.

  2. Enter the name of the Amazon S3 bucket you created.

  3. Enter the path where you would like the snapshot stored within the bucket. For example, october2023/23.

  4. Choose Submit.

    Create a new snapshot

  5. After five to ten minutes, choose Refresh on the Snapshots page to check the status. A snapshot will not be valid until the status changes from IN_PROGRESS to COMPLETED.

Apply a snapshot

Once you have created a snapshot of an environment, you can apply that snapshot to a new environment to migrate data. You will need to add a new policy to the bucket allowing the environment to read the snapshot.

Applying a snapshot copies data such as user permissions, projects, software stacks, permission profiles, and file systems with their associations to a new environment. User sessions will not be replicated. When the snapshot is applied, it checks the basic information of each resource record to determine if it already exists. For duplicate records, the snapshot skips resource creation in the new environment. For records that are similar, such as share a name or key, but other basic resource information varies, it will create a new record with a modified name and key using the following convention: RecordName_SnapshotRESVersion_ApplySnapshotID. The ApplySnapshotID looks like a timestamp and identifies each attempt to apply a snapshot.

During the snapshot application, the snapshot checks for the availability of resources. Resource not available to the new environment will not be created. For resources with a dependent resource, the snapshot checks for the availability of the dependent resource. If the dependent resource is not available, it will create the main resource without the dependent resource.

If the new environment is not as expected or fails, you can check the CloudWatch logs found in the log group /res-<env-name>/cluster-manager for details. Each log will have the [apply snapshot] tag. Once you have applied a snapshot, you can check its status from the Snapshot management page.

To add permissions to the bucket:

  1. Choose the bucket you created from the Buckets list.

  2. Choose the Permissions tab.

  3. Under Bucket policy, choose Edit.

  4. Add the following statement to the bucket policy. Replace these values with your own:

    • AWS_ACCOUNT_ID

    • RES_ENVIRONMENT_NAME

    • AWS_REGION

    • S3_BUCKET_NAME

    JSON
    {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Export-Snapshot-Policy",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::111122223333:role/{RES_ENVIRONMENT_NAME}-cluster-manager-role-{AWS_REGION}}"
            },
            "Action": [
                "s3:GetObject",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::{S3_BUCKET_NAME}",
                "arn:aws:s3:::{S3_BUCKET_NAME}/*"
            ]
        },
        {
            "Sid": "AllowSSLRequestsOnly",
            "Action": "s3:*",
            "Effect": "Deny",
            "Resource": [
                "arn:aws:s3:::{S3_BUCKET_NAME}",
                "arn:aws:s3:::{S3_BUCKET_NAME}/*"
            ],
            "Condition": {
                "Bool": {
                    "aws:SecureTransport": "false"
                }
            },
            "Principal": "*"
        }
    ]
}
To apply snapshot:

  1. Choose Apply snapshot.

  2. Enter the name of the Amazon S3 bucket containing the snapshot.

  3. Enter the file path to the snapshot within the bucket.

  4. Choose Submit.

    Apply a snapshot

  5. After five to ten minutes, choose Refresh on the Snapshot management page to check the status.