# First Steps with Amazon SageMaker AI Edge Manager
This guide demonstrates how to complete the necessary steps to register, deploy, and manage a fleet of devices, and how to satisfy Amazon SageMaker AI Edge Manager prerequisites.
**Topics**
+ [Setting Up](edge-getting-started-step1.md)
+ [Prepare Your Model for Deployment](edge-getting-started-step2.md)
+ [Register and Authenticate Your Device Fleet](edge-getting-started-step3.md)
+ [Download and Set Up Edge Manager](edge-getting-started-step4.md)
+ [Run Agent](edge-getting-started-step5.md)
# Setting Up
Before you begin using SageMaker Edge Manager to manage models on your device fleets, you must first create IAM Roles for both SageMaker AI and AWS IoT. You will also want to create at least one Amazon S3 bucket where you will store your pre-trained model, the output of your SageMaker Neo compilation job, as well as input data from your edge devices.
## Sign up for an AWS account
If you do not have an AWS account, complete the following steps to create one.
**To sign up for an AWS account**
1. Open [https://portal.aws.amazon.com/billing/signup](https://portal.aws.amazon.com/billing/signup).
1. Follow the online instructions.
Part of the sign-up procedure involves receiving a phone call or text message and entering a verification code on the phone keypad.
When you sign up for an AWS account, an *AWS account root user* is created. The root user has access to all AWS services and resources in the account. As a security best practice, assign administrative access to a user, and use only the root user to perform [tasks that require root user access](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_root-user.html#root-user-tasks).
AWS sends you a confirmation email after the sign-up process is complete. At any time, you can view your current account activity and manage your account by going to [https://aws.amazon.com/](https://aws.amazon.com/) and choosing **My Account**.
## Create a user with administrative access
After you sign up for an AWS account, secure your AWS account root user, enable AWS IAM Identity Center, and create an administrative user so that you don't use the root user for everyday tasks.
**Secure your AWS account root user**
1. Sign in to the [AWS Management Console](https://console.aws.amazon.com/) as the account owner by choosing **Root user** and entering your AWS account email address. On the next page, enter your password.
For help signing in by using root user, see [Signing in as the root user](https://docs.aws.amazon.com/signin/latest/userguide/console-sign-in-tutorials.html#introduction-to-root-user-sign-in-tutorial) in the *AWS Sign-In User Guide*.
1. Turn on multi-factor authentication (MFA) for your root user.
For instructions, see [Enable a virtual MFA device for your AWS account root user (console)](https://docs.aws.amazon.com/IAM/latest/UserGuide/enable-virt-mfa-for-root.html) in the *IAM User Guide*.
**Create a user with administrative access**
1. Enable IAM Identity Center.
For instructions, see [Enabling AWS IAM Identity Center](https://docs.aws.amazon.com//singlesignon/latest/userguide/get-set-up-for-idc.html) in the *AWS IAM Identity Center User Guide*.
1. In IAM Identity Center, grant administrative access to a user.
For a tutorial about using the IAM Identity Center directory as your identity source, see [ Configure user access with the default IAM Identity Center directory](https://docs.aws.amazon.com//singlesignon/latest/userguide/quick-start-default-idc.html) in the *AWS IAM Identity Center User Guide*.
**Sign in as the user with administrative access**
+ To sign in with your IAM Identity Center user, use the sign-in URL that was sent to your email address when you created the IAM Identity Center user.
For help signing in using an IAM Identity Center user, see [Signing in to the AWS access portal](https://docs.aws.amazon.com/signin/latest/userguide/iam-id-center-sign-in-tutorial.html) in the *AWS Sign-In User Guide*.
**Assign access to additional users**
1. In IAM Identity Center, create a permission set that follows the best practice of applying least-privilege permissions.
For instructions, see [ Create a permission set](https://docs.aws.amazon.com//singlesignon/latest/userguide/get-started-create-a-permission-set.html) in the *AWS IAM Identity Center User Guide*.
1. Assign users to a group, and then assign single sign-on access to the group.
For instructions, see [ Add groups](https://docs.aws.amazon.com//singlesignon/latest/userguide/addgroups.html) in the *AWS IAM Identity Center User Guide*.
## Create roles and storage
SageMaker Edge Manager needs access to your Amazon S3 bucket URI. To facilitate this, create an IAM role that can run SageMaker AI and has permission to access Amazon S3. Using this role, SageMaker AI can run under your account and access to your Amazon S3 bucket.
You can create an IAM role by using the IAM console, AWS SDK for Python (Boto3), or AWS CLI. The following is an example of how to create an IAM role, attach the necessary policies with the IAM console, and create an Amazon S3 bucket.
1. **Create an IAM role for Amazon SageMaker AI.**
1. Sign in to the AWS Management Console and open the IAM console at [https://console.aws.amazon.com/iam/](https://console.aws.amazon.com/iam/).
1. In the navigation pane of the IAM console, choose **Roles**, and then choose **Create role**.
1. For **Select type of trusted entity**, choose **AWS service**.
1. Choose the service that you want to allow to assume this role. In this case, choose **SageMaker AI**. Then choose **Next: Permissions**.
+ This automatically creates an IAM policy that grants access to related services such as Amazon S3, Amazon ECR, and CloudWatch Logs.
1. Choose **Next: Tags**.
1. (Optional) Add metadata to the role by attaching tags as key–value pairs. For more information about using tags in IAM, see [Tagging IAM resources](https://docs.aws.amazon.com//IAM/latest/UserGuide/id_tags.html).
1. Choose **Next: Review**.
1. Type in a **Role name**.
1. If possible, type a role name or role name suffix. Role names must be unique within your AWS account. They are not distinguished by case. For example, you cannot create roles named both `PRODROLE` and `prodrole`. Because other AWS resources might reference the role, you cannot edit the name of the role after it has been created.
1. (Optional) For **Role description**, type a description for the new role.
1. Review the role and then choose **Create role**.
Note the SageMaker AI Role ARN, which you use to create a compilation job with SageMaker Neo and a packaging job with Edge Manager. To find out the role ARN using the console, do the following:
1. Go to the IAMconsole: [https://console.aws.amazon.com/iam/](https://console.aws.amazon.com/iam/)
1. Select **Roles**.
1. Search for the role you just created by typing in the name of the role in the search field.
1. Select the role.
1. The role ARN is at the top of the **Summary** page.
1. **Create an IAM role for AWS IoT.**
The AWS IoT IAM role you create is used to authorize your thing objects. You also use the IAM role ARN to create and register device fleets with a SageMaker AI client object.
Configure an IAM role in your AWS account for the credentials provider to assume on behalf of the devices in your device fleet. Then, attach a policy to authorize your devices to interact with AWS IoT services.
Create a role for AWS IoT either programmatically or with the IAM console, similar to what you did when you created a role for SageMaker AI.
1. Sign in to the AWS Management Console and open the IAM console at [https://console.aws.amazon.com/iam/](https://console.aws.amazon.com/iam/).
1. In the navigation pane of the IAM console, choose **Roles**, and then choose **Create role**.
1. For **Select type of trusted entity**, choose **AWS service**.
1. Choose the service that you want to allow to assume this role. In this case, choose **IoT**. Select **IoT** as the **Use Case**.
1. Choose **Next: Permissions**.
1. Choose **Next: Tags**.
1. (Optional) Add metadata to the role by attaching tags as key–value pairs. For more information about using tags in IAM, see [Tagging IAM resources](https://docs.aws.amazon.com//IAM/latest/UserGuide/id_tags.html).
1. Choose **Next: Review**.
1. Type in a **Role name**. The role name must start with `SageMaker AI`.
1. (Optional) For **Role description**, type a description for the new role.
1. Review the role and then choose **Create role**.
1. Once the role is created, choose **Roles** in the IAM console. Search for the role you created by typing in role name in the **Search** field.
1. Choose your role.
1. Next, choose **Attach Policies**.
1. Search for `AmazonSageMakerEdgeDeviceFleetPolicy` in the **Search** field. Select `AmazonSageMakerEdgeDeviceFleetPolicy`.
1. Choose **Attach policy**.
1. Add the following policy statement to the trust relationship:
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {"Service": "credentials.iot.amazonaws.com"},
"Action": "sts:AssumeRole"
},
{
"Effect": "Allow",
"Principal": {"Service": "sagemaker.amazonaws.com"},
"Action": "sts:AssumeRole"
}
]
}
```
------
A trust policy is a [JSON policy document](https://docs.aws.amazon.com//IAM/latest/UserGuide/reference_policies_grammar) in which you define the principals that you trust to assume the role. For more information about trust policies, see [Roles terms and concepts](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_terms-and-concepts.html).
1. Note the AWS IoT role ARN. You use the AWS IoT Role ARN to create and register the device fleet. To find the IAM role ARN with the console:
1. Go to the IAM console: [https://console.aws.amazon.com/iam/](https://console.aws.amazon.com/iam/)
1. Choose **Roles**.
1. Search for the role you created by typing in the name of the role in the **Search** field.
1. Select the role.
1. The role ARN is on the Summary page.
1. **Create an Amazon S3 bucket.**
SageMaker Neo and Edge Manager access your pre-compiled model and compiled model from an Amazon S3 bucket. Edge Manager also stores sample data from your device fleet in Amazon S3.
1. Open the Amazon S3 console at [https://console.aws.amazon.com/s3/](https://console.aws.amazon.com/s3/).
1. Choose **Create bucket**.
1. In **Bucket name**, enter a name for your bucket.
1. In **Region**, choose the AWS Region where you want the bucket to reside.
1. In **Bucket settings for Block Public Access**, choose the settings that you want to apply to the bucket.
1. Choose **Create bucket**.
For more information about creating Amazon S3 buckets, see [Getting started with Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/GetStartedWithS3.html).
# Prepare Your Model for Deployment
In this section you will create SageMaker AI and AWS IoT client objects, download a pre-trained machine learning model, upload your model to your Amazon S3 bucket, compile your model for your target device with SageMaker Neo, and package your model so that it can be deployed with the Edge Manager agent.
1. **Import libraries and create client objects.**
This tutorial uses the AWS SDK for Python (Boto3) to create clients to interact with SageMaker AI, Amazon S3, and AWS IoT.
Import Boto3, specify your Region, and initialize the client objects you need as shown in the following example:
```
import boto3
import json
import time
AWS_REGION = 'us-west-2'# Specify your Region
bucket = 'bucket-name'
sagemaker_client = boto3.client('sagemaker', region_name=AWS_REGION)
iot_client = boto3.client('iot', region_name=AWS_REGION)
```
Define variables and assign them the role ARN you created for SageMaker AI and AWS IoT as strings:
```
# Replace with the role ARN you created for SageMaker
sagemaker_role_arn = "arn:aws:iam:::role/*"
# Replace with the role ARN you created for AWS IoT.
# Note: The name must start with 'SageMaker'
iot_role_arn = "arn:aws:iam:::role/SageMaker*"
```
1. **Train a machine learning model.**
See [Train a Model with Amazon SageMaker](https://docs.aws.amazon.com/sagemaker/latest/dg/how-it-works-training.html) for more information on how to train a machine learning model using SageMaker AI. You can optionally upload your locally trained model directly into an Amazon S3 URI bucket.
If you do not have a model yet, you can use a pre-trained model for the next steps in this tutorial. For example, you can save the MobileNet V2 models from the TensorFlow framework. MobileNet V2 is an image classification model optimized for mobile applications. For more information about MobileNet V2, see the [MobileNet GitHub README](https://github.com/tensorflow/models/tree/master/research/slim/nets/mobilenet).
Type the following into your Jupyter Notebook to save the pre-trained MobileNet V2 model:
```
# Save the MobileNet V2 model to local storage
import tensorflow as tf
model = tf.keras.applications.MobileNetV2()
model.save(“mobilenet_v2.h5”)
```
**Note**
If you do not have TensorFlow installed, you can do so by running `pip install tensorflow=2.4`
Use TensorFlow version 2.4 or lower for this tutorial.
The model will be saved into the `mobilenet_v2.h5` file. Before packaging the model, you will need to first compile your model using SageMaker Neo. See [Supported Frameworks, Devices, Systems, and Architectures](neo-supported-devices-edge.md) to check if your version of TensorFlow (or other framework of choice) is currently supported by SageMaker Neo.
SageMaker Neo requires models to be stored as a compressed TAR file. Repackage it as a compressed TAR file (\$1.tar.gz):
```
# Package MobileNet V2 model into a TAR file
import tarfile
tarfile_name='mobilenet-v2.tar.gz'
with tarfile.open(tarfile_name, mode='w:gz') as archive:
archive.add('mobilenet-v2.h5')
```
1. **Upload your model to Amazon S3.**
Once you have a machine learning model, store it in an Amazon S3 bucket. The following example uses an AWS CLI command to upload the model the to the Amazon S3 bucket you created earlier in a directory called *models*. Type in the following into your Jupyter Notebook:
```
!aws s3 cp mobilenet-v2.tar.gz s3://{bucket}/models/
```
1. **Compile your model with SageMaker Neo.**
Compile your machine learning model with SageMaker Neo for an edge device. You need to know your Amazon S3 bucket URI where you stored the trained model, the machine learning framework you used to train your model, the shape of your model’s input, and your target device.
For the MobileNet V2 model, use the following:
```
framework = 'tensorflow'
target_device = 'jetson_nano'
data_shape = '{"data":[1,3,224,224]}'
```
SageMaker Neo requires a specific model input shape and model format based on the deep learning framework you use. For more information about how to save your model, see [What input data shapes does SageMaker Neo expect?](neo-compilation-preparing-model.md#neo-job-compilation-expected-inputs). For more information about devices and frameworks supported by Neo, see [Supported Frameworks, Devices, Systems, and Architectures](neo-supported-devices-edge.md).
Use the `CreateCompilationJob` API to create a compilation job with SageMaker Neo. Provide a name to the compilation job, the SageMaker AI Role ARN, the Amazon S3 URI where your model is stored, the input shape of the model, the name of the framework, the Amazon S3 URI where you want SageMaker AI to store your compiled model, and your edge device target.
```
# Specify the path where your model is stored
model_directory = 'models'
s3_model_uri = 's3://{}/{}/{}'.format(bucket, model_directory, tarfile_name)
# Store compiled model in S3 within the 'compiled-models' directory
compilation_output_dir = 'compiled-models'
s3_output_location = 's3://{}/{}/'.format(bucket, compilation_output_dir)
# Give your compilation job a name
compilation_job_name = 'getting-started-demo'
sagemaker_client.create_compilation_job(CompilationJobName=compilation_job_name,
RoleArn=sagemaker_role_arn,
InputConfig={
'S3Uri': s3_model_uri,
'DataInputConfig': data_shape,
'Framework' : framework.upper()},
OutputConfig={
'S3OutputLocation': s3_output_location,
'TargetDevice': target_device},
StoppingCondition={'MaxRuntimeInSeconds': 900})
```
1. **Package your compiled model.**
Packaging jobs take SageMaker Neo–compiled models and make any changes necessary to deploy the model with the inference engine, Edge Manager agent. To package your model, create an edge packaging job with the `create_edge_packaging` API or the SageMaker AI console.
You need to provide the name that you used for your Neo compilation job, a name for the packaging job, a role ARN (see [Setting Up](edge-getting-started-step1.md) section), a name for the model, a model version, and the Amazon S3 bucket URI for the output of the packaging job. Note that Edge Manager packaging job names are case-sensitive. The following is an example of how to create a packaging job using the API.
```
edge_packaging_name='edge-packaging-demo'
model_name="sample-model"
model_version="1.1"
```
Define the Amazon S3 URI where you want to store the packaged model.
```
# Output directory where you want to store the output of the packaging job
packaging_output_dir = 'packaged_models'
packaging_s3_output = 's3://{}/{}'.format(bucket, packaging_output_dir)
```
Use `CreateEdgePackagingJob` to package your Neo-compiled model. Provide a name for your edge packaging job and the name you provided for your compilation job (in this example, it was stored in the variable `compilation_job_name`). Also provide a name for your model, a version for your model (this is used to help you keep track of what model version you are using), and the S3 URI where you want SageMaker AI to store the packaged model.
```
sagemaker_client.create_edge_packaging_job(
EdgePackagingJobName=edge_packaging_name,
CompilationJobName=compilation_job_name,
RoleArn=sagemaker_role_arn,
ModelName=model_name,
ModelVersion=model_version,
OutputConfig={
"S3OutputLocation": packaging_s3_output
}
)
```
# Register and Authenticate Your Device Fleet
In this section you will create your AWS IoT thing object, create a device fleet, register your device fleet so it can interact with the cloud, create X.509 certificates to authenticate your devices to AWS IoT Core, associate the role alias with AWS IoT that was generated when you created your fleet, get your AWS account-specific endpoint for the credentials provider, get an official Amazon Root CA file, and upload the Amazon CA file to Amazon S3.
1. **Create AWS IoT things.**
SageMaker Edge Manager takes advantage of the AWS IoT Core services to facilitate the connection between the edge devices and endpoints in the AWS cloud. You can take advantage of existing AWS IoT functionality after you set up your devices to work with Edge Manager.
To connect your device to AWS IoT, you need to create AWS IoT *thing objects*, create and register a client certificate with AWS IoT, and create and configure the IAM role for your devices.
First, create AWS IoT thing objects with the AWS IoT client (`iot_client`) you created earlier with Boto3. The following example shows how to create two thing objects:
```
iot_thing_name = 'sample-device'
iot_thing_type = 'getting-started-demo'
iot_client.create_thing_type(
thingTypeName=iot_thing_type
)
# Create an AWS IoT thing objects
iot_client.create_thing(
thingName=iot_thing_name,
thingTypeName=iot_thing_type
)
```
1. **Create your device fleet.**
Create a device fleet with the SageMaker AI client object defined in a previous step. You can also use the SageMaker AI console to create a device fleet.
```
import time
device_fleet_name="demo-device-fleet" + str(time.time()).split('.')[0]
device_name="sagemaker-edge-demo-device" + str(time.time()).split('.')[0]
```
Specify your IoT role ARN. This lets AWS IoT grant temporary credentials to devices.
```
device_model_directory='device_output'
s3_device_fleet_output = 's3://{}/{}'.format(bucket, device_model_directory)
sagemaker_client.create_device_fleet(
DeviceFleetName=device_fleet_name,
RoleArn=iot_role_arn, # IoT Role ARN specified in previous step
OutputConfig={
'S3OutputLocation': s3_device_fleet_output
}
)
```
An AWS IoT role alias is created when you create a device fleet. This role alias is associated with AWS IoT using the `iot_client` object in a later step.
1. **Register your device fleet.**
To interact with the cloud, you need to register your device with SageMaker Edge Manager. In this example, you register a single device with the fleet you created. To register the device, you need to provide a device name and the AWS IoT thing name as shown in the following example:
```
# Device name should be 36 characters
device_name = "sagemaker-edge-demo-device" + str(time.time()).split('.')[0]
sagemaker_client.register_devices(
DeviceFleetName=device_fleet_name,
Devices=[
{
"DeviceName": device_name,
"IotThingName": iot_thing_name
}
]
)
```
1. **Create X.509 certificates.**
After creating the AWS IoT thing object, you must create a X.509 device certificate for your thing object. This certificate authenticates your device to AWS IoT Core.
Use the following to create a private key, public key, and a X.509 certificate file using the AWS IoT client defined (`iot_client`) earlier.
```
# Creates a 2048-bit RSA key pair and issues an X.509 # certificate
# using the issued public key.
create_cert = iot_client.create_keys_and_certificate(
setAsActive=True
)
# Get certificate from dictionary object and save in its own
with open('./device.pem.crt', 'w') as f:
for line in create_cert['certificatePem'].split('\n'):
f.write(line)
f.write('\n')
# Get private key from dictionary object and save in its own
with open('./private.pem.key', 'w') as f:
for line in create_cert['keyPair']['PrivateKey'].split('\n'):
f.write(line)
f.write('\n')
# Get a private key from dictionary object and save in its own
with open('./public.pem.key', 'w') as f:
for line in create_cert['keyPair']['PublicKey'].split('\n'):
f.write(line)
f.write('\n')
```
1. **Associate the role alias with AWS IoT.**
When you create a device fleet with SageMaker AI (`sagemaker_client.create_device_fleet()`), a role alias is generated for you. An AWS IoT role alias provides a mechanism for connected devices to authenticate to AWS IoT using X.509 certificates, and then obtain short-lived AWS credentials from an IAM role that is associated with an AWS IoT role alias. The role alias allows you to change the role of the device without having to update the device. Use `DescribeDeviceFleet` to get the role alias name and ARN.
```
# Print Amazon Resource Name (ARN) and alias that has access
# to AWS Internet of Things (IoT).
sagemaker_client.describe_device_fleet(DeviceFleetName=device_fleet_name)
# Store iot role alias string in a variable
# Grabs role ARN
full_role_alias_name = sagemaker_client.describe_device_fleet(DeviceFleetName=device_fleet_name)['IotRoleAlias']
start_index = full_role_alias_name.find('SageMaker') # Find beginning of role name
role_alias_name = full_role_alias_name[start_index:]
```
Use the `iot_client` to facilitate associating the role alias generated from creating the device fleet with AWS IoT:
```
role_alias = iot_client.describe_role_alias(
roleAlias=role_alias_name)
```
For more information about IAM role alias, see [Role alias allows access to unused services](https://docs.aws.amazon.com/iot/latest/developerguide/audit-chk-role-alias-unused-svcs.html) .
You created and registered a certificate with AWS IoT earlier for successful authentication of your device. Now, you need to create and attach a policy to the certificate to authorize the request for the security token.
```
alias_policy = {
"Version": "2012-10-17",
"Statement": {
"Effect": "Allow",
"Action": "iot:AssumeRoleWithCertificate",
"Resource": role_alias['roleAliasDescription']['roleAliasArn']
}
}
policy_name = 'aliaspolicy-'+ str(time.time()).split('.')[0]
aliaspolicy = iot_client.create_policy(policyName=policy_name,
policyDocument=json.dumps(alias_policy))
# Attach policy
iot_client.attach_policy(policyName=policy_name,
target=create_cert['certificateArn'])
```
1. **Get your AWS account-specific endpoint for the credentials provider.**
Edge devices need an endpoint in order to assume credentials. Obtain your AWS account-specific endpoint for the credentials provider.
```
# Get the unique endpoint specific to your AWS account that is making the call.
iot_endpoint = iot_client.describe_endpoint(
endpointType='iot:CredentialProvider'
)
endpoint="https://{}/role-aliases/{}/credentials".format(iot_endpoint['endpointAddress'],role_alias_name)
```
1. **Get the official Amazon root CA file and upload it to the Amazon S3 bucket.**
Use the following in your Jupyter Notebook or AWS CLI (if you use your terminal, remove the ‘\$1’ magic function):
```
!wget https://www.amazontrust.com/repository/AmazonRootCA1.pem
```
Use the endpoint to make an HTTPS request to the credentials provider to return a security token. The following example command uses `curl`, but you can use any HTTP client.
```
!curl --cert device.pem.crt --key private.pem.key --cacert AmazonRootCA1.pem $endpoint
```
If the certificate is verified, upload the keys and certificate to your Amazon S3 bucket URI:
```
!aws s3 cp private.pem.key s3://{bucket}/authorization-files/
!aws s3 cp device.pem.crt s3://{bucket}/authorization-files/
!aws s3 cp AmazonRootCA1.pem s3://{bucket}/authorization-files/
```
Clean your working directory by moving your keys and certificate to a different directory:
```
# Optional - Clean up working directory
!mkdir authorization-files
!mv private.pem.key device.pem.crt AmazonRootCA1.pem authorization-files/
```
# Download and Set Up Edge Manager
The Edge Manager agent is an inference engine for your edge devices. Use the agent to make predictions with models loaded onto your edge devices. The agent also collects model metrics and captures data at specific intervals.
In this section you will set up your device with the agent. To do so, first copy a release artifact and signing root certificate from the release bucket locally to your machine. After you unzip the release artifact, upload it to Amazon S3. Next, define and save a configuration file for the agent. A template is provided for you to copy and paste. Finally, copy the release artifacts, configuration file, and credentials to your device.
1. **Download the SageMaker Edge Manager agent.**
The agent is released in binary format for supported operating systems. This example runs inference on a Jetson Nano which uses a Linux operating system and has an ARM64 architecture. For more information about what operating system and architecture supported devices use, see [Supported Devices, Chip Architectures, and Systems](neo-supported-devices-edge-devices.md).
Fetch the latest version of binaries from the SageMaker Edge Manager release bucket from the us-west-2 Region.
```
!aws s3 ls s3://sagemaker-edge-release-store-us-west-2-linux-armv8/Releases/ | sort -r
```
This returns release artifacts sorted by their version.
```
PRE 1.20210512.96da6cc/
PRE 1.20210305.a4bc999/
PRE 1.20201218.81f481f/
PRE 1.20201207.02d0e97/
```
The version has the following format: `..`. It consists of three components:
+ ``: The release version. The release version is currently set to `1`.
+ ``: The time stamp of the artifact release.
+ : The repository commit ID from which the release is built.
Copy the zipped TAR file locally or to your device directly. The following example shows how to copy the latest release artifact at the time this document was released.
```
!aws s3 cp s3://sagemaker-edge-release-store-us-west-2-linux-x64/Releases/1.20201218.81f481f/1.20201218.81f481f.tgz ./
```
Once you have the artifact, unzip the zipped TAR file. The following unzips the TAR file and stores it in a directory called `agent_demo`:
```
!mkdir agent_demo
!tar -xvzf 1.20201218.81f481f.tgz -C ./agent_demo
```
Upload the agent release artifacts to your Amazon S3 bucket. The following code example copies the content within `agent_demo` and uploads it to a directory within your Amazon S3 bucket called `agent_demo`:
```
!aws s3 cp --recursive ./agent_demo s3://{bucket}/agent_demo
```
You also need the signing root certificates from the release bucket:
```
!aws s3 cp s3://sagemaker-edge-release-store-us-west-2-linux-x64/Certificates/us-west-2/us-west-2.pem ./
```
Upload the signing root certificate to your Amazon S3 bucket:
```
!aws s3 cp us-west-2.pem s3://{bucket}/authorization-files/
```
1. **Define a SageMaker Edge Manager agent configuration file.**
First, define the agent configuration file as follows:
```
sagemaker_edge_config = {
"sagemaker_edge_core_device_name": "device_name",
"sagemaker_edge_core_device_fleet_name": "device_fleet_name",
"sagemaker_edge_core_capture_data_buffer_size": 30,
"sagemaker_edge_core_capture_data_push_period_seconds": 4,
"sagemaker_edge_core_folder_prefix": "demo_capture",
"sagemaker_edge_core_region": "us-west-2",
"sagemaker_edge_core_root_certs_path": "/agent_demo/certificates",
"sagemaker_edge_provider_aws_ca_cert_file": "/agent_demo/iot-credentials/AmazonRootCA1.pem",
"sagemaker_edge_provider_aws_cert_file": "/agent_demo/iot-credentials/device.pem.crt",
"sagemaker_edge_provider_aws_cert_pk_file": "/agent_demo/iot-credentials/private.pem.key",
"sagemaker_edge_provider_aws_iot_cred_endpoint": "endpoint",
"sagemaker_edge_provider_provider": "Aws",
"sagemaker_edge_provider_s3_bucket_name": bucket,
"sagemaker_edge_core_capture_data_destination": "Cloud"
}
```
Replace the following:
+ `"device_name"` with the name of your device (this string was stored in an earlier step in a variable named `device_name`).
+ `"device_fleet_name`" with the name of your device fleet (this string was stored an earlier step in a variable named `device_fleet_name`)
+ `"endpoint"` with your AWS account-specific endpoint for the credentials provider (this string was stored in an earlier step in a variable named `endpoint`).
Next, save it as a JSON file:
```
edge_config_file = open("sagemaker_edge_config.json", "w")
json.dump(sagemaker_edge_config, edge_config_file, indent = 6)
edge_config_file.close()
```
Upload the configuration file to your Amazon S3 bucket:
```
!aws s3 cp sagemaker_edge_config.json s3://{bucket}/
```
1. **Copy the release artifacts, configuration file, and credentials to your device.**
The following instructions are performed on the edge device itself.
**Note**
You must first install Python, the AWS SDK for Python (Boto3), and the AWS CLI on your edge device.
Open a terminal on your device. Create a folder to store the release artifacts, your credentials, and the configuration file.
```
mkdir agent_demo
cd agent_demo
```
Copy the contents of the release artifacts that you stored in your Amazon S3 bucket to your device:
```
# Copy release artifacts
aws s3 cp s3:///agent_demo/ ./ --recursive
```
(The contents of the release artifact was stored in a directory called `agent_demo` in a previous step). Replace `` and `agent_demo` with the name of your Amazon S3 bucket and the file path to your release artifacts, respectively.
Go the `/bin` directory and make the binary files executable:
```
cd bin
chmod +x sagemaker_edge_agent_binary
chmod +x sagemaker_edge_agent_client_example
cd agent_demo
```
Make a directory to store your AWS IoT credentials and copy your credentials from your Amazon S3 bucket to your edge device (use the same on you define in the variable `bucket`:
```
mkdir iot-credentials
cd iot-credentials
aws s3 cp s3:///authorization-files/AmazonRootCA1.pem ./
aws s3 cp s3:///authorization-files/device.pem.crt ./
aws s3 cp s3:///authorization-files/private.pem.key ./
cd ../
```
Make a directory to store your model signing root certificates:
```
mkdir certificates
cd certificates
aws s3 cp s3:///authorization-files/us-west-2.pem ./
cd agent_demo
```
Copy your configuration file to your device:
```
#Download config file from S3
aws s3 cp s3:///sagemaker_edge_config.json ./
cd agent_demo
```
Your `agent_demo` directory on your edge device should look similar to the following:
```
├──agent_demo
| ├── bin
| ├── sagemaker_edge_agent_binary
| └── sagemaker_edge_agent_client_example
| ├── sagemaker_edge_config.json
| ├── certificates
| └──us-west-2.pem
| ├── iot-credentials
| ├── AmazonRootCA1.pem
| ├── device.pem.crt
| └── private.pem.key
| ├── docs
| ├── api
| └── examples
| ├── ATTRIBUTIONS.txt
| ├── LICENSE.txt
| └── RELEASE_NOTES.md
```
# Run Agent
In this section you will run the agent as a binary using gRPC, and check that both your device and fleet are working and collecting sample data.
1. **Launch the agent.**
The SageMaker Edge Manager agent can be run as a standalone process in the form of an Executable and Linkable Format (ELF) executable binary or can be linked against as a Dynamic Shared Object (.dll). Running as a standalone executable binary is the preferred mode and is supported on Linux.
This example uses gRPC to run the agent. gRPC is an open source high-performance Remote Procedure Call (RPC) framework that can run in any environment. For more information about gRPC, see the [gRPC documentation](https://grpc.io/docs/).
To use gRPC, perform the following steps:
1. Define a service in a .proto file.
1. Generate server and client code using the protocol buffer compiler.
1. Use the Python (or other languages supported by gRPC) gRPC API to write the server for your service.
1. Use the Python (or other languages supported by gRPC) gRPC API to write a client for your service.
The release artifact you downloaded contains a gRPC application ready for you to run the agent. The example is located within the `/bin` directory of your release artifact. The `sagemaker_edge_agent_binary` binary executable is in this directory.
To run the agent with this example, provide the path to your socket file (.sock) and JSON .config file:
```
./bin/sagemaker_edge_agent_binary -a /tmp/sagemaker_edge_agent_example.sock -c sagemaker_edge_config.json
```
1. **Check your device.**
Check that your device is connected and sampling data. Making periodic checks, manually or automatically, allows you to check that your device or fleet is working properly.
Provide the name of the fleet to which the device belongs and the unique device identifier. From your local machine, run the following:
```
sagemaker_client.describe_device(
DeviceName=device_name,
DeviceFleetName=device_fleet_name
)
```
For the given model, you can see the name, model version, latest sample time, and when the last inference was made.
```
{
"DeviceName": "sample-device",
"DeviceFleetName": "demo-device-fleet",
"IoTThingName": "sample-thing-name-1",
"RegistrationTime": 1600977370,
"LatestHeartbeat": 1600977370,
"Models":[
{
"ModelName": "mobilenet_v2.tar.gz",
"ModelVersion": "1.1",
"LatestSampleTime": 1600977370,
"LatestInference": 1600977370
}
]
}
```
The timestamp provided by `LastetHeartbeat` indicates the last signal that was received from the device. `LatestSampleTime` and `LatestInference` describe the time stamp of the last data sample and inference, respectively.
1. **Check your fleet.**
Check that your fleet is working with `GetDeviceFleetReport`. Provide the name of the fleet the device belongs to.
```
sagemaker_client.get_device_fleet_report(
DeviceFleetName=device_fleet_name
)
```
For a given model, you can see the name, model version, latest sample time, and when the last inference was made, along with the Amazon S3 bucket URI where the data samples are stored.
```
# Sample output
{
"DeviceFleetName": "sample-device-fleet",
"DeviceFleetArn": "arn:aws:sagemaker:us-west-2:9999999999:device-fleet/sample-fleet-name",
"OutputConfig": {
"S3OutputLocation": "s3://fleet-bucket/package_output",
},
"AgentVersions":[{"Version": "1.1", "AgentCount": 2}]}
"DeviceStats": {"Connected": 2, "Registered": 2},
"Models":[{
"ModelName": "sample-model",
"ModelVersion": "1.1",
"OfflineDeviceCount": 0,
"ConnectedDeviceCount": 2,
"ActiveDeviceCount": 2,
"SamplingDeviceCount": 100
}]
}
```