# Processing data in a custom labeling workflow with AWS Lambda Processing with Lambda In this topic, you can learn how to deploy optional [AWS Lambda](https://aws.amazon.com/lambda/) functions when creating a custom labeling workflow. You can specify two types of Lambda functions to use with your custom labeling workflow. + *Pre-annotation Lambda*: This function pre-processes each data object sent to your labeling job prior to sending it to workers. + *Post-annotation Lambda*: This function processes the results once workers submit a task. If you specify multiple workers per data object, this function may include logic to consolidate annotations. If you are a new user of Lambda and Ground Truth, we recommend that you use the pages in this section as follows: 1. First, review [Using pre-annotation and post-annotation Lambda functionsUsing Lambda functions](sms-custom-templates-step3-lambda-requirements.md). 1. Then, use the page [Add required permissions to use AWS Lambda with Ground Truth](sms-custom-templates-step3-lambda-permissions.md) to learn about security and permission requirements to use your pre-annotation and post-annotation Lambda functions in a Ground Truth custom labeling job. 1. Next, you need to visit the Lambda console or use Lambda's APIs to create your functions. Use the section [Create Lambda functions using Ground Truth templates](sms-custom-templates-step3-lambda-create.md) to learn how to create Lambda functions. 1. To learn how to test your Lambda functions, see [Test pre-annotation and post-annotation Lambda functions](sms-custom-templates-step3-lambda-test.md). 1. After you create pre-processing and post-processing Lambda functions, select them from the **Lambda functions** section that comes after the code editor for your custom HTML in the Ground Truth console. To learn how to use these functions in a `CreateLabelingJob` API request, see [Create a Labeling Job (API)](sms-create-labeling-job-api.md). For a custom labeling workflow tutorial that includes example pre-annotation and post-annotation Lambda functions, see [Demo template: Annotation of images with `crowd-bounding-box`](sms-custom-templates-step2-demo1.md). **Topics** + [ # Using pre-annotation and post-annotation Lambda functions ](sms-custom-templates-step3-lambda-requirements.md) + [ # Add required permissions to use AWS Lambda with Ground Truth ](sms-custom-templates-step3-lambda-permissions.md) + [ # Create Lambda functions using Ground Truth templates ](sms-custom-templates-step3-lambda-create.md) + [ # Test pre-annotation and post-annotation Lambda functions ](sms-custom-templates-step3-lambda-test.md) # Using pre-annotation and post-annotation Lambda functions Using Lambda functions Use these topics to learn about the syntax of the requests sent to pre-annotation and post-annotation Lambda functions, and the required response syntax that Ground Truth uses in custom labeling workflows. **Topics** + [ ## Pre-annotation Lambda ](#sms-custom-templates-step3-prelambda) + [ ## Post-annotation Lambda ](#sms-custom-templates-step3-postlambda) ## Pre-annotation Lambda Before a labeling task is sent to the worker, a optional pre-annotation Lambda function can be invoked. Ground Truth sends your Lambda function a JSON formatted request to provide details about the labeling job and the data object. The following are 2 example JSON formatted requests. ------ #### [ Data object identified with "source-ref" ] ``` { "version": "2018-10-16", "labelingJobArn": "dataObject" : { "source-ref": } } ``` ------ #### [ Data object identified with "source" ] ``` { "version": "2018-10-16", "labelingJobArn": "dataObject" : { "source": } } ``` ------ The following list contains the pre-annotation request schemas. Each parameter is described below. + `version` (string): This is a version number used internally by Ground Truth. + `labelingJobArn` (string): This is the Amazon Resource Name, or ARN, of your labeling job. This ARN can be used to reference the labeling job when using Ground Truth API operations such as `DescribeLabelingJob`. + The `dataObject` (JSON object): The key contains a single JSON line, either from your input manifest file or sent from Amazon SNS. The JSON line objects in your manifest can be up to 100 kilobytes in size and contain a variety of data. For a very basic image annotation job, the `dataObject` JSON may just contain a `source-ref` key, identifying the image to be annotated. If the data object (for example, a line of text) is included directly in the input manifest file, the data object is identified with `source`. If you create a verification or adjustment job, this line may contain label data and metadata from the previous labeling job. The following tabbed examples show examples of a pre-annotation request. Each parameter in these example requests is explained below the tabbed table. ------ #### [ Data object identified with "source-ref" ] ``` { "version": "2018-10-16", "labelingJobArn": "arn:aws:sagemaker:us-west-2:111122223333:labeling-job/" "dataObject" : { "source-ref": "s3://input-data-bucket/data-object-file-name" } } ``` ------ #### [ Data object identified with "source" ] ``` { "version": "2018-10-16", "labelingJobArn": "arn:aws:sagemaker::111122223333:labeling-job/" "dataObject" : { "source": "Sue purchased 10 shares of the stock on April 10th, 2020" } } ``` ------ In return, Ground Truth requires a response formatted like the following: **Example of expected return data** ``` { "taskInput": , "isHumanAnnotationRequired": # Optional } ``` In the previous example, the `` needs to contain *all* the data your custom worker task template needs. If you're doing a bounding box task where the instructions stay the same all the time, it may just be the HTTP(S) or Amazon S3 resource for your image file. If it's a sentiment analysis task and different objects may have different choices, it is the object reference as a string and the choices as an array of strings. **Implications of `isHumanAnnotationRequired`** This value is optional because it defaults to `true`. The primary use case for explicitly setting it is when you want to exclude this data object from being labeled by human workers. If you have a mix of objects in your manifest, with some requiring human annotation and some not needing it, you can include a `isHumanAnnotationRequired` value in each data object. You can add logic to your pre-annotation Lambda to dynamically determine if an object requires annotation, and set this boolean value accordingly. ### Examples of pre-annotation Lambda functions The following basic pre-annotation Lambda function accesses the JSON object in `dataObject` from the initial request, and returns it in the `taskInput` parameter. ``` import json def lambda_handler(event, context): return { "taskInput": event['dataObject'] } ``` Assuming the input manifest file uses `"source-ref"` to identify data objects, the worker task template used in the same labeling job as this pre-annotation Lambda must include a Liquid element like the following to ingest `dataObject`: ``` {{ task.input.source-ref | grant_read_access }} ``` If the input manifest file used `source` to identify the data object, the work task template can ingest `dataObject` with the following: ``` {{ task.input.source }} ``` The following pre-annotation Lambda example includes logic to identify the key used in `dataObject`, and to point to that data object using `taskObject` in the Lambda's return statement. ``` import json def lambda_handler(event, context): # Event received print("Received event: " + json.dumps(event, indent=2)) # Get source if specified source = event['dataObject']['source'] if "source" in event['dataObject'] else None # Get source-ref if specified source_ref = event['dataObject']['source-ref'] if "source-ref" in event['dataObject'] else None # if source field present, take that otherwise take source-ref task_object = source if source is not None else source_ref # Build response object output = { "taskInput": { "taskObject": task_object }, "humanAnnotationRequired": "true" } print(output) # If neither source nor source-ref specified, mark the annotation failed if task_object is None: print(" Failed to pre-process {} !".format(event["labelingJobArn"])) output["humanAnnotationRequired"] = "false" return output ``` ## Post-annotation Lambda When all workers have annotated the data object or when [https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_HumanLoopConfig.html#SageMaker-Type-HumanLoopConfig-TaskAvailabilityLifetimeInSeconds](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_HumanLoopConfig.html#SageMaker-Type-HumanLoopConfig-TaskAvailabilityLifetimeInSeconds) has been reached, whichever comes first, Ground Truth sends those annotations to your post-annotation Lambda. This Lambda is generally used for [Annotation consolidation](sms-annotation-consolidation.md). **Note** To see an example of a post-consolidation Lambda function, see [annotation\$1consolidation\$1lambda.py](https://github.com/aws-samples/aws-sagemaker-ground-truth-recipe/blob/master/aws_sagemaker_ground_truth_sample_lambda/annotation_consolidation_lambda.py) in the [aws-sagemaker-ground-truth-recipe](https://github.com/aws-samples/aws-sagemaker-ground-truth-recipe) GitHub repository. The following code block contains the post-annotation request schema. Each parameter is described in the following bulleted list. ``` { "version": "2018-10-16", "labelingJobArn": , "labelCategories": [], "labelAttributeName": , "roleArn" : , "payload": { "s3Uri": } } ``` + `version` (string): A version number used internally by Ground Truth. + `labelingJobArn` (string): The Amazon Resource Name, or ARN, of your labeling job. This ARN can be used to reference the labeling job when using Ground Truth API operations such as `DescribeLabelingJob`. + `labelCategories` (list of strings): Includes the label categories and other attributes you either specified in the console, or that you include in the label category configuration file. + `labelAttributeName` (string): Either the name of your labeling job, or the label attribute name you specify when you create the labeling job. + `roleArn` (string): The Amazon Resource Name (ARN) of the IAM execution role you specify when you create the labeling job. + `payload` (JSON object): A JSON that includes an `s3Uri` key, which identifies the location of the annotation data for that data object in Amazon S3. The second code block below shows an example of this annotation file. The following code block contains an example of a post-annotation request. Each parameter in this example request is explained below the code block. **Example of an post-annotation Lambda request** ``` { "version": "2018-10-16", "labelingJobArn": "arn:aws:sagemaker:us-west-2:111122223333:labeling-job/labeling-job-name", "labelCategories": ["Ex Category1","Ex Category2", "Ex Category3"], "labelAttributeName": "labeling-job-attribute-name", "roleArn" : "arn:aws:iam::111122223333:role/role-name", "payload": { "s3Uri": "s3://amzn-s3-demo-bucket/annotations.json" } } ``` **Note** If no worker works on the data object and `TaskAvailabilityLifetimeInSeconds` has been reached, the data object is marked as failed and not included as part of post-annotation Lambda invocation. The following code block contains the payload schema. This is the file that is indicated by the `s3Uri` parameter in the post-annotation Lambda request `payload` JSON object. For example, if the previous code block is the post-annotation Lambda request, the following annotation file is located at `s3://amzn-s3-demo-bucket/annotations.json`. Each parameter is described in the following bulleted list. **Example of an annotation file** ``` [ { "datasetObjectId": , "dataObject": { "s3Uri": , "content": }, "annotations": [{ "workerId": , "annotationData": { "content": , "s3Uri": } }] } ] ``` + `datasetObjectId` (string): Identifies a unique ID that Ground Truth assigns to each data object you send to the labeling job. + `dataObject` (JSON object): The data object that was labeled. If the data object is included in the input manifest file and identified using the `source` key (for example, a string), `dataObject` includes a `content` key, which identifies the data object. Otherwise, the location of the data object (for example, a link or S3 URI) is identified with `s3Uri`. + `annotations` (list of JSON objects): This list contains a single JSON object for each annotation submitted by workers for that `dataObject`. A single JSON object contains a unique `workerId` that can be used to identify the worker that submitted that annotation. The `annotationData` key contains one of the following: + `content` (string): Contains the annotation data. + `s3Uri` (string): Contains an S3 URI that identifies the location of the annotation data. The following table contains examples of the content that you may find in payload for different types of annotation. ------ #### [ Named Entity Recognition Payload ] ``` [ { "datasetObjectId": "1", "dataObject": { "content": "Sift 3 cups of flour into the bowl." }, "annotations": [ { "workerId": "private.us-west-2.ef7294f850a3d9d1", "annotationData": { "content": "{\"crowd-entity-annotation\":{\"entities\":[{\"endOffset\":4,\"label\":\"verb\",\"startOffset\":0},{\"endOffset\":6,\"label\":\"number\",\"startOffset\":5},{\"endOffset\":20,\"label\":\"object\",\"startOffset\":15},{\"endOffset\":34,\"label\":\"object\",\"startOffset\":30}]}}" } } ] } ] ``` ------ #### [ Semantic Segmentation Payload ] ``` [ { "datasetObjectId": "2", "dataObject": { "s3Uri": "s3://amzn-s3-demo-bucket/gt-input-data/images/bird3.jpg" }, "annotations": [ { "workerId": "private.us-west-2.ab1234c5678a919d0", "annotationData": { "content": "{\"crowd-semantic-segmentation\":{\"inputImageProperties\":{\"height\":2000,\"width\":3020},\"labelMappings\":{\"Bird\":{\"color\":\"#2ca02c\"}},\"labeledImage\":{\"pngImageData\":\"iVBOR...\"}}}" } } ] } ] ``` ------ #### [ Bounding Box Payload ] ``` [ { "datasetObjectId": "0", "dataObject": { "s3Uri": "s3://amzn-s3-demo-bucket/gt-input-data/images/bird1.jpg" }, "annotations": [ { "workerId": "private.us-west-2.ab1234c5678a919d0", "annotationData": { "content": "{\"boundingBox\":{\"boundingBoxes\":[{\"height\":2052,\"label\":\"Bird\",\"left\":583,\"top\":302,\"width\":1375}],\"inputImageProperties\":{\"height\":2497,\"width\":3745}}}" } } ] } ] ``` ------ Your post-annotation Lambda function may contain logic similar to the following to loop through and access all annotations contained in the request. For a full example, see [annotation\$1consolidation\$1lambda.py](https://github.com/aws-samples/aws-sagemaker-ground-truth-recipe/blob/master/aws_sagemaker_ground_truth_sample_lambda/annotation_consolidation_lambda.py) in the [aws-sagemaker-ground-truth-recipe](https://github.com/aws-samples/aws-sagemaker-ground-truth-recipe) GitHub repository. In this GitHub example, you must add your own annotation consolidation logic. ``` for i in range(len(annotations)): worker_id = annotations[i]["workerId"] annotation_content = annotations[i]['annotationData'].get('content') annotation_s3_uri = annotations[i]['annotationData'].get('s3uri') annotation = annotation_content if annotation_s3_uri is None else s3_client.get_object_from_s3( annotation_s3_uri) annotation_from_single_worker = json.loads(annotation) print("{} Received Annotations from worker [{}] is [{}]" .format(log_prefix, worker_id, annotation_from_single_worker)) ``` **Tip** When you run consolidation algorithms on the data, you can use an AWS database service to store results, or you can pass the processed results back to Ground Truth. The data you return to Ground Truth is stored in consolidated annotation manifests in the S3 bucket specified for output during the configuration of the labeling job. In return, Ground Truth requires a response formatted like the following: **Example of expected return data** ``` [ { "datasetObjectId": , "consolidatedAnnotation": { "content": { "": { # ... label content } } } }, { "datasetObjectId": , "consolidatedAnnotation": { "content": { "": { # ... label content } } } } . . . ] ``` At this point, all the data you're sending to your S3 bucket, other than the `datasetObjectId`, is in the `content` object. When you return annotations in `content`, this results in an entry in your job's output manifest like the following: **Example of label format in output manifest** ``` { "source-ref"/"source" : "", "": { # ... label content from you }, "-metadata": { # This will be added by Ground Truth "job_name": , "type": "groundTruth/custom", "human-annotated": "yes", "creation_date": # Timestamp of when received from Post-labeling Lambda } } ``` Because of the potentially complex nature of a custom template and the data it collects, Ground Truth does not offer further processing of the data. # Add required permissions to use AWS Lambda with Ground Truth You may need to configure some or all the following to create and use AWS Lambda with Ground Truth. + You need to grant an IAM role or user (collectively, an IAM entity) permission to create the pre-annotation and post-annotation Lambda functions using AWS Lambda, and to choose them when creating the labeling job. + The IAM execution role specified when the labeling job is configured needs permission to invoke the pre-annotation and post-annotation Lambda functions. + The post-annotation Lambda functions may need permission to access Amazon S3. Use the following sections to learn how to create the IAM entities and grant permissions described above. **Topics** + [ ## Grant Permission to Create and Select an AWS Lambda Function ](#sms-custom-templates-step3-postlambda-create-perms) + [ ## Grant IAM Execution Role Permission to Invoke AWS Lambda Functions ](#sms-custom-templates-step3-postlambda-execution-role-perms) + [ ## Grant Post-Annotation Lambda Permissions to Access Annotation ](#sms-custom-templates-step3-postlambda-perms) ## Grant Permission to Create and Select an AWS Lambda Function If you do not require granular permissions to develop pre-annotation and post-annotation Lambda functions, you can attach the AWS managed policy `AWSLambda_FullAccess` to a user or role. This policy grants broad permissions to use all Lambda features, as well as permission to perform actions in other AWS services with which Lambda interacts. To create a more granular policy for security-sensitive use cases, refer to the documentation [Identity-based IAM policies for Lambda](https://docs.aws.amazon.com/lambda/latest/dg/access-control-identity-based.html) in the to AWS Lambda Developer Guide to learn how to create an IAM policy that fits your use case. **Policies to Use the Lambda Console** If you want to grant an IAM entity permission to use the Lambda console, see [Using the Lambda console](https://docs.aws.amazon.com/lambda/latest/dg/security_iam_id-based-policy-examples.html#security_iam_id-based-policy-examples-console) in the AWS Lambda Developer Guide. Additionally, if you want the user to be able to access and deploy the Ground Truth starter pre-annotation and post-annotation functions using the AWS Serverless Application Repository in the Lambda console, you must specify the *``* where you want to deploy the functions (this should be the same AWS Region used to create the labeling job), and add the following policy to the IAM role. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": [ "serverlessrepo:ListApplicationVersions", "serverlessrepo:GetApplication", "serverlessrepo:CreateCloudFormationTemplate" ], "Resource": "arn:aws:serverlessrepo:us-east-1:838997950401:applications/aws-sagemaker-ground-truth-recipe" }, { "Sid": "VisualEditor1", "Effect": "Allow", "Action": "serverlessrepo:SearchApplications", "Resource": "*" } ] } ``` ------ **Policies to See Lambda Functions in the Ground Truth Console** To grant an IAM entity permission to view Lambda functions in the Ground Truth console when the user is creating a custom labeling job, the entity must have the permissions described in [Grant IAM Permission to Use the Amazon SageMaker Ground Truth Console](sms-security-permission-console-access.md), including the permissions described in the section [Custom Labeling Workflow Permissions](sms-security-permission-console-access.md#sms-security-permissions-custom-workflow). ## Grant IAM Execution Role Permission to Invoke AWS Lambda Functions If you add the IAM managed policy [AmazonSageMakerGroundTruthExecution](https://console.aws.amazon.com/iam/home?#/policies/arn:aws:iam::aws:policy/AmazonSageMakerGroundTruthExecution) to the IAM execution role used to create the labeling job, this role has permission to list and invoke Lambda functions with one of the following strings in the function name: `GtRecipe`, `SageMaker`, `Sagemaker`, `sagemaker`, or `LabelingFunction`. If the pre-annotation or post-annotation Lambda function names do not include one of the terms in the preceding paragraph, or if you require more granular permission than those in the `AmazonSageMakerGroundTruthExecution` managed policy, you can add a policy similar to the following to give the execution role permission to invoke pre-annotation and post-annotation functions. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "lambda:InvokeFunction", "Resource": [ "arn:aws:lambda:us-east-1:111122223333:function:", "arn:aws:lambda:us-east-1:111122223333:function:" ] } ] } ``` ------ ## Grant Post-Annotation Lambda Permissions to Access Annotation As described in [Post-annotation Lambda](sms-custom-templates-step3-lambda-requirements.md#sms-custom-templates-step3-postlambda), the post-annotation Lambda request includes the location of the annotation data in Amazon S3. This location is identified by the `s3Uri` string in the `payload` object. To process the annotations as they come in, even for a simple pass through function, you need to assign the necessary permissions to the post-annotation [Lambda execution role](https://docs.aws.amazon.com/lambda/latest/dg/lambda-intro-execution-role.html) to read files from the Amazon S3. There are many ways that you can configure your Lambda to access annotation data in Amazon S3. Two common ways are: + Allow the Lambda execution role to assume the SageMaker AI execution role identified in `roleArn` in the post-annotation Lambda request. This SageMaker AI execution role is the one used to create the labeling job, and has access to the Amazon S3 output bucket where the annotation data is stored. + Grant the Lambda execution role permission to access the Amazon S3 output bucket directly. Use the following sections to learn how to configure these options. **Grant Lambda Permission to Assume SageMaker AI Execution Role** To allow a Lambda function to assume a SageMaker AI execution role, you must attach a policy to the Lambda function's execution role, and modify the trust relationship of the SageMaker AI execution role to allow Lambda to assume it. 1. [Attach the following IAM policy](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_manage-attach-detach.html) to your Lambda function's execution role to assume the SageMaker AI execution role identified in `Resource`. Replace `222222222222` with an [AWS account ID](https://docs.aws.amazon.com/general/latest/gr/acct-identifiers.html). Replace `sm-execution-role` with the name of the assumed role. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": { "Effect": "Allow", "Action": "sts:AssumeRole", "Resource": "arn:aws:iam::222222222222:role/sm-execution-role" } } ``` ------ 1. [Modify the trust policy](https://docs.aws.amazon.com/IAM/latest/UserGuide/roles-managingrole-editing-console.html#roles-managingrole_edit-trust-policy) of the SageMaker AI execution role to include the following `Statement`. Replace `222222222222` with an [AWS account ID](https://docs.aws.amazon.com/general/latest/gr/acct-identifiers.html). Replace `my-lambda-execution-role` with the name of the assumed role. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::222222222222:role/my-lambda-execution-role" }, "Action": "sts:AssumeRole" } ] } ``` ------ **Grant Lambda Execution Role Permission to Access S3** You can add a policy similar to the following to the post-annotation Lambda function execution role to give it S3 read permissions. Replace *amzn-s3-demo-bucket* with the name of the output bucket you specify when you create a labeling job. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:GetObject" ], "Resource": "arn:aws:s3:::amzn-s3-demo-bucket/*" } ] } ``` ------ To add S3 read permissions to a Lambda execution role in the Lambda console, use the following procedure. **Add S3 read permissions to post-annotation Lambda:** 1. Open the [**Functions** page](https://console.aws.amazon.com/lambda/home#/functions) in the Lambda console. 1. Choose the name of the post-annotation function. 1. Choose **Configuration** and then choose **Permissions**. 1. Select the **Role name** and the summary page for that role opens in the IAM console in a new tab. 1. Select **Attach policies**. 1. Do one of the following: + Search for and select **`AmazonS3ReadOnlyAccess`** to give the function permission to read all buckets and objects in the account. + If you require more granular permissions, select **Create policy** and use the policy example in the preceding section to create a policy. Note that you must navigate back to the execution role summary page after you create the policy. 1. If you used the `AmazonS3ReadOnlyAccess` managed policy, select **Attach policy**. If you created a new policy, navigate back to the Lambda execution role summary page and attach the policy you just created. # Create Lambda functions using Ground Truth templates You can create a Lambda function using the Lambda console, the AWS CLI, or an AWS SDK in a supported programming language of your choice. Use the AWS Lambda Developer Guide to learn more about each of these options: + To learn how to create a Lambda function using the console, see [Create a Lambda function with the console](https://docs.aws.amazon.com/lambda/latest/dg/getting-started-create-function.html). + To learn how to create a Lambda function using the AWS CLI, see [Using AWS Lambda with the AWS Command Line Interface](https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-awscli.html). + Select the relevant section in the table of contents to learn more about working with Lambda in the language of your choice. For example, select [Working with Python](https://docs.aws.amazon.com/lambda/latest/dg/lambda-python.html) to learn more about using Lambda with the AWS SDK for Python (Boto3). Ground Truth provides pre-annotation and post-annotation templates through an AWS Serverless Application Repository (SAR) *recipe*. Use the following procedure to select the Ground Truth recipe in the Lambda console. **Use the Ground Truth SAR recipe to create pre-annotation and post-annotation Lambda functions:** 1. Open the [**Functions** page](https://console.aws.amazon.com/lambda/home#/functions) on the Lambda console. 1. Select **Create function**. 1. Select **Browse serverless app repository**. 1. In the search text box, enter **aws-sagemaker-ground-truth-recipe** and select that app. 1. Select **Deploy**. The app may take a couple of minutes to deploy. Once the app deploys, two functions appear in the **Functions** section of the Lambda console: `serverlessrepo-aws-sagema-GtRecipePreHumanTaskFunc-` and `serverlessrepo-aws-sagema-GtRecipeAnnotationConsol-`. 1. Select one of these functions and add your custom logic in the **Code** section. 1. When you are finished making changes, select **Deploy** to deploy them. # Test pre-annotation and post-annotation Lambda functions You can test your pre-annotation and post annotation Lambda functions in the Lambda console. If you are a new user of Lambda, you can learn how to test, or *invoke*, your Lambda functions in the console using the [Create a Lambda function](https://docs.aws.amazon.com/lambda/latest/dg/getting-started-create-function.html#gettingstarted-zip-function) tutorial with the console in the AWS Lambda Developer Guide. You can use the sections on this page to learn how to test the Ground Truth pre-annotation and post-annotation templates provided through an AWS Serverless Application Repository (SAR). **Topics** + [ ## Prerequisites ](#sms-custom-templates-step3-lambda-test-pre) + [ ## Test the Pre-annotation Lambda Function ](#sms-custom-templates-step3-lambda-test-pre-annotation) + [ ## Test the Post-Annotation Lambda Function ](#sms-custom-templates-step3-lambda-test-post-annotation) ## Prerequisites You must do the following to use the tests described on this page. + You need access to the Lambda console, and you need permission to create and invoke Lambda functions. To learn how to set up these permissions, see [Grant Permission to Create and Select an AWS Lambda Function](sms-custom-templates-step3-lambda-permissions.md#sms-custom-templates-step3-postlambda-create-perms). + If you have not deployed the Ground Truth SAR recipe, use the procedure in [Create Lambda functions using Ground Truth templates](sms-custom-templates-step3-lambda-create.md) to do so. + To test the post-annotation Lambda function, you must have a data file in Amazon S3 with sample annotation data. For a simple test, you can copy and paste the following code into a file and save it as `sample-annotations.json` and [upload this file to Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/upload-objects.html). Note the S3 URI of this file—you need this information to configure the post-annotation Lambda test. ``` [{"datasetObjectId":"0","dataObject":{"content":"To train a machine learning model, you need a large, high-quality, labeled dataset. Ground Truth helps you build high-quality training datasets for your machine learning models."},"annotations":[{"workerId":"private.us-west-2.0123456789","annotationData":{"content":"{\"crowd-entity-annotation\":{\"entities\":[{\"endOffset\":8,\"label\":\"verb\",\"startOffset\":3},{\"endOffset\":27,\"label\":\"adjective\",\"startOffset\":11},{\"endOffset\":33,\"label\":\"object\",\"startOffset\":28},{\"endOffset\":51,\"label\":\"adjective\",\"startOffset\":46},{\"endOffset\":65,\"label\":\"adjective\",\"startOffset\":53},{\"endOffset\":74,\"label\":\"adjective\",\"startOffset\":67},{\"endOffset\":82,\"label\":\"adjective\",\"startOffset\":75},{\"endOffset\":102,\"label\":\"verb\",\"startOffset\":97},{\"endOffset\":112,\"label\":\"verb\",\"startOffset\":107},{\"endOffset\":125,\"label\":\"adjective\",\"startOffset\":113},{\"endOffset\":134,\"label\":\"adjective\",\"startOffset\":126},{\"endOffset\":143,\"label\":\"object\",\"startOffset\":135},{\"endOffset\":169,\"label\":\"adjective\",\"startOffset\":153},{\"endOffset\":176,\"label\":\"object\",\"startOffset\":170}]}}"}}]},{"datasetObjectId":"1","dataObject":{"content":"Sift 3 cups of flour into the bowl."},"annotations":[{"workerId":"private.us-west-2.0123456789","annotationData":{"content":"{\"crowd-entity-annotation\":{\"entities\":[{\"endOffset\":4,\"label\":\"verb\",\"startOffset\":0},{\"endOffset\":6,\"label\":\"number\",\"startOffset\":5},{\"endOffset\":20,\"label\":\"object\",\"startOffset\":15},{\"endOffset\":34,\"label\":\"object\",\"startOffset\":30}]}}"}}]},{"datasetObjectId":"2","dataObject":{"content":"Jen purchased 10 shares of the stock on Janurary 1st, 2020."},"annotations":[{"workerId":"private.us-west-2.0123456789","annotationData":{"content":"{\"crowd-entity-annotation\":{\"entities\":[{\"endOffset\":3,\"label\":\"person\",\"startOffset\":0},{\"endOffset\":13,\"label\":\"verb\",\"startOffset\":4},{\"endOffset\":16,\"label\":\"number\",\"startOffset\":14},{\"endOffset\":58,\"label\":\"date\",\"startOffset\":40}]}}"}}]},{"datasetObjectId":"3","dataObject":{"content":"The narrative was interesting, however the character development was weak."},"annotations":[{"workerId":"private.us-west-2.0123456789","annotationData":{"content":"{\"crowd-entity-annotation\":{\"entities\":[{\"endOffset\":29,\"label\":\"adjective\",\"startOffset\":18},{\"endOffset\":73,\"label\":\"adjective\",\"startOffset\":69}]}}"}}]}] ``` + You must use the directions in [Grant Post-Annotation Lambda Permissions to Access Annotation](sms-custom-templates-step3-lambda-permissions.md#sms-custom-templates-step3-postlambda-perms) to give your post-annotation Lambda function's execution role permission to assume the SageMaker AI execution role you use to create the labeling job. The post-annotation Lambda function uses the SageMaker AI execution role to access the annotation data file, `sample-annotations.json`, in S3. ## Test the Pre-annotation Lambda Function Use the following procedure to test the pre-annotation Lambda function created when you deployed the Ground Truth AWS Serverless Application Repository (SAR) recipe. **Test the Ground Truth SAR recipe pre-annotation Lambda function** 1. Open the [**Functions** page](https://console.aws.amazon.com/lambda/home#/functions) in the Lambda console. 1. Select the pre-annotation function that was deployed from the Ground Truth SAR recipe. The name of this function is similar to `serverlessrepo-aws-sagema-GtRecipePreHumanTaskFunc-`. 1. In the **Code source** section, select the arrow next to **Test**. 1. Select **Configure test event**. 1. Keep the **Create new test event** option selected. 1. Under **Event template**, select **SageMaker Ground Truth PreHumanTask**. 1. Give your test an **Event name**. 1. Select **Create**. 1. Select the arrow next to **Test** again and you should see that the test you created is selected, which is indicated with a dot by the event name. If it is not selected, select it. 1. Select **Test** to run the test. After you run the test, you can see the **Execution results**. In the **Function logs**, you should see a response similar to the following: ``` START RequestId: cd117d38-8365-4e1a-bffb-0dcd631a878f Version: $LATEST Received event: { "version": "2018-10-16", "labelingJobArn": "arn:aws:sagemaker:us-east-2:123456789012:labeling-job/example-job", "dataObject": { "source-ref": "s3://sagemakerexample/object_to_annotate.jpg" } } {'taskInput': {'taskObject': 's3://sagemakerexample/object_to_annotate.jpg'}, 'isHumanAnnotationRequired': 'true'} END RequestId: cd117d38-8365-4e1a-bffb-0dcd631a878f REPORT RequestId: cd117d38-8365-4e1a-bffb-0dcd631a878f Duration: 0.42 ms Billed Duration: 1 ms Memory Size: 128 MB Max Memory Used: 43 MB ``` In this response, we can see the Lambda function's output matches the required pre-annotation response syntax: ``` {'taskInput': {'taskObject': 's3://sagemakerexample/object_to_annotate.jpg'}, 'isHumanAnnotationRequired': 'true'} ``` ## Test the Post-Annotation Lambda Function Use the following procedure to test the post-annotation Lambda function created when you deployed the Ground Truth AWS Serverless Application Repository (SAR) recipe. **Test the Ground Truth SAR recipe post-annotation Lambda** 1. Open the [**Functions** page](https://console.aws.amazon.com/lambda/home#/functions) in the Lambda console. 1. Select the post-annotation function that was deployed from the Ground Truth SAR recipe. The name of this function is similar to `serverlessrepo-aws-sagema-GtRecipeAnnotationConsol-`. 1. In the **Code source** section, select the arrow next to **Test**. 1. Select **Configure test event**. 1. Keep the **Create new test event** option selected. 1. Under **Event template**, select **SageMaker Ground Truth AnnotationConsolidation**. 1. Give your test an **Event name**. 1. Modify the template code provided as follows: + Replace the Amazon Resource Name (ARN) in `roleArn` with the ARN of the SageMaker AI execution role you used to create the labeling job. + Replace the S3 URI in `s3Uri` with the URI of the `sample-annotations.json` file you added to Amazon S3. After you make these modifications, your test should look similar to the following: ``` { "version": "2018-10-16", "labelingJobArn": "arn:aws:sagemaker:us-east-2:123456789012:labeling-job/example-job", "labelAttributeName": "example-attribute", "roleArn": "arn:aws:iam::222222222222:role/sm-execution-role", "payload": { "s3Uri": "s3://your-bucket/sample-annotations.json" } } ``` 1. Select **Create**. 1. Select the arrow next to **Test** again and you should see that the test you created is selected, which is indicated with a dot by the event name. If it is not selected, select it. 1. Select the **Test** to run the test. After you run the test, you should see a `-- Consolidated Output --` section in the **Function Logs**, which contains a list of all annotations included in `sample-annotations.json`.