# Get Started with Amazon Mechanical Turk Get Started with Mechanical Turk Use the hands-on tutorials in this section to help you get started and learn more about Amazon Mechanical Turk. **Topics** + [ ## Prerequisites ](#get-started-prereq) + [ ## Step 1: Create a task ](#get-started-create-task) + [ ## Step 2: Check task status ](#get-started-task-status) + [ ## Step 3: Retrieve results ](#get-started-retrieve-result) + [ ## Step 4: Approve Assignments ](#get-started-approve-assignments) ## Prerequisites Before you begin, you should familiarize yourself with the basic concepts in Amazon Mechanical Turk. For more information, see [The Amazon Mechanical Turk marketplace](IntroMarketplace.md) and [Amazon Mechanical Turk core concepts](IntroCoreConcepts.md). Additional, complete the steps in [Set up Amazon Mechanical Turk](SetUpMturk.md) before completing this tutorial. ## Step 1: Create a task In this step we create a task in Mechanical Turk that asks workers to describe the current weather where they live. The task interface for this will be created using the [HTMLQuestion](https://docs.aws.amazon.com/AWSMechTurk/latest/AWSMturkAPI/ApiReference_HTMLQuestionArticle.html) data structure and we'll make use of [Crowd HTML Elements](https://docs.aws.amazon.com/AWSMechTurk/latest/AWSMturkAPI/ApiReference_HTMLQuestionArticle.html#ApiReference_HTMLQuestionArticle-crowd) to simplify the task HTML. **Important** Completing the steps in this tutorial results in a charge of \$10.60 to your account. ### Question definition The most common way to define tasks in Mechanical Turkis using the `HTMLQuestion` data structure, which is defined as XML that encapsulates the HTML that is displayed to the worker. For this task, we use the following definition. ```

Describe the current weather where you live.

 ]]> 0 ``` Note that the HTML includes a reference to the `crowd-html-elements.js` library which includes the `crowd-form` element. We use the `crowd-form` element in place of the standard `form` element because it removes the need to specify the endpoint for the form to submit results. It also automatically appends a **Submit** button if one isn't present. More information about this library can be found in [Crowd HTML Elements](https://docs.aws.amazon.com/sagemaker/latest/dg/sms-ui-template-reference.html). We've also set the value of `FrameHeight` to zero, which directs the marketplace website to render the task interface using the full browser window. ### Task attributes Next, we can define the attributes for our task. We'll use the following attributes: **** | Attribute | Value | | --- | --- | | Title | Describe the weather | | Description | Describe the current weather where you live | | Reward | 0.1 | | MaxAssignments | 5 | | LifetimeInSeconds | 14,400 | | AssignmentDurationInSeconds | 300 | | AutoApprovalDelayInSeconds | 259,200 | Here, we give an accurate description of our task and indicate that we will reward each worker with 10 cents for each successful completion. In addition, we set the `MaxAssignments` to `5` to indicate that we would like to get five responses from different workers. Finally, we set the lifetime and assignment duration to four hours and five minutes respectively. Workers have five minutes to complete the assignment before it expires and becomes available to other workers. If, after four hours, we haven't yet gotten a response, the task is automatically removed from the Mechanical Turk marketplace. We've also set the `AutoApprovalDelay` at three days which means that an assignment is automatically approved after three days if we don't take any action to approve or reject it before then. For more detail on the attributes that can be specified for a HIT, visit the [CreateHIT](https://docs.aws.amazon.com/AWSMechTurk/latest/AWSMturkAPI/ApiReference_CreateHITOperation.html) documentation. ### Post the task You can post a task using the AWS CLI or a language-specific AWS SDK. Select a tab in the following table to see an example of how you can post a task using the AWS CLI and the AWS SDK for Python (Boto3). ------ #### [ AWS CLI ] The following AWS CLI example creates a new task using [https://awscli.amazonaws.com/v2/documentation/api/latest/reference/mturk/create-hit.html](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/mturk/create-hit.html). ``` $ aws mturk create-hit \      --title "Describe the weather" \      --description "Describe the current weather where you live" \      --reward "0.10" \ --max-assignments 5 \      --lifetime-in-seconds 14400 \      --assignment-duration-in-seconds 300 \      --auto-approval-delay-in-seconds 259200 \      --question '

Describe the current weather where you live

 ]]> 0 '   ``` Using `create-hit` returns the following sample result. ``` {     "HIT": {         "HITId": "3TL87MO8CLOFYXKXNRLMZO1MOK4FL5",         "HITTypeId": "3AKE04YHPN13791QQA6BU4GMS7CHZJ",         "HITGroupId": "367GCHJ5533R84AG2MXJ0OFCCJ7M9Q",         "CreationTime": "2020-09-29T14:30:03-07:00",         "Title": "Describe the weather",         "Description": "Describe the current weather where you live",         "Question": "\n\n\n\n

Describe the current weather where you live

\n

\n\n]]>\n\n0\n",         "HITStatus": "Assignable",         "MaxAssignments": 5,         "Reward": "0.10",         "AutoApprovalDelayInSeconds": 259200,         "Expiration": "2020-09-29T18:30:03-07:00",         "AssignmentDurationInSeconds": 300,         "QualificationRequirements": [],         "HITReviewStatus": "NotReviewed",         "NumberOfAssignmentsPending": 0,         "NumberOfAssignmentsAvailable": 5,         "NumberOfAssignmentsCompleted": 0     } }  ``` ------ #### [ SDK for Python (Boto3) ] The following Python code creates a new task using [https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/mturk.html#MTurk.Client.create_hit](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/mturk.html#MTurk.Client.create_hit). This code can be run within a Jupyter Notebook or IPython as is, or can be incorporated into a Python script and executed. ``` import boto3   mturk = boto3.client('mturk')   question = """

Describe the current weather where you live

 ]]> 0 """   response = mturk.create_hit(     Title='Describe the weather',     Description='Describe the current weather where you live',     Reward='0.10',     MaxAssignments=5,     LifetimeInSeconds=14400,     AssignmentDurationInSeconds=300,     AutoApprovalDelayInSeconds=259200,     Question=question) hit_id = response['HIT']['HITId'] print('Created HIT: {}'.format(hit_id)) ``` This creates the HIT in the marketplace and displays the following. ``` Created HIT: 3QQUBC64ZDDMMJE3ZX577RS5PMNNXJ ``` After creating the HIT, capture the `HITId` that was created and proceed to Step 2: Check task status. ------ ## Step 2: Check task status In this step, we check the status of a task we created in Mechanical Turk. We poll the API for the status of our HIT using the `HITId` from the previous step. You need to capture that identifier and insert it in the following appropriate location to retrieve your results. The code block examples in this section have been spaced out for readability. ------ #### [ AWS CLI ] The following AWS CLI command retrieves the current state of a HIT using [https://awscli.amazonaws.com/v2/documentation/api/latest/reference/mturk/get-hit.html](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/mturk/get-hit.html). ``` $ aws mturk get-hit --hit-id 3QQUBC64ZDDMMJE3ZX577RS5PMNNXJ ``` Using `get-hit` immediately after creating the HIT returns the following result. The CLI returns the same information that was returned when you first created the HIT. This includes all of the attributes for the HIT and its current state. As you can see in the highlighted lines below, all five assignments are available to be requested by workers and the `HITStatus` is `Assignable`. ```   {     "HIT": {         "HITId": "3TL87MO8CLOFYXKXNRLMZO1MOK4FL5",         "HITTypeId": "3AKE04YHPN13791QQA6BU4GMS7CHZJ",         "HITGroupId": "367GCHJ5533R84AG2MXJ0OFCCJ7M9Q",         "CreationTime": "2020-09-29T14:30:03-07:00",         "Title": "Describe the weather",         "Description": "Describe the current weather where you live",         "Question": " \n\n \n\n

Describe the current weather where you live

\n

\n\n]]> \n\n0\n",         "HITStatus": "Assignable",         "MaxAssignments": 5,         "Reward": "0.10",         "AutoApprovalDelayInSeconds": 259200,         "Expiration": "2020-09-29T18:30:03-07:00",         "AssignmentDurationInSeconds": 300,         "QualificationRequirements": [],         "HITReviewStatus": "NotReviewed",         "NumberOfAssignmentsPending": 0,         "NumberOfAssignmentsAvailable": 5,         "NumberOfAssignmentsCompleted": 0     } } ``` If you run the same `get-hit` command again after about five to ten minutes, the results will likely match the following example. As you can see in the highlighted sections, the `HITStatus` is now `Reviewable` and there are no longer any assignments available or pending with workers. However, the completed count is still zero because none of the work has been approved yet. ``` {     "HIT": {         "HITId": "3TL87MO8CLOFYXKXNRLMZO1MOK4FL5",         "HITTypeId": "3AKE04YHPN13791QQA6BU4GMS7CHZJ",         "HITGroupId": "367GCHJ5533R84AG2MXJ0OFCCJ7M9Q",         "CreationTime": "2020-09-29T14:30:03-07:00",         "Title": "Describe the weather",         "Description": "Describe the current weather where you live",         "Question": " \n\n \n \n

Describe the current weather where you live

\n

\n\n]]> \n\n0 \n",         "HITStatus": "Reviewable",         "MaxAssignments": 5,         "Reward": "0.10",         "AutoApprovalDelayInSeconds": 259200,         "Expiration": "2020-09-29T18:30:03-07:00",         "AssignmentDurationInSeconds": 300,         "QualificationRequirements": [],         "HITReviewStatus": "NotReviewed",         "NumberOfAssignmentsPending": 0,         "NumberOfAssignmentsAvailable": 0,         "NumberOfAssignmentsCompleted": 0     } } ``` ------ #### [ Python SDK (Boto3) ] The following Python code retrieves the state of a HIT using [https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/mturk.html#MTurk.Client.get_hit](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/mturk.html#MTurk.Client.get_hit). This code can be run within a Jupyter Notebook or IPython as is, or can be incorporated into a Python script and executed. ``` import boto3   mturk = boto3.client('mturk')   response = mturk.get_hit(HITId='3QQUBC64ZDDMMJE3ZX577RS5PMNNXJ') response['HIT'] ``` Using `get_hit` immediately after creating the HIT returns the following result. The command returns the same information that was returned when you first created the HIT. That includes all of the attributes for the HIT and its current state. As you can see in the highlighted lines below, all five assignments are available to be requested by workers and the `HITStatus` is `Assignable`. ``` { 'HITId': '3TL87MO8CLOFYXKXNRLMZO1MOK4FL5', 'HITTypeId': '3AKE04YHPN13791QQA6BU4GMS7CHZJ', 'HITGroupId': '367GCHJ5533R84AG2MXJ0OFCCJ7M9Q', 'CreationTime': datetime.datetime(2020, 9, 29, 14, 30, 3, tzinfo=tzlocal()), 'Title': 'Describe the weather', 'Description': 'Describe the current weather where you live', 'Question': ' \n\n \n\n

Describe the current weather where you live

\n

\n\n]]>\n\n0 \n', 'HITStatus': 'Assignable', 'MaxAssignments': 5, 'Reward': '0.10', 'AutoApprovalDelayInSeconds': 259200, 'Expiration': datetime.datetime(2020, 9, 29, 18, 30, 3, tzinfo=tzlocal()), 'AssignmentDurationInSeconds': 300, 'QualificationRequirements': [], 'HITReviewStatus': 'NotReviewed', 'NumberOfAssignmentsPending': 0, 'NumberOfAssignmentsAvailable': 5, 'NumberOfAssignmentsCompleted': 0 } ``` If you run the same `get_hit` command again after about five minutes, the results will likely appear as shown below. As you can see in the highlighted sections, the `HITStatus` is now `Reviewable` and there are no longer any assignments available or pending with workers. However, the completed count is still zero because none of the work has been approved yet. ``` { 'HITId': '3TL87MO8CLOFYXKXNRLMZO1MOK4FL5',  'HITTypeId': '3AKE04YHPN13791QQA6BU4GMS7CHZJ',  'HITGroupId': '367GCHJ5533R84AG2MXJ0OFCCJ7M9Q',  'CreationTime': datetime.datetime(2020, 9, 29, 14, 30, 3, tzinfo=tzlocal()),  'Title': 'Describe the weather',  'Description': 'Describe the current weather where you live',  'Question': ' \n\n \n\n

Describe the current weather where you live

\n

\n\n]]>\n\n0 \n',  'HITStatus': 'Reviewable',  'MaxAssignments': 5,  'Reward': '0.10',  'AutoApprovalDelayInSeconds': 259200,  'Expiration': datetime.datetime(2020, 9, 29, 18, 30, 3, tzinfo=tzlocal()),  'AssignmentDurationInSeconds': 300,  'QualificationRequirements': [],  'HITReviewStatus': 'NotReviewed',  'NumberOfAssignmentsPending': 0,  'NumberOfAssignmentsAvailable': 0,  'NumberOfAssignmentsCompleted': 0 } ``` Now that we've confirmed that the HIT is in the `Reviewable` state, we can proceed to Step 3: Retrieve Results. ------ ## Step 3: Retrieve results In this step, we retrieve the results of a task we created in Mechanical Turk. We use the `HITId` from that was generated in Step 1. You need to capture that identifier and insert it in the appropriate location below to retrieve your results. Before running the commands in this step, you should confirm the `HITStatus` is `Reviewable` as shown in Step 2, or you may get incomplete results. ------ #### [ AWS CLI ] The following AWS CLI command retrieves all of the submitted assignments for your HIT using [https://awscli.amazonaws.com/v2/documentation/api/latest/reference/mturk/list-assignments-for-hit.html](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/mturk/list-assignments-for-hit.html). ``` $ aws mturk list-assignments-for-hit --hit-id 3TL87MO8CLOFYXKXNRLMZO1MOK4FL5 ``` Using `list-assignments-for-hit` returns an array of results similar to those shown below. Each assignment has an `AssignmentId` and includes information about the Worker who submitted it, when they first accepted it, and when they submitted their answer. The answer information is captured in a `[QuestionFormAnswers](https://docs.aws.amazon.com/AWSMechTurk/latest/AWSMturkAPI/ApiReference_QuestionFormAnswersDataStructureArticle.html)` XML data structure that you can read in to extract the results. For example, the first assignment below was submitted by the worker with `WorkerId` A1KYPXUBSBWJBY, it took them 25 seconds to complete it, and their answer was "Its currently raining lightly". Scrolling through the other assignments, you can see how other workers answered this question. ``` {     "Assignments": [         {             "AssignmentId": "3IOEN3P9S7I9CGLBJQ50ANAQJXB16B",             "WorkerId": "AIDACKCEVSQ6C2EXAMPLE",             "HITId": "3TL87MO8CLOFYXKXNRLMZO1MOK4FL5",             "AssignmentStatus": "Submitted",             "AutoApprovalTime": "2020-10-02T14:39:42-07:00",             "AcceptTime": "2020-09-29T14:39:17-07:00",             "SubmitTime": "2020-09-29T14:39:42-07:00",             "Answer": "taskAnswers[{\"weather\":\"Its currently raining lightly\"}]"         },         {             "AssignmentId": "32AT8R96GL8U8BA6SRINMUBFCJ5USP",             "WorkerId": "DAAICKCEVSQ6C2EXAMPLE",             "HITId": "3TL87MO8CLOFYXKXNRLMZO1MOK4FL5",             "AssignmentStatus": "Submitted",             "AutoApprovalTime": "2020-10-02T14:44:47-07:00",             "AcceptTime": "2020-09-29T14:42:45-07:00",             "SubmitTime": "2020-09-29T14:44:47-07:00",             "Answer": "taskAnswers[{\"weather\":\"It is currently chilly and cloudy outside and looks like it may rain in a little bit.\"}]"         },         {             "AssignmentId": "3FTOP5WARFNLTMF07QVP5MWL0BPJ0W",             "WorkerId": "DDAEBCKCEVSQ6C2EXAMPLE",             "HITId": "3TL87MO8CLOFYXKXNRLMZO1MOK4FL5",             "AssignmentStatus": "Submitted",             "AutoApprovalTime": "2020-10-02T14:48:12-07:00",             "AcceptTime": "2020-09-29T14:43:28-07:00",             "SubmitTime": "2020-09-29T14:48:12-07:00",             "Answer": "taskAnswers[{\"weather\":\"Currently indianapolis weather very cool, because northwest area are most of this situation landing. In currently approximately cloud is 15 to 18'C.\"}]"         },         {             "AssignmentId": "3DI28L7YXADDPVEQP8OYMB230VZE19",             "WorkerId": "AICACKCEVSQ6C2EXAMPLE",             "HITId": "3TL87MO8CLOFYXKXNRLMZO1MOK4FL5",             "AssignmentStatus": "Submitted",             "AutoApprovalTime": "2020-10-02T14:54:52-07:00",             "AcceptTime": "2020-09-29T14:53:05-07:00",             "SubmitTime": "2020-09-29T14:54:52-07:00",             "Answer": "taskAnswers[{\"weather\":\"32\"}]"         },         {             "AssignmentId": "3LO69W1SU3COZGELODW56TWTBHJLG3",             "WorkerId": "SDLKSDEVSQ6C2EXAMPLE",             "HITId": "3TL87MO8CLOFYXKXNRLMZO1MOK4FL5",             "AssignmentStatus": "Submitted",             "AutoApprovalTime": "2020-10-02T14:58:14-07:00",             "AcceptTime": "2020-09-29T14:57:39-07:00",             "SubmitTime": "2020-09-29T14:58:14-07:00",             "Answer": "taskAnswers[{\"weather\":\"Hot and dry, with poor air quality.\"}]"         }     ] }   ``` Note that the AWS CLI supports various output formats as well as a option [https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-output.html#cli-usage-output-filter](https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-output.html#cli-usage-output-filter) that can be used to filter your results. For example, the following only returns a list of the `WorkerId`s that completed the HIT. ``` $ aws mturk list-assignments-for-hit \ --hit-id 3TL87MO8CLOFYXKXNRLMZO1MOK4FL5 \ --query 'Assignments[*].WorkerId'   ``` ------ #### [ SDK for Python (Boto3) ] The following Python code retrieves all of the submitted assignments for your HIT using [https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/mturk.html#MTurk.Client.list_assignments_for_hit](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/mturk.html#MTurk.Client.list_assignments_for_hit) This code can be run within a Jupyter Notebook or IPython as is, or can be incorporated into a Python script and executed. ``` import boto3   mturk = boto3.client('mturk')   response = mturk.list_assignments_for_hit(HITId='3TL87MO8CLOFYXKXNRLMZO1MOK4FL5') response['Assignments']   ``` Using `list_assignments_for_hit` returns an array of results similar to those shown below. Each assignment has an `AssignmentId` and includes information about the worker who submitted it, when they first accepted it, and when they submitted their answer. The answer information is captured in a `[QuestionFormAnswers](https://docs.aws.amazon.com/AWSMechTurk/latest/AWSMturkAPI/ApiReference_QuestionFormAnswersDataStructureArticle.html)` XML data structure that you can read in to extract the results. For example, the first assignment below was submitted by the Worker with `WorkerId` AIDACKCEVSQ6C2EXAMPLE, it took them 25 seconds to complete it, and their answer was "Its currently raining lightly". Scrolling through the other assignments, you can see how other workers answered this question. ``` [ {'AssignmentId': '3IOEN3P9S7I9CGLBJQ50ANAQJXB16B',   'WorkerId': 'AIDACKCEVSQ6C2EXAMPLE,   'HITId': '3TL87MO8CLOFYXKXNRLMZO1MOK4FL5',   'AssignmentStatus': 'Submitted',   'AutoApprovalTime': datetime.datetime(2020, 10, 2, 14, 39, 42, tzinfo=tzlocal()),   'AcceptTime': datetime.datetime(2020, 9, 29, 14, 39, 17, tzinfo=tzlocal()),   'SubmitTime': datetime.datetime(2020, 9, 29, 14, 39, 42, tzinfo=tzlocal()),   'Answer': 'taskAnswers[{"weather":"Its currently raining lightly"}]' },  {'AssignmentId': '32AT8R96GL8U8BA6SRINMUBFCJ5USP',   'WorkerId': 'DAAICKCEVSQ6C2EXAMPLE,   'HITId': '3TL87MO8CLOFYXKXNRLMZO1MOK4FL5',   'AssignmentStatus': 'Submitted',   'AutoApprovalTime': datetime.datetime(2020, 10, 2, 14, 44, 47, tzinfo=tzlocal()),   'AcceptTime': datetime.datetime(2020, 9, 29, 14, 42, 45, tzinfo=tzlocal()),   'SubmitTime': datetime.datetime(2020, 9, 29, 14, 44, 47, tzinfo=tzlocal()),   'Answer': 'taskAnswers[{"weather":"It is currently chilly and cloudy outside and looks like it may rain in a little bit."}]' },  {'AssignmentId': '3FTOP5WARFNLTMF07QVP5MWL0BPJ0W',   'WorkerId': 'DDAEBCKCEVSQ6C2EXAMPLE,   'HITId': '3TL87MO8CLOFYXKXNRLMZO1MOK4FL5',   'AssignmentStatus': 'Submitted',   'AutoApprovalTime': datetime.datetime(2020, 10, 2, 14, 48, 12, tzinfo=tzlocal()),   'AcceptTime': datetime.datetime(2020, 9, 29, 14, 43, 28, tzinfo=tzlocal()),   'SubmitTime': datetime.datetime(2020, 9, 29, 14, 48, 12, tzinfo=tzlocal()),   'Answer': 'taskAnswers[{"weather":"Currently indianapolis weather very cool, because northwest area are most of this situation landing. In currently approximately cloud is 15 to 18\'C."}]' },  {'AssignmentId': '3DI28L7YXADDPVEQP8OYMB230VZE19',   'WorkerId': 'AICACKCEVSQ6C2EXAMPLE,   'HITId': '3TL87MO8CLOFYXKXNRLMZO1MOK4FL5',   'AssignmentStatus': 'Submitted',   'AutoApprovalTime': datetime.datetime(2020, 10, 2, 14, 54, 52, tzinfo=tzlocal()),   'AcceptTime': datetime.datetime(2020, 9, 29, 14, 53, 5, tzinfo=tzlocal()),   'SubmitTime': datetime.datetime(2020, 9, 29, 14, 54, 52, tzinfo=tzlocal()),   'Answer': 'taskAnswers[{"weather":"32"}]' },  {'AssignmentId': '3LO69W1SU3COZGELODW56TWTBHJLG3',   'WorkerId': 'SDLKSDEVSQ6C2EXAMPLE,   'HITId': '3TL87MO8CLOFYXKXNRLMZO1MOK4FL5',   'AssignmentStatus': 'Submitted',   'AutoApprovalTime': datetime.datetime(2020, 10, 2, 14, 58, 14, tzinfo=tzlocal()),   'AcceptTime': datetime.datetime(2020, 9, 29, 14, 57, 39, tzinfo=tzlocal()),   'SubmitTime': datetime.datetime(2020, 9, 29, 14, 58, 14, tzinfo=tzlocal()),   'Answer': 'taskAnswers[{"weather":"Hot and dry, with poor air quality."}]' } ]   ``` This information is returned in a Python dict that can be used to access any of the relevant values for each assignment. In addition, you can use the [ElementTree](https://docs.python.org/3.8/library/xml.etree.elementtree.html) library as shown below to parse the Answer XML into a format that can be more easily viewed. This code works well with tasks that use the `crowd-form` element as we did in this in this tutorial, but may need to be modified if you use a standard `form` element. ``` import xml.etree.ElementTree as ET import json   answers = [] namespace = {'mt': 'http://mechanicalturk.amazonaws.com/AWSMechanicalTurkDataSchemas/2005-10-01/QuestionFormAnswers.xsd'}   for assignment in response['Assignments']:     assignment_answer = ET.fromstring(assignment['Answer']).find('mt:Answer', namespace)     answers.append(json.loads(assignment_answer.find('mt:FreeText', namespace).text)) print(answers) ``` Now that we've reviewed the results of our HIT, we can proceed to Step 4: Approve Assignments. ------ ## Step 4: Approve Assignments In this step, we approve the assignments submitted by workers so that the reward is transferred to their account. In Step 1, we set the Auto Approval Delay so that if we do nothing, workers are paid automatically after three days. However, it is always a best practice to approve work quickly if at all possible so that workers don't have to wait for the time to expire. Alternatively, if you plan to approve all of the assignments that are submitted, the Auto Approval Delay can be set to 0 and you can skip this step. We use the same `HITId` that was generated in Step 1. You will need to capture that identifier and insert it in the appropriate location below to approve the results. ------ #### [ AWS CLI ] To approve the assignments, we need to start by retrieving a list of the `AssignmentId`s. We begin by getting a list of the `AssignmentId`s using a variation of the query we used in Step 3 to retrieve results. ``` $ aws mturk list-assignments-for-hit \ --hit-id 3TL87MO8CLOFYXKXNRLMZO1MOK4FL5 \ --query 'Assignments[*].AssignmentId' ``` This returns a list of the `AssignmentId`s for our HIT as shown below. ``` [     "3IOEN3P9S7I9CGLBJQ50ANAQJXB16B",     "32AT8R96GL8U8BA6SRINMUBFCJ5USP",     "3FTOP5WARFNLTMF07QVP5MWL0BPJ0W",     "3DI28L7YXADDPVEQP8OYMB230VZE19",     "3LO69W1SU3COZGELODW56TWTBHJLG3" ] ``` Now we can use the [https://awscli.amazonaws.com/v2/documentation/api/latest/reference/mturk/approve-assignment.html](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/mturk/approve-assignment.html) operation to approve each of the assignments. ``` $ aws mturk approve-assignment --assignment-id 3IOEN3P9S7I9CGLBJQ50ANAQJXB16B $ aws mturk approve-assignment –-assignment-id 32AT8R96GL8U8BA6SRINMUBFCJ5USP $ aws mturk approve-assignment –-assignment-id 3FTOP5WARFNLTMF07QVP5MWL0BPJ0W $ aws mturk approve-assignment –-assignment-id 3DI28L7YXADDPVEQP8OYMB230VZE19 $ aws mturk approve-assignment –-assignment-id 3LO69W1SU3COZGELODW56TWTBHJLG3 ``` ------ #### [ SDK for Python (Boto3) ] To approve the assignments, retrieve the list of assignments using the `HITId` and iterating through the results to call the [https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/mturk.html#MTurk.Client.approve_assignment](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/mturk.html#MTurk.Client.approve_assignment) operation. ``` import boto3   mturk = boto3.client('mturk')   response = mturk.list_assignments_for_hit(HITId='3TL87MO8CLOFYXKXNRLMZO1MOK4FL5') for assignment in response['Assignments']:     mturk.approve_assignment(AssignmentId=assignment['AssignmentId']) ``` ------