# Evaluating and improving your adapters Once you have finished the training process and created your adapter, it's important to evaluate how well the adapter is extracting information from your documents. **Performance metrics** Three metrics are provided in the Amazon Textract console to assist you in analyzing your adapter's performance: 1. Precision - Precision measures the percentage of extracted information (predictions) that are correct. The higher the precision rating, the fewer false positives there are. 1. Recall - Recall measures the percentage of total relevant items that are successfully identified and extracted by the model. The higher the recall value, the fewer false negatives there are. 1. F1 Score - The F1 score combines precision and recall into a single metric, providing a balanced measurement for overall extraction accuracy. The values for these measurements range from 0 to 1, with 1 being perfect extraction. These metrics are calculated by comparing the adapter's extractions to the "ground truth" annotations on the test set. By analyzing the F1, precision, and recall, you can determine where the adapter needs improvement. For example, low precision means many of the model’s predictions are false positives, therefore the adapter is extracting irrelevant data. In contrast, a low recall value means that the model is missing relevant data. Using these insights, you can refine the training data and retrain the adapter to increase performance. You can also check the performance of your model by testing it with new documents and queries that you specify. Use the **Try Adapter** option in the console to get predictions for these documents. This way, you can evaluate the adapter with your own test queries and documents and see real-world examples of how the adapter is performing. You can also retrieve metrics for an adapter version by using the [GetAdapterVersion](API_GetAdapterVersion.md) operation using an SDK or the CLI. Get a list of adapters that you want to retrieve metrics for by using the [ListAdapterVersions](API_ListAdapterVersions.md) API operation. Delete an adapter you no longer need with the [DeleteAdapterVersion](API_DeleteAdapterVersion.md) operation. **Improving your model** Adapter deployment is an iterative process, as you’ll likely need to retrain several times to reach your target level of accuracy. After you create and train your adapter, you’ll want to test and evaluate your adapter’s performance on various metrics and queries. If your adapter’s accuracy is lacking in any area, add new examples of those documents to increase the adapter’s performance for those queries. Try to provide the adapter with additional, varied examples that reflect the cases where it struggles. Providing your adapter with representative, varied documents enables it to handle diverse real-world examples. After adding new documents to your training set, retrain the adapter. Then re-evaluate on your test set and queries. Repeat this process until the adapter reaches your desired level of performance. Precision, recall, and F1 scores should gradually improve over successive training iterations. # List adapter versions An Amazon Textract adapter can have a number of different versions associated with it. In order to see which adapter versions associated with a given adapter, you can call the [ListAdapterVersions](API_ListAdapterVersions.md) operation. The operation will return all versions of an adapter unless provided with filtering criteria using of the optional arguments such as AdapterId, AfterCreationTime, BeforeCreationTIme, Statuses, or MaxResults. To see a list of your adapter versions with the console: + Sign in to the Amazon Textract console. + Select **Custom Queries** from the left navigation panel. + From the list of your adapters, select the adapter. + View the adapter versions in the **Adapter versions** box. To create an adapter with the AWS CLI or AWS SDK: + If you haven't already done so, install and configure the AWS CLI and the AWS SDKs. For more information, see [Step 2: Set Up the AWS CLI and AWS SDKs](setup-awscli-sdk.md). + Use the following code to create an adapter: ------ #### [ CLI ] ``` aws textract list-adapter-versions ``` ------ # Get an Adapter version You can retrieve configuration information and the current status of an adapter version by calling the [GetAdapterVersion](API_GetAdapterVersion.md) operation. When calling GetAdapterVersion, specify the AdapterId and the AdapterVersion. This returns information about the specified adapter version so that you can check the current operational status and configuration options. To see details for your adapter using the console: + Sign in to the Amazon Textract console. + Select **Custom Queries** from the left navigation panel. + From the list of your adapters, select the adapter. + Select the adapter version in the **Adapter versions** box. To see details for your adapter using the AWS CLI or AWS SDK + If you haven't already done so, install and configure the AWS CLI and the AWS SDKs. For more information, see [Step 2: Set Up the AWS CLI and AWS SDKs](setup-awscli-sdk.md). + Use the following code to create an adapter: ------ #### [ CLI ] ``` aws textract get-adapter-version \ --adapter-id "abcdef123456" \ --adapter-version "1" ``` ------ # Delete adapter version You can delete an adapter version you’re no longer using by calling [DeleteAdapterVersion](API_DeleteAdapterVersion.md). To delete an adapter version you provide the DeleteAdapterVersion operation with both the adapter’s AdapterId and the specific AdapterVersion that you want to delete. Note that you cannot delete adapter versions with an "IN\$1PROGRESS" status. To delete an adapter version with the console: + Sign in to the Amazon Textract console. + Select **Custom Queries** from the left navigation panel. + From the list of your adapters, select the adapter. + Select **Delete** and follow the instructions. + Select the adapter version that you want to delete from the list of versions in the **Adapter** versions box. + Select **Delete** and follow the instructions to delete your adapter. To delete an adapter with the AWS CLI or AWS SDK: + If you haven't already done so, install and configure the AWS CLI and the AWS SDKs. For more information, see [Step 2: Set Up the AWS CLI and AWS SDKs](setup-awscli-sdk.md). + Use the following code to create an adapter: ------ #### [ CLI ] ``` aws textract delete-adapter-version \ --adapter-id "abcdef123456" \ --adapter-version "1" ``` ------