# Monitoring Amazon DocumentDB Monitoring Amazon DocumentDB Monitoring your AWS services is an important part of keeping your systems healthy and functioning optimally. It's wise to collect monitoring data from all parts of your AWS solution so that you can more easily debug and fix failures or degradations, should they occur. Before you begin monitoring your AWS solutions, we recommend that you consider and formulate answers for the following questions: + What are your monitoring goals? + What resources are you going to monitor? + How frequently will you monitor these resources? + What monitoring tools will you use? + Who is responsible for doing the monitoring? + Who is to be notified and by what means if something goes wrong? To understand your current performance patterns, identify performance anomalies, and formulate methods to address issues, you should establish baseline performance metrics for various times and under differing load conditions. As you monitor your AWS solution, we recommend that you store your historical monitoring data for future reference and for establishing your baselines. In general, acceptable values for performance metrics depend on what your baseline looks like and what your application is doing. Investigate consistent or trending variances from your baseline. The following is advice about specific types of metrics: + **High CPU or RAM use** — High values for CPU or RAM use might be appropriate, provided that they are in keeping with your goals for your application (like throughput or concurrency) and are expected. + **Storage volume consumption** —Investigate storage consumption (`VolumeBytesUsed`) if space that is used is consistently at or above 85 percent of the total storage volume space. Determine whether you can delete data from the storage volume or archive data to a different system to free up space. For more information, see [Amazon DocumentDB storage](how-it-works.md#how-it-works.storage) and [Amazon DocumentDB Quotas and limits](limits.md). + **Network traffic** — For network traffic, talk with your system administrator to understand what the expected throughput is for your domain network and internet connection. Investigate network traffic if throughput is consistently lower than expected. + **Database connections** — Consider constraining database connections if you see high numbers of user connections in conjunction with decreases in instance performance and response time. The best number of user connections for your instance will vary based on your instance class and the complexity of the operations being performed. + **IOPS metrics**—The expected values for IOPS metrics depend on disk specification and server configuration, so use your baseline to know what is typical. Investigate if values are consistently different from your baseline. For best IOPS performance, make sure that your typical working set fits into memory to minimize read and write operations. Amazon DocumentDB (with MongoDB compatibility) provides a variety of Amazon CloudWatch metrics that you can monitor to determine the health and performance of your Amazon DocumentDB clusters and instances. You can view Amazon DocumentDB metrics using various tools, including the Amazon DocumentDB console, AWS CLI, CloudWatch API, and Performance Insights. **Topics** + [Monitoring a cluster's status](monitoring_docdb-cluster_status.md) + [Monitoring an instance's status](monitoring_docdb-instance_status.md) + [ # Viewing Amazon DocumentDB recommendations ](view-docdb-recommendations.md) + [Event subscriptions](event-subscriptions.md) + [ # Monitoring Amazon DocumentDB with CloudWatch ](cloud_watch.md) + [Logging Amazon DocumentDB API calls with CloudTrail](logging-with-cloudtrail.md) + [Profiling operations](profiling.md) + [Monitoring with Performance Insights](performance-insights.md) # Monitoring an Amazon DocumentDB cluster's status Monitoring a cluster's status The status of a cluster indicates the health of the cluster. You can view the status of a cluster by using the Amazon DocumentDB console or the AWS CLI `describe-db-clusters` command. **Topics** + [ ## Cluster status values ](#monitoring_docdb-status_values) + [ ## Monitoring a cluster's status ](#monitor-cluster-status) ## Cluster status values The following table lists the valid values for a cluster's status. | Cluster Status | Description | | --- | --- | | active | The cluster is active. This status applies to elastic clusters only. | | available | The cluster is healthy and available. This status applies to instance-based clusters only. | | backing-up | The cluster is currently being backed up. | | creating | The cluster is being created. It is inaccessible while it is being created. | | deleting | The cluster is being deleted. It is inaccessible while it is being deleted. | | failing-over | A failover from the primary instance to an Amazon DocumentDB replica is being performed. | | inaccessible-encryption-credentials | The AWS KMS key used to encrypt or decrypt the cluster can't be accessed. | | maintenance | A maintenance update is being applied to the cluster. This status is used for cluster-level maintenance that Amazon DocumentDB schedules well in advance. | | migrating | A cluster snapshot is being restored to a cluster. | | migration-failed | A migration failed. | | modifying | The cluster is being modified because of a customer request to modify the cluster. | | renaming | The cluster is being renamed because of a customer request to rename it. | | resetting-master-credentials | The master credentials for the cluster are being reset because of a customer request to reset them. | | upgrading | The cluster engine version is being upgraded. | ## Monitoring a cluster's status ------ #### [ Using the AWS Management Console ] When using the AWS Management Console to determine the status of a cluster, use the following procedure. 1. Sign in to the AWS Management Console, and open the Amazon DocumentDB console at [https://console.aws.amazon.com/docdb](https://console.aws.amazon.com/docdb). 1. In the navigation pane, choose **Clusters**. 1. In the Clusters navigation box, you'll see the column **Cluster identifier**. Your instances are listed under clusters, similar to the screenshot below. ![\[Clusters table showing how an instance is nested under a cluster.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/choose-clusters.png) 1. In the **Cluster identifier** column, find the name of the instance that you are interested in. Then, to find the status of the instance, read across that row to the **Status** column, as shown below. ![\[Cluster instance showing available status.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/db-cluster-status-con.png) ------ #### [ Using the AWS CLI ] When using the AWS CLI to determine the status of a cluster, use the `describe-db-clusters` operation. The following code finds the status of the cluster `sample-cluster`. For Linux, macOS, or Unix: ``` aws docdb describe-db-clusters \ --db-cluster-identifier sample-cluster \ --query 'DBClusters[*].[DBClusterIdentifier,Status]' ``` For Windows: ``` aws docdb describe-db-clusters ^ --db-cluster-identifier sample-cluster ^ --query 'DBClusters[*].[DBClusterIdentifier,Status]' ``` Output from this operation looks something like the following. ``` [ [ "sample-cluster", "available" ] ] ``` ------ # Monitoring an Amazon DocumentDB instance's status Monitoring an instance's status Amazon DocumentDB provides information about the current condition of each configured instance in the database. There are three types of status that you can view for an Amazon DocumentDB instance: + Instance status: This status is shown in the **Status** column of the cluster table in the AWS Management Console and shows the current life cycle condition of the instance. The values shown in the **Status** column are derived from the `Status` field of the `DescribeDBCluster` API response. + Instance health status: This status is shown in the **Instance health** column of the cluster table in the AWS Management Console and shows whether the database engine, the component responsible for managing and retrieving data, is running. The values shown in the **Instance health** column are based on the Amazon CloudWatch `EngineUptime` system metric. + Maintenance status: This status is shown in the **Maintenance** column of the cluster table in the AWS Management Console and indicates the status of any maintenance event that needs to be applied to an instance. Maintenance status is independent of the other instance status' and is derived from the `PendingMaintenanceAction` API. For more information about maintenance status, see [Maintaining Amazon DocumentDB](https://docs.aws.amazon.com/documentdb/latest/developerguide/db-instance-maintain.html). **Topics** + [ ## Instance status values ](#monitoring_docdb-instance_status-values) + [ ## Monitoring instance status using the AWS Management Console or AWS CLI ](#monitoring-instance-status) + [ ## Instance health status values ](#instance-health-status-values) + [ ## Monitoring instance health status using the AWS Management Console ](#monitoring-instance-health-status) ## Instance status values The following table lists the possible status values for instances and how you are billed for each status. It shows if you will be billed for the instance and storage, only storage, or not billed. For all instance statuses, you are always billed for backup usage. | Instance status | Billed | Description | | --- | --- | --- | | available | Billed | The instance is healthy and available. | | backing-up | Billed | The instance is currently being backed up. | | configuring-log-exports | Billed | Publishing log files to Amazon CloudWatch Logs is being enabled or disabled for this instance. | | creating | Not billed | The instance is being created. The instance is not accessible while it is being created. | | deleting | Not billed | The instance is being deleted. | | failed | Not billed | The instance has failed and Amazon DocumentDB was unable to recover it. To recover the data, perform a point-in-time restore to the latest restorable time of the instance. | | inaccessible-encryption-credentials | Not billed | The AWS KMS key that is used to encrypt or decrypt the instance could not be accessed. | | incompatible-network | Not billed | Amazon DocumentDB is attempting to perform a recovery action on an instance but is unable to do so because the VPC is in a state that is preventing the action from being completed. This status can occur if, for example, all available IP addresses in a subnet were in use and Amazon DocumentDB was unable to get an IP address for the instance. | | maintenance | Billed | Amazon DocumentDB is applying a maintenance update to the instance. This status is used for instance-level maintenance that Amazon DocumentDB schedules well in advance. We're evaluating ways to expose additional maintenance actions to customers through this status. | | modifying | Billed | The instance is being modified because of a request to modify the instance. | | rebooting | Billed | The instance is being rebooted because of a request or an Amazon DocumentDB process that requires the rebooting of the instance. | | renaming | Billed | The instance is being renamed because of a request to rename it. | | resetting-master-credentials | Billed | The master credentials for the instance are being reset because of a request to reset them. | | restore-error | Billed | The instance encountered an error attempting to restore to a point-in-time or from a snapshot. | | starting | Billed for storage | The instance is starting. | | stopped | Billed for storage | The instance is stopped. | | stopping | Billed for storage | The instance is being stopped. | | storage-full | Billed | The instance has reached its storage capacity allocation. This is a critical status and should be remedied immediately; scale up your storage by modifying the instance. Set Amazon CloudWatch alarms to warn you when storage space is getting low so you don't run into this situation. | ## Monitoring instance status using the AWS Management Console or AWS CLI Use the AWS Management Console or AWS CLI to monitor the status of you instance. ------ #### [ Using the AWS Management Console ] When using the AWS Management Console to determine the status of a cluster, use the following procedure. 1. Sign in to the AWS Management Console, and open the Amazon DocumentDB console at [https://console.aws.amazon.com/docdb](https://console.aws.amazon.com/docdb). 1. In the navigation pane, choose **Clusters**. **Note** Note that in the Clusters navigation box, the column **Cluster identifier** shows both clusters and instances. Instances are listed underneath clusters, similar to the image below. ![\[List of clusters and instances on the Clusters page in the Amazon DocumentDB console.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/clusters.png) 1. Find the name of the instance that you are interested in. Then, to find the status of the instance, read across that row to the **Status** column, as shown following. ![\[Status column showing available status for clusters and instances on the Clusters page.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/instance-status.png) ------ #### [ Using the AWS CLI ] When using the AWS CLI to determine the status of a cluster, use the `describe-db-instances` operation. The following code finds the status of the instance `sample-cluster-instance-01`. For Linux, macOS, or Unix: ``` aws docdb describe-db-instances \ --db-instance-identifier sample-cluster-instance-01 \ --query 'DBInstances[*].[DBInstanceIdentifier,DBInstanceStatus]' ``` For Windows: ``` aws docdb describe-db-instances ^ --db-instance-identifier sample-cluster-instance-01 ^ --query 'DBInstances[*].[DBInstanceIdentifier,DBInstanceStatus]' ``` Output from this operation looks something like the following. ``` [ [ "sample-cluster-instance-01", "available" ] ] ``` ------ ## Instance health status values The following table lists the possible health status values for instances. The **Instance health** column, located in the **Clusters** table in the AWS Management Console, shows whether the database engine, the component responsible for storing, managing, and retrieving data, is operating normally. This column also indicates if the `EngineUptime` system metric, available in CloudWatch, is showing the health status of each instance. | Instance health status | Description | | --- | --- | | healthy | Database engine is running in the Amazon DocumentDB instance. | | unhealthy | Database engine is not running or has restarted less than a minute ago. | ## Monitoring instance health status using the AWS Management Console Use the AWS Management Console to monitor the health status of you instance. When using AWS Management Console, use the following steps to understand the instance's health status. 1. Sign in to the AWS Management Console, and open the Amazon DocumentDB console at [https://console.aws.amazon.com/docdb](https://console.aws.amazon.com/docdb). 1. In the navigation pane, choose **Clusters**. **Note** In the **Clusters** navigation box, the column **Cluster identifier** shows both clusters and instances. Instances are listed underneath clusters, similar to the image below. ![\[List of clusters and instances on the Clusters page in the Amazon DocumentDB console.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/clusters.png) 1. Find the name of the instance that you are interested in. Then, to find the status of the instance, read across that row to the **Instance health** column, as shown in the following image: ![\[Instance health column showing healthy and unhealthy states for instances listed on the Clusters page.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/health-status-1.png) **Note** Instance health status polling occurs every 60 seconds and is based on the CloudWatch `EngineUptime` system metric. The values in the **Instance health** column are automatically updated. # Viewing Amazon DocumentDB recommendations Amazon DocumentDB provides a list of automated recommendations for database resources, such as instances and clusters. These recommendations provide best practice guidance by analyzing your cluster and instance configurations. As an example of these recommendations, see the following: | Type | Description | Recommendation | Additional information | | --- | --- | --- | --- | | One instance | Cluster only contains one instance | Performance and availability: we recommend adding another instance with the same instance class in a different Availability Zone. | [Amazon DocumentDB High Availability and Replication](https://docs.aws.amazon.com/documentdb/latest/developerguide/replication.html) | Amazon DocumentDB generates recommendations for a resource when the resource is created or modified. Amazon DocumentDB also periodically scans your resources and generates recommendations. **To view and take action on Amazon DocumentDB recommendations** 1. Sign in to the AWS Management Console, and open the Amazon DocumentDB console at [https://console.aws.amazon.com/docdb](https://console.aws.amazon.com/docdb). 1. In the navigation pane, choose **Recommendations**: ![\[Amazon DocumentDB console navigation pane with Recommendations option selected.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/recommendations-nav-1.png) 1. In the **Recommendations** dialog, expand the section of interest and select the recommended task. In the example below, the recommended task applies to an Amazon DocumentDB cluster with only one instance. The recommendation is to add another instance to improve performance and availability. ![\[The Recommendations form showing a selected recommended task for an Amazon DocumentDB cluster.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/recommendations-1.png) 1. Click **Apply now**. For this example, the **Add instances** dialog appears: ![\[The Add instances form with options for instance settings.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/add-instances-1.png) 1. Modify your new instance's settings and click **Create**. # Using Amazon DocumentDB event subscriptions Event subscriptions Amazon DocumentDB uses Amazon Simple Notification Service (Amazon SNS) to provide notifications when an Amazon DocumentDB event occurs. These notifications can be in any form that is supported by Amazon SNS for an AWS Region, such as an email, a text message, or a call to an HTTP endpoint. Amazon DocumentDB groups these events into categories that you can subscribe to so that you can be notified when an event in that category occurs. You can subscribe to an event category for an instance, cluster, snapshot, cluster snapshot, or for a parameter group. For example, if you subscribe to the Backup category for a given instance, you are notified whenever a backup-related event occurs that affects the instance. You also receive notification when an event subscription changes. Events occur at both the cluster and the instance level, so you can receive events if you subscribe to a cluster or an instance. Event subscriptions are sent to the addresses you provide when you create the subscription. You might want to create several different subscriptions, such as a subscription that receives all event notifications and another subscription that includes only critical events for your production instances. You can easily turn off notification without deleting a subscription. To do so, set the **Enabled** radio button to **No** in the Amazon DocumentDB console. **Important** Amazon DocumentDB doesn't guarantee the order of events sent in an event stream. The event order is subject to change. Amazon DocumentDB uses the Amazon Resource Name (ARN) of an Amazon SNS topic to identify each subscription. The Amazon DocumentDB console creates the ARN for you when you create the subscription. Billing for Amazon DocumentDB event subscriptions is through Amazon SNS. Amazon SNS fees apply when using event notification. For more information, see Amazon Simple Notification Service Pricing. Other than Amazon SNS charges, Amazon DocumentDB does not bill for event subscriptions. **Topics** + [Subscribing to events](event-subscriptions.subscribe.md) + [Manage subscriptions](event-subscriptions.managing.md) + [Categories and messages](event-subscriptions.categories-messages.md) # Subscribing to Amazon DocumentDB events Subscribing to events You can use the Amazon DocumentDB console to subscribe to event subscriptions, as follows: 1. Sign in to the AWS Management Console at [https://console.aws.amazon.com/docdb](https://console.aws.amazon.com/docdb). 1. In the navigation pane, choose **Event subscriptions**. ![\[Amazon DocumentDB console navigation pane with Event Subscriptions option highlighted.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/event-subs/subscribe-event-subs.png) 1. In the **Event subscriptions** pane, choose **Create event subscription**. ![\[Event Subscriptions pane highlighting the Create event subscription button in the upper-right corner.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/event-subs/subscribe-create.png) 1. In the **Create event subscription** dialog box, do the following: + For **Name**, enter a name for the event notification subscription. ![\[The Create event subscription form showing the Details section and the Name input field.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/event-subs/subscribe-name.png) + For **Target**, choose where you want to send notifications to. You can choose an existing **ARN** or choose **New Email Topic** to enter the name of a topic and a list of recipients. ![\[The Target section with options to specify where to send notifications to.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/event-subs/subscribe-target.png) + For **Source**, choose a source type. Depending on the source type you selected, choose the event categories and the sources that you want to receive event notifications from. ![\[The Source section to select a source type to receive event notifications from.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/event-subs/subscribe-source.png) + Choose **Create**. ![\[The Source section with the Create button in the lower-right corner.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/event-subs/subscribe-create-2.png) # Managing Amazon DocumentDB event notification subscriptions Manage subscriptions If you choose **Event subscriptions** in the navigation pane of the Amazon DocumentDB console, you can view subscription categories and a list of your current subscriptions. You can also modify or delete a specific subscription. ## To modify your current Amazon DocumentDB event notification subscriptions 1. Sign in to the AWS Management Console at [https://console.aws.amazon.com/docdb](https://console.aws.amazon.com/docdb). 1. In the navigation pane, choose **Event subscriptions**. The **Event subscriptions** pane shows all your event notification subscriptions. ![\[Amazon DocumentDB console navigation pane with Event Subscriptions option highlighted.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/event-subs/modify-event-subs.png) 1. In the **Event subscriptions** pane, choose the subscription that you want to modify and choose **Edit**. ![\[The Event subscriptions pane showing a selected subscription and the Edit button.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/event-subs/modify-edit.png) 1. Make your changes to the subscription in either the **Target** or **Source** section. You can add or remove source identifiers by selecting or deselecting them in the Source section. ![\[The Modify event subscription form highlighting the Target section.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/event-subs/modify-target.png) 1. Choose **Modify**. The Amazon DocumentDB console indicates that the subscription is being modified. ![\[The end of the Modify event subscription form with the Modify button highlighted.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/event-subs/modify-button.png) ## Deleting an Amazon DocumentDB event notification subscription You can delete a subscription when you no longer need it. All subscribers to the topic will no longer receive event notifications specified by the subscription. 1. Sign in to the AWS Management Console at [https://console.aws.amazon.com/docdb](https://console.aws.amazon.com/docdb). 1. In the navigation pane, choose **Event subscriptions**. ![\[Amazon DocumentDB console navigation pane with Event Subscriptions option highlighted.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/event-subs/delete-event-subs.png) 1. In the **Event subscriptions** pane, choose the subscription that you want to delete. ![\[The Event subscriptions pane showing a selected subscription.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/event-subs/delete-select.png) 1. Choose **Delete**. ![\[The Event subscriptions pane highlighting the Delete button.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/event-subs/delete-delete.png) 1. A pop-up window will appear asking you if you want to permanently delete this notification. Choose **Delete**. ![\[A dialog box confirming deletion of the event subscription with the Delete button highlighted in the lower-right corner.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/event-subs/delete-delete-2.png) # Amazon DocumentDB event categories and messages Categories and messages Amazon DocumentDB generates a significant number of events in categories that you can subscribe to using the console. Each category applies to a source type, which can be an instance, cluster, snapshot, or parameter group. **Note** Amazon DocumentDB uses existing Amazon RDS event definitions and IDs. ## Amazon DocumentDB events originating from instances | Category | Description | | --- | --- | | availability | The instance restarted. | | availability | The instance shutdown. | | configuration change | Applying modification to an instance class. | | configuration change | Finished applying modification to an instance class. | | configuration change | Reset primary credentials. | | creation | Instance created. | | deletion | Instance deleted | | failure | The instance has failed due to an incompatible configuration or an underlying storage issue. Begin a point-in-time-restore for the instance. | | notification | Instance stopped. | | notification | Instance started. | | notification | Instance is being started due to it exceeding the maximum allowed time being stopped. | | recovery | Recovery of the instance has started. Recovery time will vary with the amount of data to be recovered. | | recovery | Recovery of the instance is complete. | | security patching | The operating system update is available for your instance. For information about applying updates, see [Maintaining Amazon DocumentDB](https://docs.aws.amazon.com/documentdb/latest/developerguide/db-instance-maintain.html). | ## Amazon DocumentDB events originating from a cluster | Category | Description | | --- | --- | | creation | Cluster created | | deletion | Cluster deleted. | | failover | Promoting previous primary again. | | failover | Completed failover to instance. | | failover | Started failover to DB instance: %s | | failover | Started same AZ failover to DB instance: %s | | failover | Started cross AZ failover to DB instance: %s | | maintenance | Cluster has been patched. | | maintenance | Database cluster is in a state that cannot be upgraded: %s | | notification | The cluster stopped. | | notification | The cluster started. | | notification | The cluster stop failed. | | notification | The cluster is being started due to it exceeding the maximum allowed time being stopped. | | notification | Renamed cluster from %s to %s. | ## Amazon DocumentDB events originating from cluster snapshot The following table shows the event category and a list of events when an Amazon DocumentDB cluster snapshot is the source type. | Category | Description | | --- | --- | | backup | Creating manual cluster snapshot. | | backup | Manual cluster snapshot created. | | backup | Creating automated cluster snapshot. | | backup | Automated cluster snapshot created. | ## Amazon DocumentDB events originating from parameter group The following table shows the event category and a list of events when a parameter group is the source type. | Category | Description | | --- | --- | | configuration change | Updated parameter %s to %s with apply method %s | # Monitoring Amazon DocumentDB with CloudWatch Amazon DocumentDB (with MongoDB compatibility) integrates with Amazon CloudWatch so that you can gather and analyze operational metrics for your clusters. You can monitor these metrics using the CloudWatch console, the Amazon DocumentDB console, the AWS Command Line Interface (AWS CLI), or the CloudWatch API. CloudWatch also lets you set alarms so that you can be notified if a metric value breaches a threshold that you specify. You can even set up Amazon CloudWatch Events to take corrective action if a breach occurs. For more information about using CloudWatch and alarms, see the [ Amazon CloudWatch documentation](https://docs.aws.amazon.com/cloudwatch/index.html). **Topics** + [ ## Amazon DocumentDB metrics ](#cloud_watch-metrics_list) + [ ## Viewing CloudWatch data ](#cloud_watch-view_data) + [ ## Amazon DocumentDB dimensions ](#cloud_watch-metrics_dimensions) + [ ## Monitoring Opcounter metrics ](#cloud_watch-monitoring_opcounters) + [ ## Monitoring database connections ](#cloud_watch-monitoring_connections) ## Amazon DocumentDB metrics To monitor the health and performance of your Amazon DocumentDB cluster and instances, you can view the following metrics in the Amazon DocumentDB console. **Note** Metrics in the following tables apply to both instance-based and elastic clusters. **Topics** + [ ### Resource utilization metrics ](#resource-utilization) + [ ### Latency metrics ](#latency-metrics) + [ ### NVMe-backed instance metrics ](#nvme-metrics) + [ ### Operations metrics ](#operations-metrics) + [ ### Throughput metrics ](#throughput-metrics) + [ ### System metrics ](#system-metrics) + [ ### T3 instance metrics ](#t3-instance-metrics) ### Resource utilization metrics | Metric | Description | | --- | --- | | BackupRetentionPeriodStorageUsed | The total amount of backup storage in bytes used to support the point-in-time restore feature within the Amazon DocumentDB's retention window. Included in the total reported by the TotalBackupStorageBilled metric. Computed separately for each Amazon DocumentDB cluster. | | ChangeStreamLogSize | The amount of storage used by your cluster to store the change stream log in megabytes. This value is a subset of the total storage for the cluster (VolumeBytesUsed) and affects the cost of the cluster. For storage pricing information, see the [ Amazon DocumentDB product page](https://aws.amazon.com//documentdb/pricing). The change stream log size is a function of how much change is happening on your cluster and the change stream long retention duration. For more information on change streams, see [Using change streams with Amazon DocumentDB](change_streams.md). | | CPUUtilization | The percentage of CPU used by an instance. | | DatabaseConnections | The number of connections (active and idle) open on an instance taken at a 1-minute frequency. | | DatabaseConnectionsMax | The maximum number of open database connections (active and idle) on an instance in a 1-minute period. | | DatabaseConnectionsLimit | The maximum number of concurrent database connections (active and idle) allowed on an instance at any given time. | | DatabaseCursors | The number of cursors open on an instance taken at a 1-minute frequency. | | DatabaseCursorsMax | The maximum number of open cursors on an instance in a 1-minute period. | | DatabaseCursorsLimit | The maximum number of cursors allowed on an instance at any given time. | | DatabaseCursorsTimedOut | The number of cursors that timed out in a 1-minute period. | | FreeableMemory | The amount of available random access memory, in bytes. | | FreeLocalStorage | This metric reports the amount of storage available to each instance for temporary tables and logs. This value depends on the instance class. You can increase the amount of free storage space for an instance by choosing a larger instance class for your instance. (This doesn't apply to DocumentDB Serverless.) | | LowMemThrottleQueueDepth | The queue depth for requests that are throttled due to low available memory taken at a 1-minute frequency. | | LowMemThrottleMaxQueueDepth | The maximum queue depth for requests that are throttled due to low available memory in a 1-minute period. | | LowMemNumOperationsThrottled | The number of requests that are throttled due to low available memory in a 1-minute period. | | SnapshotStorageUsed | The total amount of backup storage in bytes consumed by all snapshots for a given Amazon DocumentDB cluster outside its backup retention window. Included in the total reported by the TotalBackupStorageBilled metric. Computed separately for each Amazon DocumentDB cluster. | | SwapUsage | The amount of swap space used on the instance. | | TotalBackupStorageBilled | The total amount of backup storage in bytes for which you are billed for a given Amazon DocumentDB cluster. Includes the backup storage measured by the BackupRetentionPeriodStorageUsed and SnapshotStorageUsed metrics. Computed separately for each Amazon DocumentDB cluster. | | TransactionsOpen | The number of transactions open on an instance taken at a 1-minute frequency. | | TransactionsOpenMax | The maximum number of transactions open on an instance in a 1-minute period. | | TransactionsOpenLimit | The maximum number of concurrent transactions allowed on an instance at any given time. | | VolumeBytesUsed | The amount of storage used by your cluster in bytes. This value affects the cost of the cluster. For pricing information, see the [ Amazon DocumentDB product page](https://aws.amazon.com//documentdb/pricing). | ### Latency metrics | Metric | Description | | --- | --- | | DBClusterReplicaLagMaximum | The maximum amount of lag, in milliseconds, between the primary instance and each Amazon DocumentDB instance in the cluster. | | DBClusterReplicaLagMinimum | The minimum amount of lag, in milliseconds, between the primary instance and each replica instance in the cluster. | | DBInstanceReplicaLag | The amount of lag, in milliseconds, when replicating updates from the primary instance to a replica instance. | | ReadLatency | The average amount of time taken per disk I/O operation. | | WriteLatency | The average amount of time, in milliseconds, taken per disk I/O operation. | ### NVMe-backed instance metrics | Metric | Description | | --- | --- | | NVMeStorageCacheHitRatio | The percentage of requests that are served by the tiered cache. | | FreeNVMeStorage | The amount of available Ephemeral NVMe storage. | | ReadIOPSNVMeStorage | The average number of disk read I/O operations to Ephemeral NVMe storage. | | ReadLatencyNVMeStorage | The average amount of time taken per disk read I/O operation for Ephemeral NVMe storage. | | ReadThroughputNVMeStorage | The average number of bytes read from disk per second for Ephemeral NVMe storage. | | WriteIOPSNVMeStorage | The average number of disk write I/O operations to Ephemeral NVMe storage. | | WriteLatencyNVMeStorage | The average amount of time taken per disk write I/O operation for Ephemeral NVMe storage. | | WriteThroughputNVMeStorage | The average number of bytes written to disk per second for Ephemeral NVMe storage. | ### Operations metrics | Metric | Description | | --- | --- | | DocumentsDeleted | The number of deleted documents in a 1-minute period. | | DocumentsInserted | The number of inserted documents in a 1-minute period. | | DocumentsReturned | The number of returned documents in a 1-minute period. | | DocumentsUpdated | The number of updated documents in a 1-minute period. | | OpcountersCommand | The number of commands issued in a 1-minute period. | | OpcountersDelete | The number of delete operations issued in a 1-minute period. | | OpcountersGetmore | The number of getmores issued in a 1-minute period. | | OpcountersInsert | The number of insert operations issued in a 1-minute period. | | OpcountersQuery | The number of queries issued in a 1-minute period. | | OpcountersUpdate | The number of update operations issued in a 1-minute period. | | TransactionsStarted | The number of transactions started on an instance in a 1-minute period. | | TransactionsCommitted | The number of transactions committed on an instance in a 1-minute period. | | TransactionsAborted | The number of transactions aborted on an instance in a 1-minute period. | | TTLDeletedDocuments | The number of documents deleted by a TTLMonitor in a 1-minute period. | ### Throughput metrics | Metric | Description | | --- | --- | | NetworkReceiveThroughput | The amount of network throughput, in bytes per second, received from clients by each instance in the cluster. This throughput doesn't include network traffic between instances in the cluster and the cluster volume. | | NetworkThroughput | The amount of network throughput, in bytes per second, both received from and transmitted to clients by each instance in the Amazon DocumentDB cluster. This throughput doesn't include network traffic between instances in the cluster and the cluster volume. | | NetworkTransmitThroughput | The amount of network throughput, in bytes per second, sent to clients by each instance in the cluster. This throughput doesn't include network traffic between instances in the cluster and the cluster volume. | | ReadIOPS | The average number of disk read I/O operations per second. Amazon DocumentDB reports read and write IOPS separately, and on 1-minute intervals. | | ReadThroughput | The average number of bytes read from disk per second. | | StorageNetworkReceiveThroughput | The amount of network throughput, in bytes per second, received from the Amazon DocumentDB cluster storage volume by each instance in the cluster. | | StorageNetworkTransmitThroughput | The amount of network throughput, in bytes per second, sent to the Amazon DocumentDB cluster storage volume by each instance in the cluster. | | StorageNetworkThroughput | The amount of network throughput, in bytes per second, received and sent to the Amazon DocumentDB cluster storage volume by each instance in the Amazon DocumentDB cluster. | | VolumeReadIOPs | The average number of billed read I/O operations from a cluster volume, reported at 5-minute intervals. Billed read operations are calculated at the cluster volume level, aggregated from all instances in the cluster, and then reported at 5-minute intervals. The value is calculated by taking the value of the read operations metric over a 5-minute period. You can determine the amount of billed read operations per second by taking the value of the billed read operations metric and dividing by 300 seconds. For example, if the `VolumeReadIOPs` returns 13,686, then the billed read operations per second is 45 (13,686 / 300 = 45.62). You accrue billed read operations for queries that request database pages that are not present in the buffer cache and therefore must be loaded from storage. You might see spikes in billed read operations as query results are read from storage and then loaded into the buffer cache. | | VolumeWriteIOPs | The average number of billed write I/O operations from a cluster volume, reported at 5-minute intervals. Billed write operations are calculated at the cluster volume level, aggregated from all instances in the cluster, and then reported at 5-minute intervals. The value is calculated by taking the value of the write operations metric over a 5-minute period. You can determine the amount of billed write operations per second by taking the value of the billed write operations metric and dividing by 300 seconds. For example, if the `VolumeWriteIOPs` returns 13,686, then the billed write operations per second is 45 (13,686 / 300 = 45.62). Note that `VolumeReadIOPs` and `VolumeWriteIOPs` metrics are calculated by the DocumentDB storage layer and it includes IOs performed by the primary and replica instances. The data is aggregated every 20-30 minutes and then reported at 5-minute intervals, thus emitting the same data point for the metric in the time period. If you are looking for a metric to correlate to your insert operations over a 1-minute interval, you can use the instance level WriteIOPS metric. The metric is available in the monitoring tab of your Amazon DocumentDB primary instance. | | WriteIOPS | The average number of disk write I/O operations per second. When used on a cluster level, WriteIOPs are evaluated across all instances in the cluster. Read and write IOPS are reported separately, on 1-minute intervals. | | WriteThroughput | The average number of bytes written to disk per second. | ### System metrics | Metric | Description | | --- | --- | | AvailableMVCCIds | A counter that shows the number of remaining write operations available before reaching zero. When this counter reaches zero, your cluster will enter read-only mode until IDs are reclaimed and recycled. The counter decreases with each write operation and increases as garbage collection recycles old MVCC IDs. | | BufferCacheHitRatio | The percentage of requests that are served by the buffer cache. | | DiskQueueDepth | The number of I/O operations that are waiting to be written to or read from disk. | | EngineUptime | The amount of time, in seconds, that the instance has been running. | | IndexBufferCacheHitRatio | The percentage of index requests that are served by the buffer cache. You might see a spike greater than 100 percent for the metric right after you drop an index, collection or database. This will automatically be corrected after 60 seconds. This limitation will be fixed in a future patch update. | | LongestActiveGCRuntime | Duration in seconds of the longest active garbage collection process. Updates every minute and tracks only active operations, excluding processes that complete within the one-minute window. | ### T3 instance metrics | Metric | Description | | --- | --- | | CPUCreditUsage | The number of CPU credits spent during the measurement period. | | CPUCreditBalance | The number of CPU credits that an instance has accrued. This balance is depleted when the CPU bursts and CPU credits are spent more quickly than they are earned. | | CPUSurplusCreditBalance | The number of surplus CPU credits spent to sustain CPU performance when the CPUCreditBalance value is zero. | | CPUSurplusCreditsCharged | The number of surplus CPU credits exceeding the maximum number of CPU credits that can be earned in a 24-hour period, and thus attracting an additional charge. For more information, see [ Monitoring your CPU credits](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/burstable-performance-instances-monitoring-cpu-credits.html). | ## Viewing CloudWatch data You can view Amazon CloudWatch data using the CloudWatch console, the Amazon DocumentDB console, AWS Command Line Interface (AWS CLI), or the CloudWatch API. ------ #### [ Using the AWS Management Console ] To view CloudWatch metrics using the Amazon DocumentDB Management Console, complete the following steps. 1. Sign in to the AWS Management Console, and open the Amazon DocumentDB console at [https://console.aws.amazon.com/docdb](https://console.aws.amazon.com/docdb). 1. In the navigation pane, choose **Clusters**. **Tip** If you don't see the navigation pane on the left side of your screen, choose the menu icon (![\[Hamburger menu icon with three horizontal lines.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/docdb-menu-icon.png)) in the upper-left corner of the page. 1. In the Clusters navigation box, you’ll see the column **Cluster Identifier**. Your instances are listed under clusters, similar to the screenshot below. ![\[Clusters table showing how an instance is nested under a cluster.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/choose-clusters.png) 1. From the list of instances, choose the name of the instance that you want metrics for. 1. In the resulting instance summary page, choose the **Monitoring** tab to view graphical representations of your Amazon DocumentDB instance's metrics. Because a graph must be generated for each metric, it might take a few minutes for the **CloudWatch** graphs to populate. The following image shows the graphical representations of two CloudWatch metrics in the Amazon DocumentDB console, `WriteIOPS` and `ReadIOPS`. ![\[Two line graphs representing the WriteIOPS and ReadIOPS CloudWatch metrics in the Amazon DocumentDB console.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/cw-metrics-con.png) ------ #### [ Using the CloudWatch Management Console ] To view CloudWatch metrics using the CloudWatch Management Console, complete the following steps. 1. Sign in to the AWS Management Console, and open the Amazon DocumentDB console at [https://console.aws.amazon.com/cloudwatch](https://console.aws.amazon.com/cloudwatch). 1. In the navigation pane, choose **Metrics**. Then, from the list of service names, choose **DocDB**. 1. Choose a metric dimension (for example, **Cluster Metrics**). 1. The **All metrics** tab displays all metrics for that dimension in **DocDB**. 1. To sort the table, use the column heading. 1. To graph a metric, select the check box next to the metric. To select all metrics, select the check box in the heading row of the table. 1. To filter by metric, hover over the metric name and select the dropdown arrow next to the metric name. Then, choose **Add to search**, as shown in the image below. ![\[The All metrics tab listing metrics, with a dropdown list appearing for a metric name.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/cloudwatch-filter-metrics.png) ------ #### [ Using the AWS CLI ] To view CloudWatch data for Amazon DocumentDB, use the CloudWatch `get-metric-statistics` operation with the following parameters. **Parameters** + **--namespace** — Required. The service namespace for which you want CloudWatch metrics. For Amazon DocumentDB, this must be `AWS/DocDB`. + **--metric-name** — Required. The name of the metric for which you want data. + **--start-time** — Required. The timestamp that determines the first data point to return. The value specified is inclusive; results include data points with the specified timestamp. The timestamp must be in ISO 8601 UTC format (for example, 2016-10-03T23:00:00Z). + **--end-time** — Required. The timestamp that determines the last data point to return. The value specified is inclusive; results include data points with the specified timestamp. The timestamp must be in ISO 8601 UTC format (for example, 2016-10-03T23:00:00Z). + **--period** — Required. The granularity, in seconds, of the returned data points. For metrics with regular resolution, a period can be as short as one minute (60 seconds) and must be a multiple of 60. For high-resolution metrics that are collected at intervals of less than one minute, the period can be 1, 5, 10, 30, 60, or any multiple of 60. + **--dimensions** — Optional. If the metric contains multiple dimensions, you must include a value for each dimension. CloudWatch treats each unique combination of dimensions as a separate metric. If a specific combination of dimensions was not published, you can't retrieve statistics for it. You must specify the same dimensions that were used when the metrics were created. + **--statistics** — Optional. The metric statistics, other than percentile. For percentile statistics, use `ExtendedStatistics`. When calling `GetMetricStatistics`, you must specify either `Statistics` or `ExtendedStatistics`, but not both. **Permitted values:** + `SampleCount` + `Average` + `Sum` + `Minimum` + `Maximum` + **--extended-statistics** — Optional. The `percentile` statistics. Specify values between p0.0 and p100. When calling `GetMetricStatistics`, you must specify either `Statistics` or `ExtendedStatistics`, but not both. + **--unit** — Optional. The unit for a given metric. Metrics may be reported in multiple units. Not supplying a unit results in all units being returned. If you specify only a unit that the metric does not report, the results of the call are null. **Possible values:** + `Seconds` + `Microseconds` + `Milliseconds` + `Bytes` + `Kilobytes` + `Megabytes` + `Gigabytes` + `Terabytes` + `Bits` + `Kilobytes` + `Megabits` + `Gigabits` + `Terabits` + `Percent` + `Count` + `Bytes/Second` + `Kilobytes/Second` + `Megabytes/Second` + `Gigabytes/Second` + `Terabytes/Second` + `Bits/Second` + `Kilobits/Second` + `Megabits/Second` + `Gigabits/Second` + `Terabits/Second` + `Count/Second` + `None` **Example** The following example finds the maximum `CPUUtilization` for a 2-hour period taking a sample every 60 seconds. For Linux, macOS, or Unix: ``` aws cloudwatch get-metric-statistics \ --namespace AWS/DocDB \ --dimensions \ Name=DBInstanceIdentifier,Value=docdb-2019-01-09-23-55-38 \ --metric-name CPUUtilization \ --start-time 2019-02-11T05:00:00Z \ --end-time 2019-02-11T07:00:00Z \ --period 60 \ --statistics Maximum ``` For Windows: ``` aws cloudwatch get-metric-statistics ^ --namespace AWS/DocDB ^ --dimensions ^ Name=DBInstanceIdentifier,Value=docdb-2019-01-09-23-55-38 ^ --metric-name CPUUtilization ^ --start-time 2019-02-11T05:00:00Z ^ --end-time 2019-02-11T07:00:00Z ^ --period 60 ^ --statistics Maximum ``` Output from this operation look something like the following. ``` { "Label": "CPUUtilization", "Datapoints": [ { "Unit": "Percent", "Maximum": 4.49152542374361, "Timestamp": "2019-02-11T05:51:00Z" }, { "Unit": "Percent", "Maximum": 4.25000000000485, "Timestamp": "2019-02-11T06:44:00Z" }, ********* some output omitted for brevity ********* { "Unit": "Percent", "Maximum": 4.33333333331878, "Timestamp": "2019-02-11T06:07:00Z" } ] } ``` ------ ## Amazon DocumentDB dimensions The metrics for Amazon DocumentDB are qualified by the values for the account or operation. You can use the CloudWatch console to retrieve Amazon DocumentDB data filtered by any of the dimensions in the following table. | Dimension | Description | | --- | --- | | DBClusterIdentifier | Filters the data that you request for a specific Amazon DocumentDB cluster. | | DBClusterIdentifier, Role | Filters the data that you request for a specific Amazon DocumentDB cluster, aggregating the metric by instance role (WRITER/READER). For example, you can aggregate metrics for all READER instances that belong to a cluster. | | DBInstanceIdentifier | Filters the data that you request for a specific database instance. | ## Monitoring Opcounter metrics Opcounter metrics have a non-zero value (usually \$150) for idle clusters. This is because Amazon DocumentDB performs periodic health checks, internal operations, and metrics collection tasks. ## Monitoring database connections When you view the number of connections by using database engine commands such as `db.runCommand( { serverStatus: 1 })`, you might see up to 10 more connections than you see in `DatabaseConnections` through CloudWatch. This occurs because Amazon DocumentDB performs periodic health checks and metrics collection tasks that don't get accounted for in `DatabaseConnections`. `DatabaseConnections` represents customer-initiated connections only. # Logging Amazon DocumentDB API calls with AWS CloudTrail Logging Amazon DocumentDB API calls with CloudTrail Amazon DocumentDB (with MongoDB compatibility) is integrated with AWS CloudTrail, a service that provides a record of actions taken by users, roles, or an AWS service in Amazon DocumentDB (with MongoDB compatibility). CloudTrail captures all AWS CLI API calls for Amazon DocumentDB as events, including calls from the Amazon DocumentDB console and from code calls to the Amazon DocumentDB SDK. If you create a trail, you can enable continuous delivery of CloudTrail events to an Amazon S3 bucket, including events for Amazon DocumentDB. If you don't configure a trail, you can still view the most recent events on the CloudTrail console in **Event history**. Using the information collected by CloudTrail, you can determine the request that was made to Amazon DocumentDB (with MongoDB compatibility), the IP address from which the request was made, who made the request, when it was made, and other details. **Important** For certain management features, Amazon DocumentDB uses operational technology that is shared with Amazon Relational Database Service (Amazon RDS). Amazon DocumentDB console, AWS CLI, and API calls are logged as calls made to the Amazon RDS API. To learn more about AWS CloudTrail, see [AWS CloudTrail User Guide](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/). ## Amazon DocumentDB information in CloudTrail Amazon DocumentDB Information in CloudTrail CloudTrail is enabled on your AWS account when you create the account. When activity occurs in Amazon DocumentDB (with MongoDB compatibility), that activity is recorded in a CloudTrail event along with other AWS service events in **Event history**. You can view, search, and download recent events in your AWS account. For more information, see [Viewing Events with CloudTrail Event History](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/view-cloudtrail-events.html). For an ongoing record of events in your AWS account, including events for Amazon DocumentDB (with MongoDB compatibility), create a trail. A trail enables CloudTrail to deliver log files to an Amazon S3 bucket. By default, when you create a trail in the console, the trail applies to all AWS Regions. The trail logs events from all Regions in the AWS partition and delivers the log files to the Amazon S3 bucket that you specify. Additionally, you can configure other AWS services to further analyze and act upon the event data collected in CloudTrail logs. For more information, see the following topics in the *AWS CloudTrail User Guide*: + [Overview for Creating a Trail](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail.html) + [CloudTrail Supported Services and Integrations](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-aws-service-specific-topics.html#cloudtrail-aws-service-specific-topics-integrations) + [Configuring Amazon SNS Notifications for CloudTrail](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/configure-sns-notifications-for-cloudtrail.html) + [Receiving CloudTrail Log Files from Multiple Regions](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/receive-cloudtrail-log-files-from-multiple-regions.html) + [Receiving CloudTrail Log Files from Multiple Accounts](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-receive-logs-from-multiple-accounts.html) Every event or log entry includes information about who generated the request. The identity information helps you determine the following: + Whether the request was made with root or user credentials. + Whether the request was made with temporary security credentials for a role or federated user. + Whether the request was made by another AWS service. For more information, see the [CloudTrail userIdentity Element](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-event-reference-user-identity.html). # Profiling Amazon DocumentDB operations Profiling operations You can use the profiler in Amazon DocumentDB (with MongoDB compatibility) to log the execution time and details of operations that were performed on your cluster. The profiler is useful for monitoring the slowest operations on your cluster to help you improve individual query performance and overall cluster performance. By default, the profiler feature is disabled. When enabled, the profiler logs operations that are taking longer than a customer-defined threshold value (for example, 100 ms) to Amazon CloudWatch Logs. Logged details include the profiled command, time, plan summary, and client metadata. After the operations are logged to CloudWatch Logs, you can use CloudWatch Logs Insights to analyze, monitor, and archive your Amazon DocumentDB profiling data. Common queries are provided in the section [Common queries](#profiling.common-queries). When enabled, the profiler uses additional resources in your cluster. We recommend that you start with a high threshold value (for example, 500 ms) and gradually lower the value to identify slow operations. Starting with a threshold value of 50 ms can cause performance issues on your cluster for high throughput applications. The profiler is enabled at the cluster level and works on all instances and databases in a cluster. Amazon DocumentDB logs operations to Amazon CloudWatch Logs on a best-effort basis. Although Amazon DocumentDB imposes no additional charge to enable the profiler, you are charged the standard rates for the usage of CloudWatch Logs. For information about CloudWatch Logs pricing, see [Amazon CloudWatch pricing](https://aws.amazon.com/cloudwatch/pricing/). **Topics** + [ ## Supported operations ](#profiling.supported-commands) + [ ## Limitations ](#profiling.limitations) + [Enabling the profiler](#profiling.enable-profiling) + [Disabling the profiler](#profiling.disable-profiling) + [ ## Disabling profiler logs export ](#profiling.disabling-logs-export) + [Accessing your profiler logs](#profiling.accessing) + [ ## Common queries ](#profiling.common-queries) ## Supported operations Amazon DocumentDB profiler supports the following operations: + `aggregate` + `count` + `delete` + `distinct` + `find` (OP\$1QUERY and command) + `findAndModify` + `insert` + `update` ## Limitations The slow query profiler is only able to emit profiler logs if the entire result set of the query is able to fit in one batch, and if the result set is under 16MB (maximum BSON size). Result sets greater than 16MB are automatically split into multiple batches. Most drivers or shells may set a default batch size that is small. You can specify the batch size as part of your query. For the purpose of capturing slow query logs, we recommend a batch size that exceeds the size of your expected result set. If you are unsure of the result set size, or if it varies, you can also set the batch size to a large number (e.g., 100k). However, using a larger batch size means more results will have to be retrieved from the database before a response is sent to the client. For some queries, that may create longer delays before you get results. If you do not plan to consume the entire result set, it is possible that you will spend more I/Os to process the query and throw away the result. ## Enabling the Amazon DocumentDB profiler Enabling the profiler Enabling the profiler on a cluster is a three-step process. Ensure that all steps are completed, or profiling logs will not be sent to CloudWatch Logs. Profiler is set at the cluster level and is performed on all of the cluster's databases and instances. **To enable the profiler on a cluster** 1. Because you can't modify a default cluster parameter group, ensure that you have an available custom cluster parameter group. For more information, see [Creating Amazon DocumentDB cluster parameter groups](cluster_parameter_groups-create.md). 1. Using an available custom cluster parameter group, modify the following parameters: `profiler`, `profiler_threshold_ms`, and `profiler_sampling_rate`. For more information, see [Modifying Amazon DocumentDB cluster parameter groups](cluster_parameter_groups-modify.md). 1. Create or modify your cluster to use the custom cluster parameter group and to enable exporting `profiler` logs to CloudWatch Logs. The following sections show how to implement these steps using the AWS Management Console and the AWS Command Line Interface (AWS CLI). ------ #### [ Using the AWS Management Console ] 1. Before you begin, create a Amazon DocumentDB cluster and a custom cluster parameter group if you don't already have one. For more information, see [Creating Amazon DocumentDB cluster parameter groups](cluster_parameter_groups-create.md) and [Creating an Amazon DocumentDB cluster](db-cluster-create.md). 1. Using an available custom cluster parameter group, modify the following parameters. For more information, see [Modifying Amazon DocumentDB cluster parameter groups](cluster_parameter_groups-modify.md). + `profiler` — Enables or disables query profiling. Permitted values are `enabled ` and `disabled`. The default value is `disabled`. To enable profiling, set the value to `enabled`. + `profiler_threshold_ms` — When `profiler` is set to `enabled`, all commands that are taking longer than ` profiler_threshold_ms` are logged to CloudWatch. Permitted values are `[50-INT_MAX]`. The default value is `100`. + `profiler_sampling_rate` — The fraction of slow operations that should be profiled or logged. Permitted values are `[0.0-1.0]`. The default value is `1.0`. 1. Modify your cluster to use the custom cluster parameter group and set the profiler log exports to publish to Amazon CloudWatch. 1. In the navigation pane, choose **Clusters ** to add your custom parameter group to a cluster. 1. Choose the button to the left of the name of the cluster that you want to associate your parameter group with. Select **Actions**, and then **Modify** to modify your cluster. 1. Under **Cluster options**, choose the custom parameter group from the step above to add it to your cluster. 1. Under **Log exports**, select **Profiler logs** to publish to Amazon CloudWatch. 1. Choose **Continue** to view a summary of your modifications. 1. After verifying your changes, you can apply them immediately or during the next maintenance window under **Scheduling of modifications **. 1. Choose **Modify cluster** to update your cluster with your new parameter group. ------ #### [ Using the AWS CLI ] The following procedure enables the profiler on all supported operations for the cluster `sample-cluster`. 1. Before you begin, ensure that you have an available custom cluster parameter group by running the following command, and reviewing the output for a cluster parameter group that doesn't have `default` in the name and has `docdb3.6` as the parameter group family. If you don't have a non-default cluster parameter group, see [Creating Amazon DocumentDB cluster parameter groups](cluster_parameter_groups-create.md). ``` aws docdb describe-db-cluster-parameter-groups \ --query 'DBClusterParameterGroups[*].[DBClusterParameterGroupName,DBParameterGroupFamily]' ``` In the following output, only `sample-parameter-group ` meets both criteria. ``` [ [ "default.docdb3.6", "docdb3.6" ], [ "sample-parameter-group", "docdb3.6" ] ] ``` 1. Using your custom cluster parameter group, modify the following parameters: + `profiler` — Enables or disables query profiling. Permitted values are `enabled ` and `disabled`. The default value is `disabled`. To enable profiling, set the value to `enabled`. + `profiler_threshold_ms` — When `profiler` is set to `enabled`, all commands taking longer than `profiler_threshold_ms` are logged to CloudWatch. Permitted values are `[50-INT_MAX]`. The default value is `100`. + `profiler_sampling_rate` — The fraction of slow operations that should be profiled or logged. Permitted values are `[0.0-1.0]`. The default value is `1.0`. ``` aws docdb modify-db-cluster-parameter-group \ --db-cluster-parameter-group-name sample-parameter-group \ --parameters ParameterName=profiler,ParameterValue=enabled,ApplyMethod=immediate \ ParameterName=profiler_threshold_ms,ParameterValue=100,ApplyMethod=immediate \ ParameterName=profiler_sampling_rate,ParameterValue=0.5,ApplyMethod=immediate ``` 1. Modify your Amazon DocumentDB cluster so that it uses the `sample-parameter-group` custom cluster parameter group from the previous step and sets the parameter `--enable-cloudwatch-logs-exports` to `profiler`. The following code modifies the cluster `sample-cluster` to use the `sample-parameter-group` from the previous step, and adds `profiler` to the enabled CloudWatch Logs exports. ``` aws docdb modify-db-cluster \ --db-cluster-identifier sample-cluster \ --db-cluster-parameter-group-name sample-parameter-group \ --cloudwatch-logs-export-configuration '{"EnableLogTypes":["profiler"]}' ``` Output from this operation looks something like the following. ``` { "DBCluster": { "AvailabilityZones": [ "us-east-1c", "us-east-1b", "us-east-1a" ], "BackupRetentionPeriod": 1, "DBClusterIdentifier": "sample-cluster", "DBClusterParameterGroup": "sample-parameter-group", "DBSubnetGroup": "default", "Status": "available", "EarliestRestorableTime": "2020-04-07T02:05:12.479Z", "Endpoint": "sample-cluster.node.us-east-1.docdb.amazonaws.com", "ReaderEndpoint": "sample-cluster.node.us-east-1.docdb.amazonaws.com", "MultiAZ": false, "Engine": "docdb", "EngineVersion": "3.6.0", "LatestRestorableTime": "2020-04-08T22:08:59.317Z", "Port": 27017, "MasterUsername": "test", "PreferredBackupWindow": "02:00-02:30", "PreferredMaintenanceWindow": "tue:09:50-tue:10:20", "DBClusterMembers": [ { "DBInstanceIdentifier": "sample-instance-1", "IsClusterWriter": true, "DBClusterParameterGroupStatus": "in-sync", "PromotionTier": 1 }, { "DBInstanceIdentifier": "sample-instance-2", "IsClusterWriter": true, "DBClusterParameterGroupStatus": "in-sync", "PromotionTier": 1 } ], "VpcSecurityGroups": [ { "VpcSecurityGroupId": "sg-abcd0123", "Status": "active" } ], "HostedZoneId": "ABCDEFGHIJKLM", "StorageEncrypted": true, "KmsKeyId": "arn:aws:kms:us-east-1::key/sample-key", "DbClusterResourceId": "cluster-ABCDEFGHIJKLMNOPQRSTUVWXYZ", "DBClusterArn": "arn:aws:rds:us-east-1::cluster:sample-cluster", "AssociatedRoles": [], "ClusterCreateTime": "2020-01-10T22:13:38.261Z", "EnabledCloudwatchLogsExports": [ "profiler" ], "DeletionProtection": true } } ``` ------ ## Disabling the Amazon DocumentDB profiler Disabling the profiler To disable the profiler, you disable both the `profiler` parameter and the export of `profiler` logs to CloudWatch Logs. ### Disabling the Profiler You can disable the `profiler` parameter using either the AWS Management Console or AWS CLI, as follows. ------ #### [ Using the AWS Management Console ] The following procedure uses the AWS Management Console to disable Amazon DocumentDB `profiler`. 1. Sign in to the AWS Management Console, and open the Amazon DocumentDB console at [https://console.aws.amazon.com/docdb](https://console.aws.amazon.com/docdb). 1. In the navigation pane, choose **Parameter groups**. Then choose the name of the cluster parameter group that you want to disable the profiler on. 1. In the resulting **Cluster parameters ** page, select the button to the left of the `profiler` parameter and choose **Edit**. 1. In the **Modify profiler** dialog box, choose `disabled` in the list. 1. Choose **Modify cluster parameter**. ------ #### [ Using the AWS CLI ] To disable `profiler` on a cluster using the AWS CLI, modify the cluster as follows. ``` aws docdb modify-db-cluster-parameter-group \ --db-cluster-parameter-group-name sample-parameter-group \ --parameters ParameterName=profiler,ParameterValue=disabled,ApplyMethod=immediate ``` ------ ## Disabling profiler logs export You can disable exporting `profiler` logs to CloudWatch Logs by using either the AWS Management Console or AWS CLI, as follows. ------ #### [ Using the AWS Management Console ] The following procedure uses the AWS Management Console to disable Amazon DocumentDB exporting logs to CloudWatch. 1. Open the Amazon DocumentDB console at [https://console.aws.amazon.com/docdb](https://console.aws.amazon.com/docdb). 1. In the navigation pane, choose **Clusters**. Choose the button to the left of the name of the cluster for which you want to disable exporting logs. 1. On the **Actions** menu, choose **Modify**. 1. Scroll down to the **Log exports** section and unselect **Profiler logs**. 1. Choose **Continue**. 1. Review your changes, and then choose when you want this change applied to your cluster: + **Apply during the next scheduled maintenance window** + **Apply immediately** 1. Choose **Modify cluster**. ------ #### [ Using the AWS CLI ] The following code modifies the cluster `sample-cluster ` and disables CloudWatch profiler logs. **Example** For Linux, macOS, or Unix: ``` aws docdb modify-db-cluster \ --db-cluster-identifier sample-cluster \ --cloudwatch-logs-export-configuration '{"DisableLogTypes":["profiler"]}' ``` For Windows: ``` aws docdb modify-db-cluster ^ --db-cluster-identifier sample-cluster ^ --cloudwatch-logs-export-configuration '{"DisableLogTypes":["profiler"]}' ``` Output from this operation looks something like the following. ``` { "DBCluster": { "AvailabilityZones": [ "us-east-1c", "us-east-1b", "us-east-1a" ], "BackupRetentionPeriod": 1, "DBClusterIdentifier": "sample-cluster", "DBClusterParameterGroup": "sample-parameter-group", "DBSubnetGroup": "default", "Status": "available", "EarliestRestorableTime": "2020-04-08T02:05:17.266Z", "Endpoint": "sample-cluster.node.us-east-1.docdb.amazonaws.com", "ReaderEndpoint": "sample-cluster.node.us-east-1.docdb.amazonaws.com", "MultiAZ": false, "Engine": "docdb", "EngineVersion": "3.6.0", "LatestRestorableTime": "2020-04-09T05:14:44.356Z", "Port": 27017, "MasterUsername": "test", "PreferredBackupWindow": "02:00-02:30", "PreferredMaintenanceWindow": "tue:09:50-tue:10:20", "DBClusterMembers": [ { "DBInstanceIdentifier": "sample-instance-1", "IsClusterWriter": true, "DBClusterParameterGroupStatus": "in-sync", "PromotionTier": 1 }, { "DBInstanceIdentifier": "sample-instance-2", "IsClusterWriter": true, "DBClusterParameterGroupStatus": "in-sync", "PromotionTier": 1 } ], "VpcSecurityGroups": [ { "VpcSecurityGroupId": "sg-abcd0123", "Status": "active" } ], "HostedZoneId": "ABCDEFGHIJKLM", "StorageEncrypted": true, "KmsKeyId": "arn:aws:kms:us-east-1::key/sample-key", "DbClusterResourceId": "cluster-ABCDEFGHIJKLMNOPQRSTUVWXYZ", "DBClusterArn": "arn:aws:rds:us-east-1::cluster:sample-cluster", "AssociatedRoles": [], "ClusterCreateTime": "2020-01-10T22:13:38.261Z", "DeletionProtection": true } } ``` ------ ## Accessing your Amazon DocumentDB profiler logs Accessing your profiler logs Follow these steps to access your profile logs on Amazon CloudWatch. 1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/). 1. Make sure that you are in the same Region as your Amazon DocumentDB cluster. 1. In the navigation pane, choose **Logs**. 1. To find the profiler logs for your cluster, in the list, choose `/aws/docdb/yourClusterName/profiler`. The profile logs for each of your instances are available under each of the respective instance names. ## Common queries The following are some common queries you can use to analyze your profiled commands. For more information about CloudWatch Logs Insights, see [ Analyzing Log Data with CloudWatch Logs Insights](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AnalyzingLogData.html) and [ Sample Queries](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_QuerySyntax-examples.html). ### Get the 10 slowest operations on a specified collection ``` filter ns="test.foo" | sort millis desc | limit 10 ``` ### Get all the update operations on a collection that took more than 60 ms ``` filter millis > 60 and op = "update" ``` ### Get the 10 slowest operations in the last month ``` sort millis desc | limit 10 ``` ### Get all the queries with a COLLSCAN plan summary ``` filter planSummary="COLLSCAN" ``` # Monitoring with Performance Insights Monitoring with Performance Insights Performance Insights adds to the existing Amazon DocumentDB monitoring features to illustrate your cluster performance and help you analyze any issues that affect it. With the Performance Insights dashboard, you can visualize the database load and filter the load by waits, query statements, hosts, or application. **Note** Performance Insights is only available for Amazon DocumentDB 3.6, 4.0, 5.0, and 8.0 instance-based clusters. **How is it useful?** + Visualize database performance — Visualize the load to determine when and where the load is on the database + Determine what is causing load on database — Determine which queries, hosts, and applications are contributing to the load on your instance + Determine when there is load on your database — Zoom in on the Performance Insights dashboard to focus on specific events or zoom out to look at trends across a larger time span + Alert on database load — Access new database load metrics automatically from CloudWatch where you can monitor the DB load metrics alongside other Amazon DocumentDB metrics and set alerts on them **What are the limitations of Amazon DocumentDB Performance Insights?** + Performance Insights in the AWS GovCloud (US-East) and AWS GovCloud (US-West) regions are not available + Performance Insights for Amazon DocumentDB retains up to 7 days of performance data + Queries longer than 1,024 bytes are not aggregated in Performance Insights **Topics** + [Performance Insights concepts](performance-insights-concepts.md) + [Enabling and disabling Performance Insights](performance-insights-enabling.md) + [Configuring access policies for Performance Insights](performance-insights-policies.md) + [Analyzing metrics with the Performance Insights dashboard](performance-insights-analyzing.md) + [Retrieving metrics with the Performance Insights API](performance-insights-metrics.md) + [Amazon CloudWatch metrics for Performance Insights](performance-insights-cloudwatch.md) + [Performance Insights for counter metrics](performance-insights-counter-metrics.md) # Performance Insights concepts Performance Insights concepts **Topics** + [Average active sessions](#performance-insights-concepts-sessions) + [Dimensions](#performance-insights-concepts-dimensions) + [Max vCPU](#performance-insights-concepts-maxvcpu) ## Average active sessions Average active sessions Database load (DB load) measures the level of activity in your database. The key metric in Performance Insights is `DB Load`, which is collected every second. The unit for the `DBLoad` metric is the *Average Active Sessions (AAS)* for an Amazon DocumentDB instance. An *active* session is a connection that has submitted work to the Amazon DocumentDB instance and is waiting for a response. For example, if you submit a query to an Amazon DocumentDB instance, the database session is active while the instance is processing the query. To obtain the average active sessions, Performance Insights samples the number of sessions concurrently running a query. The AAS is the total number of sessions divided by the total number of samples. The following table shows five consecutive samples of a running query. | Sample | Number of sessions running query | AAS | Calculation | | --- | --- | --- | --- | | 1 | 2 | 2 | 2 sessions / 1 sample | | 2 | 0 | 1 | 2 sessions / 2 samples | | 3 | 4 | 2 | 6 sessions / 3 samples | | 4 | 0 | 1.5 | 6 sessions / 4 samples | | 5 | 4 | 2 | 10 sessions / 5 samples | In the preceding example, the DB Load for the time interval from 1-5 is 2 AAS. An increase in DB load means that, on average, more sessions are running on the database. ## Dimensions Dimensions The `DB Load` metric is different from the other time-series metrics because you can break it into subcomponents called dimensions. You can think of dimensions as categories for the different characteristics of the `DB Load` metric. When you are diagnosing performance issues, the most useful dimensions are **wait states** and **top query**. **wait states** A *wait state *causes a query statement to wait for a specific event to happen before it can continue running. For example, a query statement might wait until a locked resource is unlocked. By combining `DB Load` with wait states, you can get a complete picture of the session state. Here are various Amazon DocumentDB wait states: | Amazon DocumentDB wait state | Wait State Description | | --- | --- | | Latch | The Latch wait state occurs when the session is waiting to page the buffer pool. Frequent paging in and out of the buffer pool can happen more often when there are frequent large queries being processed by the system, collection scans, or when the buffer pool is too small to handle the working set. | | CPU | The CPU wait state occurs when the session is waiting on CPU. | | CollectionLock | The CollectionLock wait state occurs when the session is waiting to acquire a lock on the collection. These events occur when there are DDL operations on the collection. | | DocumentLock | The DocumentLock wait state occurs when the session is waiting to acquire a lock on a document. High number of concurrent writes to the same document will contribute to more DocumentLock wait states on that document. | | SystemLock | The SystemLock wait state occurs when the session is waiting on the system. This can occur when there are frequent long running queries, long running transactions, or high concurrency on the system. | | IO | The IO wait state occurs when the session waiting on IO to complete. | | BufferLock | The BufferLock wait state occurs when the session is waiting to acquire a lock on a shared page in the buffer. BufferLock wait states can be prolonged if other processes are holding open cursors on the requested pages. | | LowMemThrottle | The LowMemThrottle wait state occurs when the session is waiting due to heavy memory pressure on the Amazon DocumentDB instance. If this state persists for a long time, consider scaling up the instance to provide additional memory. For more information, see [Resource Governor](https://docs.aws.amazon.com/documentdb/latest/developerguide/how-it-works.html). | | BackgroundActivity | The BackgroundActivity wait state occurs when the session is waiting on internal system processes. | | Other | The Other wait state is an internal wait state. If this state persists for a long time, consider terminating this query. For more information, see [How Do I Find and Terminate Long Running or Blocked Queries?](https://docs.aws.amazon.com/documentdb/latest/developerguide/user_diagnostics.html#user_diagnostics-query_terminating.html) | **Top queries** Whereas wait states show bottlenecks, top queries show which queries are contributing the most to DB load. For example, many queries might be currently running on the database, but a single query might consume 99% of the DB load. In this case, the high load might indicate a problem with the query. ## Max vCPU Max vCPU In the dashboard, the **Database load** chart collects, aggregates, and displays session information. To see whether active sessions are exceeding the maximum CPU, look at their relationship to the **Max vCPU** line. The **Max vCPU** value is determined by the number of vCPU (virtual CPU) cores for your Amazon DocumentDB instance. If the DB load is often above the **Max vCPU** line, and the primary wait state is CPU, the CPU is overloaded. In this case, you might want to throttle connections to the instance, tune any queries with a high CPU load, or consider a larger instance class. High and consistent instances of any wait state indicate that there might be bottlenecks or resource contention issues to resolve. This can be true even if the DB load doesn't cross the **Max vCPU** line. # Enabling and disabling Performance Insights Enabling and disabling Performance Insights To use Performance Insights, enable it on your DB instance. You can disable it later if necessary. Enabling and disabling Performance Insights doesn't cause downtime, a reboot, or a failover. The Performance Insights agent consumes limited CPU and memory on the DB host. When the DB load is high, the agent limits the performance impact by collecting data less frequently. ## Enabling Performance Insights when creating a cluster Enabling Performance Insights when creating a cluster In the console, you can enable or disable Performance Insights when you create or modify a new DB instance. ### Using the AWS Management Console In the console, you can enable Performance Insights when you create an Amazon DocumentDB cluster. When you create a new Amazon DocumentDB cluster, enable Performance Insights by choosing **Enable Performance Insights** in the **Performance Insights** section. **Console instructions** 1. To create a cluster, follow the instructions for [Creating an Amazon DocumentDB cluster.](https://docs.aws.amazon.com/documentdb/latest/developerguide/db-cluster-create.html) 1. Select **Enable Performance Insights** in the Performance Insights section. ![\[The Performance Insights section with Enable Performance Insights selected.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/performance-insights/select-performance-insights.png) **Note** The Performance Insights data retention period will be seven days. ** AWS KMS key** — Specify your AWS KMS key. Performance Insights encrypts all potentially sensitive data using your AWS KMS key. Data is encrypted in flight and at rest. For more information, see Configuring an AWS AWS KMS policy for Performance Insights. ## Enabling and disabling when modifying an instance Enabling and disabling when modifying an instance You can modify a DB instance to enable or disable Performance Insights using the console or AWS CLI. ------ #### [ Using the AWS Management Console ] **Console instructions** 1. Sign in to the AWS Management Console, and open the Amazon DocumentDB console at [https://console.aws.amazon.com/docdb](https://console.aws.amazon.com/docdb). 1. Choose **Clusters**. 1. Choose a DB instance, and choose **Modify**. 1. In the Performance Insights section, choose either **Enable Performance Insights** or **Disable Performance Insights**. **Note** If you choose **Enable Performance Insights**, you can specify your AWS AWS KMS key. Performance Insights encrypts all potentially sensitive data using your AWS KMS key. Data is encrypted in flight and at rest. For more information, see [Encrypting Amazon DocumentDB Data at Rest](https://docs.aws.amazon.com/documentdb/latest/developerguide/encryption-at-rest.html). 1. Choose **Continue**. 1. For **Scheduling of Modifications**, choose **Apply immediately**. If you choose **Apply during the next scheduled maintenance window**, your instance ignores this setting and enables Performance Insights immediately. 1. Choose **Modify instance**. ------ #### [ Using the AWS CLI ] When you use the `create-db-instance` or `modify-db-instance` AWS AWS CLI commands, you can enable Performance Insights by specifying `--enable-performance-insights`, or disable it by specifying `--no-enable-performance-insights`. The following procedure describes how to enable or disable Performance Insights for a DB instance using the AWS AWS CLI. **AWS AWS CLI instructions** Call the `modify-db-instance` AWS AWS CLI command and provide the following values: + `--db-instance-identifer` — The name of the DB instance + `--enable-performance-insights` to enable or `--no-enable-performance-insights` to disable **Example** The following example enables Performance Insights for `sample-db-instance`: ``` aws docdb modify-db-instance \ --db-instance-identifier sample-db-instance \ --enable-performance-insights ``` ``` aws docdb modify-db-instance ^ --db-instance-identifier sample-db-instance ^ --enable-performance-insights ``` ------ # Configuring access policies for Performance Insights Configuring access policies for Performance Insights To access Performance Insights, you must have the appropriate permissions from AWS Identity and Access Management (IAM). You have the following options for granting access: + Attach the `AmazonRDSPerformanceInsightsReadOnly` managed policy to a permission set or role. + Create a custom IAM policy and attach it to a permission set or role. Also, if you specified a customer managed key when you turned on Performance Insights, make sure that users in your account have the `kms:Decrypt` and `kms:GenerateDataKey` permissions on the KMS key. **Note** For encryption-at-rest with AWS KMS keys and security groups management, Amazon DocumentDB leverages operational technology that is shared with [Amazon RDS](https://aws.amazon.com/rds). ## Attaching the AmazonRDSPerformanceInsightsReadOnly policy to an IAM principal `AmazonRDSPerformanceInsightsReadOnly` is an AWS-managed policy that grants access to all read-only operations of the Amazon DocumentDB Performance Insights API. Currently, all operations in this API are read-only. If you attach `AmazonRDSPerformanceInsightsReadOnly` to a permission set or role, the recipient can use Performance Insights with other console features. ## Creating a custom IAM policy for Performance Insights For users who don't have the `AmazonRDSPerformanceInsightsReadOnly` policy, you can grant access to Performance Insights by creating or modifying a user-managed IAM policy. When you attach the policy to a permission set or role, the recipient can use Performance Insights. **To create a custom policy** 1. Open the IAM console at [https://console.aws.amazon.com/iam/](https://console.aws.amazon.com/iam/). 1. In the navigation pane, choose **Policies**. 1. Choose **Create policy**. 1. On the **Create Policy** page, choose the JSON tab. 1. Copy and paste the following text, replacing *us-east-1* with the name of your AWS Region and *111122223333* with your customer account number. ------ #### [ JSON ] **** ``` { "Version":"2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "rds:DescribeDBInstances", "Resource": "*" }, { "Effect": "Allow", "Action": "rds:DescribeDBClusters", "Resource": "*" }, { "Effect": "Allow", "Action": "pi:DescribeDimensionKeys", "Resource": "arn:aws:pi:us-east-1:111122223333:metrics/rds/*" }, { "Effect": "Allow", "Action": "pi:GetDimensionKeyDetails", "Resource": "arn:aws:pi:us-east-1:111122223333:metrics/rds/*" }, { "Effect": "Allow", "Action": "pi:GetResourceMetadata", "Resource": "arn:aws:pi:us-east-1:111122223333:metrics/rds/*" }, { "Effect": "Allow", "Action": "pi:GetResourceMetrics", "Resource": "arn:aws:pi:us-east-1:111122223333:metrics/rds/*" }, { "Effect": "Allow", "Action": "pi:ListAvailableResourceDimensions", "Resource": "arn:aws:pi:us-east-1:111122223333:metrics/rds/*" }, { "Effect": "Allow", "Action": "pi:ListAvailableResourceMetrics", "Resource": "arn:aws:pi:us-east-1:111122223333:metrics/rds/*" } ] } ``` ------ 1. Choose **Review policy**. 1. Provide a name for the policy and optionally a description, and then choose **Create policy**. You can now attach the policy to a permission set or role. The following procedure assumes that you already have a user available for this purpose. **To attach the policy to a user** 1. Open the IAM console at [https://console.aws.amazon.com/iam/](https://console.aws.amazon.com/iam/). 1. In the navigation pane, choose **Users**. 1. Choose an existing user from the list. **Important** To use Performance Insights, make sure that you have access to Amazon DocumentDB in addition to the custom policy. For example, the **AmazonDocDBReadOnlyAccess** predefined policy provides read-only access to Amazon DocDB.For more information, see [Managing access using policies](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAM.html#security_iam_access-manage). 1. On the **Summary** page, choose **Add permissions**. 1. Choose **Attach existing policies directly**. For **Search**, type the first few characters of your policy name, as shown following. ![\[Choose a Policy\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/performance-insights/pi-add-permissions.png) 1. Choose your policy, and then choose **Next: Review**. 1. Choose **Add permissions**. ## Configuring an AWS KMS policy for Performance Insights Performance Insights uses an AWS KMS key to encrypt sensitive data. When you enable Performance Insights through the API or the console, you have the following options: + Choose the default AWS managed key. Amazon DocumentDB uses the AWS managed key for your new DB instance. Amazon DocumentDB creates an AWS managed key for your AWS account. Your AWS account has a different AWS managed key for Amazon DocumentDB for each AWS Region. + Choose a customer managed key. If you specify a customer managed key, users in your account that call the Performance Insights API need the `kms:Decrypt` and `kms:GenerateDataKey` permissions on the KMS key. You can configure these permissions through IAM policies. However, we recommend that you manage these permissions through your KMS key policy. For more information, see [ Using key policies in AWS KMS](https://docs.aws.amazon.com/kms/latest/developerguide/key-policies.html). **Example** The following sample key policy shows how to add statements to your KMS key policy. These statements allow access to Performance Insights. Depending on how you use the AWS KMS, you might want to change some restrictions. Before adding statements to your policy, remove all comments. # Analyzing metrics with the Performance Insights dashboard Analyzing metrics with the Performance Insights dashboard The Performance Insights dashboard contains database performance information to help you analyze and troubleshoot performance issues. On the main dashboard page, you can view information about the database load (DB load). You can "slice" DB load by dimensions such as wait states or query. **Topics** + [Overview of the Performance Insights dashboard](performance-insights-dashboard-overview.md) + [Opening the Performance Insights dashboard](performance-insights-dashboard-opening.md) + [Analyzing database load by wait states](performance-insights-analyzing-db-load.md) + [Overview of the Top queries tab](performance-insights-top-queries.md) + [Zooming in on the database load chart](performance-insights-zoom-db-load.md) # Overview of the Performance Insights dashboard Overview of the Performance Insights dashboard The dashboard is the easiest way to interact with Performance Insights. The following example shows the dashboard for an Amazon DocumentDB instance. By default, the Performance Insights dashboard shows data for the last hour. ![\[Performance Insights dashboard showing CPU utilization and database load over time for an Amazon DocumentDB instance.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/performance-insights/overview-dashboard.png) The dashboard is divided into the following parts: 1. **Counter metrics** – Shows data for specific performance counter metrics. 1. **Database load** – Shows how the DB load compares to DB instance capacity as represented by the **Max vCPU** line. 1. **Top dimensions** – Shows the top dimensions contributing to DB load. These dimensions include `waits`, `queries`, `hosts`, `databases`, and `applications`. **Topics** + [Counter metrics chart](#performance-insights-overview-metrics) + [Database load chart](#performance-insights-overview-db-load-chart) + [Top dimensions table](#performance-insights-overview-top-dimensions) ## Counter metrics chart Counter metrics chart With counter metrics, you can customize the Performance Insights dashboard to include up to 10 additional graphs. These graphs show a selection of dozens of operating system metrics. You can correlate this information with DB load to help identify and analyze performance problems. The **Counter metrics** chart displays data for performance counters. ![\[Counter metrics chart showing CPU utilization over time.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/performance-insights/counter-metrics.png) To change the performance counters, choose **Manage metrics**. You can select multiple **OS metrics** as shown in the following screenshot. To see details for any metric, hover over the metric name. ![\[Performance Insights dashboard metric selection interface with OS metrics options.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/performance-insights/overview-os-metrics.png) ## Database load chart Database load chart The **Database load** chart shows how the database activity compares to instance capacity as represented by the **Max vCPU** line. By default, the stacked line chart represents DB load as average active sessions per unit of time. The DB load is sliced (grouped) by wait states. ![\[Database load chart showing average active sessions over time, with CPU usage spiking near the end.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/performance-insights/database-load.png) **DB load sliced by dimensions** You can choose to display load as active sessions grouped by any supported dimensions. The following image shows the dimensions for the Amazon DocumentDB instance. ![\[Graph showing database load with various "Slice by" options displayed in a dropdown list.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/performance-insights/database-load-sliced.png) **DB load details for a dimension item** To see details about a DB load item within a dimension, hover over the item name. The following image shows details for a query statement. ![\[Bar graph showing database load with additional details displayed upon hovering over an item name.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/performance-insights/database-load-details.png) To see details for any item for the selected time period in the legend, hover over that item. ![\[Bar graph showing database load with additional details displayed upon hovering over a bar.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/performance-insights/database-load-hover.png) ## Top dimensions table Top dimensions table The **Top dimensions table** slices DB load by different dimensions. A dimension is a category or "slice by" for different characteristics of DB load. If the dimension is query, **Top queries **shows the query statements that contribute most to the DB load. Choose any of the following dimension tabs. ![\[The Top queries dimensions tab showing the two top queries.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/performance-insights/top-dimensions.png) The following table provides a brief description of each tab. | Tab | Description | | --- | --- | | Top waits | The event for which the database backend is waiting | | Top queries | The query statements that are currently running | | Top hosts | The host IP and port of the connected client | | Top databases | The name of the database to which the client is connected | | Top applications | The name of the application that is connected to the database | To learn how to analyze queries by using the **Top queries** tab, see [Overview of the Top queries tab](performance-insights-top-queries.md). # Opening the Performance Insights dashboard Opening the Performance Insights dashboard **To view the Performance Insights dashboard in the AWS Management Console, use the following steps:** 1. Open the Performance Insights console at [https://console.aws.amazon.com/docdb/](https://console.aws.amazon.com/docdb/home#performance-insights). 1. Choose a DB instance. The Performance Insights dashboard is shown for that Amazon DocumentDB instance. For Amazon DocumentDB instances with Performance Insights enabled, you can also reach the dashboard by choosing the **Sessions** item in the list of instances. Under **Current activity**, the **Sessions** item shows the database load in average active sessions over the last five minutes. The bar graphically shows the load. When the bar is empty, the instance is idle. As the load increases, the bar fills with blue. When the load passes the number of virtual CPUs (vCPUs) on the instance class, the bar turns red, indicating a potential bottleneck. ![\[The Clusters page showing an Amazon DocumentDB regional cluster and the CPU and current activity of each cluster instance.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/performance-insights/opening-clusters.png) 1. (Optional) Choose a different time interval by selecting a button in the upper right. For example, to change the interval to 1 hour, select **1h**. ![\[Time interval buttons ranging from five minutes to one week.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/performance-insights/opening-time.png) In the following screenshot, the DB load interval is 1 hour. ![\[Bar graph showing database load measured in average active sessions.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/performance-insights/opening-db-load.png) 1. To refresh your data automatically, enable **Auto refresh**. ![\[The auto refresh button enabled, appearing next to the time interval buttons.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/performance-insights/opening-auto-refresh.png) The Performance Insight dashboard automatically refreshes with new data. The refresh rate depends on the amount of data displayed: + 5 minutes refreshes every 5 seconds. + 1 hour refreshes every minute. + 5 hours refreshes every minute. + 24 hours refreshes every 5 minutes. + 1 week refreshes every hour. # Analyzing database load by wait states Analyzing database load by wait states If the **Database load (DB load)** chart shows a bottleneck, you can find out where the load is coming from. To do so, look at the top load items table below the **Database load** chart. Choose a particular item, like a query or an application, to drill down into that item and see details about it. DB load grouped by waits and top queries typically provides the most insight into performance issues. DB load grouped by waits shows if there are any resource or concurrency bottlenecks in the database. In this case, the **Top queries** tab of the top load items table shows which queries are driving that load. Your typical workflow for diagnosing performance issues is as follows: 1. Review the **Database load** chart and see if there are any incidents of database load exceeding the **Max CPU** line. 1. If there is, look at the **Database load** chart and identify which wait state or states are primarily responsible. 1. Identify the digest queries causing the load by seeing which of the queries the **Top queries** tab on the top load items table are contributing most to those wait states. You can identify these by the **Load by Wait (AAS)** column. 1. Choose one of these digest queries in the **Top queries** tab to expand it and see the child queries that it is composed of. You can also see which hosts or applications are contributing the most load by selecting **Top hosts** or **Top applications**, respectively. Application names are specified in the connection string to the Amazon DocumentDB instance. `Unknown` indicates that the application field was not specified. For example, in the following dashboard, **CPU** waits account for most of the DB load. Selecting the top query under **Top queries** will scope the Database load chart to focus on the most load that is being contributed by the select query. ![\[Database load chart showing CPU usage spike. A corresponding Top queries tab shows queries contributing the most to wait states.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/performance-insights/db-load-1.png) ![\[Database load chart showing CPU usage spike for the query contributing the most to wait states. A corresponding Top queries tab shows that query's child queries.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/performance-insights/db-load-2.png) # Overview of the Top queries tab Overview of the Top queries tab By default, the **Top query** tab shows the queries that are contributing the most to DB load. You can analyze the query text to help tune your queries. **Topics** + [Query digests](#performance-insights-top-queries-digests) + [Load by waits (AAS)](#performance-insights-top-queries-aas) + [Viewing detailed query information](#performance-insights-top-queries-query-info) + [Accessing statement query text](#performance-insights-top-queries-accessing-text) + [Viewing and downloading statement query text](#performance-insights-top-queries-viewing-downloading) ## Query digests Query digests A *query digest* is a composite of multiple actual queries that are structurally similar but might have different literal values. The digest replaces hardcoded values with a question mark. For example, a query digest might look like this: ``` {"find":"customerscollection","filter":{"FirstName":"?"},"sort":{"key":{"$numberInt":"?"}},"limit":{"$numberInt":"?"}} ``` This digest might include the following child queries: ``` {"find":"customerscollection","filter":{"FirstName":"Karrie"},"sort":{"key":{"$numberInt":"1"}},"limit":{"$numberInt":"3"}} {"find":"customerscollection","filter":{"FirstName":"Met"},"sort":{"key":{"$numberInt":"1"}},"limit":{"$numberInt":"3"}} {"find":"customerscollection","filter":{"FirstName":"Rashin"},"sort":{"key":{"$numberInt":"1"}},"limit":{"$numberInt":"3"}} ``` To see the literal query statements in a digest, select the query, and then choose the plus symbol (`+`). In the following screenshot, the selected query is a digest. ![\[The Top queries table showing an expanded query digest with one child query selected.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/performance-insights/top-queries-literal.png) **Note** A query digest groups similar query statements, but does not redact sensitive information. ## Load by waits (AAS) Load by waits (AAS) In **Top queries**, the **Load by waits (AAS)** column illustrates the percentage of the database load associated with each top load item. This column reflects the load for that item by whatever grouping is currently selected in the **DB load chart**. For example, you might group the **DB load chart** by wait states. In this case, the **DB Load by Waits** bar is sized, segmented, and color-coded to show how much of a given wait state that query is contributing to. It also shows which wait states are affecting the selected query. ![\[Bar chart showing database load grouped by CPU, IO, and latch wait states. The corresponding table shows the top queries based on load by wait.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/performance-insights/top-queries-aas.png) ## Viewing detailed query information Viewing detailed query information In the **Top query** table, you can open a *digest statement* to view its information. The information appears in the bottom pane. ![\[The Top queries table showing a selected query statement and its query information below.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/performance-insights/top-queries-detailed.png) The following types of identifiers (IDs) are associated with query statements: 1. **Support query ID** – A hash value of the query ID. This value is only for referencing a query ID when you are working with AWS Support. AWS Support doesn't have access to your actual query IDs and query text. 1. **Support digest ID** – A hash value of the digest ID. This value is only for referencing a digest ID when you are working with AWS Support. AWS Support doesn't have access to your actual digest IDs and query text. ## Accessing statement query text Accessing statement query text By default, each row in the **Top queries** table shows 500 bytes of query text for each query statement. When a digest statement exceeds 500 bytes, you can view more text by opening the statement in the Performance Insights dashboard. In this case, the maximum length for the displayed query is 1 KB. If you view a full query statement, you can also choose **Download**. ## Viewing and downloading statement query text Viewing and downloading statement query text In the Performance Insights dashboard, you can view or download query text. **To view more query text in the Performance Insights dashboard** 1. Open the Amazon DocumentDB console at: [https://console.aws.amazon.com/docdb/](https://console.aws.amazon.com/docdb/) 1. In the navigation pane, choose **Performance Insights**. 1. Choose a DB instance. The Performance Insights dashboard is displayed for that DB instance. Query statements with text larger than 500 bytes will look like the following image: ![\[The Top queries table with a child query selected.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/performance-insights/top-queries-statement.png) 1. Examine the query information section to view more of the query text. ![\[The Query information section showing the full text of the selected query.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/performance-insights/top-queries-query-text.png) The Performance Insights dashboard can display up to 1 KB for each full query statement. **Note** To copy or download the query statement, disable any pop-up blockers. # Zooming in on the database load chart Zooming in on the database load chart You can use other features of the Performance Insights user interface to help analyze performance data. **Click-and-Drag Zoom In** In the Performance Insights interface, you can choose a small portion of the load chart and zoom in on the detail. ![\[Bar chart showing database load, with a portion of it highlighted for zooming in.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/performance-insights/pi-zoom-1.png) To zoom in on a portion of the load chart, choose the start time and drag to the end of the time period you want. When you do this, the selected area is highlighted. When you release the mouse, the load chart zooms in on the selected area, and the **Top *items*** table is recalculated. ![\[Database load bar chart showing zoomed-in portion, with corresponding Top waits table below.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/performance-insights/pi-zoom-2.png) # Retrieving metrics with the Performance Insights API Retrieving metrics with the Performance Insights API When Performance Insights is enabled, the API provides visibility into instance performance. Amazon CloudWatch Logs provides the authoritative source for vended monitoring metrics for AWS services. Performance Insights offers a domain-specific view of database load measured as average active sessions (AAS). This metric appears to API consumers as a two-dimensional time-series dataset. The time dimension of the data provides DB load data for each time point in the queried time range. Each time point decomposes overall load in relation to the requested dimensions, such as `Query`, `Wait-state`, `Application`, or `Host`, measured at that time point. Amazon DocumentDB Performance Insights monitors your Amazon DocumentDB DB instance so that you can analyze and troubleshoot database performance. One way to view Performance Insights data is in the AWS Management Console. Performance Insights also provides a public API so that you can query your own data. You can use the API to do the following: + Offload data into a database + Add Performance Insights data to existing monitoring dashboards + Build monitoring tools To use the Performance Insights API, enable Performance Insights on one of your Amazon DocumentDB instances. For information about enabling Performance Insights, see [Enabling and disabling Performance Insights](performance-insights-enabling.md). For more information about the Performance Insights API, see the [ Performance Insights API Reference](https://docs.aws.amazon.com/performance-insights/latest/APIReference/Welcome.html). The Performance Insights API provides the following operations. **** | Performance Insights action | AWS CLI command | Description | | --- | --- | --- | | [https://docs.aws.amazon.com/performance-insights/latest/APIReference/API_DescribeDimensionKeys.html](https://docs.aws.amazon.com/performance-insights/latest/APIReference/API_DescribeDimensionKeys.html) | [https://docs.aws.amazon.com/cli/latest/reference/pi/describe-dimension-keys.html](https://docs.aws.amazon.com/cli/latest/reference/pi/describe-dimension-keys.html) | Retrieves the top N dimension keys for a metric for a specific time period. | | [https://docs.aws.amazon.com/performance-insights/latest/APIReference/API_GetDimensionKeyDetails.html](https://docs.aws.amazon.com/performance-insights/latest/APIReference/API_GetDimensionKeyDetails.html) | [https://docs.aws.amazon.com/cli/latest/reference/pi/get-dimension-key-details.html](https://docs.aws.amazon.com/cli/latest/reference/pi/get-dimension-key-details.html) | Retrieves the attributes of the specified dimension group for a DB instance or data source. For example, if you specify a query ID, and if the dimension details are available, `GetDimensionKeyDetails` retrieves the full text of the dimension `db.query.statement` associated with this ID. This operation is useful because `GetResourceMetrics` and `DescribeDimensionKeys` don't support retrieval of large query statement text. | | [GetResourceMetadata](https://docs.aws.amazon.com/performance-insights/latest/APIReference/API_GetResourceMetadata.html) | [https://docs.aws.amazon.com/cli/latest/reference/pi/get-resource-metadata.html](https://docs.aws.amazon.com/cli/latest/reference/pi/get-resource-metadata.html) | Retrieve the metadata for different features. For example, the metadata might indicate that a feature is turned on or off on a specific DB instance. | | [https://docs.aws.amazon.com/performance-insights/latest/APIReference/API_GetResourceMetrics.html](https://docs.aws.amazon.com/performance-insights/latest/APIReference/API_GetResourceMetrics.html) | [https://docs.aws.amazon.com/cli/latest/reference/pi/get-resource-metrics.html](https://docs.aws.amazon.com/cli/latest/reference/pi/get-resource-metrics.html) | Retrieves Performance Insights metrics for a set of data sources over a time period. You can provide specific dimension groups and dimensions, and provide aggregation and filtering criteria for each group. | | [ListAvailableResourceDimensions](https://docs.aws.amazon.com/performance-insights/latest/APIReference/API_ListAvailableResourceDimensions.html) | [https://docs.aws.amazon.com/cli/latest/reference/pi/list-available-resource-dimensions.html](https://docs.aws.amazon.com/cli/latest/reference/pi/list-available-resource-dimensions.html) | Retrieve the dimensions that can be queried for each specified metric type on a specified instance. | | [ListAvailableResourceMetrics](https://docs.aws.amazon.com/performance-insights/latest/APIReference/API_ListAvailableResourceMetrics.html) | [https://docs.aws.amazon.com/cli/latest/reference/pi/list-available-resource-metrics.html](https://docs.aws.amazon.com/cli/latest/reference/pi/list-available-resource-metrics.html) | Retrieve all available metrics of the specified metric types that can be queried for a specified DB instance. | **Topics** + [ ## AWS CLI for Performance Insights ](#performance-insights-metrics-CLI) + [ ## Retrieving time-series metrics ](#performance-insights-metrics-time-series) + [ ## AWS CLI examples for Performance Insights ](#performance-insights-metrics-api-examples) ## AWS CLI for Performance Insights You can view Performance Insights data using the AWS CLI. You can view help for the AWS CLI commands for Performance Insights by entering the following on the command line. ``` aws pi help ``` If you don't have the AWS CLI installed, see [Installing the AWS Command Line Interface](https://docs.aws.amazon.com/cli/latest/userguide/installing.html) in the *AWS CLI User Guide *for information about installing it. ## Retrieving time-series metrics The `GetResourceMetrics` operation retrieves one or more time-series metrics from the Performance Insights data. `GetResourceMetrics` requires a metric and time period, and returns a response with a list of data points. For example, the AWS Management Console uses `GetResourceMetrics` to populate the **Counter Metrics** chart and the **Database Load** chart, as seen in the following image. ![\[Counter Metrics and Database Load charts\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/performance-insights/perf-insights-api-charts.png) All metrics returned by `GetResourceMetrics` are standard time-series metrics, with the exception of `db.load`. This metric is displayed in the **Database Load** chart. The `db.load` metric is different from the other time-series metrics because you can break it into subcomponents called *dimensions*. In the previous image, `db.load` is broken down and grouped by the waits states that make up the `db.load`. **Note** `GetResourceMetrics` can also return the `db.sampleload` metric, but the `db.load` metric is appropriate in most cases. For information about the counter metrics returned by `GetResourceMetrics`, see [Performance Insights for counter metrics](performance-insights-counter-metrics.md). The following calculations are supported for the metrics: + Average – The average value for the metric over a period of time. Append `.avg` to the metric name. + Minimum – The minimum value for the metric over a period of time. Append `.min` to the metric name. + Maximum – The maximum value for the metric over a period of time. Append `.max` to the metric name. + Sum – The sum of the metric values over a period of time. Append `.sum` to the metric name. + Sample count – The number of times the metric was collected over a period of time. Append `.sample_count` to the metric name. For example, assume that a metric is collected for 300 seconds (5 minutes), and that the metric is collected one time each minute. The values for each minute are 1, 2, 3, 4, and 5. In this case, the following calculations are returned: + Average – 3 + Minimum – 1 + Maximum – 5 + Sum – 15 + Sample count – 5 For information about using the `get-resource-metrics` AWS CLI command, see [https://docs.aws.amazon.com/cli/latest/reference/pi/get-resource-metrics.html](https://docs.aws.amazon.com/cli/latest/reference/pi/get-resource-metrics.html). For the `--metric-queries` option, specify one or more queries that you want to get results for. Each query consists of a mandatory `Metric` and optional `GroupBy` and `Filter` parameters. The following is an example of a `--metric-queries` option specification. ``` { "Metric": "string", "GroupBy": { "Group": "string", "Dimensions": ["string", ...], "Limit": integer }, "Filter": {"string": "string" ...} ``` ## AWS CLI examples for Performance Insights The following examples show how to use the AWS CLI for Performance Insights. **Topics** + [ ### Retrieving counter metrics ](#performance-insights-metrics-api-examples.CounterMetrics) + [ ### Retrieving the DB load average for top wait states ](#performance-insights-metrics-api-examples.DBLoadAverage) + [ ### Retrieving the DB load average for top Query ](#performance-insights-metrics-api-examples.topquery) + [ ### Retrieving the DB load average filtered by Query ](#performance-insights-metrics-api-examples.DBLoadAverageByQuery) ### Retrieving counter metrics The following screenshot shows two counter metrics charts in the AWS Management Console. ![\[Counter Metrics charts.\]](http://docs.aws.amazon.com/documentdb/latest/developerguide/images/performance-insights/perf-insights-api-counters-charts.png) The following example shows how to gather the same data that the AWS Management Console uses to generate the two counter metric charts. For Linux, macOS, or Unix: ``` aws pi get-resource-metrics \ --service-type DOCDB \ --identifier db-ID \ --start-time 2022-03-13T8:00:00Z \ --end-time 2022-03-13T9:00:00Z \ --period-in-seconds 60 \ --metric-queries '[{"Metric": "os.cpuUtilization.user.avg" }, {"Metric": "os.cpuUtilization.idle.avg"}]' ``` For Windows: ``` aws pi get-resource-metrics ^ --service-type DOCDB ^ --identifier db-ID ^ --start-time 2022-03-13T8:00:00Z ^ --end-time 2022-03-13T9:00:00Z ^ --period-in-seconds 60 ^ --metric-queries '[{"Metric": "os.cpuUtilization.user.avg" }, {"Metric": "os.cpuUtilization.idle.avg"}]' ``` You can also make a command easier to read by specifying a file for the `--metrics-query` option. The following example uses a file called query.json for the option. The file has the following contents. ``` [ { "Metric": "os.cpuUtilization.user.avg" }, { "Metric": "os.cpuUtilization.idle.avg" } ] ``` Run the following command to use the file. For Linux, macOS, or Unix: ``` aws pi get-resource-metrics \ --service-type DOCDB \ --identifier db-ID \ --start-time 2022-03-13T8:00:00Z \ --end-time 2022-03-13T9:00:00Z \ --period-in-seconds 60 \ --metric-queries file://query.json ``` For Windows: ``` aws pi get-resource-metrics ^ --service-type DOCDB ^ --identifier db-ID ^ --start-time 2022-03-13T8:00:00Z ^ --end-time 2022-03-13T9:00:00Z ^ --period-in-seconds 60 ^ --metric-queries file://query.json ``` The preceding example specifies the following values for the options: + `--service-type` – `DOCDB` for Amazon DocumentDB + `--identifier` – The resource ID for the DB instance + `--start-time` and `--end-time` – The ISO 8601 `DateTime` values for the period to query, with multiple supported formats It queries for a one-hour time range: + `--period-in-seconds` – `60` for a per-minute query + `--metric-queries` – An array of two queries, each just for one metric. The metric name uses dots to classify the metric in a useful category, with the final element being a function. In the example, the function is `avg` for each query. As with Amazon CloudWatch, the supported functions are `min`, `max`, `total`, and `avg`. The response looks similar to the following. ``` { "AlignedStartTime": "2022-03-13T08:00:00+00:00", "AlignedEndTime": "2022-03-13T09:00:00+00:00", "Identifier": "db-NQF3TTMFQ3GTOKIMJODMC3KQQ4", "MetricList": [ { "Key": { "Metric": "os.cpuUtilization.user.avg" }, "DataPoints": [ { "Timestamp": "2022-03-13T08:01:00+00:00", //Minute1 "Value": 3.6 }, { "Timestamp": "2022-03-13T08:02:00+00:00", //Minute2 "Value": 2.6 }, //.... 60 datapoints for the os.cpuUtilization.user.avg metric { "Key": { "Metric": "os.cpuUtilization.idle.avg" }, "DataPoints": [ { "Timestamp": "2022-03-13T08:01:00+00:00", "Value": 92.7 }, { "Timestamp": "2022-03-13T08:02:00+00:00", "Value": 93.7 }, //.... 60 datapoints for the os.cpuUtilization.user.avg metric ] } ] //end of MetricList } //end of response ``` The response has an `Identifier`, `AlignedStartTime`, and `AlignedEndTime`. B the `--period-in-seconds` value was `60`, the start and end times have been aligned to the minute. If the `--period-in-seconds` was `3600`, the start and end times would have been aligned to the hour. The `MetricList` in the response has a number of entries, each with a `Key` and a `DataPoints` entry. Each `DataPoint` has a `Timestamp` and a `Value`. Each `Datapoints` list has 60 data points because the queries are for per-minute data over an hour, with `Timestamp1/Minute1`, `Timestamp2/Minute2`, and so on, up to `Timestamp60/Minute60`. Because the query is for two different counter metrics, there are two elements in the response `MetricList`. ### Retrieving the DB load average for top wait states The following example is the same query that the AWS Management Console uses to generate a stacked area line graph. This example retrieves the `db.load.avg` for the last hour with load divided according to the top seven wait states. The command is the same as the command in [Retrieving counter metrics](#performance-insights-metrics-api-examples.CounterMetrics). However, the query.json file has the following contents. ``` [ { "Metric": "db.load.avg", "GroupBy": { "Group": "db.wait_state", "Limit": 7 } } ] ``` Run the following command. For Linux, macOS, or Unix: ``` aws pi get-resource-metrics \ --service-type DOCDB \ --identifier db-ID \ --start-time 2022-03-13T8:00:00Z \ --end-time 2022-03-13T9:00:00Z \ --period-in-seconds 60 \ --metric-queries file://query.json ``` For Windows: ``` aws pi get-resource-metrics ^ --service-type DOCDB ^ --identifier db-ID ^ --start-time 2022-03-13T8:00:00Z ^ --end-time 2022-03-13T9:00:00Z ^ --period-in-seconds 60 ^ --metric-queries file://query.json ``` The example specifies the metric of `db.load.avg` and a `GroupBy` of the top seven wait states. For details about valid values for this example, see [DimensionGroup](https://docs.aws.amazon.com/performance-insights/latest/APIReference/API_DimensionGroup.html) in the *Performance Insights API Reference.* The response looks similar to the following. ``` { "AlignedStartTime": "2022-04-04T06:00:00+00:00", "AlignedEndTime": "2022-04-04T06:15:00+00:00", "Identifier": "db-NQF3TTMFQ3GTOKIMJODMC3KQQ4", "MetricList": [ {//A list of key/datapoints "Key": { //A Metric with no dimensions. This is the total db.load.avg "Metric": "db.load.avg" }, "DataPoints": [ //Each list of datapoints has the same timestamps and same number of items { "Timestamp": "2022-04-04T06:01:00+00:00",//Minute1 "Value": 0.0 }, { "Timestamp": "2022-04-04T06:02:00+00:00",//Minute2 "Value": 0.0 }, //... 60 datapoints for the total db.load.avg key ] }, { "Key": { //Another key. This is db.load.avg broken down by CPU "Metric": "db.load.avg", "Dimensions": { "db.wait_state.name": "CPU" } }, "DataPoints": [ { "Timestamp": "2022-04-04T06:01:00+00:00",//Minute1 "Value": 0.0 }, { "Timestamp": "2022-04-04T06:02:00+00:00",//Minute2 "Value": 0.0 }, //... 60 datapoints for the CPU key ] },//... In total we have 3 key/datapoints entries, 1) total, 2-3) Top Wait States ] //end of MetricList } //end of response ``` In this response, there are three entries in the `MetricList`. There is one entry for the total `db.load.avg`, and three entries each for the `db.load.avg` divided according to one of the top three wait states. Since there was a grouping dimension (unlike the first example), there must be one key for each grouping of the metric. There can't be only one key for each metric, as in the basic counter metric use case. ### Retrieving the DB load average for top Query The following example groups `db.wait_state` by the top 10 query statements. There are two different groups for query statements: + `db.query` – The full query statement, such as `{"find":"customers","filter":{"FirstName":"Jesse"},"sort":{"key":{"$numberInt":"1"}}}` + `db.query_tokenized` – The tokenized query statement, such as `{"find":"customers","filter":{"FirstName":"?"},"sort":{"key":{"$numberInt":"?"}},"limit":{"$numberInt":"?"}}` When analyzing database performance, it can be useful to consider query statements that only differ by their parameters as one logic item. So, you can use `db.query_tokenized` when querying. However, especially when you're interested in `explain()`, sometimes it's more useful to examine full query statements with parameters. There is a parent-child relationship between tokenized and full queries, with multiple full queries (children) grouped under the same tokenized query (parent). The command in this example is the similar to the command in [Retrieving the DB load average for top wait states](#performance-insights-metrics-api-examples.DBLoadAverage). However, the query.json file has the following contents. ``` [ { "Metric": "db.load.avg", "GroupBy": { "Group": "db.query_tokenized", "Limit": 10 } } ] ``` The following example uses `db.query_tokenized`. For Linux, macOS, or Unix: ``` aws pi get-resource-metrics \ --service-type DOCDB \ --identifier db-ID \ --start-time 2022-03-13T8:00:00Z \ --end-time 2022-03-13T9:00:00Z \ --period-in-seconds 3600 \ --metric-queries file://query.json ``` For Windows: ``` aws pi get-resource-metrics ^ --service-type DOCDB ^ --identifier db-ID ^ --start-time 2022-03-13T8:00:00Z ^ --end-time 2022-03-13T9:00:00Z ^ --period-in-seconds 3600 ^ --metric-queries file://query.json ``` This example queries over 1 hour, with a one minute period-in-seconds. The example specifies the metric of `db.load.avg` and a `GroupBy` of the top seven wait states. For details about valid values for this example, see [DimensionGroup](https://docs.aws.amazon.com/performance-insights/latest/APIReference/API_DimensionGroup.html) in the *Performance Insights API Reference.* The response looks similar to the following. ``` { "AlignedStartTime": "2022-04-04T06:00:00+00:00", "AlignedEndTime": "2022-04-04T06:15:00+00:00", "Identifier": "db-NQF3TTMFQ3GTOKIMJODMC3KQQ4", "MetricList": [ {//A list of key/datapoints "Key": { "Metric": "db.load.avg" }, "DataPoints": [ //... 60 datapoints for the total db.load.avg key ] }, { "Key": {//Next key are the top tokenized queries "Metric": "db.load.avg", "Dimensions": { "db.query_tokenized.db_id": "pi-1064184600", "db.query_tokenized.id": "77DE8364594EXAMPLE", "db.query_tokenized.statement": "{\"find\":\"customers\",\"filter\":{\"FirstName\":\"?\"},\"sort\":{\"key\":{\"$numberInt\":\"?\"}},\"limit\" :{\"$numberInt\":\"?\"},\"$db\":\"myDB\",\"$readPreference\":{\"mode\":\"primary\"}}" } }, "DataPoints": [ //... 60 datapoints ] }, // In total 11 entries, 10 Keys of top tokenized queries, 1 total key ] //End of MetricList } //End of response ``` This response has 11 entries in the `MetricList` (1 total, 10 top tokenized query), with each entry having 24 per-hour `DataPoints`. For tokenized queries, there are three entries in each dimensions list: + `db.query_tokenized.statement` – The tokenized query statement. + `db.query_tokenized.db_id ` – The synthetic ID that Performance Insights generates for you. This example returns the `pi-1064184600` synthetic ID. + `db.query_tokenized.id` – The ID of the query inside Performance Insights. In the AWS Management Console, this ID is called the Support ID. It's named this because the ID is data that AWS Support can examine to help you troubleshoot an issue with your database. AWS takes the security and privacy of your data extremely seriously, and almost all data is stored encrypted with your AWS KMS key. Therefore, nobody inside AWS can look at this data. In the example preceding, both the `tokenized.statement` and the `tokenized.db_id` are stored encrypted. If you have an issue with your database, AWS Support can help you by referencing the Support ID. When querying, it might be convenient to specify a `Group` in `GroupBy`. However, for finer-grained control over the data that's returned, specify the list of dimensions. For example, if all that is needed is the `db.query_tokenized.statement`, then a `Dimensions` attribute can be added to the query.json file. ``` [ { "Metric": "db.load.avg", "GroupBy": { "Group": "db.query_tokenized", "Dimensions":["db.query_tokenized.statement"], "Limit": 10 } } ] ``` ### Retrieving the DB load average filtered by Query The corresponding API query in this example is similar to the command in [Retrieving the DB load average for top Query](#performance-insights-metrics-api-examples.topquery). However, the query.json file has the following contents. ``` [ { "Metric": "db.load.avg", "GroupBy": { "Group": "db.wait_state", "Limit": 5 }, "Filter": { "db.query_tokenized.id": "AKIAIOSFODNN7EXAMPLE" } } ] ``` In this response, all values are filtered according to the contribution of tokenized query AKIAIOSFODNN7EXAMPLE specified in the query.json file. The keys also might follow a different order than a query without a filter, because it's the top five wait states that affected the filtered query. # Amazon CloudWatch metrics for Performance Insights Amazon CloudWatch metrics for Performance Insights Performance Insights automatically publishes metrics to Amazon CloudWatch. The same data can be queried from Performance Insights, but having the metrics in CloudWatch makes it easy to add CloudWatch alarms. It also makes it easy to add the metrics to existing CloudWatch Dashboards. | Metric | Description | | --- | --- | | DBLoad | The number of active sessions for Amazon DocumentDB. Typically, you want the data for the average number of active sessions. In Performance Insights, this data is queried as `db.load.avg`. | | DBLoadCPU | The number of active sessions where the wait state type is CPU. In Performance Insights, this data is queried as `db.load.avg`, filtered by the wait state type `CPU`. | | DBLoadNonCPU | The number of active sessions where the wait state type is not CPU. | **Note** These metrics are published to CloudWatch only if there is load on the DB instance. You can examine these metrics using the CloudWatch console, the AWS CLI, or the CloudWatch API. For example, you can get the statistics for the `DBLoad` metric by running the [get-metric-statistics](https://docs.aws.amazon.com/cli/latest/reference/cloudwatch/get-metric-statistics.html) command. ``` aws cloudwatch get-metric-statistics \ --region ap-south-1 \ --namespace AWS/DocDB \ --metric-name DBLoad \ --period 360 \ --statistics Average \ --start-time 2022-03-14T8:00:00Z \ --end-time 2022-03-14T9:00:00Z \ --dimensions Name=DBInstanceIdentifier,Value=documentdbinstance ``` This example generates output similar to the following. ``` { "Datapoints": [ { "Timestamp": "2022-03-14T08:42:00Z", "Average": 1.0, "Unit": "None" }, { "Timestamp": "2022-03-14T08:24:00Z", "Average": 2.0, "Unit": "None" }, { "Timestamp": "2022-03-14T08:54:00Z", "Average": 6.0, "Unit": "None" }, { "Timestamp": "2022-03-14T08:36:00Z", "Average": 5.7, "Unit": "None" }, { "Timestamp": "2022-03-14T08:06:00Z", "Average": 4.0, "Unit": "None" }, { "Timestamp": "2022-03-14T08:00:00Z", "Average": 5.2, "Unit": "None" } ], "Label": "DBLoad" } ``` You can use the `DB_PERF_INSIGHTS` metric math function in the CloudWatch console to query Amazon DocumentDB Performance Insights counter metrics. The `DB_PERF_INSIGHTS` function also includes the `DBLoad` metric at sub-minute intervals. You can set CloudWatch alarms on these metrics. For more details on how to create an alarm, see [Create an alarm on Performance Insights counter metrics from an AWS database](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_alarm_database_performance_insights.html). For more information about CloudWatch, see [What is Amazon CloudWatch?](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/WhatIsCloudWatch.html) in the *Amazon CloudWatch User Guide*. # Performance Insights for counter metrics Performance Insights for counter metrics Counter metrics are operating system metrics in the Performance Insights dashboard. To help identify and analyze performance problems, you can correlate counter metrics with DB load. ## Performance Insights operating system counters Performance Insights operating system counters The following operating system counters are available with Amazon DocumentDB Performance Insights. | Counter | Type | Metric | | --- | --- | --- | | active | memory | os.memory.active | | buffers | memory | os.memory.buffers | | cached | memory | os.memory.cached | | dirty | memory | os.memory.dirty | | free | memory | os.memory.free | | inactive | memory | os.memory.inactive | | mapped | memory | os.memory.mapped | | pageTables | memory | os.memory.pageTables | | slab | memory | os.memory.slab | | total | memory | os.memory.total | | writeback | memory | os.memory.writeback | | idle | cpuUtilization | os.cpuUtilization.idle | | system | cpuUtilization | os.cpuUtilization.system | | total | cpuUtilization | os.cpuUtilization.total | | user | cpuUtilization | os.cpuUtilization.user | | wait | cpuUtilization | os.cpuUtilization.wait | | one | loadAverageMinute | os.loadAverageMinute.one | | fifteen | loadAverageMinute | os.loadAverageMinute.fifteen | | five | loadAverageMinute | os.loadAverageMinute.five | | cached | swap | os.swap.cached | | free | swap | os.swap.free | | in | swap | os.swap.in | | out | swap | os.swap.out | | total | swap | os.swap.total | | rx | network | os.network.rx | | tx | network | os.network.tx | | numVCPUs | general | os.general.numVCPUs |