filterIndex

Use filterIndex to return indexed data only, by forcing a query to scan only log groups that are indexed on a field that you specify in the query. For these log groups that are indexed on this field, it further optimizes the query by skipping the log groups that do not have any log events containing the field specified in the query for the indexed field. It further reduces scanned volume by attempting to scan only log events from these log groups that match the value specified in the query for this field index. For more information about field indexes and how to create them, see Create field indexes to improve query performance and reduce scan volume.

Using filterIndex with indexed fields can help you query log groups that include petabytes of log data efficiently by limiting the actual search space to log groups and log events that have field indexes.

For example, suppose that you have created a field index for IPaddress in some of the log groups in your account. You can then create the following query and choose to query all log groups in the account to find log events that include the value 198.51.100.0 in the IPaddress field.

fields @timestamp, @message
| filterIndex IPaddress = "198.51.100.0"
| limit 20

The filterIndex command causes this query to attempt to skip all log groups that are not indexed for IPaddress. Additionally, within the log groups that are indexed, the query skips log events that have an IPaddress field but not observed 198.51.100.0 as the value for that field.

Use the IN operator to expand the results to any of multiple values for the indexed fields. The following example finds logs events that include either the value 198.51.100.0 or 198.51.100.1 in the IPaddress field.

fields @timestamp, @message 
| filterIndex IPaddress in ["198.51.100.0", "198.51.100.1"]
| limit 20