# Catalog objects API Catalog objects The Catalog objects API describes the data types and API related to working with catalogs in AWS Glue. **Topics** + [ # Catalogs API ](aws-glue-api-catalog-catalogs.md) + [ # Database API ](aws-glue-api-catalog-databases.md) + [ # Table API ](aws-glue-api-catalog-tables.md) + [ # Partition API ](aws-glue-api-catalog-partitions.md) + [ # Connections API ](aws-glue-api-catalog-connections.md) + [ # User-defined Function API ](aws-glue-api-catalog-functions.md) + [ # Importing an Athena catalog to AWS Glue ](aws-glue-api-catalog-migration.md) # Catalogs API Catalogs The Catalogs API describes the APIs for creating, deleting, locating, updating, and listing catalogs. ## Data types  —  data types  — + [Catalog structure](#aws-glue-api-catalog-catalogs-Catalog) + [CatalogInput structure](#aws-glue-api-catalog-catalogs-CatalogInput) + [TargetRedshiftCatalog structure](#aws-glue-api-catalog-catalogs-TargetRedshiftCatalog) + [CatalogProperties structure](#aws-glue-api-catalog-catalogs-CatalogProperties) + [CatalogPropertiesOutput structure](#aws-glue-api-catalog-catalogs-CatalogPropertiesOutput) + [DataLakeAccessProperties structure](#aws-glue-api-catalog-catalogs-DataLakeAccessProperties) + [IcebergOptimizationProperties structure](#aws-glue-api-catalog-catalogs-IcebergOptimizationProperties) + [DataLakeAccessPropertiesOutput structure](#aws-glue-api-catalog-catalogs-DataLakeAccessPropertiesOutput) + [IcebergOptimizationPropertiesOutput structure](#aws-glue-api-catalog-catalogs-IcebergOptimizationPropertiesOutput) + [FederatedCatalog structure](#aws-glue-api-catalog-catalogs-FederatedCatalog) ## Catalog structure Catalog The catalog object represents a logical grouping of databases in the AWS Glue Data Catalog or a federated source. You can now create a Redshift-federated catalog or a catalog containing resource links to Redshift databases in another account or region. **Fields** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the catalog. To grant access to the default catalog, this field should not be provided. + `Name` – *Required:* UTF-8 string, not less than 1 or more than 64 bytes long, matching the [Custom string pattern #25](aws-glue-api-common.md#regex_25). The name of the catalog. Cannot be the same as the account ID. + `ResourceArn` – UTF-8 string. The Amazon Resource Name (ARN) assigned to the catalog resource. + `Description` – Description string, not more than 2048 bytes long, matching the [URI address multi-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-uri). Description string, not more than 2048 bytes long, matching the URI address multi-line string pattern. A description of the catalog. + `Parameters` – A map array of key-value pairs. Each key is a Key string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Each value is a UTF-8 string, not more than 512000 bytes long. A map array of key-value pairs that define parameters and properties of the catalog. + `CreateTime` – Timestamp. The time at which the catalog was created. + `UpdateTime` – Timestamp. The time at which the catalog was last updated. + `TargetRedshiftCatalog` – A [TargetRedshiftCatalog](#aws-glue-api-catalog-catalogs-TargetRedshiftCatalog) object. A `TargetRedshiftCatalog` object that describes a target catalog for database resource linking. + `FederatedCatalog` – A [FederatedCatalog](#aws-glue-api-catalog-catalogs-FederatedCatalog) object. A `FederatedCatalog` object that points to an entity outside the AWS Glue Data Catalog. + `CatalogProperties` – A [CatalogPropertiesOutput](#aws-glue-api-catalog-catalogs-CatalogPropertiesOutput) object. A `CatalogProperties` object that specifies data lake access properties and other custom properties. + `CreateTableDefaultPermissions` – An array of [PrincipalPermissions](aws-glue-api-catalog-databases.md#aws-glue-api-catalog-databases-PrincipalPermissions) objects. An array of `PrincipalPermissions` objects. Creates a set of default permissions on the table(s) for principals. Used by AWS Lake Formation. Not used in the normal course of AWS Glue operations. + `CreateDatabaseDefaultPermissions` – An array of [PrincipalPermissions](aws-glue-api-catalog-databases.md#aws-glue-api-catalog-databases-PrincipalPermissions) objects. An array of `PrincipalPermissions` objects. Creates a set of default permissions on the database(s) for principals. Used by AWS Lake Formation. Not used in the normal course of AWS Glue operations. + `AllowFullTableExternalDataAccess` – UTF-8 string (valid values: `True` \$1 `False`). Allows third-party engines to access data in Amazon S3 locations that are registered with Lake Formation. ## CatalogInput structure CatalogInput A structure that describes catalog properties. **Fields** + `Description` – Description string, not more than 2048 bytes long, matching the [URI address multi-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-uri). Description string, not more than 2048 bytes long, matching the URI address multi-line string pattern. A description of the catalog. + `FederatedCatalog` – A [FederatedCatalog](#aws-glue-api-catalog-catalogs-FederatedCatalog) object. A `FederatedCatalog` object. A `FederatedCatalog` structure that references an entity outside the AWS Glue Data Catalog, for example a Redshift database. + `Parameters` – A map array of key-value pairs. Each key is a Key string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Each value is a UTF-8 string, not more than 512000 bytes long. A map array of key-value pairs that define the parameters and properties of the catalog. + `TargetRedshiftCatalog` – A [TargetRedshiftCatalog](#aws-glue-api-catalog-catalogs-TargetRedshiftCatalog) object. A `TargetRedshiftCatalog` object that describes a target catalog for resource linking. + `CatalogProperties` – A [CatalogProperties](#aws-glue-api-catalog-catalogs-CatalogProperties) object. A `CatalogProperties` object that specifies data lake access properties and other custom properties. + `CreateTableDefaultPermissions` – An array of [PrincipalPermissions](aws-glue-api-catalog-databases.md#aws-glue-api-catalog-databases-PrincipalPermissions) objects. An array of `PrincipalPermissions` objects. Creates a set of default permissions on the table(s) for principals. Used by AWS Lake Formation. Typically should be explicitly set as an empty list. + `CreateDatabaseDefaultPermissions` – An array of [PrincipalPermissions](aws-glue-api-catalog-databases.md#aws-glue-api-catalog-databases-PrincipalPermissions) objects. An array of `PrincipalPermissions` objects. Creates a set of default permissions on the database(s) for principals. Used by AWS Lake Formation. Typically should be explicitly set as an empty list. + `AllowFullTableExternalDataAccess` – UTF-8 string (valid values: `True` \$1 `False`). Allows third-party engines to access data in Amazon S3 locations that are registered with Lake Formation. ## TargetRedshiftCatalog structure TargetRedshiftCatalog A structure that describes a target catalog for resource linking. **Fields** + `CatalogArn` – *Required:* UTF-8 string. The Amazon Resource Name (ARN) of the catalog resource. ## CatalogProperties structure CatalogProperties A structure that specifies data lake access properties and other custom properties. **Fields** + `DataLakeAccessProperties` – A [DataLakeAccessProperties](#aws-glue-api-catalog-catalogs-DataLakeAccessProperties) object. A `DataLakeAccessProperties` object that specifies properties to configure data lake access for your catalog resource in the AWS Glue Data Catalog. + `IcebergOptimizationProperties` – An [IcebergOptimizationProperties](#aws-glue-api-catalog-catalogs-IcebergOptimizationProperties) object. A structure that specifies Iceberg table optimization properties for the catalog. This includes configuration for compaction, retention, and orphan file deletion operations that can be applied to Iceberg tables in this catalog. + `CustomProperties` – A map array of key-value pairs. Each key is a Key string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Each value is a UTF-8 string, not more than 512000 bytes long. Additional key-value properties for the catalog, such as column statistics optimizations. ## CatalogPropertiesOutput structure CatalogPropertiesOutput Property attributes that include configuration properties for the catalog resource. **Fields** + `DataLakeAccessProperties` – A [DataLakeAccessPropertiesOutput](#aws-glue-api-catalog-catalogs-DataLakeAccessPropertiesOutput) object. A `DataLakeAccessProperties` object with input properties to configure data lake access for your catalog resource in the AWS Glue Data Catalog. + `IcebergOptimizationProperties` – An [IcebergOptimizationPropertiesOutput](#aws-glue-api-catalog-catalogs-IcebergOptimizationPropertiesOutput) object. An `IcebergOptimizationPropertiesOutput` object that specifies Iceberg table optimization settings for the catalog, including configurations for compaction, retention, and orphan file deletion operations. + `CustomProperties` – A map array of key-value pairs. Each key is a Key string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Each value is a UTF-8 string, not more than 512000 bytes long. Additional key-value properties for the catalog, such as column statistics optimizations. ## DataLakeAccessProperties structure DataLakeAccessProperties Input properties to configure data lake access for your catalog resource in the AWS Glue Data Catalog. **Fields** + `DataLakeAccess` – Boolean. Turns on or off data lake access for Apache Spark applications that access Amazon Redshift databases in the Data Catalog from any non-Redshift engine, such as Amazon Athena, Amazon EMR, or AWS Glue ETL. + `DataTransferRole` – UTF-8 string, matching the [Custom string pattern #53](aws-glue-api-common.md#regex_53). A role that will be assumed by AWS Glue for transferring data into/out of the staging bucket during a query. + `KmsKey` – UTF-8 string. An encryption key that will be used for the staging bucket that will be created along with the catalog. + `CatalogType` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Specifies a federated catalog type for the native catalog resource. The currently supported type is `aws:redshift`. ## IcebergOptimizationProperties structure IcebergOptimizationProperties A structure that specifies Iceberg table optimization properties for the catalog, including configurations for compaction, retention, and orphan file deletion operations. **Fields** + `RoleArn` – UTF-8 string, matching the [Custom string pattern #53](aws-glue-api-common.md#regex_53). The Amazon Resource Name (ARN) of the IAM role that will be assumed to perform Iceberg table optimization operations. + `Compaction` – A map array of key-value pairs. Each key is a Key string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Each value is a UTF-8 string, not more than 512000 bytes long. A map of key-value pairs that specify configuration parameters for Iceberg table compaction operations, which optimize the layout of data files to improve query performance. + `Retention` – A map array of key-value pairs. Each key is a Key string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Each value is a UTF-8 string, not more than 512000 bytes long. A map of key-value pairs that specify configuration parameters for Iceberg table retention operations, which manage the lifecycle of table snapshots to control storage costs. + `OrphanFileDeletion` – A map array of key-value pairs. Each key is a Key string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Each value is a UTF-8 string, not more than 512000 bytes long. A map of key-value pairs that specify configuration parameters for Iceberg orphan file deletion operations, which identify and remove files that are no longer referenced by the table metadata. ## DataLakeAccessPropertiesOutput structure DataLakeAccessPropertiesOutput The output properties of the data lake access configuration for your catalog resource in the AWS Glue Data Catalog. **Fields** + `DataLakeAccess` – Boolean. Turns on or off data lake access for Apache Spark applications that access Amazon Redshift databases in the Data Catalog. + `DataTransferRole` – UTF-8 string, matching the [Custom string pattern #53](aws-glue-api-common.md#regex_53). A role that will be assumed by AWS Glue for transferring data into/out of the staging bucket during a query. + `KmsKey` – UTF-8 string. An encryption key that will be used for the staging bucket that will be created along with the catalog. + `ManagedWorkgroupName` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The managed Redshift Serverless compute name that is created for your catalog resource. + `ManagedWorkgroupStatus` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The managed Redshift Serverless compute status. + `RedshiftDatabaseName` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The default Redshift database resource name in the managed compute. + `StatusMessage` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). A message that gives more detailed information about the managed workgroup status. + `CatalogType` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Specifies a federated catalog type for the native catalog resource. The currently supported type is `aws:redshift`. ## IcebergOptimizationPropertiesOutput structure IcebergOptimizationPropertiesOutput A structure that contains the output properties of Iceberg table optimization configuration for your catalog resource in the AWS Glue Data Catalog. **Fields** + `RoleArn` – UTF-8 string, matching the [Custom string pattern #53](aws-glue-api-common.md#regex_53). The Amazon Resource Name (ARN) of the IAM role that is used to perform Iceberg table optimization operations. + `Compaction` – A map array of key-value pairs. Each key is a Key string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Each value is a UTF-8 string, not more than 512000 bytes long. A map of key-value pairs that specify configuration parameters for Iceberg table compaction operations, which optimize the layout of data files to improve query performance. + `Retention` – A map array of key-value pairs. Each key is a Key string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Each value is a UTF-8 string, not more than 512000 bytes long. A map of key-value pairs that specify configuration parameters for Iceberg table retention operations, which manage the lifecycle of table snapshots to control storage costs. + `OrphanFileDeletion` – A map array of key-value pairs. Each key is a Key string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Each value is a UTF-8 string, not more than 512000 bytes long. A map of key-value pairs that specify configuration parameters for Iceberg orphan file deletion operations, which identify and remove files that are no longer referenced by the table metadata. + `LastUpdatedTime` – Timestamp. The timestamp when the Iceberg optimization properties were last updated. ## FederatedCatalog structure FederatedCatalog A catalog that points to an entity outside the AWS Glue Data Catalog. **Fields** + `Identifier` – UTF-8 string, not less than 1 or more than 512 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). A unique identifier for the federated catalog. + `ConnectionName` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the connection to an external data source, for example a Redshift-federated catalog. + `ConnectionType` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The type of connection used to access the federated catalog, specifying the protocol or method for connection to the external data source. ## Operations  —  operations  — + [CreateCatalog action (Python: create\$1catalog)](#aws-glue-api-catalog-catalogs-CreateCatalog) + [UpdateCatalog action (Python: update\$1catalog)](#aws-glue-api-catalog-catalogs-UpdateCatalog) + [DeleteCatalog action (Python: delete\$1catalog)](#aws-glue-api-catalog-catalogs-DeleteCatalog) + [GetCatalog action (Python: get\$1catalog)](#aws-glue-api-catalog-catalogs-GetCatalog) + [GetCatalogs action (Python: get\$1catalogs)](#aws-glue-api-catalog-catalogs-GetCatalogs) ## CreateCatalog action (Python: create\$1catalog) CreateCatalog (create\$1catalog) Creates a new catalog in the AWS Glue Data Catalog. **Request** + `Name` – *Required:* UTF-8 string, not less than 1 or more than 64 bytes long, matching the [Custom string pattern #25](aws-glue-api-common.md#regex_25). The name of the catalog to create. + `CatalogInput` – *Required:* A [CatalogInput](#aws-glue-api-catalog-catalogs-CatalogInput) object. A `CatalogInput` object that defines the metadata for the catalog. + `Tags` – A map array of key-value pairs, not more than 50 pairs. Each key is a UTF-8 string, not less than 1 or more than 128 bytes long. Each value is a UTF-8 string, not more than 256 bytes long. A map array of key-value pairs, not more than 50 pairs. Each key is a UTF-8 string, not less than 1 or more than 128 bytes long. Each value is a UTF-8 string, not more than 256 bytes long. The tags you assign to the catalog. **Response** + *No Response parameters.* **Errors** + `InvalidInputException` + `AlreadyExistsException` + `ResourceNumberLimitExceededException` + `InternalServiceException` + `OperationTimeoutException` + `GlueEncryptionException` + `ConcurrentModificationException` + `AccessDeniedException` + `EntityNotFoundException` + `FederatedResourceAlreadyExistsException` + `FederationSourceException` ## UpdateCatalog action (Python: update\$1catalog) UpdateCatalog (update\$1catalog) Updates an existing catalog's properties in the AWS Glue Data Catalog. **Request** + `CatalogId` – *Required:* Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the catalog. + `CatalogInput` – *Required:* A [CatalogInput](#aws-glue-api-catalog-catalogs-CatalogInput) object. A `CatalogInput` object specifying the new properties of an existing catalog. **Response** + *No Response parameters.* **Errors** + `EntityNotFoundException` + `InvalidInputException` + `InternalServiceException` + `OperationTimeoutException` + `GlueEncryptionException` + `ConcurrentModificationException` + `AccessDeniedException` + `FederationSourceException` ## DeleteCatalog action (Python: delete\$1catalog) DeleteCatalog (delete\$1catalog) Removes the specified catalog from the AWS Glue Data Catalog. After completing this operation, you no longer have access to the databases, tables (and all table versions and partitions that might belong to the tables) and the user-defined functions in the deleted catalog. AWS Glue deletes these "orphaned" resources asynchronously in a timely manner, at the discretion of the service. To ensure the immediate deletion of all related resources before calling the `DeleteCatalog` operation, use `DeleteTableVersion` (or `BatchDeleteTableVersion`), `DeletePartition` (or `BatchDeletePartition`), `DeleteTable` (or `BatchDeleteTable`), `DeleteUserDefinedFunction` and `DeleteDatabase` to delete any resources that belong to the catalog. **Request** + `CatalogId` – *Required:* Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the catalog. **Response** + *No Response parameters.* **Errors** + `EntityNotFoundException` + `InvalidInputException` + `InternalServiceException` + `OperationTimeoutException` + `GlueEncryptionException` + `ConcurrentModificationException` + `AccessDeniedException` + `FederationSourceException` ## GetCatalog action (Python: get\$1catalog) GetCatalog (get\$1catalog) The name of the Catalog to retrieve. This should be all lowercase. **Request** + `CatalogId` – *Required:* Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the parent catalog in which the catalog resides. If none is provided, the AWS Account Number is used by default. **Response** + `Catalog` – A [Catalog](#aws-glue-api-catalog-catalogs-Catalog) object. A `Catalog` object. The definition of the specified catalog in the AWS Glue Data Catalog. **Errors** + `InvalidInputException` + `EntityNotFoundException` + `InternalServiceException` + `OperationTimeoutException` + `GlueEncryptionException` + `AccessDeniedException` + `EntityNotFoundException` + `FederationSourceException` + `FederationSourceRetryableException` ## GetCatalogs action (Python: get\$1catalogs) GetCatalogs (get\$1catalogs) Retrieves all catalogs defined in a catalog in the AWS Glue Data Catalog. For a Redshift-federated catalog use case, this operation returns the list of catalogs mapped to Redshift databases in the Redshift namespace catalog. **Request** + `ParentCatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the parent catalog in which the catalog resides. If none is provided, the AWS Account Number is used by default. + `NextToken` – UTF-8 string. A continuation token, if this is a continuation call. + `MaxResults` – Number (integer), not less than 1 or more than 1000. The maximum number of catalogs to return in one response. + `Recursive` – Boolean. Whether to list all catalogs across the catalog hierarchy, starting from the `ParentCatalogId`. Defaults to `false` . When `true`, all catalog objects in the `ParentCatalogID` hierarchy are enumerated in the response. + `IncludeRoot` – Boolean. Whether to list the default catalog in the account and region in the response. Defaults to `false`. When `true` and `ParentCatalogId = NULL | AWS Account ID`, all catalogs and the default catalog are enumerated in the response. When the `ParentCatalogId` is not equal to null, and this attribute is passed as `false` or `true`, an `InvalidInputException` is thrown. **Response** + `CatalogList` – *Required:* An array of [Catalog](#aws-glue-api-catalog-catalogs-Catalog) objects. An array of `Catalog` objects. A list of `Catalog` objects from the specified parent catalog. + `NextToken` – UTF-8 string. A continuation token for paginating the returned list of tokens, returned if the current segment of the list is not the last. **Errors** + `InvalidInputException` + `InternalServiceException` + `OperationTimeoutException` + `GlueEncryptionException` + `AccessDeniedException` + `EntityNotFoundException` + `FederationSourceException` + `FederationSourceRetryableException` # Database API Databases The Database API describes database data types, and includes the API for creating, deleting, locating, updating, and listing databases. ## Data types  —  data types  — + [Database structure](#aws-glue-api-catalog-databases-Database) + [DatabaseInput structure](#aws-glue-api-catalog-databases-DatabaseInput) + [PrincipalPermissions structure](#aws-glue-api-catalog-databases-PrincipalPermissions) + [DataLakePrincipal structure](#aws-glue-api-catalog-databases-DataLakePrincipal) + [DatabaseIdentifier structure](#aws-glue-api-catalog-databases-DatabaseIdentifier) + [FederatedDatabase structure](#aws-glue-api-catalog-databases-FederatedDatabase) ## Database structure Database The `Database` object represents a logical grouping of tables that might reside in a Hive metastore or an RDBMS. **Fields** + `Name` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the database. For Hive compatibility, this is folded to lowercase when it is stored. + `Description` – Description string, not more than 2048 bytes long, matching the [URI address multi-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-uri). A description of the database. + `LocationUri` – Uniform resource identifier (uri), not less than 1 or more than 1024 bytes long, matching the [URI address multi-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-uri). The location of the database (for example, an HDFS path). + `Parameters` – A map array of key-value pairs. Each key is a Key string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Each value is a UTF-8 string, not more than 512000 bytes long. These key-value pairs define parameters and properties of the database. + `CreateTime` – Timestamp. The time at which the metadata database was created in the catalog. + `CreateTableDefaultPermissions` – An array of [PrincipalPermissions](#aws-glue-api-catalog-databases-PrincipalPermissions) objects. Creates a set of default permissions on the table for principals. Used by AWS Lake Formation. Not used in the normal course of AWS Glue operations. + `TargetDatabase` – A [DatabaseIdentifier](#aws-glue-api-catalog-databases-DatabaseIdentifier) object. A `DatabaseIdentifier` structure that describes a target database for resource linking. + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog in which the database resides. + `FederatedDatabase` – A [FederatedDatabase](#aws-glue-api-catalog-databases-FederatedDatabase) object. A `FederatedDatabase` structure that references an entity outside the AWS Glue Data Catalog. ## DatabaseInput structure DatabaseInput The structure used to create or update a database. **Fields** + `Name` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the database. For Hive compatibility, this is folded to lowercase when it is stored. + `Description` – Description string, not more than 2048 bytes long, matching the [URI address multi-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-uri). A description of the database. + `LocationUri` – Uniform resource identifier (uri), not less than 1 or more than 1024 bytes long, matching the [URI address multi-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-uri). The location of the database (for example, an HDFS path). + `Parameters` – A map array of key-value pairs. Each key is a Key string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Each value is a UTF-8 string, not more than 512000 bytes long. These key-value pairs define parameters and properties of the database. These key-value pairs define parameters and properties of the database. + `CreateTableDefaultPermissions` – An array of [PrincipalPermissions](#aws-glue-api-catalog-databases-PrincipalPermissions) objects. Creates a set of default permissions on the table for principals. Used by AWS Lake Formation. Not used in the normal course of AWS Glue operations. + `TargetDatabase` – A [DatabaseIdentifier](#aws-glue-api-catalog-databases-DatabaseIdentifier) object. A `DatabaseIdentifier` structure that describes a target database for resource linking. + `FederatedDatabase` – A [FederatedDatabase](#aws-glue-api-catalog-databases-FederatedDatabase) object. A `FederatedDatabase` structure that references an entity outside the AWS Glue Data Catalog. ## PrincipalPermissions structure PrincipalPermissions Permissions granted to a principal. **Fields** + `Principal` – A [DataLakePrincipal](#aws-glue-api-catalog-databases-DataLakePrincipal) object. The principal who is granted permissions. + `Permissions` – An array of UTF-8 strings. The permissions that are granted to the principal. ## DataLakePrincipal structure DataLakePrincipal The AWS Lake Formation principal. **Fields** + `DataLakePrincipalIdentifier` – UTF-8 string, not less than 1 or more than 255 bytes long. An identifier for the AWS Lake Formation principal. ## DatabaseIdentifier structure DatabaseIdentifier A structure that describes a target database for resource linking. **Fields** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog in which the database resides. + `DatabaseName` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the catalog database. + `Region` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Region of the target database. ## FederatedDatabase structure FederatedDatabase A database that points to an entity outside the AWS Glue Data Catalog. **Fields** + `Identifier` – UTF-8 string, not less than 1 or more than 512 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). A unique identifier for the federated database. + `ConnectionName` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the connection to the external metastore. + `ConnectionType` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The type of connection used to access the federated database, such as JDBC, ODBC, or other supported connection protocols. ## Operations  —  operations  — + [CreateDatabase action (Python: create\$1database)](#aws-glue-api-catalog-databases-CreateDatabase) + [UpdateDatabase action (Python: update\$1database)](#aws-glue-api-catalog-databases-UpdateDatabase) + [DeleteDatabase action (Python: delete\$1database)](#aws-glue-api-catalog-databases-DeleteDatabase) + [GetDatabase action (Python: get\$1database)](#aws-glue-api-catalog-databases-GetDatabase) + [GetDatabases action (Python: get\$1databases)](#aws-glue-api-catalog-databases-GetDatabases) ## CreateDatabase action (Python: create\$1database) CreateDatabase (create\$1database) Creates a new database in a Data Catalog. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog in which to create the database. If none is provided, the AWS account ID is used by default. + `DatabaseInput` – *Required:* A [DatabaseInput](#aws-glue-api-catalog-databases-DatabaseInput) object. The metadata for the database. + `Tags` – A map array of key-value pairs, not more than 50 pairs. Each key is a UTF-8 string, not less than 1 or more than 128 bytes long. Each value is a UTF-8 string, not more than 256 bytes long. The tags you assign to the database. **Response** + *No Response parameters.* **Errors** + `InvalidInputException` + `AlreadyExistsException` + `ResourceNumberLimitExceededException` + `InternalServiceException` + `OperationTimeoutException` + `GlueEncryptionException` + `ConcurrentModificationException` + `FederatedResourceAlreadyExistsException` + `FederationSourceException` + `FederationSourceRetryableException` ## UpdateDatabase action (Python: update\$1database) UpdateDatabase (update\$1database) Updates an existing database definition in a Data Catalog. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog in which the metadata database resides. If none is provided, the AWS account ID is used by default. + `Name` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the database to update in the catalog. For Hive compatibility, this is folded to lowercase. + `DatabaseInput` – *Required:* A [DatabaseInput](#aws-glue-api-catalog-databases-DatabaseInput) object. A `DatabaseInput` object specifying the new definition of the metadata database in the catalog. **Response** + *No Response parameters.* **Errors** + `EntityNotFoundException` + `InvalidInputException` + `InternalServiceException` + `OperationTimeoutException` + `GlueEncryptionException` + `ConcurrentModificationException` + `FederationSourceException` + `FederationSourceRetryableException` + `AlreadyExistsException` ## DeleteDatabase action (Python: delete\$1database) DeleteDatabase (delete\$1database) Removes a specified database from a Data Catalog. **Note** After completing this operation, you no longer have access to the tables (and all table versions and partitions that might belong to the tables) and the user-defined functions in the deleted database. AWS Glue deletes these "orphaned" resources asynchronously in a timely manner, at the discretion of the service. To ensure the immediate deletion of all related resources, before calling `DeleteDatabase`, use `DeleteTableVersion` or `BatchDeleteTableVersion`, `DeletePartition` or `BatchDeletePartition`, `DeleteUserDefinedFunction`, and `DeleteTable` or `BatchDeleteTable`, to delete any resources that belong to the database. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog in which the database resides. If none is provided, the AWS account ID is used by default. + `Name` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the database to delete. For Hive compatibility, this must be all lowercase. **Response** + *No Response parameters.* **Errors** + `EntityNotFoundException` + `InvalidInputException` + `InternalServiceException` + `OperationTimeoutException` + `ConcurrentModificationException` + `FederationSourceException` + `FederationSourceRetryableException` ## GetDatabase action (Python: get\$1database) GetDatabase (get\$1database) Retrieves the definition of a specified database. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog in which the database resides. If none is provided, the AWS account ID is used by default. + `Name` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the database to retrieve. For Hive compatibility, this should be all lowercase. **Response** + `Database` – A [Database](#aws-glue-api-catalog-databases-Database) object. The definition of the specified database in the Data Catalog. **Errors** + `InvalidInputException` + `EntityNotFoundException` + `InternalServiceException` + `OperationTimeoutException` + `GlueEncryptionException` + `FederationSourceException` + `FederationSourceRetryableException` ## GetDatabases action (Python: get\$1databases) GetDatabases (get\$1databases) Retrieves all databases defined in a given Data Catalog. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog from which to retrieve `Databases`. If none is provided, the AWS account ID is used by default. + `NextToken` – UTF-8 string. A continuation token, if this is a continuation call. + `MaxResults` – Number (integer), not less than 1 or more than 100. The maximum number of databases to return in one response. + `ResourceShareType` – UTF-8 string (valid values: `FOREIGN` \$1 `ALL` \$1 `FEDERATED`). Allows you to specify that you want to list the databases shared with your account. The allowable values are `FEDERATED`, `FOREIGN` or `ALL`. + If set to `FEDERATED`, will list the federated databases (referencing an external entity) shared with your account. + If set to `FOREIGN`, will list the databases shared with your account. + If set to `ALL`, will list the databases shared with your account, as well as the databases in yor local account. + `AttributesToGet` – An array of UTF-8 strings. Specifies the database fields returned by the `GetDatabases` call. This parameter doesn't accept an empty list. The request must include the `NAME`. **Response** + `DatabaseList` – *Required:* An array of [Database](#aws-glue-api-catalog-databases-Database) objects. A list of `Database` objects from the specified catalog. + `NextToken` – UTF-8 string. A continuation token for paginating the returned list of tokens, returned if the current segment of the list is not the last. **Errors** + `InvalidInputException` + `InternalServiceException` + `OperationTimeoutException` + `GlueEncryptionException` + `EntityNotFoundException` + `FederationSourceException` + `FederationSourceRetryableException` # Table API Tables The Table API describes data types and operations associated with tables. ## Data types  —  data types  — + [Table structure](#aws-glue-api-catalog-tables-Table) + [TableInput structure](#aws-glue-api-catalog-tables-TableInput) + [FederatedTable structure](#aws-glue-api-catalog-tables-FederatedTable) + [Column structure](#aws-glue-api-catalog-tables-Column) + [StorageDescriptor structure](#aws-glue-api-catalog-tables-StorageDescriptor) + [SchemaReference structure](#aws-glue-api-catalog-tables-SchemaReference) + [SerDeInfo structure](#aws-glue-api-catalog-tables-SerDeInfo) + [Order structure](#aws-glue-api-catalog-tables-Order) + [SkewedInfo structure](#aws-glue-api-catalog-tables-SkewedInfo) + [TableVersion structure](#aws-glue-api-catalog-tables-TableVersion) + [TableError structure](#aws-glue-api-catalog-tables-TableError) + [TableVersionError structure](#aws-glue-api-catalog-tables-TableVersionError) + [SortCriterion structure](#aws-glue-api-catalog-tables-SortCriterion) + [TableIdentifier structure](#aws-glue-api-catalog-tables-TableIdentifier) + [KeySchemaElement structure](#aws-glue-api-catalog-tables-KeySchemaElement) + [PartitionIndex structure](#aws-glue-api-catalog-tables-PartitionIndex) + [PartitionIndexDescriptor structure](#aws-glue-api-catalog-tables-PartitionIndexDescriptor) + [BackfillError structure](#aws-glue-api-catalog-tables-BackfillError) + [IcebergInput structure](#aws-glue-api-catalog-tables-IcebergInput) + [OpenTableFormatInput structure](#aws-glue-api-catalog-tables-OpenTableFormatInput) + [ViewDefinition structure](#aws-glue-api-catalog-tables-ViewDefinition) + [ViewDefinitionInput structure](#aws-glue-api-catalog-tables-ViewDefinitionInput) + [ViewRepresentation structure](#aws-glue-api-catalog-tables-ViewRepresentation) + [ViewRepresentationInput structure](#aws-glue-api-catalog-tables-ViewRepresentationInput) + [UpdateOpenTableFormatInput structure](#aws-glue-api-catalog-tables-UpdateOpenTableFormatInput) + [UpdateIcebergInput structure](#aws-glue-api-catalog-tables-UpdateIcebergInput) + [CreateIcebergTableInput structure](#aws-glue-api-catalog-tables-CreateIcebergTableInput) + [UpdateIcebergTableInput structure](#aws-glue-api-catalog-tables-UpdateIcebergTableInput) + [IcebergSortOrder structure](#aws-glue-api-catalog-tables-IcebergSortOrder) + [IcebergSortField structure](#aws-glue-api-catalog-tables-IcebergSortField) + [IcebergPartitionSpec structure](#aws-glue-api-catalog-tables-IcebergPartitionSpec) + [IcebergPartitionField structure](#aws-glue-api-catalog-tables-IcebergPartitionField) + [IcebergSchema structure](#aws-glue-api-catalog-tables-IcebergSchema) + [IcebergStructField structure](#aws-glue-api-catalog-tables-IcebergStructField) + [IcebergTableUpdate structure](#aws-glue-api-catalog-tables-IcebergTableUpdate) + [AuditContext structure](#aws-glue-api-catalog-tables-AuditContext) ## Table structure Table Represents a collection of related data organized in columns and rows. **Fields** + `Name` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The table name. For Hive compatibility, this must be entirely lowercase. + `DatabaseName` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the database where the table metadata resides. For Hive compatibility, this must be all lowercase. + `Description` – Description string, not more than 2048 bytes long, matching the [URI address multi-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-uri). A description of the table. + `Owner` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The owner of the table. + `CreateTime` – Timestamp. The time when the table definition was created in the Data Catalog. + `UpdateTime` – Timestamp. The last time that the table was updated. + `LastAccessTime` – Timestamp. The last time that the table was accessed. This is usually taken from HDFS, and might not be reliable. + `LastAnalyzedTime` – Timestamp. The last time that column statistics were computed for this table. + `Retention` – Number (integer), not more than None. The retention time for this table. + `StorageDescriptor` – A [StorageDescriptor](aws-glue-api-catalog-partitions.md#aws-glue-api-catalog-partitions-StorageDescriptor) object. A storage descriptor containing information about the physical storage of this table. + `PartitionKeys` – An array of [Column](#aws-glue-api-catalog-tables-Column) objects. A list of columns by which the table is partitioned. Only primitive types are supported as partition keys. When you create a table used by Amazon Athena, and you do not specify any `partitionKeys`, you must at least set the value of `partitionKeys` to an empty list. For example: `"PartitionKeys": []` + `ViewOriginalText` – UTF-8 string, not more than 409600 bytes long. Included for Apache Hive compatibility. Not used in the normal course of AWS Glue operations. If the table is a `VIRTUAL_VIEW`, certain Athena configuration encoded in base64. + `ViewExpandedText` – UTF-8 string, not more than 409600 bytes long. Included for Apache Hive compatibility. Not used in the normal course of AWS Glue operations. + `TableType` – UTF-8 string, not more than 255 bytes long. The type of this table. AWS Glue will create tables with the `EXTERNAL_TABLE` type. Other services, such as Athena, may create tables with additional table types. AWS Glue related table types: EXTERNAL\$1TABLE Hive compatible attribute - indicates a non-Hive managed table. GOVERNED Used by AWS Lake Formation. The AWS Glue Data Catalog understands `GOVERNED`. + `Parameters` – A map array of key-value pairs. Each key is a Key string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Each value is a UTF-8 string, not more than 512000 bytes long. These key-value pairs define properties associated with the table. + `CreatedBy` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The person or entity who created the table. + `IsRegisteredWithLakeFormation` – Boolean. Indicates whether the table has been registered with AWS Lake Formation. + `TargetTable` – A [TableIdentifier](#aws-glue-api-catalog-tables-TableIdentifier) object. A `TableIdentifier` structure that describes a target table for resource linking. + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog in which the table resides. + `VersionId` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the table version. + `FederatedTable` – A [FederatedTable](#aws-glue-api-catalog-tables-FederatedTable) object. A `FederatedTable` structure that references an entity outside the AWS Glue Data Catalog. + `ViewDefinition` – A [ViewDefinition](#aws-glue-api-catalog-tables-ViewDefinition) object. A structure that contains all the information that defines the view, including the dialect or dialects for the view, and the query. + `IsMultiDialectView` – Boolean. Specifies whether the view supports the SQL dialects of one or more different query engines and can therefore be read by those engines. ## TableInput structure TableInput A structure used to define a table. **Fields** + `Name` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The table name. For Hive compatibility, this is folded to lowercase when it is stored. + `Description` – Description string, not more than 2048 bytes long, matching the [URI address multi-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-uri). A description of the table. + `Owner` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The table owner. Included for Apache Hive compatibility. Not used in the normal course of AWS Glue operations. + `LastAccessTime` – Timestamp. The last time that the table was accessed. + `LastAnalyzedTime` – Timestamp. The last time that column statistics were computed for this table. + `Retention` – Number (integer), not more than None. The retention time for this table. + `StorageDescriptor` – A [StorageDescriptor](aws-glue-api-catalog-partitions.md#aws-glue-api-catalog-partitions-StorageDescriptor) object. A storage descriptor containing information about the physical storage of this table. + `PartitionKeys` – An array of [Column](#aws-glue-api-catalog-tables-Column) objects. A list of columns by which the table is partitioned. Only primitive types are supported as partition keys. When you create a table used by Amazon Athena, and you do not specify any `partitionKeys`, you must at least set the value of `partitionKeys` to an empty list. For example: `"PartitionKeys": []` + `ViewOriginalText` – UTF-8 string, not more than 409600 bytes long. Included for Apache Hive compatibility. Not used in the normal course of AWS Glue operations. If the table is a `VIRTUAL_VIEW`, certain Athena configuration encoded in base64. + `ViewExpandedText` – UTF-8 string, not more than 409600 bytes long. Included for Apache Hive compatibility. Not used in the normal course of AWS Glue operations. + `TableType` – UTF-8 string, not more than 255 bytes long. The type of this table. AWS Glue will create tables with the `EXTERNAL_TABLE` type. Other services, such as Athena, may create tables with additional table types. AWS Glue related table types: EXTERNAL\$1TABLE Hive compatible attribute - indicates a non-Hive managed table. GOVERNED Used by AWS Lake Formation. The AWS Glue Data Catalog understands `GOVERNED`. + `Parameters` – A map array of key-value pairs. Each key is a Key string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Each value is a UTF-8 string, not more than 512000 bytes long. These key-value pairs define properties associated with the table. + `TargetTable` – A [TableIdentifier](#aws-glue-api-catalog-tables-TableIdentifier) object. A `TableIdentifier` structure that describes a target table for resource linking. + `ViewDefinition` – A [ViewDefinitionInput](#aws-glue-api-catalog-tables-ViewDefinitionInput) object. A structure that contains all the information that defines the view, including the dialect or dialects for the view, and the query. ## FederatedTable structure FederatedTable A table that points to an entity outside the AWS Glue Data Catalog. **Fields** + `Identifier` – UTF-8 string, not less than 1 or more than 512 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). A unique identifier for the federated table. + `DatabaseIdentifier` – UTF-8 string, not less than 1 or more than 512 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). A unique identifier for the federated database. + `ConnectionName` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the connection to the external metastore. + `ConnectionType` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The type of connection used to access the federated table, specifying the protocol or method for connecting to the external data source. ## Column structure Column A column in a `Table`. **Fields** + `Name` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the `Column`. + `Type` – UTF-8 string, not more than 131072 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The data type of the `Column`. + `Comment` – Comment string, not more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). A free-form text comment. + `Parameters` – A map array of key-value pairs. Each key is a Key string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Each value is a UTF-8 string, not more than 512000 bytes long. These key-value pairs define properties associated with the column. ## StorageDescriptor structure StorageDescriptor Describes the physical storage of table data. **Fields** + `Columns` – An array of [Column](#aws-glue-api-catalog-tables-Column) objects. A list of the `Columns` in the table. + `Location` – Location string, not more than 2056 bytes long, matching the [URI address multi-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-uri). The physical location of the table. By default, this takes the form of the warehouse location, followed by the database location in the warehouse, followed by the table name. + `AdditionalLocations` – An array of UTF-8 strings. A list of locations that point to the path where a Delta table is located. + `InputFormat` – Format string, not more than 128 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The input format: `SequenceFileInputFormat` (binary), or `TextInputFormat`, or a custom format. + `OutputFormat` – Format string, not more than 128 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The output format: `SequenceFileOutputFormat` (binary), or `IgnoreKeyTextOutputFormat`, or a custom format. + `Compressed` – Boolean. `True` if the data in the table is compressed, or `False` if not. + `NumberOfBuckets` – Number (integer). Must be specified if the table contains any dimension columns. + `SerdeInfo` – A [SerDeInfo](aws-glue-api-catalog-partitions.md#aws-glue-api-catalog-partitions-SerDeInfo) object. The serialization/deserialization (SerDe) information. + `BucketColumns` – An array of UTF-8 strings. A list of reducer grouping columns, clustering columns, and bucketing columns in the table. + `SortColumns` – An array of [Order](#aws-glue-api-catalog-tables-Order) objects. A list specifying the sort order of each bucket in the table. + `Parameters` – A map array of key-value pairs. Each key is a Key string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Each value is a UTF-8 string, not more than 512000 bytes long. The user-supplied properties in key-value form. + `SkewedInfo` – A [SkewedInfo](aws-glue-api-catalog-partitions.md#aws-glue-api-catalog-partitions-SkewedInfo) object. The information about values that appear frequently in a column (skewed values). + `StoredAsSubDirectories` – Boolean. `True` if the table data is stored in subdirectories, or `False` if not. + `SchemaReference` – A [SchemaReference](aws-glue-api-catalog-partitions.md#aws-glue-api-catalog-partitions-SchemaReference) object. An object that references a schema stored in the AWS Glue Schema Registry. When creating a table, you can pass an empty list of columns for the schema, and instead use a schema reference. ## SchemaReference structure SchemaReference An object that references a schema stored in the AWS Glue Schema Registry. **Fields** + `SchemaId` – A [SchemaId](aws-glue-api-schema-registry-api.md#aws-glue-api-schema-registry-api-SchemaId) object. A structure that contains schema identity fields. Either this or the `SchemaVersionId` has to be provided. + `SchemaVersionId` – UTF-8 string, not less than 36 or more than 36 bytes long, matching the [Custom string pattern #45](aws-glue-api-common.md#regex_45). The unique ID assigned to a version of the schema. Either this or the `SchemaId` has to be provided. + `SchemaVersionNumber` – Number (long), not less than 1 or more than 100000. The version number of the schema. ## SerDeInfo structure SerDeInfo Information about a serialization/deserialization program (SerDe) that serves as an extractor and loader. **Fields** + `Name` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Name of the SerDe. + `SerializationLibrary` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Usually the class that implements the SerDe. An example is `org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe`. + `Parameters` – A map array of key-value pairs. Each key is a Key string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Each value is a UTF-8 string, not more than 512000 bytes long. These key-value pairs define initialization parameters for the SerDe. ## Order structure Order Specifies the sort order of a sorted column. **Fields** + `Column` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the column. + `SortOrder` – *Required:* Number (integer), not more than 1. Indicates that the column is sorted in ascending order (`== 1`), or in descending order (`==0`). ## SkewedInfo structure SkewedInfo Specifies skewed values in a table. Skewed values are those that occur with very high frequency. **Fields** + `SkewedColumnNames` – An array of UTF-8 strings. A list of names of columns that contain skewed values. + `SkewedColumnValues` – An array of UTF-8 strings. A list of values that appear so frequently as to be considered skewed. + `SkewedColumnValueLocationMaps` – A map array of key-value pairs. Each key is a UTF-8 string. Each value is a UTF-8 string. A mapping of skewed values to the columns that contain them. ## TableVersion structure TableVersion Specifies a version of a table. **Fields** + `Table` – A [Table](#aws-glue-api-catalog-tables-Table) object. The table in question. + `VersionId` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID value that identifies this table version. A `VersionId` is a string representation of an integer. Each version is incremented by 1. ## TableError structure TableError An error record for table operations. **Fields** + `TableName` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the table. For Hive compatibility, this must be entirely lowercase. + `ErrorDetail` – An [ErrorDetail](aws-glue-api-common.md#aws-glue-api-common-ErrorDetail) object. The details about the error. ## TableVersionError structure TableVersionError An error record for table-version operations. **Fields** + `TableName` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the table in question. + `VersionId` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID value of the version in question. A `VersionID` is a string representation of an integer. Each version is incremented by 1. + `ErrorDetail` – An [ErrorDetail](aws-glue-api-common.md#aws-glue-api-common-ErrorDetail) object. The details about the error. ## SortCriterion structure SortCriterion Specifies a field to sort by and a sort order. **Fields** + `FieldName` – Value string, not less than 1 or more than 1024 bytes long. The name of the field on which to sort. + `Sort` – UTF-8 string (valid values: `ASC="ASCENDING"` \$1 `DESC="DESCENDING"`). An ascending or descending sort. ## TableIdentifier structure TableIdentifier A structure that describes a target table for resource linking. **Fields** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog in which the table resides. + `DatabaseName` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the catalog database that contains the target table. + `Name` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the target table. + `Region` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Region of the target table. ## KeySchemaElement structure KeySchemaElement A partition key pair consisting of a name and a type. **Fields** + `Name` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of a partition key. + `Type` – *Required:* UTF-8 string, not more than 131072 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The type of a partition key. ## PartitionIndex structure PartitionIndex A structure for a partition index. **Fields** + `Keys` – *Required:* An array of UTF-8 strings, at least 1 string. The keys for the partition index. + `IndexName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the partition index. ## PartitionIndexDescriptor structure PartitionIndexDescriptor A descriptor for a partition index in a table. **Fields** + `IndexName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the partition index. + `Keys` – *Required:* An array of [KeySchemaElement](#aws-glue-api-catalog-tables-KeySchemaElement) objects, at least 1 structure. A list of one or more keys, as `KeySchemaElement` structures, for the partition index. + `IndexStatus` – *Required:* UTF-8 string (valid values: `CREATING` \$1 `ACTIVE` \$1 `DELETING` \$1 `FAILED`). The status of the partition index. The possible statuses are: + CREATING: The index is being created. When an index is in a CREATING state, the index or its table cannot be deleted. + ACTIVE: The index creation succeeds. + FAILED: The index creation fails. + DELETING: The index is deleted from the list of indexes. + `BackfillErrors` – An array of [BackfillError](#aws-glue-api-catalog-tables-BackfillError) objects. A list of errors that can occur when registering partition indexes for an existing table. ## BackfillError structure BackfillError A list of errors that can occur when registering partition indexes for an existing table. These errors give the details about why an index registration failed and provide a limited number of partitions in the response, so that you can fix the partitions at fault and try registering the index again. The most common set of errors that can occur are categorized as follows: + EncryptedPartitionError: The partitions are encrypted. + InvalidPartitionTypeDataError: The partition value doesn't match the data type for that partition column. + MissingPartitionValueError: The partitions are encrypted. + UnsupportedPartitionCharacterError: Characters inside the partition value are not supported. For example: U\$10000 , U\$10001, U\$10002. + InternalError: Any error which does not belong to other error codes. **Fields** + `Code` – UTF-8 string (valid values: `ENCRYPTED_PARTITION_ERROR` \$1 `INTERNAL_ERROR` \$1 `INVALID_PARTITION_TYPE_DATA_ERROR` \$1 `MISSING_PARTITION_VALUE_ERROR` \$1 `UNSUPPORTED_PARTITION_CHARACTER_ERROR`). The error code for an error that occurred when registering partition indexes for an existing table. + `Partitions` – An array of [PartitionValueList](aws-glue-api-catalog-partitions.md#aws-glue-api-catalog-partitions-PartitionValueList) objects. A list of a limited number of partitions in the response. ## IcebergInput structure IcebergInput A structure that defines an Apache Iceberg metadata table to create in the catalog. **Fields** + `MetadataOperation` – *Required:* UTF-8 string (valid values: `CREATE`). A required metadata operation. Can only be set to `CREATE`. + `Version` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The table version for the Iceberg table. Defaults to 2. + `CreateIcebergTableInput` – A [CreateIcebergTableInput](#aws-glue-api-catalog-tables-CreateIcebergTableInput) object. The configuration parameters required to create a new Iceberg table in the AWS Glue Data Catalog, including table properties and metadata specifications. ## OpenTableFormatInput structure OpenTableFormatInput A structure representing an open format table. **Fields** + `IcebergInput` – An [IcebergInput](#aws-glue-api-catalog-tables-IcebergInput) object. Specifies an `IcebergInput` structure that defines an Apache Iceberg metadata table. ## ViewDefinition structure ViewDefinition A structure containing details for representations. **Fields** + `IsProtected` – Boolean. You can set this flag as true to instruct the engine not to push user-provided operations into the logical plan of the view during query planning. However, setting this flag does not guarantee that the engine will comply. Refer to the engine's documentation to understand the guarantees provided, if any. + `Definer` – UTF-8 string, not less than 20 or more than 2048 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The definer of a view in SQL. + `SubObjects` – An array of UTF-8 strings, not more than 10 strings. A list of table Amazon Resource Names (ARNs). + `Representations` – An array of [ViewRepresentation](#aws-glue-api-catalog-tables-ViewRepresentation) objects, not less than 1 or more than 1000 structures. A list of representations. ## ViewDefinitionInput structure ViewDefinitionInput A structure containing details for creating or updating an AWS Glue view. **Fields** + `IsProtected` – Boolean. You can set this flag as true to instruct the engine not to push user-provided operations into the logical plan of the view during query planning. However, setting this flag does not guarantee that the engine will comply. Refer to the engine's documentation to understand the guarantees provided, if any. + `Definer` – UTF-8 string, not less than 20 or more than 2048 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The definer of a view in SQL. + `Representations` – An array of [ViewRepresentationInput](#aws-glue-api-catalog-tables-ViewRepresentationInput) objects, not less than 1 or more than 10 structures. A list of structures that contains the dialect of the view, and the query that defines the view. + `SubObjects` – An array of UTF-8 strings, not more than 10 strings. A list of base table ARNs that make up the view. ## ViewRepresentation structure ViewRepresentation A structure that contains the dialect of the view, and the query that defines the view. **Fields** + `Dialect` – UTF-8 string (valid values: `REDSHIFT` \$1 `ATHENA` \$1 `SPARK`). The dialect of the query engine. + `DialectVersion` – UTF-8 string, not less than 1 or more than 255 bytes long. The version of the dialect of the query engine. For example, 3.0.0. + `ViewOriginalText` – UTF-8 string, not more than 409600 bytes long. The `SELECT` query provided by the customer during `CREATE VIEW DDL`. This SQL is not used during a query on a view (`ViewExpandedText` is used instead). `ViewOriginalText` is used for cases like `SHOW CREATE VIEW` where users want to see the original DDL command that created the view. + `ViewExpandedText` – UTF-8 string, not more than 409600 bytes long. The expanded SQL for the view. This SQL is used by engines while processing a query on a view. Engines may perform operations during view creation to transform `ViewOriginalText` to `ViewExpandedText`. For example: + Fully qualified identifiers: `SELECT * from table1 -> SELECT * from db1.table1` + `ValidationConnection` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the connection to be used to validate the specific representation of the view. + `IsStale` – Boolean. Dialects marked as stale are no longer valid and must be updated before they can be queried in their respective query engines. ## ViewRepresentationInput structure ViewRepresentationInput A structure containing details of a representation to update or create a Lake Formation view. **Fields** + `Dialect` – UTF-8 string (valid values: `REDSHIFT` \$1 `ATHENA` \$1 `SPARK`). A parameter that specifies the engine type of a specific representation. + `DialectVersion` – UTF-8 string, not less than 1 or more than 255 bytes long. A parameter that specifies the version of the engine of a specific representation. + `ViewOriginalText` – UTF-8 string, not more than 409600 bytes long. A string that represents the original SQL query that describes the view. + `ValidationConnection` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the connection to be used to validate the specific representation of the view. + `ViewExpandedText` – UTF-8 string, not more than 409600 bytes long. A string that represents the SQL query that describes the view with expanded resource ARNs ## UpdateOpenTableFormatInput structure UpdateOpenTableFormatInput Input parameters for updating open table format tables in AWS GlueData Catalog, serving as a wrapper for format-specific update operations such as Apache Iceberg. **Fields** + `UpdateIcebergInput` – An [UpdateIcebergInput](#aws-glue-api-catalog-tables-UpdateIcebergInput) object. Apache Iceberg-specific update parameters that define the table modifications to be applied, including schema changes, partition specifications, and table properties. ## UpdateIcebergInput structure UpdateIcebergInput Input parameters specific to updating Apache Iceberg tables in AWS Glue Data Catalog, containing the update operations to be applied to an existing Iceberg table. **Fields** + `UpdateIcebergTableInput` – *Required:* An [UpdateIcebergTableInput](#aws-glue-api-catalog-tables-UpdateIcebergTableInput) object. The specific update operations to be applied to the Iceberg table, containing a list of updates that define the new state of the table including schema, partitions, and properties. ## CreateIcebergTableInput structure CreateIcebergTableInput The configuration parameters required to create a new Iceberg table in the AWS Glue Data Catalog, including table properties and metadata specifications. **Fields** + `Location` – *Required:* Location string, not more than 2056 bytes long, matching the [URI address multi-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-uri). The S3 location where the Iceberg table data will be stored. + `Schema` – *Required:* An [IcebergSchema](#aws-glue-api-catalog-tables-IcebergSchema) object. The schema definition that specifies the structure, field types, and metadata for the Iceberg table. + `PartitionSpec` – An [IcebergPartitionSpec](#aws-glue-api-catalog-tables-IcebergPartitionSpec) object. The partitioning specification that defines how the Iceberg table data will be organized and partitioned for optimal query performance. + `WriteOrder` – An [IcebergSortOrder](#aws-glue-api-catalog-tables-IcebergSortOrder) object. The sort order specification that defines how data should be ordered within each partition to optimize query performance. + `Properties` – Key-value pairs of additional table properties and configuration settings for the Iceberg table. ## UpdateIcebergTableInput structure UpdateIcebergTableInput Contains the update operations to be applied to an existing Iceberg table inAWS Glue Data Catalog, defining the new state of the table metadata. **Fields** + `Updates` – *Required:* An array of [IcebergTableUpdate](#aws-glue-api-catalog-tables-IcebergTableUpdate) objects. The list of table update operations that specify the changes to be made to the Iceberg table, including schema modifications, partition specifications, and table properties. ## IcebergSortOrder structure IcebergSortOrder Defines the sort order specification for an Iceberg table, determining how data should be ordered within partitions to optimize query performance. **Fields** + `OrderId` – *Required:* Number (integer). The unique identifier for this sort order specification within the Iceberg table's metadata. + `Fields` – *Required:* An array of [IcebergSortField](#aws-glue-api-catalog-tables-IcebergSortField) objects. The list of fields and their sort directions that define the ordering criteria for the Iceberg table data. ## IcebergSortField structure IcebergSortField Defines a single field within an Iceberg sort order specification, including the source field, transformation, sort direction, and null value ordering. **Fields** + `SourceId` – *Required:* Number (integer). The identifier of the source field from the table schema that this sort field is based on. + `Transform` – *Required:* UTF-8 string. The transformation function applied to the source field before sorting, such as identity, bucket, or truncate. + `Direction` – *Required:* UTF-8 string (valid values: `asc="ASC"` \$1 `desc="DESC"`). The sort direction for this field, either ascending or descending. + `NullOrder` – *Required:* UTF-8 string (valid values: `nulls-first="NULLS_FIRST"` \$1 `nulls-last="NULLS_LAST"`). The ordering behavior for null values in this field, specifying whether nulls should appear first or last in the sort order. ## IcebergPartitionSpec structure IcebergPartitionSpec Defines the partitioning specification for an Iceberg table, determining how table data will be organized and partitioned for optimal query performance. **Fields** + `Fields` – *Required:* An array of [IcebergPartitionField](#aws-glue-api-catalog-tables-IcebergPartitionField) objects. The list of partition fields that define how the table data should be partitioned, including source fields and their transformations. + `SpecId` – Number (integer). The unique identifier for this partition specification within the Iceberg table's metadata history. ## IcebergPartitionField structure IcebergPartitionField Defines a single partition field within an Iceberg partition specification, including the source field, transformation function, partition name, and unique identifier. **Fields** + `SourceId` – *Required:* Number (integer). The identifier of the source field from the table schema that this partition field is based on. + `Transform` – *Required:* UTF-8 string. The transformation function applied to the source field to create the partition, such as identity, bucket, truncate, year, month, day, or hour. + `Name` – *Required:* UTF-8 string, not less than 1 or more than 1024 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the partition field as it will appear in the partitioned table structure. + `FieldId` – Number (integer). The unique identifier assigned to this partition field within the Iceberg table's partition specification. ## IcebergSchema structure IcebergSchema Defines the schema structure for an Iceberg table, including field definitions, data types, and schema metadata. **Fields** + `SchemaId` – Number (integer). The unique identifier for this schema version within the Iceberg table's schema evolution history. + `IdentifierFieldIds` – An array of signed 32-bit integers. The list of field identifiers that uniquely identify records in the table, used for row-level operations and deduplication. + `Type` – UTF-8 string (valid values: `struct="STRUCT"`). The root type of the schema structure, typically "struct" for Iceberg table schemas. + `Fields` – *Required:* An array of [IcebergStructField](#aws-glue-api-catalog-tables-IcebergStructField) objects. The list of field definitions that make up the table schema, including field names, types, and metadata. ## IcebergStructField structure IcebergStructField Defines a single field within an Iceberg table schema, including its identifier, name, data type, nullability, and documentation. **Fields** + `Id` – *Required:* Number (integer). The unique identifier assigned to this field within the Iceberg table schema, used for schema evolution and field tracking. + `Name` – *Required:* UTF-8 string, not less than 1 or more than 1024 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the field as it appears in the table schema and query operations. + `Type` – *Required:* An empty-structure named `IcebergDocument`. The data type definition for this field, specifying the structure and format of the data it contains. + `Required` – *Required:* Boolean. Indicates whether this field is required (non-nullable) or optional (nullable) in the table schema. + `Doc` – Comment string, not more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Optional documentation or description text that provides additional context about the purpose and usage of this field. ## IcebergTableUpdate structure IcebergTableUpdate Defines a complete set of updates to be applied to an Iceberg table, including schema changes, partitioning modifications, sort order adjustments, location updates, and property changes. **Fields** + `Schema` – *Required:* An [IcebergSchema](#aws-glue-api-catalog-tables-IcebergSchema) object. The updated schema definition for the Iceberg table, specifying any changes to field structure, data types, or schema metadata. + `PartitionSpec` – An [IcebergPartitionSpec](#aws-glue-api-catalog-tables-IcebergPartitionSpec) object. The updated partitioning specification that defines how the table data should be reorganized and partitioned. + `SortOrder` – An [IcebergSortOrder](#aws-glue-api-catalog-tables-IcebergSortOrder) object. The updated sort order specification that defines how data should be ordered within partitions for optimal query performance. + `Location` – *Required:* Location string, not more than 2056 bytes long, matching the [URI address multi-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-uri). The updated S3 location where the Iceberg table data will be stored. + `Properties` – Updated key-value pairs of table properties and configuration settings for the Iceberg table. ## AuditContext structure AuditContext A structure containing the Lake Formation audit context. **Fields** + `AdditionalAuditContext` – UTF-8 string, not more than 2048 bytes long. A string containing the additional audit context information. + `RequestedColumns` – An array of UTF-8 strings. The requested columns for audit. + `AllColumnsRequested` – Boolean. All columns request for audit. ## Operations  —  operations  — + [CreateTable action (Python: create\$1table)](#aws-glue-api-catalog-tables-CreateTable) + [UpdateTable action (Python: update\$1table)](#aws-glue-api-catalog-tables-UpdateTable) + [DeleteTable action (Python: delete\$1table)](#aws-glue-api-catalog-tables-DeleteTable) + [BatchDeleteTable action (Python: batch\$1delete\$1table)](#aws-glue-api-catalog-tables-BatchDeleteTable) + [GetTable action (Python: get\$1table)](#aws-glue-api-catalog-tables-GetTable) + [GetTables action (Python: get\$1tables)](#aws-glue-api-catalog-tables-GetTables) + [GetTableVersion action (Python: get\$1table\$1version)](#aws-glue-api-catalog-tables-GetTableVersion) + [GetTableVersions action (Python: get\$1table\$1versions)](#aws-glue-api-catalog-tables-GetTableVersions) + [DeleteTableVersion action (Python: delete\$1table\$1version)](#aws-glue-api-catalog-tables-DeleteTableVersion) + [BatchDeleteTableVersion action (Python: batch\$1delete\$1table\$1version)](#aws-glue-api-catalog-tables-BatchDeleteTableVersion) + [SearchTables action (Python: search\$1tables)](#aws-glue-api-catalog-tables-SearchTables) + [GetPartitionIndexes action (Python: get\$1partition\$1indexes)](#aws-glue-api-catalog-tables-GetPartitionIndexes) + [CreatePartitionIndex action (Python: create\$1partition\$1index)](#aws-glue-api-catalog-tables-CreatePartitionIndex) + [DeletePartitionIndex action (Python: delete\$1partition\$1index)](#aws-glue-api-catalog-tables-DeletePartitionIndex) + [GetColumnStatisticsForTable action (Python: get\$1column\$1statistics\$1for\$1table)](#aws-glue-api-catalog-tables-GetColumnStatisticsForTable) + [UpdateColumnStatisticsForTable action (Python: update\$1column\$1statistics\$1for\$1table)](#aws-glue-api-catalog-tables-UpdateColumnStatisticsForTable) + [DeleteColumnStatisticsForTable action (Python: delete\$1column\$1statistics\$1for\$1table)](#aws-glue-api-catalog-tables-DeleteColumnStatisticsForTable) ## CreateTable action (Python: create\$1table) CreateTable (create\$1table) Creates a new table definition in the Data Catalog. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog in which to create the `Table`. If none is supplied, the AWS account ID is used by default. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The catalog database in which to create the new table. For Hive compatibility, this name is entirely lowercase. + `Name` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The unique identifier for the table within the specified database that will be created in the AWS Glue Data Catalog. + `TableInput` – A [TableInput](#aws-glue-api-catalog-tables-TableInput) object. The `TableInput` object that defines the metadata table to create in the catalog. + `PartitionIndexes` – An array of [PartitionIndex](#aws-glue-api-catalog-tables-PartitionIndex) objects, not more than 3 structures. A list of partition indexes, `PartitionIndex` structures, to create in the table. + `TransactionId` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Custom string pattern #44](aws-glue-api-common.md#regex_44). The ID of the transaction. + `OpenTableFormatInput` – An [OpenTableFormatInput](#aws-glue-api-catalog-tables-OpenTableFormatInput) object. Specifies an `OpenTableFormatInput` structure when creating an open format table. **Response** + *No Response parameters.* **Errors** + `AlreadyExistsException` + `InvalidInputException` + `EntityNotFoundException` + `ResourceNumberLimitExceededException` + `InternalServiceException` + `OperationTimeoutException` + `GlueEncryptionException` + `ConcurrentModificationException` + `ResourceNotReadyException` + `FederationSourceException` + `FederationSourceRetryableException` ## UpdateTable action (Python: update\$1table) UpdateTable (update\$1table) Updates a metadata table in the Data Catalog. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog where the table resides. If none is provided, the AWS account ID is used by default. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the catalog database in which the table resides. For Hive compatibility, this name is entirely lowercase. + `Name` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The unique identifier for the table within the specified database that will be created in the AWS Glue Data Catalog. + `TableInput` – A [TableInput](#aws-glue-api-catalog-tables-TableInput) object. An updated `TableInput` object to define the metadata table in the catalog. + `SkipArchive` – Boolean. By default, `UpdateTable` always creates an archived version of the table before updating it. However, if `skipArchive` is set to true, `UpdateTable` does not create the archived version. + `TransactionId` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Custom string pattern #44](aws-glue-api-common.md#regex_44). The transaction ID at which to update the table contents. + `VersionId` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The version ID at which to update the table contents. + `ViewUpdateAction` – UTF-8 string (valid values: `ADD` \$1 `REPLACE` \$1 `ADD_OR_REPLACE` \$1 `DROP`). The operation to be performed when updating the view. + `Force` – Boolean. A flag that can be set to true to ignore matching storage descriptor and subobject matching requirements. + `UpdateOpenTableFormatInput` – An [UpdateOpenTableFormatInput](#aws-glue-api-catalog-tables-UpdateOpenTableFormatInput) object. Input parameters for updating open table format tables in AWS GlueData Catalog, serving as a wrapper for format-specific update operations such as Apache Iceberg. **Response** + *No Response parameters.* **Errors** + `EntityNotFoundException` + `InvalidInputException` + `InternalServiceException` + `OperationTimeoutException` + `ConcurrentModificationException` + `ResourceNumberLimitExceededException` + `GlueEncryptionException` + `ResourceNotReadyException` + `FederationSourceException` + `FederationSourceRetryableException` + `AlreadyExistsException` ## DeleteTable action (Python: delete\$1table) DeleteTable (delete\$1table) Removes a table definition from the Data Catalog. **Note** After completing this operation, you no longer have access to the table versions and partitions that belong to the deleted table. AWS Glue deletes these "orphaned" resources asynchronously in a timely manner, at the discretion of the service. To ensure the immediate deletion of all related resources, before calling `DeleteTable`, use `DeleteTableVersion` or `BatchDeleteTableVersion`, and `DeletePartition` or `BatchDeletePartition`, to delete any resources that belong to the table. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog where the table resides. If none is provided, the AWS account ID is used by default. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the catalog database in which the table resides. For Hive compatibility, this name is entirely lowercase. + `Name` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the table to be deleted. For Hive compatibility, this name is entirely lowercase. + `TransactionId` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Custom string pattern #44](aws-glue-api-common.md#regex_44). The transaction ID at which to delete the table contents. **Response** + *No Response parameters.* **Errors** + `EntityNotFoundException` + `InvalidInputException` + `InternalServiceException` + `OperationTimeoutException` + `ConcurrentModificationException` + `ResourceNotReadyException` + `FederationSourceException` + `FederationSourceRetryableException` ## BatchDeleteTable action (Python: batch\$1delete\$1table) BatchDeleteTable (batch\$1delete\$1table) Deletes multiple tables at once. **Note** After completing this operation, you no longer have access to the table versions and partitions that belong to the deleted table. AWS Glue deletes these "orphaned" resources asynchronously in a timely manner, at the discretion of the service. To ensure the immediate deletion of all related resources, before calling `BatchDeleteTable`, use `DeleteTableVersion` or `BatchDeleteTableVersion`, and `DeletePartition` or `BatchDeletePartition`, to delete any resources that belong to the table. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog where the table resides. If none is provided, the AWS account ID is used by default. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the catalog database in which the tables to delete reside. For Hive compatibility, this name is entirely lowercase. + `TablesToDelete` – *Required:* An array of UTF-8 strings, not more than 100 strings. A list of the table to delete. + `TransactionId` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Custom string pattern #44](aws-glue-api-common.md#regex_44). The transaction ID at which to delete the table contents. **Response** + `Errors` – An array of [TableError](#aws-glue-api-catalog-tables-TableError) objects. A list of errors encountered in attempting to delete the specified tables. **Errors** + `InvalidInputException` + `EntityNotFoundException` + `InternalServiceException` + `OperationTimeoutException` + `GlueEncryptionException` + `ResourceNotReadyException` ## GetTable action (Python: get\$1table) GetTable (get\$1table) Retrieves the `Table` definition in a Data Catalog for a specified table. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog where the table resides. If none is provided, the AWS account ID is used by default. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the database in the catalog in which the table resides. For Hive compatibility, this name is entirely lowercase. + `Name` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the table for which to retrieve the definition. For Hive compatibility, this name is entirely lowercase. + `TransactionId` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Custom string pattern #44](aws-glue-api-common.md#regex_44). The transaction ID at which to read the table contents. + `QueryAsOfTime` – Timestamp. The time as of when to read the table contents. If not set, the most recent transaction commit time will be used. Cannot be specified along with `TransactionId`. + `AuditContext` – An [AuditContext](#aws-glue-api-catalog-tables-AuditContext) object. A structure containing the Lake Formation [audit context](https://docs.aws.amazon.com/glue/latest/webapi/API_AuditContext.html). + `IncludeStatusDetails` – Boolean. Specifies whether to include status details related to a request to create or update an AWS Glue Data Catalog view. **Response** + `Table` – A [Table](#aws-glue-api-catalog-tables-Table) object. The `Table` object that defines the specified table. **Errors** + `EntityNotFoundException` + `InvalidInputException` + `InternalServiceException` + `OperationTimeoutException` + `GlueEncryptionException` + `ResourceNotReadyException` + `FederationSourceException` + `FederationSourceRetryableException` ## GetTables action (Python: get\$1tables) GetTables (get\$1tables) Retrieves the definitions of some or all of the tables in a given `Database`. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog where the tables reside. If none is provided, the AWS account ID is used by default. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The database in the catalog whose tables to list. For Hive compatibility, this name is entirely lowercase. + `Expression` – UTF-8 string, not more than 2048 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). A regular expression pattern. If present, only those tables whose names match the pattern are returned. + `NextToken` – UTF-8 string. A continuation token, included if this is a continuation call. + `MaxResults` – Number (integer), not less than 1 or more than 100. The maximum number of tables to return in a single response. + `TransactionId` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Custom string pattern #44](aws-glue-api-common.md#regex_44). The transaction ID at which to read the table contents. + `QueryAsOfTime` – Timestamp. The time as of when to read the table contents. If not set, the most recent transaction commit time will be used. Cannot be specified along with `TransactionId`. + `AuditContext` – An [AuditContext](#aws-glue-api-catalog-tables-AuditContext) object. A structure containing the Lake Formation [audit context](https://docs.aws.amazon.com/glue/latest/webapi/API_AuditContext.html). + `IncludeStatusDetails` – Boolean. Specifies whether to include status details related to a request to create or update an AWS Glue Data Catalog view. + `AttributesToGet` – An array of UTF-8 strings. Specifies the table fields returned by the `GetTables` call. This parameter doesn't accept an empty list. The request must include `NAME`. The following are the valid combinations of values: + `NAME` - Names of all tables in the database. + `NAME`, `TABLE_TYPE` - Names of all tables and the table types. **Response** + `TableList` – An array of [Table](#aws-glue-api-catalog-tables-Table) objects. A list of the requested `Table` objects. + `NextToken` – UTF-8 string. A continuation token, present if the current list segment is not the last. **Errors** + `EntityNotFoundException` + `InvalidInputException` + `OperationTimeoutException` + `InternalServiceException` + `GlueEncryptionException` + `FederationSourceException` + `FederationSourceRetryableException` ## GetTableVersion action (Python: get\$1table\$1version) GetTableVersion (get\$1table\$1version) Retrieves a specified version of a table. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog where the tables reside. If none is provided, the AWS account ID is used by default. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The database in the catalog in which the table resides. For Hive compatibility, this name is entirely lowercase. + `TableName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the table. For Hive compatibility, this name is entirely lowercase. + `VersionId` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID value of the table version to be retrieved. A `VersionID` is a string representation of an integer. Each version is incremented by 1. **Response** + `TableVersion` – A [TableVersion](#aws-glue-api-catalog-tables-TableVersion) object. The requested table version. **Errors** + `EntityNotFoundException` + `InvalidInputException` + `InternalServiceException` + `OperationTimeoutException` + `GlueEncryptionException` ## GetTableVersions action (Python: get\$1table\$1versions) GetTableVersions (get\$1table\$1versions) Retrieves a list of strings that identify available versions of a specified table. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog where the tables reside. If none is provided, the AWS account ID is used by default. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The database in the catalog in which the table resides. For Hive compatibility, this name is entirely lowercase. + `TableName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the table. For Hive compatibility, this name is entirely lowercase. + `NextToken` – UTF-8 string. A continuation token, if this is not the first call. + `MaxResults` – Number (integer), not less than 1 or more than 100. The maximum number of table versions to return in one response. **Response** + `TableVersions` – An array of [TableVersion](#aws-glue-api-catalog-tables-TableVersion) objects. A list of strings identifying available versions of the specified table. + `NextToken` – UTF-8 string. A continuation token, if the list of available versions does not include the last one. **Errors** + `EntityNotFoundException` + `InvalidInputException` + `InternalServiceException` + `OperationTimeoutException` + `GlueEncryptionException` ## DeleteTableVersion action (Python: delete\$1table\$1version) DeleteTableVersion (delete\$1table\$1version) Deletes a specified version of a table. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog where the tables reside. If none is provided, the AWS account ID is used by default. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The database in the catalog in which the table resides. For Hive compatibility, this name is entirely lowercase. + `TableName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the table. For Hive compatibility, this name is entirely lowercase. + `VersionId` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the table version to be deleted. A `VersionID` is a string representation of an integer. Each version is incremented by 1. **Response** + *No Response parameters.* **Errors** + `EntityNotFoundException` + `InvalidInputException` + `InternalServiceException` + `OperationTimeoutException` ## BatchDeleteTableVersion action (Python: batch\$1delete\$1table\$1version) BatchDeleteTableVersion (batch\$1delete\$1table\$1version) Deletes a specified batch of versions of a table. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog where the tables reside. If none is provided, the AWS account ID is used by default. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The database in the catalog in which the table resides. For Hive compatibility, this name is entirely lowercase. + `TableName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the table. For Hive compatibility, this name is entirely lowercase. + `VersionIds` – *Required:* An array of UTF-8 strings, not more than 100 strings. A list of the IDs of versions to be deleted. A `VersionId` is a string representation of an integer. Each version is incremented by 1. **Response** + `Errors` – An array of [TableVersionError](#aws-glue-api-catalog-tables-TableVersionError) objects. A list of errors encountered while trying to delete the specified table versions. **Errors** + `EntityNotFoundException` + `InvalidInputException` + `InternalServiceException` + `OperationTimeoutException` ## SearchTables action (Python: search\$1tables) SearchTables (search\$1tables) Searches a set of tables based on properties in the table metadata as well as on the parent database. You can search against text or filter conditions. You can only get tables that you have access to based on the security policies defined in Lake Formation. You need at least a read-only access to the table for it to be returned. If you do not have access to all the columns in the table, these columns will not be searched against when returning the list of tables back to you. If you have access to the columns but not the data in the columns, those columns and the associated metadata for those columns will be included in the search. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). A unique identifier, consisting of `account_id`. + `NextToken` – UTF-8 string. A continuation token, included if this is a continuation call. + `Filters` – An array of [PropertyPredicate](aws-glue-api-common.md#aws-glue-api-common-PropertyPredicate) objects. A list of key-value pairs, and a comparator used to filter the search results. Returns all entities matching the predicate. The `Comparator` member of the `PropertyPredicate` struct is used only for time fields, and can be omitted for other field types. Also, when comparing string values, such as when `Key=Name`, a fuzzy match algorithm is used. The `Key` field (for example, the value of the `Name` field) is split on certain punctuation characters, for example, -, :, \$1, etc. into tokens. Then each token is exact-match compared with the `Value` member of `PropertyPredicate`. For example, if `Key=Name` and `Value=link`, tables named `customer-link` and `xx-link-yy` are returned, but `xxlinkyy` is not returned. + `SearchText` – Value string, not less than 1 or more than 1024 bytes long. A string used for a text search. Specifying a value in quotes filters based on an exact match to the value. + `SortCriteria` – An array of [SortCriterion](#aws-glue-api-catalog-tables-SortCriterion) objects, not more than 1 structures. A list of criteria for sorting the results by a field name, in an ascending or descending order. + `MaxResults` – Number (integer), not less than 1 or more than 1000. The maximum number of tables to return in a single response. + `ResourceShareType` – UTF-8 string (valid values: `FOREIGN` \$1 `ALL` \$1 `FEDERATED`). Allows you to specify that you want to search the tables shared with your account. The allowable values are `FOREIGN` or `ALL`. + If set to `FOREIGN`, will search the tables shared with your account. + If set to `ALL`, will search the tables shared with your account, as well as the tables in yor local account. + `IncludeStatusDetails` – Boolean. Specifies whether to include status details related to a request to create or update an AWS Glue Data Catalog view. **Response** + `NextToken` – UTF-8 string. A continuation token, present if the current list segment is not the last. + `TableList` – An array of [Table](#aws-glue-api-catalog-tables-Table) objects. A list of the requested `Table` objects. The `SearchTables` response returns only the tables that you have access to. **Errors** + `InternalServiceException` + `InvalidInputException` + `OperationTimeoutException` ## GetPartitionIndexes action (Python: get\$1partition\$1indexes) GetPartitionIndexes (get\$1partition\$1indexes) Retrieves the partition indexes associated with a table. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The catalog ID where the table resides. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Specifies the name of a database from which you want to retrieve partition indexes. + `TableName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Specifies the name of a table for which you want to retrieve the partition indexes. + `NextToken` – UTF-8 string. A continuation token, included if this is a continuation call. **Response** + `PartitionIndexDescriptorList` – An array of [PartitionIndexDescriptor](#aws-glue-api-catalog-tables-PartitionIndexDescriptor) objects. A list of index descriptors. + `NextToken` – UTF-8 string. A continuation token, present if the current list segment is not the last. **Errors** + `InternalServiceException` + `OperationTimeoutException` + `InvalidInputException` + `EntityNotFoundException` + `ConflictException` ## CreatePartitionIndex action (Python: create\$1partition\$1index) CreatePartitionIndex (create\$1partition\$1index) Creates a specified partition index in an existing table. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The catalog ID where the table resides. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Specifies the name of a database in which you want to create a partition index. + `TableName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Specifies the name of a table in which you want to create a partition index. + `PartitionIndex` – *Required:* A [PartitionIndex](#aws-glue-api-catalog-tables-PartitionIndex) object. Specifies a `PartitionIndex` structure to create a partition index in an existing table. **Response** + *No Response parameters.* **Errors** + `AlreadyExistsException` + `InvalidInputException` + `EntityNotFoundException` + `ResourceNumberLimitExceededException` + `InternalServiceException` + `OperationTimeoutException` + `GlueEncryptionException` ## DeletePartitionIndex action (Python: delete\$1partition\$1index) DeletePartitionIndex (delete\$1partition\$1index) Deletes a specified partition index from an existing table. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The catalog ID where the table resides. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Specifies the name of a database from which you want to delete a partition index. + `TableName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Specifies the name of a table from which you want to delete a partition index. + `IndexName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the partition index to be deleted. **Response** + *No Response parameters.* **Errors** + `InternalServiceException` + `OperationTimeoutException` + `InvalidInputException` + `EntityNotFoundException` + `ConflictException` + `GlueEncryptionException` ## GetColumnStatisticsForTable action (Python: get\$1column\$1statistics\$1for\$1table) GetColumnStatisticsForTable (get\$1column\$1statistics\$1for\$1table) Retrieves table statistics of columns. The Identity and Access Management (IAM) permission required for this operation is `GetTable`. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog where the partitions in question reside. If none is supplied, the AWS account ID is used by default. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the catalog database where the partitions reside. + `TableName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the partitions' table. + `ColumnNames` – *Required:* An array of UTF-8 strings, not more than 100 strings. A list of the column names. **Response** + `ColumnStatisticsList` – An array of [ColumnStatistics](aws-glue-api-common.md#aws-glue-api-common-ColumnStatistics) objects. List of ColumnStatistics. + `Errors` – An array of [ColumnError](aws-glue-api-common.md#aws-glue-api-common-ColumnError) objects. List of ColumnStatistics that failed to be retrieved. **Errors** + `EntityNotFoundException` + `InvalidInputException` + `InternalServiceException` + `OperationTimeoutException` + `GlueEncryptionException` ## UpdateColumnStatisticsForTable action (Python: update\$1column\$1statistics\$1for\$1table) UpdateColumnStatisticsForTable (update\$1column\$1statistics\$1for\$1table) Creates or updates table statistics of columns. The Identity and Access Management (IAM) permission required for this operation is `UpdateTable`. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog where the partitions in question reside. If none is supplied, the AWS account ID is used by default. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the catalog database where the partitions reside. + `TableName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the partitions' table. + `ColumnStatisticsList` – *Required:* An array of [ColumnStatistics](aws-glue-api-common.md#aws-glue-api-common-ColumnStatistics) objects, not more than 25 structures. A list of the column statistics. **Response** + `Errors` – An array of [ColumnStatisticsError](aws-glue-api-common.md#aws-glue-api-common-ColumnStatisticsError) objects. List of ColumnStatisticsErrors. **Errors** + `EntityNotFoundException` + `InvalidInputException` + `InternalServiceException` + `OperationTimeoutException` + `GlueEncryptionException` ## DeleteColumnStatisticsForTable action (Python: delete\$1column\$1statistics\$1for\$1table) DeleteColumnStatisticsForTable (delete\$1column\$1statistics\$1for\$1table) Retrieves table statistics of columns. The Identity and Access Management (IAM) permission required for this operation is `DeleteTable`. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog where the partitions in question reside. If none is supplied, the AWS account ID is used by default. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the catalog database where the partitions reside. + `TableName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the partitions' table. + `ColumnName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the column. **Response** + *No Response parameters.* **Errors** + `EntityNotFoundException` + `InvalidInputException` + `InternalServiceException` + `OperationTimeoutException` + `GlueEncryptionException` # Partition API Partitions The Partition API describes data types and operations used to work with partitions. ## Data types  —  data types  — + [Partition structure](#aws-glue-api-catalog-partitions-Partition) + [PartitionInput structure](#aws-glue-api-catalog-partitions-PartitionInput) + [PartitionSpecWithSharedStorageDescriptor structure](#aws-glue-api-catalog-partitions-PartitionSpecWithSharedStorageDescriptor) + [PartitionListComposingSpec structure](#aws-glue-api-catalog-partitions-PartitionListComposingSpec) + [PartitionSpecProxy structure](#aws-glue-api-catalog-partitions-PartitionSpecProxy) + [PartitionValueList structure](#aws-glue-api-catalog-partitions-PartitionValueList) + [Segment structure](#aws-glue-api-catalog-partitions-Segment) + [PartitionError structure](#aws-glue-api-catalog-partitions-PartitionError) + [BatchUpdatePartitionFailureEntry structure](#aws-glue-api-catalog-partitions-BatchUpdatePartitionFailureEntry) + [BatchUpdatePartitionRequestEntry structure](#aws-glue-api-catalog-partitions-BatchUpdatePartitionRequestEntry) + [StorageDescriptor structure](#aws-glue-api-catalog-partitions-StorageDescriptor) + [SchemaReference structure](#aws-glue-api-catalog-partitions-SchemaReference) + [SerDeInfo structure](#aws-glue-api-catalog-partitions-SerDeInfo) + [SkewedInfo structure](#aws-glue-api-catalog-partitions-SkewedInfo) ## Partition structure Partition Represents a slice of table data. **Fields** + `Values` – An array of UTF-8 strings. The values of the partition. + `DatabaseName` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the catalog database in which to create the partition. + `TableName` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the database table in which to create the partition. + `CreationTime` – Timestamp. The time at which the partition was created. + `LastAccessTime` – Timestamp. The last time at which the partition was accessed. + `StorageDescriptor` – A [StorageDescriptor](#aws-glue-api-catalog-partitions-StorageDescriptor) object. Provides information about the physical location where the partition is stored. + `Parameters` – A map array of key-value pairs. Each key is a Key string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Each value is a UTF-8 string, not more than 512000 bytes long. These key-value pairs define partition parameters. + `LastAnalyzedTime` – Timestamp. The last time at which column statistics were computed for this partition. + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog in which the partition resides. ## PartitionInput structure PartitionInput The structure used to create and update a partition. **Fields** + `Values` – An array of UTF-8 strings. The values of the partition. Although this parameter is not required by the SDK, you must specify this parameter for a valid input. The values for the keys for the new partition must be passed as an array of String objects that must be ordered in the same order as the partition keys appearing in the Amazon S3 prefix. Otherwise AWS Glue will add the values to the wrong keys. + `LastAccessTime` – Timestamp. The last time at which the partition was accessed. + `StorageDescriptor` – A [StorageDescriptor](#aws-glue-api-catalog-partitions-StorageDescriptor) object. Provides information about the physical location where the partition is stored. + `Parameters` – A map array of key-value pairs. Each key is a Key string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Each value is a UTF-8 string, not more than 512000 bytes long. These key-value pairs define partition parameters. + `LastAnalyzedTime` – Timestamp. The last time at which column statistics were computed for this partition. ## PartitionSpecWithSharedStorageDescriptor structure PartitionSpecWithSharedStorageDescriptor A partition specification for partitions that share a physical location. **Fields** + `StorageDescriptor` – A [StorageDescriptor](#aws-glue-api-catalog-partitions-StorageDescriptor) object. The shared physical storage information. + `Partitions` – An array of [Partition](#aws-glue-api-catalog-partitions-Partition) objects. A list of the partitions that share this physical location. ## PartitionListComposingSpec structure PartitionListComposingSpec Lists the related partitions. **Fields** + `Partitions` – An array of [Partition](#aws-glue-api-catalog-partitions-Partition) objects. A list of the partitions in the composing specification. ## PartitionSpecProxy structure PartitionSpecProxy Provides a root path to specified partitions. **Fields** + `DatabaseName` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The catalog database in which the partitions reside. + `TableName` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the table that contains the partitions. + `RootPath` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The root path of the proxy for addressing the partitions. + `PartitionSpecWithSharedSD` – A [PartitionSpecWithSharedStorageDescriptor](#aws-glue-api-catalog-partitions-PartitionSpecWithSharedStorageDescriptor) object. A specification of partitions that share the same physical storage location. + `PartitionListComposingSpec` – A [PartitionListComposingSpec](#aws-glue-api-catalog-partitions-PartitionListComposingSpec) object. Specifies a list of partitions. ## PartitionValueList structure PartitionValueList Contains a list of values defining partitions. **Fields** + `Values` – *Required:* An array of UTF-8 strings. The list of values. ## Segment structure Segment Defines a non-overlapping region of a table's partitions, allowing multiple requests to be run in parallel. **Fields** + `SegmentNumber` – *Required:* Number (integer), not more than None. The zero-based index number of the segment. For example, if the total number of segments is 4, `SegmentNumber` values range from 0 through 3. + `TotalSegments` – *Required:* Number (integer), not less than 1 or more than 10. The total number of segments. ## PartitionError structure PartitionError Contains information about a partition error. **Fields** + `PartitionValues` – An array of UTF-8 strings. The values that define the partition. + `ErrorDetail` – An [ErrorDetail](aws-glue-api-common.md#aws-glue-api-common-ErrorDetail) object. The details about the partition error. ## BatchUpdatePartitionFailureEntry structure BatchUpdatePartitionFailureEntry Contains information about a batch update partition error. **Fields** + `PartitionValueList` – An array of UTF-8 strings, not more than 100 strings. A list of values defining the partitions. + `ErrorDetail` – An [ErrorDetail](aws-glue-api-common.md#aws-glue-api-common-ErrorDetail) object. The details about the batch update partition error. ## BatchUpdatePartitionRequestEntry structure BatchUpdatePartitionRequestEntry A structure that contains the values and structure used to update a partition. **Fields** + `PartitionValueList` – *Required:* An array of UTF-8 strings, not more than 100 strings. A list of values defining the partitions. + `PartitionInput` – *Required:* A [PartitionInput](#aws-glue-api-catalog-partitions-PartitionInput) object. The structure used to update a partition. ## StorageDescriptor structure StorageDescriptor Describes the physical storage of table data. **Fields** + `Columns` – An array of [Column](aws-glue-api-catalog-tables.md#aws-glue-api-catalog-tables-Column) objects. A list of the `Columns` in the table. + `Location` – Location string, not more than 2056 bytes long, matching the [URI address multi-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-uri). The physical location of the table. By default, this takes the form of the warehouse location, followed by the database location in the warehouse, followed by the table name. + `AdditionalLocations` – An array of UTF-8 strings. A list of locations that point to the path where a Delta table is located. + `InputFormat` – Format string, not more than 128 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The input format: `SequenceFileInputFormat` (binary), or `TextInputFormat`, or a custom format. + `OutputFormat` – Format string, not more than 128 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The output format: `SequenceFileOutputFormat` (binary), or `IgnoreKeyTextOutputFormat`, or a custom format. + `Compressed` – Boolean. `True` if the data in the table is compressed, or `False` if not. + `NumberOfBuckets` – Number (integer). Must be specified if the table contains any dimension columns. + `SerdeInfo` – A [SerDeInfo](#aws-glue-api-catalog-partitions-SerDeInfo) object. The serialization/deserialization (SerDe) information. + `BucketColumns` – An array of UTF-8 strings. A list of reducer grouping columns, clustering columns, and bucketing columns in the table. + `SortColumns` – An array of [Order](aws-glue-api-catalog-tables.md#aws-glue-api-catalog-tables-Order) objects. A list specifying the sort order of each bucket in the table. + `Parameters` – A map array of key-value pairs. Each key is a Key string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Each value is a UTF-8 string, not more than 512000 bytes long. The user-supplied properties in key-value form. + `SkewedInfo` – A [SkewedInfo](#aws-glue-api-catalog-partitions-SkewedInfo) object. The information about values that appear frequently in a column (skewed values). + `StoredAsSubDirectories` – Boolean. `True` if the table data is stored in subdirectories, or `False` if not. + `SchemaReference` – A [SchemaReference](#aws-glue-api-catalog-partitions-SchemaReference) object. An object that references a schema stored in the AWS Glue Schema Registry. When creating a table, you can pass an empty list of columns for the schema, and instead use a schema reference. ## SchemaReference structure SchemaReference An object that references a schema stored in the AWS Glue Schema Registry. **Fields** + `SchemaId` – A [SchemaId](aws-glue-api-schema-registry-api.md#aws-glue-api-schema-registry-api-SchemaId) object. A structure that contains schema identity fields. Either this or the `SchemaVersionId` has to be provided. + `SchemaVersionId` – UTF-8 string, not less than 36 or more than 36 bytes long, matching the [Custom string pattern #45](aws-glue-api-common.md#regex_45). The unique ID assigned to a version of the schema. Either this or the `SchemaId` has to be provided. + `SchemaVersionNumber` – Number (long), not less than 1 or more than 100000. The version number of the schema. ## SerDeInfo structure SerDeInfo Information about a serialization/deserialization program (SerDe) that serves as an extractor and loader. **Fields** + `Name` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Name of the SerDe. + `SerializationLibrary` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Usually the class that implements the SerDe. An example is `org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe`. + `Parameters` – A map array of key-value pairs. Each key is a Key string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Each value is a UTF-8 string, not more than 512000 bytes long. These key-value pairs define initialization parameters for the SerDe. ## SkewedInfo structure SkewedInfo Specifies skewed values in a table. Skewed values are those that occur with very high frequency. **Fields** + `SkewedColumnNames` – An array of UTF-8 strings. A list of names of columns that contain skewed values. + `SkewedColumnValues` – An array of UTF-8 strings. A list of values that appear so frequently as to be considered skewed. + `SkewedColumnValueLocationMaps` – A map array of key-value pairs. Each key is a UTF-8 string. Each value is a UTF-8 string. A mapping of skewed values to the columns that contain them. ## Operations  —  operations  — + [CreatePartition action (Python: create\$1partition)](#aws-glue-api-catalog-partitions-CreatePartition) + [BatchCreatePartition action (Python: batch\$1create\$1partition)](#aws-glue-api-catalog-partitions-BatchCreatePartition) + [UpdatePartition action (Python: update\$1partition)](#aws-glue-api-catalog-partitions-UpdatePartition) + [DeletePartition action (Python: delete\$1partition)](#aws-glue-api-catalog-partitions-DeletePartition) + [BatchDeletePartition action (Python: batch\$1delete\$1partition)](#aws-glue-api-catalog-partitions-BatchDeletePartition) + [GetPartition action (Python: get\$1partition)](#aws-glue-api-catalog-partitions-GetPartition) + [GetPartitions action (Python: get\$1partitions)](#aws-glue-api-catalog-partitions-GetPartitions) + [BatchGetPartition action (Python: batch\$1get\$1partition)](#aws-glue-api-catalog-partitions-BatchGetPartition) + [BatchUpdatePartition action (Python: batch\$1update\$1partition)](#aws-glue-api-catalog-partitions-BatchUpdatePartition) + [GetColumnStatisticsForPartition action (Python: get\$1column\$1statistics\$1for\$1partition)](#aws-glue-api-catalog-partitions-GetColumnStatisticsForPartition) + [UpdateColumnStatisticsForPartition action (Python: update\$1column\$1statistics\$1for\$1partition)](#aws-glue-api-catalog-partitions-UpdateColumnStatisticsForPartition) + [DeleteColumnStatisticsForPartition action (Python: delete\$1column\$1statistics\$1for\$1partition)](#aws-glue-api-catalog-partitions-DeleteColumnStatisticsForPartition) ## CreatePartition action (Python: create\$1partition) CreatePartition (create\$1partition) Creates a new partition. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The AWS account ID of the catalog in which the partition is to be created. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the metadata database in which the partition is to be created. + `TableName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the metadata table in which the partition is to be created. + `PartitionInput` – *Required:* A [PartitionInput](#aws-glue-api-catalog-partitions-PartitionInput) object. A `PartitionInput` structure defining the partition to be created. **Response** + *No Response parameters.* **Errors** + `InvalidInputException` + `AlreadyExistsException` + `ResourceNumberLimitExceededException` + `InternalServiceException` + `EntityNotFoundException` + `OperationTimeoutException` + `GlueEncryptionException` ## BatchCreatePartition action (Python: batch\$1create\$1partition) BatchCreatePartition (batch\$1create\$1partition) Creates one or more partitions in a batch operation. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the catalog in which the partition is to be created. Currently, this should be the AWS account ID. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the metadata database in which the partition is to be created. + `TableName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the metadata table in which the partition is to be created. + `PartitionInputList` – *Required:* An array of [PartitionInput](#aws-glue-api-catalog-partitions-PartitionInput) objects, not more than 100 structures. A list of `PartitionInput` structures that define the partitions to be created. **Response** + `Errors` – An array of [PartitionError](#aws-glue-api-catalog-partitions-PartitionError) objects. The errors encountered when trying to create the requested partitions. **Errors** + `InvalidInputException` + `AlreadyExistsException` + `ResourceNumberLimitExceededException` + `InternalServiceException` + `EntityNotFoundException` + `OperationTimeoutException` + `GlueEncryptionException` ## UpdatePartition action (Python: update\$1partition) UpdatePartition (update\$1partition) Updates a partition. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog where the partition to be updated resides. If none is provided, the AWS account ID is used by default. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the catalog database in which the table in question resides. + `TableName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the table in which the partition to be updated is located. + `PartitionValueList` – *Required:* An array of UTF-8 strings, not more than 100 strings. List of partition key values that define the partition to update. + `PartitionInput` – *Required:* A [PartitionInput](#aws-glue-api-catalog-partitions-PartitionInput) object. The new partition object to update the partition to. The `Values` property can't be changed. If you want to change the partition key values for a partition, delete and recreate the partition. **Response** + *No Response parameters.* **Errors** + `EntityNotFoundException` + `InvalidInputException` + `InternalServiceException` + `OperationTimeoutException` + `GlueEncryptionException` ## DeletePartition action (Python: delete\$1partition) DeletePartition (delete\$1partition) Deletes a specified partition. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog where the partition to be deleted resides. If none is provided, the AWS account ID is used by default. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the catalog database in which the table in question resides. + `TableName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the table that contains the partition to be deleted. + `PartitionValues` – *Required:* An array of UTF-8 strings. The values that define the partition. **Response** + *No Response parameters.* **Errors** + `EntityNotFoundException` + `InvalidInputException` + `InternalServiceException` + `OperationTimeoutException` ## BatchDeletePartition action (Python: batch\$1delete\$1partition) BatchDeletePartition (batch\$1delete\$1partition) Deletes one or more partitions in a batch operation. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog where the partition to be deleted resides. If none is provided, the AWS account ID is used by default. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the catalog database in which the table in question resides. + `TableName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the table that contains the partitions to be deleted. + `PartitionsToDelete` – *Required:* An array of [PartitionValueList](#aws-glue-api-catalog-partitions-PartitionValueList) objects, not more than 25 structures. A list of `PartitionInput` structures that define the partitions to be deleted. **Response** + `Errors` – An array of [PartitionError](#aws-glue-api-catalog-partitions-PartitionError) objects. The errors encountered when trying to delete the requested partitions. **Errors** + `InvalidInputException` + `EntityNotFoundException` + `InternalServiceException` + `OperationTimeoutException` ## GetPartition action (Python: get\$1partition) GetPartition (get\$1partition) Retrieves information about a specified partition. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog where the partition in question resides. If none is provided, the AWS account ID is used by default. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the catalog database where the partition resides. + `TableName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the partition's table. + `PartitionValues` – *Required:* An array of UTF-8 strings. The values that define the partition. **Response** + `Partition` – A [Partition](#aws-glue-api-catalog-partitions-Partition) object. The requested information, in the form of a `Partition` object. **Errors** + `EntityNotFoundException` + `InvalidInputException` + `InternalServiceException` + `OperationTimeoutException` + `GlueEncryptionException` + `FederationSourceException` + `FederationSourceRetryableException` ## GetPartitions action (Python: get\$1partitions) GetPartitions (get\$1partitions) Retrieves information about the partitions in a table. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog where the partitions in question reside. If none is provided, the AWS account ID is used by default. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the catalog database where the partitions reside. + `TableName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the partitions' table. + `Expression` – Predicate string, not more than 2048 bytes long, matching the [URI address multi-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-uri). An expression that filters the partitions to be returned. The expression uses SQL syntax similar to the SQL `WHERE` filter clause. The SQL statement parser [JSQLParser](http://jsqlparser.sourceforge.net/home.php) parses the expression. *Operators*: The following are the operators that you can use in the `Expression` API call: = Checks whether the values of the two operands are equal; if yes, then the condition becomes true. Example: Assume 'variable a' holds 10 and 'variable b' holds 20. (a = b) is not true. < > Checks whether the values of two operands are equal; if the values are not equal, then the condition becomes true. Example: (a < > b) is true. > Checks whether the value of the left operand is greater than the value of the right operand; if yes, then the condition becomes true. Example: (a > b) is not true. < Checks whether the value of the left operand is less than the value of the right operand; if yes, then the condition becomes true. Example: (a < b) is true. >= Checks whether the value of the left operand is greater than or equal to the value of the right operand; if yes, then the condition becomes true. Example: (a >= b) is not true. <= Checks whether the value of the left operand is less than or equal to the value of the right operand; if yes, then the condition becomes true. Example: (a <= b) is true. AND, OR, IN, BETWEEN, LIKE, NOT, IS NULL Logical operators. *Supported Partition Key Types*: The following are the supported partition keys. + `string` + `date` + `timestamp` + `int` + `bigint` + `long` + `tinyint` + `smallint` + `decimal` If an type is encountered that is not valid, an exception is thrown. The following list shows the valid operators on each type. When you define a crawler, the `partitionKey` type is created as a `STRING`, to be compatible with the catalog partitions. *Sample API Call*: **Example** The table `twitter_partition` has three partitions: ``` year = 2015 year = 2016 year = 2017 ``` **Example** Get partition `year` equal to 2015 ``` aws glue get-partitions --database-name dbname --table-name twitter_partition --expression "year*=*'2015'" ``` **Example** Get partition `year` between 2016 and 2018 (exclusive) ``` aws glue get-partitions --database-name dbname --table-name twitter_partition --expression "year>'2016' AND year<'2018'" ``` **Example** Get partition `year` between 2015 and 2018 (inclusive). The following API calls are equivalent to each other: ``` aws glue get-partitions --database-name dbname --table-name twitter_partition --expression "year>='2015' AND year<='2018'" aws glue get-partitions --database-name dbname --table-name twitter_partition --expression "year BETWEEN 2015 AND 2018" aws glue get-partitions --database-name dbname --table-name twitter_partition --expression "year IN (2015,2016,2017,2018)" ``` **Example** A wildcard partition filter, where the following call output is partition year=2017. A regular expression is not supported in `LIKE`. ``` aws glue get-partitions --database-name dbname --table-name twitter_partition --expression "year LIKE '%7'" ``` + `NextToken` – UTF-8 string. A continuation token, if this is not the first call to retrieve these partitions. + `Segment` – A [Segment](#aws-glue-api-catalog-partitions-Segment) object. The segment of the table's partitions to scan in this request. + `MaxResults` – Number (integer), not less than 1 or more than 1000. The maximum number of partitions to return in a single response. + `ExcludeColumnSchema` – Boolean. When true, specifies not returning the partition column schema. Useful when you are interested only in other partition attributes such as partition values or location. This approach avoids the problem of a large response by not returning duplicate data. + `TransactionId` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Custom string pattern #44](aws-glue-api-common.md#regex_44). The transaction ID at which to read the partition contents. + `QueryAsOfTime` – Timestamp. The time as of when to read the partition contents. If not set, the most recent transaction commit time will be used. Cannot be specified along with `TransactionId`. **Response** + `Partitions` – An array of [Partition](#aws-glue-api-catalog-partitions-Partition) objects. A list of requested partitions. + `NextToken` – UTF-8 string. A continuation token, if the returned list of partitions does not include the last one. **Errors** + `EntityNotFoundException` + `InvalidInputException` + `OperationTimeoutException` + `InternalServiceException` + `GlueEncryptionException` + `InvalidStateException` + `ResourceNotReadyException` + `FederationSourceException` + `FederationSourceRetryableException` ## BatchGetPartition action (Python: batch\$1get\$1partition) BatchGetPartition (batch\$1get\$1partition) Retrieves partitions in a batch request. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog where the partitions in question reside. If none is supplied, the AWS account ID is used by default. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the catalog database where the partitions reside. + `TableName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the partitions' table. + `PartitionsToGet` – *Required:* An array of [PartitionValueList](#aws-glue-api-catalog-partitions-PartitionValueList) objects, not more than 1000 structures. A list of partition values identifying the partitions to retrieve. **Response** + `Partitions` – An array of [Partition](#aws-glue-api-catalog-partitions-Partition) objects. A list of the requested partitions. + `UnprocessedKeys` – An array of [PartitionValueList](#aws-glue-api-catalog-partitions-PartitionValueList) objects, not more than 1000 structures. A list of the partition values in the request for which partitions were not returned. **Errors** + `InvalidInputException` + `EntityNotFoundException` + `OperationTimeoutException` + `InternalServiceException` + `GlueEncryptionException` + `InvalidStateException` + `FederationSourceException` + `FederationSourceRetryableException` ## BatchUpdatePartition action (Python: batch\$1update\$1partition) BatchUpdatePartition (batch\$1update\$1partition) Updates one or more partitions in a batch operation. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the catalog in which the partition is to be updated. Currently, this should be the AWS account ID. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the metadata database in which the partition is to be updated. + `TableName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the metadata table in which the partition is to be updated. + `Entries` – *Required:* An array of [BatchUpdatePartitionRequestEntry](#aws-glue-api-catalog-partitions-BatchUpdatePartitionRequestEntry) objects, not less than 1 or more than 100 structures. A list of up to 100 `BatchUpdatePartitionRequestEntry` objects to update. **Response** + `Errors` – An array of [BatchUpdatePartitionFailureEntry](#aws-glue-api-catalog-partitions-BatchUpdatePartitionFailureEntry) objects. The errors encountered when trying to update the requested partitions. A list of `BatchUpdatePartitionFailureEntry` objects. **Errors** + `InvalidInputException` + `EntityNotFoundException` + `OperationTimeoutException` + `InternalServiceException` + `GlueEncryptionException` ## GetColumnStatisticsForPartition action (Python: get\$1column\$1statistics\$1for\$1partition) GetColumnStatisticsForPartition (get\$1column\$1statistics\$1for\$1partition) Retrieves partition statistics of columns. The Identity and Access Management (IAM) permission required for this operation is `GetPartition`. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog where the partitions in question reside. If none is supplied, the AWS account ID is used by default. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the catalog database where the partitions reside. + `TableName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the partitions' table. + `PartitionValues` – *Required:* An array of UTF-8 strings. A list of partition values identifying the partition. + `ColumnNames` – *Required:* An array of UTF-8 strings, not more than 100 strings. A list of the column names. **Response** + `ColumnStatisticsList` – An array of [ColumnStatistics](aws-glue-api-common.md#aws-glue-api-common-ColumnStatistics) objects. List of ColumnStatistics that failed to be retrieved. + `Errors` – An array of [ColumnError](aws-glue-api-common.md#aws-glue-api-common-ColumnError) objects. Error occurred during retrieving column statistics data. **Errors** + `EntityNotFoundException` + `InvalidInputException` + `InternalServiceException` + `OperationTimeoutException` + `GlueEncryptionException` ## UpdateColumnStatisticsForPartition action (Python: update\$1column\$1statistics\$1for\$1partition) UpdateColumnStatisticsForPartition (update\$1column\$1statistics\$1for\$1partition) Creates or updates partition statistics of columns. The Identity and Access Management (IAM) permission required for this operation is `UpdatePartition`. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog where the partitions in question reside. If none is supplied, the AWS account ID is used by default. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the catalog database where the partitions reside. + `TableName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the partitions' table. + `PartitionValues` – *Required:* An array of UTF-8 strings. A list of partition values identifying the partition. + `ColumnStatisticsList` – *Required:* An array of [ColumnStatistics](aws-glue-api-common.md#aws-glue-api-common-ColumnStatistics) objects, not more than 25 structures. A list of the column statistics. **Response** + `Errors` – An array of [ColumnStatisticsError](aws-glue-api-common.md#aws-glue-api-common-ColumnStatisticsError) objects. Error occurred during updating column statistics data. **Errors** + `EntityNotFoundException` + `InvalidInputException` + `InternalServiceException` + `OperationTimeoutException` + `GlueEncryptionException` ## DeleteColumnStatisticsForPartition action (Python: delete\$1column\$1statistics\$1for\$1partition) DeleteColumnStatisticsForPartition (delete\$1column\$1statistics\$1for\$1partition) Delete the partition column statistics of a column. The Identity and Access Management (IAM) permission required for this operation is `DeletePartition`. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog where the partitions in question reside. If none is supplied, the AWS account ID is used by default. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the catalog database where the partitions reside. + `TableName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the partitions' table. + `PartitionValues` – *Required:* An array of UTF-8 strings. A list of partition values identifying the partition. + `ColumnName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Name of the column. **Response** + *No Response parameters.* **Errors** + `EntityNotFoundException` + `InvalidInputException` + `InternalServiceException` + `OperationTimeoutException` + `GlueEncryptionException` # Connections API Connections The Connections API describes the data types and API related to working with connections in AWS Glue. **Topics** + [ # Connection API ](aws-glue-api-catalog-connections-connections.md) + [ # Connection Types API ](aws-glue-api-catalog-connections-connections-type.md) + [ # Connection Metadata and Preview API ](aws-glue-api-catalog-connections-connections-metadata.md) # Connection API Connections The Connection API describes AWS Glue connection data types, and the API for creating, deleting, updating, and listing connections. ## Data types  —  data types  — + [Connection structure](#aws-glue-api-catalog-connections-connections-Connection) + [ConnectionInput structure](#aws-glue-api-catalog-connections-connections-ConnectionInput) + [TestConnectionInput structure](#aws-glue-api-catalog-connections-connections-TestConnectionInput) + [PhysicalConnectionRequirements structure](#aws-glue-api-catalog-connections-connections-PhysicalConnectionRequirements) + [GetConnectionsFilter structure](#aws-glue-api-catalog-connections-connections-GetConnectionsFilter) + [AuthenticationConfiguration structure](#aws-glue-api-catalog-connections-connections-AuthenticationConfiguration) + [AuthenticationConfigurationInput structure](#aws-glue-api-catalog-connections-connections-AuthenticationConfigurationInput) + [OAuth2Properties structure](#aws-glue-api-catalog-connections-connections-OAuth2Properties) + [OAuth2PropertiesInput structure](#aws-glue-api-catalog-connections-connections-OAuth2PropertiesInput) + [OAuth2ClientApplication structure](#aws-glue-api-catalog-connections-connections-OAuth2ClientApplication) + [AuthorizationCodeProperties structure](#aws-glue-api-catalog-connections-connections-AuthorizationCodeProperties) + [BasicAuthenticationCredentials structure](#aws-glue-api-catalog-connections-connections-BasicAuthenticationCredentials) + [OAuth2Credentials structure](#aws-glue-api-catalog-connections-connections-OAuth2Credentials) ## Connection structure Connection Defines a connection to a data source. **Fields** + `Name` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the connection definition. + `Description` – Description string, not more than 2048 bytes long, matching the [URI address multi-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-uri). The description of the connection. + `ConnectionType` – UTF-8 string (valid values: `JDBC` \$1 `SFTP` \$1 `MONGODB` \$1 `KAFKA` \$1 `NETWORK` \$1 `MARKETPLACE` \$1 `CUSTOM` \$1 `SALESFORCE` \$1 `VIEW_VALIDATION_REDSHIFT` \$1 `VIEW_VALIDATION_ATHENA` \$1 `GOOGLEADS` \$1 `GOOGLESHEETS` \$1 `GOOGLEANALYTICS4` \$1 `SERVICENOW` \$1 `MARKETO` \$1 `SAPODATA` \$1 `ZENDESK` \$1 `JIRACLOUD` \$1 `NETSUITEERP` \$1 `HUBSPOT` \$1 `FACEBOOKADS` \$1 `INSTAGRAMADS` \$1 `ZOHOCRM` \$1 `SALESFORCEPARDOT` \$1 `SALESFORCEMARKETINGCLOUD` \$1 `ADOBEANALYTICS` \$1 `SLACK` \$1 `LINKEDIN` \$1 `MIXPANEL` \$1 `ASANA` \$1 `STRIPE` \$1 `SMARTSHEET` \$1 `DATADOG` \$1 `WOOCOMMERCE` \$1 `INTERCOM` \$1 `SNAPCHATADS` \$1 `PAYPAL` \$1 `QUICKBOOKS` \$1 `FACEBOOKPAGEINSIGHTS` \$1 `FRESHDESK` \$1 `TWILIO` \$1 `DOCUSIGNMONITOR` \$1 `FRESHSALES` \$1 `ZOOM` \$1 `GOOGLESEARCHCONSOLE` \$1 `SALESFORCECOMMERCECLOUD` \$1 `SAPCONCUR` \$1 `DYNATRACE` \$1 `MICROSOFTDYNAMIC365FINANCEANDOPS` \$1 `MICROSOFTTEAMS` \$1 `BLACKBAUDRAISEREDGENXT` \$1 `MAILCHIMP` \$1 `GITLAB` \$1 `PENDO` \$1 `PRODUCTBOARD` \$1 `CIRCLECI` \$1 `PIPEDIVE` \$1 `SENDGRID` \$1 `AZURECOSMOS` \$1 `AZURESQL` \$1 `BIGQUERY` \$1 `BLACKBAUD` \$1 `CLOUDERAHIVE` \$1 `CLOUDERAIMPALA` \$1 `CLOUDWATCH` \$1 `CLOUDWATCHMETRICS` \$1 `CMDB` \$1 `DATALAKEGEN2` \$1 `DB2` \$1 `DB2AS400` \$1 `DOCUMENTDB` \$1 `DOMO` \$1 `DYNAMODB` \$1 `GOOGLECLOUDSTORAGE` \$1 `HBASE` \$1 `KUSTOMER` \$1 `MICROSOFTDYNAMICS365CRM` \$1 `MONDAY` \$1 `MYSQL` \$1 `OKTA` \$1 `OPENSEARCH` \$1 `ORACLE` \$1 `PIPEDRIVE` \$1 `POSTGRESQL` \$1 `SAPHANA` \$1 `SQLSERVER` \$1 `SYNAPSE` \$1 `TERADATA` \$1 `TERADATANOS` \$1 `TIMESTREAM` \$1 `TPCDS` \$1 `VERTICA`). The type of the connection. Currently, SFTP is not supported. + `MatchCriteria` – An array of UTF-8 strings, not more than 10 strings. A list of criteria that can be used in selecting this connection. + `ConnectionProperties` – A map array of key-value pairs, not more than 100 pairs. Each key is a UTF-8 string (valid values: `HOST` \$1 `PORT` \$1 `USERNAME="USER_NAME"` \$1 `PASSWORD` \$1 `ENCRYPTED_PASSWORD` \$1 `JDBC_DRIVER_JAR_URI` \$1 `JDBC_DRIVER_CLASS_NAME` \$1 `JDBC_ENGINE` \$1 `JDBC_ENGINE_VERSION` \$1 `CONFIG_FILES` \$1 `INSTANCE_ID` \$1 `JDBC_CONNECTION_URL` \$1 `JDBC_ENFORCE_SSL` \$1 `CUSTOM_JDBC_CERT` \$1 `SKIP_CUSTOM_JDBC_CERT_VALIDATION` \$1 `CUSTOM_JDBC_CERT_STRING` \$1 `CONNECTION_URL` \$1 `KAFKA_BOOTSTRAP_SERVERS` \$1 `KAFKA_SSL_ENABLED` \$1 `KAFKA_CUSTOM_CERT` \$1 `KAFKA_SKIP_CUSTOM_CERT_VALIDATION` \$1 `KAFKA_CLIENT_KEYSTORE` \$1 `KAFKA_CLIENT_KEYSTORE_PASSWORD` \$1 `KAFKA_CLIENT_KEY_PASSWORD` \$1 `ENCRYPTED_KAFKA_CLIENT_KEYSTORE_PASSWORD` \$1 `ENCRYPTED_KAFKA_CLIENT_KEY_PASSWORD` \$1 `KAFKA_SASL_MECHANISM` \$1 `KAFKA_SASL_PLAIN_USERNAME` \$1 `KAFKA_SASL_PLAIN_PASSWORD` \$1 `ENCRYPTED_KAFKA_SASL_PLAIN_PASSWORD` \$1 `KAFKA_SASL_SCRAM_USERNAME` \$1 `KAFKA_SASL_SCRAM_PASSWORD` \$1 `KAFKA_SASL_SCRAM_SECRETS_ARN` \$1 `ENCRYPTED_KAFKA_SASL_SCRAM_PASSWORD` \$1 `KAFKA_SASL_GSSAPI_KEYTAB` \$1 `KAFKA_SASL_GSSAPI_KRB5_CONF` \$1 `KAFKA_SASL_GSSAPI_SERVICE` \$1 `KAFKA_SASL_GSSAPI_PRINCIPAL` \$1 `SECRET_ID` \$1 `CONNECTOR_URL` \$1 `CONNECTOR_TYPE` \$1 `CONNECTOR_CLASS_NAME` \$1 `ENDPOINT` \$1 `ENDPOINT_TYPE` \$1 `ROLE_ARN` \$1 `REGION` \$1 `WORKGROUP_NAME` \$1 `CLUSTER_IDENTIFIER` \$1 `DATABASE`). Each value is a Value string, not less than 1 or more than 1024 bytes long. These key-value pairs define parameters for the connection when using the version 1 Connection schema: + `HOST` - The host URI: either the fully qualified domain name (FQDN) or the IPv4 address of the database host. + `PORT` - The port number, between 1024 and 65535, of the port on which the database host is listening for database connections. + `USER_NAME` - The name under which to log in to the database. The value string for `USER_NAME` is "`USERNAME`". + `PASSWORD` - A password, if one is used, for the user name. + `ENCRYPTED_PASSWORD` - When you enable connection password protection by setting `ConnectionPasswordEncryption` in the Data Catalog encryption settings, this field stores the encrypted password. + `JDBC_DRIVER_JAR_URI` - The Amazon Simple Storage Service (Amazon S3) path of the JAR file that contains the JDBC driver to use. + `JDBC_DRIVER_CLASS_NAME` - The class name of the JDBC driver to use. + `JDBC_ENGINE` - The name of the JDBC engine to use. + `JDBC_ENGINE_VERSION` - The version of the JDBC engine to use. + `CONFIG_FILES` - (Reserved for future use.) + `INSTANCE_ID` - The instance ID to use. + `JDBC_CONNECTION_URL` - The URL for connecting to a JDBC data source. + `JDBC_ENFORCE_SSL` - A case-insensitive Boolean string (true, false) specifying whether Secure Sockets Layer (SSL) with hostname matching is enforced for the JDBC connection on the client. The default is false. + `CUSTOM_JDBC_CERT` - An Amazon S3 location specifying the customer's root certificate. AWS Glue uses this root certificate to validate the customer's certificate when connecting to the customer database. AWS Glue only handles X.509 certificates. The certificate provided must be DER-encoded and supplied in Base64 encoding PEM format. + `SKIP_CUSTOM_JDBC_CERT_VALIDATION` - By default, this is `false`. AWS Glue validates the Signature algorithm and Subject Public Key Algorithm for the customer certificate. The only permitted algorithms for the Signature algorithm are SHA256withRSA, SHA384withRSA or SHA512withRSA. For the Subject Public Key Algorithm, the key length must be at least 2048. You can set the value of this property to `true` to skip AWS Glue's validation of the customer certificate. + `CUSTOM_JDBC_CERT_STRING` - A custom JDBC certificate string which is used for domain match or distinguished name match to prevent a man-in-the-middle attack. In Oracle database, this is used as the `SSL_SERVER_CERT_DN`; in Microsoft SQL Server, this is used as the `hostNameInCertificate`. + `CONNECTION_URL` - The URL for connecting to a general (non-JDBC) data source. + `SECRET_ID` - The secret ID used for the secret manager of credentials. + `CONNECTOR_URL` - The connector URL for a MARKETPLACE or CUSTOM connection. + `CONNECTOR_TYPE` - The connector type for a MARKETPLACE or CUSTOM connection. + `CONNECTOR_CLASS_NAME` - The connector class name for a MARKETPLACE or CUSTOM connection. + `KAFKA_BOOTSTRAP_SERVERS` - A comma-separated list of host and port pairs that are the addresses of the Apache Kafka brokers in a Kafka cluster to which a Kafka client will connect to and bootstrap itself. + `KAFKA_SSL_ENABLED` - Whether to enable or disable SSL on an Apache Kafka connection. Default value is "true". + `KAFKA_CUSTOM_CERT` - The Amazon S3 URL for the private CA cert file (.pem format). The default is an empty string. + `KAFKA_SKIP_CUSTOM_CERT_VALIDATION` - Whether to skip the validation of the CA cert file or not. AWS Glue validates for three algorithms: SHA256withRSA, SHA384withRSA and SHA512withRSA. Default value is "false". + `KAFKA_CLIENT_KEYSTORE` - The Amazon S3 location of the client keystore file for Kafka client side authentication (Optional). + `KAFKA_CLIENT_KEYSTORE_PASSWORD` - The password to access the provided keystore (Optional). + `KAFKA_CLIENT_KEY_PASSWORD` - A keystore can consist of multiple keys, so this is the password to access the client key to be used with the Kafka server side key (Optional). + `ENCRYPTED_KAFKA_CLIENT_KEYSTORE_PASSWORD` - The encrypted version of the Kafka client keystore password (if the user has the AWS Glue encrypt passwords setting selected). + `ENCRYPTED_KAFKA_CLIENT_KEY_PASSWORD` - The encrypted version of the Kafka client key password (if the user has the AWS Glue encrypt passwords setting selected). + `KAFKA_SASL_MECHANISM` - `"SCRAM-SHA-512"`, `"GSSAPI"`, `"AWS_MSK_IAM"`, or `"PLAIN"`. These are the supported [SASL Mechanisms](https://www.iana.org/assignments/sasl-mechanisms/sasl-mechanisms.xhtml). + `KAFKA_SASL_PLAIN_USERNAME` - A plaintext username used to authenticate with the "PLAIN" mechanism. + `KAFKA_SASL_PLAIN_PASSWORD` - A plaintext password used to authenticate with the "PLAIN" mechanism. + `ENCRYPTED_KAFKA_SASL_PLAIN_PASSWORD` - The encrypted version of the Kafka SASL PLAIN password (if the user has the AWS Glue encrypt passwords setting selected). + `KAFKA_SASL_SCRAM_USERNAME` - A plaintext username used to authenticate with the "SCRAM-SHA-512" mechanism. + `KAFKA_SASL_SCRAM_PASSWORD` - A plaintext password used to authenticate with the "SCRAM-SHA-512" mechanism. + `ENCRYPTED_KAFKA_SASL_SCRAM_PASSWORD` - The encrypted version of the Kafka SASL SCRAM password (if the user has the AWS Glue encrypt passwords setting selected). + `KAFKA_SASL_SCRAM_SECRETS_ARN` - The Amazon Resource Name of a secret in AWS Secrets Manager. + `KAFKA_SASL_GSSAPI_KEYTAB` - The S3 location of a Kerberos `keytab` file. A keytab stores long-term keys for one or more principals. For more information, see [MIT Kerberos Documentation: Keytab](https://web.mit.edu/kerberos/krb5-latest/doc/basic/keytab_def.html). + `KAFKA_SASL_GSSAPI_KRB5_CONF` - The S3 location of a Kerberos `krb5.conf` file. A krb5.conf stores Kerberos configuration information, such as the location of the KDC server. For more information, see [MIT Kerberos Documentation: krb5.conf](https://web.mit.edu/kerberos/krb5-1.12/doc/admin/conf_files/krb5_conf.html). + `KAFKA_SASL_GSSAPI_SERVICE` - The Kerberos service name, as set with `sasl.kerberos.service.name` in your [Kafka Configuration](https://kafka.apache.org/documentation/#brokerconfigs_sasl.kerberos.service.name). + `KAFKA_SASL_GSSAPI_PRINCIPAL` - The name of the Kerberos princial used by AWS Glue. For more information, see [Kafka Documentation: Configuring Kafka Brokers](https://kafka.apache.org/documentation/#security_sasl_kerberos_clientconfig). + `ROLE_ARN` - The role to be used for running queries. + `REGION` - The AWS Region where queries will be run. + `WORKGROUP_NAME` - The name of an Amazon Redshift serverless workgroup or Amazon Athena workgroup in which queries will run. + `CLUSTER_IDENTIFIER` - The cluster identifier of an Amazon Redshift cluster in which queries will run. + `DATABASE` - The Amazon Redshift database that you are connecting to. + `SparkProperties` – A map array of key-value pairs. Each key is a UTF-8 string, not less than 1 or more than 128 bytes long. Each value is a UTF-8 string, not less than 1 or more than 2048 bytes long. Connection properties specific to the Spark compute environment. + `AthenaProperties` – A map array of key-value pairs. Each key is a UTF-8 string, not less than 1 or more than 128 bytes long. Each value is a UTF-8 string, not less than 1 or more than 2048 bytes long. Connection properties specific to the Athena compute environment. + `PythonProperties` – A map array of key-value pairs. Each key is a UTF-8 string, not less than 1 or more than 128 bytes long. Each value is a UTF-8 string, not less than 1 or more than 2048 bytes long. Connection properties specific to the Python compute environment. + `PhysicalConnectionRequirements` – A [PhysicalConnectionRequirements](#aws-glue-api-catalog-connections-connections-PhysicalConnectionRequirements) object. The physical connection requirements, such as virtual private cloud (VPC) and `SecurityGroup`, that are needed to make this connection successfully. + `CreationTime` – Timestamp. The timestamp of the time that this connection definition was created. + `LastUpdatedTime` – Timestamp. The timestamp of the last time the connection definition was updated. + `LastUpdatedBy` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The user, group, or role that last updated this connection definition. + `Status` – UTF-8 string (valid values: `READY` \$1 `IN_PROGRESS` \$1 `FAILED`). The status of the connection. Can be one of: `READY`, `IN_PROGRESS`, or `FAILED`. + `StatusReason` – UTF-8 string, not less than 1 or more than 16384 bytes long. The reason for the connection status. + `LastConnectionValidationTime` – Timestamp. A timestamp of the time this connection was last validated. + `AuthenticationConfiguration` – An [AuthenticationConfiguration](#aws-glue-api-catalog-connections-connections-AuthenticationConfiguration) object. The authentication properties of the connection. + `ConnectionSchemaVersion` – Number (integer), not less than 1 or more than 2. The version of the connection schema for this connection. Version 2 supports properties for specific compute environments. + `CompatibleComputeEnvironments` – An array of UTF-8 strings. A list of compute environments compatible with the connection. ## ConnectionInput structure ConnectionInput A structure that is used to specify a connection to create or update. **Fields** + `Name` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the connection. + `Description` – Description string, not more than 2048 bytes long, matching the [URI address multi-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-uri). The description of the connection. + `ConnectionType` – *Required:* UTF-8 string (valid values: `JDBC` \$1 `SFTP` \$1 `MONGODB` \$1 `KAFKA` \$1 `NETWORK` \$1 `MARKETPLACE` \$1 `CUSTOM` \$1 `SALESFORCE` \$1 `VIEW_VALIDATION_REDSHIFT` \$1 `VIEW_VALIDATION_ATHENA` \$1 `GOOGLEADS` \$1 `GOOGLESHEETS` \$1 `GOOGLEANALYTICS4` \$1 `SERVICENOW` \$1 `MARKETO` \$1 `SAPODATA` \$1 `ZENDESK` \$1 `JIRACLOUD` \$1 `NETSUITEERP` \$1 `HUBSPOT` \$1 `FACEBOOKADS` \$1 `INSTAGRAMADS` \$1 `ZOHOCRM` \$1 `SALESFORCEPARDOT` \$1 `SALESFORCEMARKETINGCLOUD` \$1 `ADOBEANALYTICS` \$1 `SLACK` \$1 `LINKEDIN` \$1 `MIXPANEL` \$1 `ASANA` \$1 `STRIPE` \$1 `SMARTSHEET` \$1 `DATADOG` \$1 `WOOCOMMERCE` \$1 `INTERCOM` \$1 `SNAPCHATADS` \$1 `PAYPAL` \$1 `QUICKBOOKS` \$1 `FACEBOOKPAGEINSIGHTS` \$1 `FRESHDESK` \$1 `TWILIO` \$1 `DOCUSIGNMONITOR` \$1 `FRESHSALES` \$1 `ZOOM` \$1 `GOOGLESEARCHCONSOLE` \$1 `SALESFORCECOMMERCECLOUD` \$1 `SAPCONCUR` \$1 `DYNATRACE` \$1 `MICROSOFTDYNAMIC365FINANCEANDOPS` \$1 `MICROSOFTTEAMS` \$1 `BLACKBAUDRAISEREDGENXT` \$1 `MAILCHIMP` \$1 `GITLAB` \$1 `PENDO` \$1 `PRODUCTBOARD` \$1 `CIRCLECI` \$1 `PIPEDIVE` \$1 `SENDGRID` \$1 `AZURECOSMOS` \$1 `AZURESQL` \$1 `BIGQUERY` \$1 `BLACKBAUD` \$1 `CLOUDERAHIVE` \$1 `CLOUDERAIMPALA` \$1 `CLOUDWATCH` \$1 `CLOUDWATCHMETRICS` \$1 `CMDB` \$1 `DATALAKEGEN2` \$1 `DB2` \$1 `DB2AS400` \$1 `DOCUMENTDB` \$1 `DOMO` \$1 `DYNAMODB` \$1 `GOOGLECLOUDSTORAGE` \$1 `HBASE` \$1 `KUSTOMER` \$1 `MICROSOFTDYNAMICS365CRM` \$1 `MONDAY` \$1 `MYSQL` \$1 `OKTA` \$1 `OPENSEARCH` \$1 `ORACLE` \$1 `PIPEDRIVE` \$1 `POSTGRESQL` \$1 `SAPHANA` \$1 `SQLSERVER` \$1 `SYNAPSE` \$1 `TERADATA` \$1 `TERADATANOS` \$1 `TIMESTREAM` \$1 `TPCDS` \$1 `VERTICA`). The type of the connection. Currently, these types are supported: + `JDBC` - Designates a connection to a database through Java Database Connectivity (JDBC). `JDBC` Connections use the following ConnectionParameters. + Required: All of (`HOST`, `PORT`, `JDBC_ENGINE`) or `JDBC_CONNECTION_URL`. + Required: All of (`USERNAME`, `PASSWORD`) or `SECRET_ID`. + Optional: `JDBC_ENFORCE_SSL`, `CUSTOM_JDBC_CERT`, `CUSTOM_JDBC_CERT_STRING`, `SKIP_CUSTOM_JDBC_CERT_VALIDATION`. These parameters are used to configure SSL with JDBC. + `KAFKA` - Designates a connection to an Apache Kafka streaming platform. `KAFKA` Connections use the following ConnectionParameters. + Required: `KAFKA_BOOTSTRAP_SERVERS`. + Optional: `KAFKA_SSL_ENABLED`, `KAFKA_CUSTOM_CERT`, `KAFKA_SKIP_CUSTOM_CERT_VALIDATION`. These parameters are used to configure SSL with `KAFKA`. + Optional: `KAFKA_CLIENT_KEYSTORE`, `KAFKA_CLIENT_KEYSTORE_PASSWORD`, `KAFKA_CLIENT_KEY_PASSWORD`, `ENCRYPTED_KAFKA_CLIENT_KEYSTORE_PASSWORD`, `ENCRYPTED_KAFKA_CLIENT_KEY_PASSWORD`. These parameters are used to configure TLS client configuration with SSL in `KAFKA`. + Optional: `KAFKA_SASL_MECHANISM`. Can be specified as `SCRAM-SHA-512`, `GSSAPI`, or `AWS_MSK_IAM`. + Optional: `KAFKA_SASL_SCRAM_USERNAME`, `KAFKA_SASL_SCRAM_PASSWORD`, `ENCRYPTED_KAFKA_SASL_SCRAM_PASSWORD`. These parameters are used to configure SASL/SCRAM-SHA-512 authentication with `KAFKA`. + Optional: `KAFKA_SASL_GSSAPI_KEYTAB`, `KAFKA_SASL_GSSAPI_KRB5_CONF`, `KAFKA_SASL_GSSAPI_SERVICE`, `KAFKA_SASL_GSSAPI_PRINCIPAL`. These parameters are used to configure SASL/GSSAPI authentication with `KAFKA`. + `MONGODB` - Designates a connection to a MongoDB document database. `MONGODB` Connections use the following ConnectionParameters. + Required: `CONNECTION_URL`. + Required: All of (`USERNAME`, `PASSWORD`) or `SECRET_ID`. + `VIEW_VALIDATION_REDSHIFT` - Designates a connection used for view validation by Amazon Redshift. + `VIEW_VALIDATION_ATHENA` - Designates a connection used for view validation by Amazon Athena. + `NETWORK` - Designates a network connection to a data source within an Amazon Virtual Private Cloud environment (Amazon VPC). `NETWORK` Connections do not require ConnectionParameters. Instead, provide a PhysicalConnectionRequirements. + `MARKETPLACE` - Uses configuration settings contained in a connector purchased from AWS Marketplace to read from and write to data stores that are not natively supported by AWS Glue. `MARKETPLACE` Connections use the following ConnectionParameters. + Required: `CONNECTOR_TYPE`, `CONNECTOR_URL`, `CONNECTOR_CLASS_NAME`, `CONNECTION_URL`. + Required for `JDBC` `CONNECTOR_TYPE` connections: All of (`USERNAME`, `PASSWORD`) or `SECRET_ID`. + `CUSTOM` - Uses configuration settings contained in a custom connector to read from and write to data stores that are not natively supported by AWS Glue. For more information on the connection parameters needed for a particular connector, see the documentation for the connector in [Adding an AWS Glue connection](https://docs.aws.amazon.com/glue/latest/dg/console-connections.html)in the AWS Glue User Guide. `SFTP` is not supported. For more information about how optional ConnectionProperties are used to configure features in AWS Glue, consult [AWS Glue connection properties](https://docs.aws.amazon.com/glue/latest/dg/connection-defining.html). For more information about how optional ConnectionProperties are used to configure features in AWS Glue Studio, consult [Using connectors and connections](https://docs.aws.amazon.com/glue/latest/ug/connectors-chapter.html). + `MatchCriteria` – An array of UTF-8 strings, not more than 10 strings. A list of criteria that can be used in selecting this connection. + `ConnectionProperties` – *Required:* A map array of key-value pairs, not more than 100 pairs. Each key is a UTF-8 string (valid values: `HOST` \$1 `PORT` \$1 `USERNAME="USER_NAME"` \$1 `PASSWORD` \$1 `ENCRYPTED_PASSWORD` \$1 `JDBC_DRIVER_JAR_URI` \$1 `JDBC_DRIVER_CLASS_NAME` \$1 `JDBC_ENGINE` \$1 `JDBC_ENGINE_VERSION` \$1 `CONFIG_FILES` \$1 `INSTANCE_ID` \$1 `JDBC_CONNECTION_URL` \$1 `JDBC_ENFORCE_SSL` \$1 `CUSTOM_JDBC_CERT` \$1 `SKIP_CUSTOM_JDBC_CERT_VALIDATION` \$1 `CUSTOM_JDBC_CERT_STRING` \$1 `CONNECTION_URL` \$1 `KAFKA_BOOTSTRAP_SERVERS` \$1 `KAFKA_SSL_ENABLED` \$1 `KAFKA_CUSTOM_CERT` \$1 `KAFKA_SKIP_CUSTOM_CERT_VALIDATION` \$1 `KAFKA_CLIENT_KEYSTORE` \$1 `KAFKA_CLIENT_KEYSTORE_PASSWORD` \$1 `KAFKA_CLIENT_KEY_PASSWORD` \$1 `ENCRYPTED_KAFKA_CLIENT_KEYSTORE_PASSWORD` \$1 `ENCRYPTED_KAFKA_CLIENT_KEY_PASSWORD` \$1 `KAFKA_SASL_MECHANISM` \$1 `KAFKA_SASL_PLAIN_USERNAME` \$1 `KAFKA_SASL_PLAIN_PASSWORD` \$1 `ENCRYPTED_KAFKA_SASL_PLAIN_PASSWORD` \$1 `KAFKA_SASL_SCRAM_USERNAME` \$1 `KAFKA_SASL_SCRAM_PASSWORD` \$1 `KAFKA_SASL_SCRAM_SECRETS_ARN` \$1 `ENCRYPTED_KAFKA_SASL_SCRAM_PASSWORD` \$1 `KAFKA_SASL_GSSAPI_KEYTAB` \$1 `KAFKA_SASL_GSSAPI_KRB5_CONF` \$1 `KAFKA_SASL_GSSAPI_SERVICE` \$1 `KAFKA_SASL_GSSAPI_PRINCIPAL` \$1 `SECRET_ID` \$1 `CONNECTOR_URL` \$1 `CONNECTOR_TYPE` \$1 `CONNECTOR_CLASS_NAME` \$1 `ENDPOINT` \$1 `ENDPOINT_TYPE` \$1 `ROLE_ARN` \$1 `REGION` \$1 `WORKGROUP_NAME` \$1 `CLUSTER_IDENTIFIER` \$1 `DATABASE`). Each value is a Value string, not less than 1 or more than 1024 bytes long. These key-value pairs define parameters for the connection. + `SparkProperties` – A map array of key-value pairs. Each key is a UTF-8 string, not less than 1 or more than 128 bytes long. Each value is a UTF-8 string, not less than 1 or more than 2048 bytes long. Connection properties specific to the Spark compute environment. + `AthenaProperties` – A map array of key-value pairs. Each key is a UTF-8 string, not less than 1 or more than 128 bytes long. Each value is a UTF-8 string, not less than 1 or more than 2048 bytes long. Connection properties specific to the Athena compute environment. + `PythonProperties` – A map array of key-value pairs. Each key is a UTF-8 string, not less than 1 or more than 128 bytes long. Each value is a UTF-8 string, not less than 1 or more than 2048 bytes long. Connection properties specific to the Python compute environment. + `PhysicalConnectionRequirements` – A [PhysicalConnectionRequirements](#aws-glue-api-catalog-connections-connections-PhysicalConnectionRequirements) object. The physical connection requirements, such as virtual private cloud (VPC) and `SecurityGroup`, that are needed to successfully make this connection. + `AuthenticationConfiguration` – An [AuthenticationConfigurationInput](#aws-glue-api-catalog-connections-connections-AuthenticationConfigurationInput) object. The authentication properties of the connection. + `ValidateCredentials` – Boolean. A flag to validate the credentials during create connection. Default is true. + `ValidateForComputeEnvironments` – An array of UTF-8 strings. The compute environments that the specified connection properties are validated against. ## TestConnectionInput structure TestConnectionInput A structure that is used to specify testing a connection to a service. **Fields** + `ConnectionType` – *Required:* UTF-8 string (valid values: `JDBC` \$1 `SFTP` \$1 `MONGODB` \$1 `KAFKA` \$1 `NETWORK` \$1 `MARKETPLACE` \$1 `CUSTOM` \$1 `SALESFORCE` \$1 `VIEW_VALIDATION_REDSHIFT` \$1 `VIEW_VALIDATION_ATHENA` \$1 `GOOGLEADS` \$1 `GOOGLESHEETS` \$1 `GOOGLEANALYTICS4` \$1 `SERVICENOW` \$1 `MARKETO` \$1 `SAPODATA` \$1 `ZENDESK` \$1 `JIRACLOUD` \$1 `NETSUITEERP` \$1 `HUBSPOT` \$1 `FACEBOOKADS` \$1 `INSTAGRAMADS` \$1 `ZOHOCRM` \$1 `SALESFORCEPARDOT` \$1 `SALESFORCEMARKETINGCLOUD` \$1 `ADOBEANALYTICS` \$1 `SLACK` \$1 `LINKEDIN` \$1 `MIXPANEL` \$1 `ASANA` \$1 `STRIPE` \$1 `SMARTSHEET` \$1 `DATADOG` \$1 `WOOCOMMERCE` \$1 `INTERCOM` \$1 `SNAPCHATADS` \$1 `PAYPAL` \$1 `QUICKBOOKS` \$1 `FACEBOOKPAGEINSIGHTS` \$1 `FRESHDESK` \$1 `TWILIO` \$1 `DOCUSIGNMONITOR` \$1 `FRESHSALES` \$1 `ZOOM` \$1 `GOOGLESEARCHCONSOLE` \$1 `SALESFORCECOMMERCECLOUD` \$1 `SAPCONCUR` \$1 `DYNATRACE` \$1 `MICROSOFTDYNAMIC365FINANCEANDOPS` \$1 `MICROSOFTTEAMS` \$1 `BLACKBAUDRAISEREDGENXT` \$1 `MAILCHIMP` \$1 `GITLAB` \$1 `PENDO` \$1 `PRODUCTBOARD` \$1 `CIRCLECI` \$1 `PIPEDIVE` \$1 `SENDGRID` \$1 `AZURECOSMOS` \$1 `AZURESQL` \$1 `BIGQUERY` \$1 `BLACKBAUD` \$1 `CLOUDERAHIVE` \$1 `CLOUDERAIMPALA` \$1 `CLOUDWATCH` \$1 `CLOUDWATCHMETRICS` \$1 `CMDB` \$1 `DATALAKEGEN2` \$1 `DB2` \$1 `DB2AS400` \$1 `DOCUMENTDB` \$1 `DOMO` \$1 `DYNAMODB` \$1 `GOOGLECLOUDSTORAGE` \$1 `HBASE` \$1 `KUSTOMER` \$1 `MICROSOFTDYNAMICS365CRM` \$1 `MONDAY` \$1 `MYSQL` \$1 `OKTA` \$1 `OPENSEARCH` \$1 `ORACLE` \$1 `PIPEDRIVE` \$1 `POSTGRESQL` \$1 `SAPHANA` \$1 `SQLSERVER` \$1 `SYNAPSE` \$1 `TERADATA` \$1 `TERADATANOS` \$1 `TIMESTREAM` \$1 `TPCDS` \$1 `VERTICA`). The type of connection to test. This operation is only available for the `JDBC` or `SALESFORCE` connection types. + `ConnectionProperties` – *Required:* A map array of key-value pairs, not more than 100 pairs. Each key is a UTF-8 string (valid values: `HOST` \$1 `PORT` \$1 `USERNAME="USER_NAME"` \$1 `PASSWORD` \$1 `ENCRYPTED_PASSWORD` \$1 `JDBC_DRIVER_JAR_URI` \$1 `JDBC_DRIVER_CLASS_NAME` \$1 `JDBC_ENGINE` \$1 `JDBC_ENGINE_VERSION` \$1 `CONFIG_FILES` \$1 `INSTANCE_ID` \$1 `JDBC_CONNECTION_URL` \$1 `JDBC_ENFORCE_SSL` \$1 `CUSTOM_JDBC_CERT` \$1 `SKIP_CUSTOM_JDBC_CERT_VALIDATION` \$1 `CUSTOM_JDBC_CERT_STRING` \$1 `CONNECTION_URL` \$1 `KAFKA_BOOTSTRAP_SERVERS` \$1 `KAFKA_SSL_ENABLED` \$1 `KAFKA_CUSTOM_CERT` \$1 `KAFKA_SKIP_CUSTOM_CERT_VALIDATION` \$1 `KAFKA_CLIENT_KEYSTORE` \$1 `KAFKA_CLIENT_KEYSTORE_PASSWORD` \$1 `KAFKA_CLIENT_KEY_PASSWORD` \$1 `ENCRYPTED_KAFKA_CLIENT_KEYSTORE_PASSWORD` \$1 `ENCRYPTED_KAFKA_CLIENT_KEY_PASSWORD` \$1 `KAFKA_SASL_MECHANISM` \$1 `KAFKA_SASL_PLAIN_USERNAME` \$1 `KAFKA_SASL_PLAIN_PASSWORD` \$1 `ENCRYPTED_KAFKA_SASL_PLAIN_PASSWORD` \$1 `KAFKA_SASL_SCRAM_USERNAME` \$1 `KAFKA_SASL_SCRAM_PASSWORD` \$1 `KAFKA_SASL_SCRAM_SECRETS_ARN` \$1 `ENCRYPTED_KAFKA_SASL_SCRAM_PASSWORD` \$1 `KAFKA_SASL_GSSAPI_KEYTAB` \$1 `KAFKA_SASL_GSSAPI_KRB5_CONF` \$1 `KAFKA_SASL_GSSAPI_SERVICE` \$1 `KAFKA_SASL_GSSAPI_PRINCIPAL` \$1 `SECRET_ID` \$1 `CONNECTOR_URL` \$1 `CONNECTOR_TYPE` \$1 `CONNECTOR_CLASS_NAME` \$1 `ENDPOINT` \$1 `ENDPOINT_TYPE` \$1 `ROLE_ARN` \$1 `REGION` \$1 `WORKGROUP_NAME` \$1 `CLUSTER_IDENTIFIER` \$1 `DATABASE`). Each value is a Value string, not less than 1 or more than 1024 bytes long. The key-value pairs that define parameters for the connection. JDBC connections use the following connection properties: + Required: All of (`HOST`, `PORT`, `JDBC_ENGINE`) or `JDBC_CONNECTION_URL`. + Required: All of (`USERNAME`, `PASSWORD`) or `SECRET_ID`. + Optional: `JDBC_ENFORCE_SSL`, `CUSTOM_JDBC_CERT`, `CUSTOM_JDBC_CERT_STRING`, `SKIP_CUSTOM_JDBC_CERT_VALIDATION`. These parameters are used to configure SSL with JDBC. SALESFORCE connections require the `AuthenticationConfiguration` member to be configured. + `AuthenticationConfiguration` – An [AuthenticationConfigurationInput](#aws-glue-api-catalog-connections-connections-AuthenticationConfigurationInput) object. A structure containing the authentication configuration in the TestConnection request. Required for a connection to Salesforce using OAuth authentication. ## PhysicalConnectionRequirements structure PhysicalConnectionRequirements The OAuth client app in GetConnection response. **Fields** + `SubnetId` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The subnet ID used by the connection. + `SecurityGroupIdList` – An array of UTF-8 strings, not more than 50 strings. The security group ID list used by the connection. + `AvailabilityZone` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The connection's Availability Zone. ## GetConnectionsFilter structure GetConnectionsFilter Filters the connection definitions that are returned by the `GetConnections` API operation. **Fields** + `MatchCriteria` – An array of UTF-8 strings, not more than 10 strings. A criteria string that must match the criteria recorded in the connection definition for that connection definition to be returned. + `ConnectionType` – UTF-8 string (valid values: `JDBC` \$1 `SFTP` \$1 `MONGODB` \$1 `KAFKA` \$1 `NETWORK` \$1 `MARKETPLACE` \$1 `CUSTOM` \$1 `SALESFORCE` \$1 `VIEW_VALIDATION_REDSHIFT` \$1 `VIEW_VALIDATION_ATHENA` \$1 `GOOGLEADS` \$1 `GOOGLESHEETS` \$1 `GOOGLEANALYTICS4` \$1 `SERVICENOW` \$1 `MARKETO` \$1 `SAPODATA` \$1 `ZENDESK` \$1 `JIRACLOUD` \$1 `NETSUITEERP` \$1 `HUBSPOT` \$1 `FACEBOOKADS` \$1 `INSTAGRAMADS` \$1 `ZOHOCRM` \$1 `SALESFORCEPARDOT` \$1 `SALESFORCEMARKETINGCLOUD` \$1 `ADOBEANALYTICS` \$1 `SLACK` \$1 `LINKEDIN` \$1 `MIXPANEL` \$1 `ASANA` \$1 `STRIPE` \$1 `SMARTSHEET` \$1 `DATADOG` \$1 `WOOCOMMERCE` \$1 `INTERCOM` \$1 `SNAPCHATADS` \$1 `PAYPAL` \$1 `QUICKBOOKS` \$1 `FACEBOOKPAGEINSIGHTS` \$1 `FRESHDESK` \$1 `TWILIO` \$1 `DOCUSIGNMONITOR` \$1 `FRESHSALES` \$1 `ZOOM` \$1 `GOOGLESEARCHCONSOLE` \$1 `SALESFORCECOMMERCECLOUD` \$1 `SAPCONCUR` \$1 `DYNATRACE` \$1 `MICROSOFTDYNAMIC365FINANCEANDOPS` \$1 `MICROSOFTTEAMS` \$1 `BLACKBAUDRAISEREDGENXT` \$1 `MAILCHIMP` \$1 `GITLAB` \$1 `PENDO` \$1 `PRODUCTBOARD` \$1 `CIRCLECI` \$1 `PIPEDIVE` \$1 `SENDGRID` \$1 `AZURECOSMOS` \$1 `AZURESQL` \$1 `BIGQUERY` \$1 `BLACKBAUD` \$1 `CLOUDERAHIVE` \$1 `CLOUDERAIMPALA` \$1 `CLOUDWATCH` \$1 `CLOUDWATCHMETRICS` \$1 `CMDB` \$1 `DATALAKEGEN2` \$1 `DB2` \$1 `DB2AS400` \$1 `DOCUMENTDB` \$1 `DOMO` \$1 `DYNAMODB` \$1 `GOOGLECLOUDSTORAGE` \$1 `HBASE` \$1 `KUSTOMER` \$1 `MICROSOFTDYNAMICS365CRM` \$1 `MONDAY` \$1 `MYSQL` \$1 `OKTA` \$1 `OPENSEARCH` \$1 `ORACLE` \$1 `PIPEDRIVE` \$1 `POSTGRESQL` \$1 `SAPHANA` \$1 `SQLSERVER` \$1 `SYNAPSE` \$1 `TERADATA` \$1 `TERADATANOS` \$1 `TIMESTREAM` \$1 `TPCDS` \$1 `VERTICA`). The type of connections to return. Currently, SFTP is not supported. + `ConnectionSchemaVersion` – Number (integer), not less than 1 or more than 2. Denotes if the connection was created with schema version 1 or 2. ## AuthenticationConfiguration structure AuthenticationConfiguration A structure containing the authentication configuration. **Fields** + `AuthenticationType` – UTF-8 string (valid values: `BASIC` \$1 `OAUTH2` \$1 `CUSTOM` \$1 `IAM`). A structure containing the authentication configuration. + `SecretArn` – UTF-8 string, matching the [Custom string pattern #36](aws-glue-api-common.md#regex_36). The secret manager ARN to store credentials. + `KmsKeyArn` – UTF-8 string, matching the [Custom string pattern #42](aws-glue-api-common.md#regex_42). The Amazon Resource Name (ARN) of the KMS key used to encrypt sensitive authentication information. This key is used to protect credentials and other sensitive data stored within the authentication configuration. + `OAuth2Properties` – An [OAuth2Properties](#aws-glue-api-catalog-connections-connections-OAuth2Properties) object. The properties for OAuth2 authentication. ## AuthenticationConfigurationInput structure AuthenticationConfigurationInput A structure containing the authentication configuration in the CreateConnection request. **Fields** + `AuthenticationType` – UTF-8 string (valid values: `BASIC` \$1 `OAUTH2` \$1 `CUSTOM` \$1 `IAM`). A structure containing the authentication configuration in the CreateConnection request. + `OAuth2Properties` – An [OAuth2PropertiesInput](#aws-glue-api-catalog-connections-connections-OAuth2PropertiesInput) object. The properties for OAuth2 authentication in the CreateConnection request. + `SecretArn` – UTF-8 string, matching the [Custom string pattern #36](aws-glue-api-common.md#regex_36). The secret manager ARN to store credentials in the CreateConnection request. + `KmsKeyArn` – UTF-8 string, matching the [Custom string pattern #42](aws-glue-api-common.md#regex_42). The ARN of the KMS key used to encrypt the connection. Only taken an as input in the request and stored in the Secret Manager. + `BasicAuthenticationCredentials` – A [BasicAuthenticationCredentials](#aws-glue-api-catalog-connections-connections-BasicAuthenticationCredentials) object. The credentials used when the authentication type is basic authentication. + `CustomAuthenticationCredentials` – A map array of key-value pairs. Each key is a UTF-8 string, not less than 1 or more than 128 bytes long. Each value is a UTF-8 string, not less than 1 or more than 2048 bytes long. The credentials used when the authentication type is custom authentication. ## OAuth2Properties structure OAuth2Properties A structure containing properties for OAuth2 authentication. **Fields** + `OAuth2GrantType` – UTF-8 string (valid values: `AUTHORIZATION_CODE` \$1 `CLIENT_CREDENTIALS` \$1 `JWT_BEARER`). The OAuth2 grant type. For example, `AUTHORIZATION_CODE`, `JWT_BEARER`, or `CLIENT_CREDENTIALS`. + `OAuth2ClientApplication` – An [OAuth2ClientApplication](#aws-glue-api-catalog-connections-connections-OAuth2ClientApplication) object. The client application type. For example, AWS\$1MANAGED or USER\$1MANAGED. + `TokenUrl` – UTF-8 string, not more than 256 bytes long, matching the [Custom string pattern #40](aws-glue-api-common.md#regex_40). The URL of the provider's authentication server, to exchange an authorization code for an access token. + `TokenUrlParametersMap` – A map array of key-value pairs. Each key is a UTF-8 string, not less than 1 or more than 128 bytes long. Each value is a UTF-8 string, not less than 1 or more than 512 bytes long. A map of parameters that are added to the token `GET` request. ## OAuth2PropertiesInput structure OAuth2PropertiesInput A structure containing properties for OAuth2 in the CreateConnection request. **Fields** + `OAuth2GrantType` – UTF-8 string (valid values: `AUTHORIZATION_CODE` \$1 `CLIENT_CREDENTIALS` \$1 `JWT_BEARER`). The OAuth2 grant type in the CreateConnection request. For example, `AUTHORIZATION_CODE`, `JWT_BEARER`, or `CLIENT_CREDENTIALS`. + `OAuth2ClientApplication` – An [OAuth2ClientApplication](#aws-glue-api-catalog-connections-connections-OAuth2ClientApplication) object. The client application type in the CreateConnection request. For example, `AWS_MANAGED` or `USER_MANAGED`. + `TokenUrl` – UTF-8 string, not more than 256 bytes long, matching the [Custom string pattern #40](aws-glue-api-common.md#regex_40). The URL of the provider's authentication server, to exchange an authorization code for an access token. + `TokenUrlParametersMap` – A map array of key-value pairs. Each key is a UTF-8 string, not less than 1 or more than 128 bytes long. Each value is a UTF-8 string, not less than 1 or more than 512 bytes long. A map of parameters that are added to the token `GET` request. + `AuthorizationCodeProperties` – An [AuthorizationCodeProperties](#aws-glue-api-catalog-connections-connections-AuthorizationCodeProperties) object. The set of properties required for the the OAuth2 `AUTHORIZATION_CODE` grant type. + `OAuth2Credentials` – An [OAuth2Credentials](#aws-glue-api-catalog-connections-connections-OAuth2Credentials) object. The credentials used when the authentication type is OAuth2 authentication. ## OAuth2ClientApplication structure OAuth2ClientApplication The OAuth2 client app used for the connection. **Fields** + `UserManagedClientApplicationClientId` – UTF-8 string, not more than 2048 bytes long, matching the [Custom string pattern #37](aws-glue-api-common.md#regex_37). The client application clientID if the ClientAppType is `USER_MANAGED`. + `AWSManagedClientApplicationReference` – UTF-8 string, not more than 2048 bytes long, matching the [Custom string pattern #37](aws-glue-api-common.md#regex_37). The reference to the SaaS-side client app that is AWS managed. ## AuthorizationCodeProperties structure AuthorizationCodeProperties The set of properties required for the the OAuth2 `AUTHORIZATION_CODE` grant type workflow. **Fields** + `AuthorizationCode` – UTF-8 string, not less than 1 or more than 4096 bytes long, matching the [Custom string pattern #37](aws-glue-api-common.md#regex_37). An authorization code to be used in the third leg of the `AUTHORIZATION_CODE` grant workflow. This is a single-use code which becomes invalid once exchanged for an access token, thus it is acceptable to have this value as a request parameter. + `RedirectUri` – UTF-8 string, not more than 512 bytes long, matching the [Custom string pattern #41](aws-glue-api-common.md#regex_41). The redirect URI where the user gets redirected to by authorization server when issuing an authorization code. The URI is subsequently used when the authorization code is exchanged for an access token. ## BasicAuthenticationCredentials structure BasicAuthenticationCredentials For supplying basic auth credentials when not providing a `SecretArn` value. **Fields** + `Username` – UTF-8 string, not more than 512 bytes long, matching the [Custom string pattern #37](aws-glue-api-common.md#regex_37). The username to connect to the data source. + `Password` – UTF-8 string, not more than 512 bytes long, matching the [Custom string pattern #33](aws-glue-api-common.md#regex_33). The password to connect to the data source. ## OAuth2Credentials structure OAuth2Credentials The credentials used when the authentication type is OAuth2 authentication. **Fields** + `UserManagedClientApplicationClientSecret` – UTF-8 string, not more than 512 bytes long, matching the [Custom string pattern #38](aws-glue-api-common.md#regex_38). The client application client secret if the client application is user managed. + `AccessToken` – UTF-8 string, not more than 4096 bytes long, matching the [Custom string pattern #38](aws-glue-api-common.md#regex_38). The access token used when the authentication type is OAuth2. + `RefreshToken` – UTF-8 string, not more than 4096 bytes long, matching the [Custom string pattern #38](aws-glue-api-common.md#regex_38). The refresh token used when the authentication type is OAuth2. + `JwtToken` – UTF-8 string, not more than 8000 bytes long, matching the [Custom string pattern #39](aws-glue-api-common.md#regex_39). The JSON Web Token (JWT) used when the authentication type is OAuth2. ## Operations  —  operations  — + [CreateConnection action (Python: create\$1connection)](#aws-glue-api-catalog-connections-connections-CreateConnection) + [DeleteConnection action (Python: delete\$1connection)](#aws-glue-api-catalog-connections-connections-DeleteConnection) + [GetConnection action (Python: get\$1connection)](#aws-glue-api-catalog-connections-connections-GetConnection) + [GetConnections action (Python: get\$1connections)](#aws-glue-api-catalog-connections-connections-GetConnections) + [UpdateConnection action (Python: update\$1connection)](#aws-glue-api-catalog-connections-connections-UpdateConnection) + [TestConnection action (Python: test\$1connection)](#aws-glue-api-catalog-connections-connections-TestConnection) + [BatchDeleteConnection action (Python: batch\$1delete\$1connection)](#aws-glue-api-catalog-connections-connections-BatchDeleteConnection) ## CreateConnection action (Python: create\$1connection) CreateConnection (create\$1connection) Creates a connection definition in the Data Catalog. Connections used for creating federated resources require the IAM `glue:PassConnection` permission. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog in which to create the connection. If none is provided, the AWS account ID is used by default. + `ConnectionInput` – *Required:* A [ConnectionInput](#aws-glue-api-catalog-connections-connections-ConnectionInput) object. A `ConnectionInput` object defining the connection to create. + `Tags` – A map array of key-value pairs, not more than 50 pairs. Each key is a UTF-8 string, not less than 1 or more than 128 bytes long. Each value is a UTF-8 string, not more than 256 bytes long. The tags you assign to the connection. **Response** + `CreateConnectionStatus` – UTF-8 string (valid values: `READY` \$1 `IN_PROGRESS` \$1 `FAILED`). The status of the connection creation request. The request can take some time for certain authentication types, for example when creating an OAuth connection with token exchange over VPC. **Errors** + `AlreadyExistsException` + `InvalidInputException` + `OperationTimeoutException` + `ResourceNumberLimitExceededException` + `GlueEncryptionException` ## DeleteConnection action (Python: delete\$1connection) DeleteConnection (delete\$1connection) Deletes a connection from the Data Catalog. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog in which the connection resides. If none is provided, the AWS account ID is used by default. + `ConnectionName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the connection to delete. **Response** + *No Response parameters.* **Errors** + `EntityNotFoundException` + `OperationTimeoutException` ## GetConnection action (Python: get\$1connection) GetConnection (get\$1connection) Retrieves a connection definition from the Data Catalog. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog in which the connection resides. If none is provided, the AWS account ID is used by default. + `Name` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the connection definition to retrieve. + `HidePassword` – Boolean. Allows you to retrieve the connection metadata without returning the password. For instance, the AWS Glue console uses this flag to retrieve the connection, and does not display the password. Set this parameter when the caller might not have permission to use the AWS KMS key to decrypt the password, but it does have permission to access the rest of the connection properties. + `ApplyOverrideForComputeEnvironment` – UTF-8 string (valid values: `SPARK` \$1 `ATHENA` \$1 `PYTHON`). For connections that may be used in multiple services, specifies returning properties for the specified compute environment. **Response** + `Connection` – A [Connection](#aws-glue-api-catalog-connections-connections-Connection) object. The requested connection definition. **Errors** + `EntityNotFoundException` + `OperationTimeoutException` + `InvalidInputException` + `GlueEncryptionException` ## GetConnections action (Python: get\$1connections) GetConnections (get\$1connections) Retrieves a list of connection definitions from the Data Catalog. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog in which the connections reside. If none is provided, the AWS account ID is used by default. + `Filter` – A [GetConnectionsFilter](#aws-glue-api-catalog-connections-connections-GetConnectionsFilter) object. A filter that controls which connections are returned. + `HidePassword` – Boolean. Allows you to retrieve the connection metadata without returning the password. For instance, the AWS Glue console uses this flag to retrieve the connection, and does not display the password. Set this parameter when the caller might not have permission to use the AWS KMS key to decrypt the password, but it does have permission to access the rest of the connection properties. + `NextToken` – UTF-8 string. A continuation token, if this is a continuation call. + `MaxResults` – Number (integer), not less than 1 or more than 1000. The maximum number of connections to return in one response. **Response** + `ConnectionList` – An array of [Connection](#aws-glue-api-catalog-connections-connections-Connection) objects. A list of requested connection definitions. + `NextToken` – UTF-8 string. A continuation token, if the list of connections returned does not include the last of the filtered connections. **Errors** + `EntityNotFoundException` + `OperationTimeoutException` + `InvalidInputException` + `GlueEncryptionException` ## UpdateConnection action (Python: update\$1connection) UpdateConnection (update\$1connection) Updates a connection definition in the Data Catalog. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog in which the connection resides. If none is provided, the AWS account ID is used by default. + `Name` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the connection definition to update. + `ConnectionInput` – *Required:* A [ConnectionInput](#aws-glue-api-catalog-connections-connections-ConnectionInput) object. A `ConnectionInput` object that redefines the connection in question. **Response** + *No Response parameters.* **Errors** + `InvalidInputException` + `EntityNotFoundException` + `OperationTimeoutException` + `InvalidInputException` + `GlueEncryptionException` ## TestConnection action (Python: test\$1connection) TestConnection (test\$1connection) Tests a connection to a service to validate the service credentials that you provide. You can either provide an existing connection name or a `TestConnectionInput` for testing a non-existing connection input. Providing both at the same time will cause an error. If the action is successful, the service sends back an HTTP 200 response. **Request** + `ConnectionName` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Optional. The name of the connection to test. If only name is provided, the operation will get the connection and use that for testing. + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The catalog ID where the connection resides. + `TestConnectionInput` – A [TestConnectionInput](#aws-glue-api-catalog-connections-connections-TestConnectionInput) object. A structure that is used to specify testing a connection to a service. **Response** + *No Response parameters.* **Errors** + `InvalidInputException` + `OperationTimeoutException` + `ResourceNumberLimitExceededException` + `GlueEncryptionException` + `FederationSourceException` + `AccessDeniedException` + `EntityNotFoundException` + `ConflictException` + `InternalServiceException` ## BatchDeleteConnection action (Python: batch\$1delete\$1connection) BatchDeleteConnection (batch\$1delete\$1connection) Deletes a list of connection definitions from the Data Catalog. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog in which the connections reside. If none is provided, the AWS account ID is used by default. + `ConnectionNameList` – *Required:* An array of UTF-8 strings, not more than 25 strings. A list of names of the connections to delete. **Response** + `Succeeded` – An array of UTF-8 strings. A list of names of the connection definitions that were successfully deleted. + `Errors` – A map array of key-value pairs. Each key is a UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). Each value is a An [ErrorDetail](aws-glue-api-common.md#aws-glue-api-common-ErrorDetail) object. A map of the names of connections that were not successfully deleted to error details. **Errors** + `InternalServiceException` + `OperationTimeoutException` # Connection Types API Connection types The Connection Type API describes AWS Glue APIs related to describing connection types. ## Connection management APIs  —  connection management apis  — + [DescribeConnectionType action (Python: describe\$1connection\$1type)](#aws-glue-api-catalog-connections-connections-type-DescribeConnectionType) + [ListConnectionTypes action (Python: list\$1connection\$1types)](#aws-glue-api-catalog-connections-connections-type-ListConnectionTypes) + [ConnectionTypeBrief structure](#aws-glue-api-catalog-connections-connections-type-ConnectionTypeBrief) + [ConnectionTypeVariant structure](#aws-glue-api-catalog-connections-connections-type-ConnectionTypeVariant) ## DescribeConnectionType action (Python: describe\$1connection\$1type) DescribeConnectionType (describe\$1connection\$1type) The `DescribeConnectionType` API provides full details of the supported options for a given connection type in AWS Glue. **Request** + `ConnectionType` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the connection type to be described. **Response** + `ConnectionType` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the connection type. + `Description` – UTF-8 string, not more than 1024 bytes long. A description of the connection type. + `Capabilities` – A [Capabilities](#aws-glue-api-catalog-connections-connections-type-Capabilities) object. The supported authentication types, data interface types (compute environments), and data operations of the connector. + `ConnectionProperties` – A map array of key-value pairs. Each key is a UTF-8 string, not less than 1 or more than 128 bytes long. Each value is a A [Property](#aws-glue-api-catalog-connections-connections-type-Property) object. Connection properties which are common across compute environments. + `ConnectionOptions` – A map array of key-value pairs. Each key is a UTF-8 string, not less than 1 or more than 128 bytes long. Each value is a A [Property](#aws-glue-api-catalog-connections-connections-type-Property) object. Returns properties that can be set when creating a connection in the `ConnectionInput.ConnectionProperties`. `ConnectionOptions` defines parameters that can be set in a Spark ETL script in the connection options map passed to a dataframe. + `AuthenticationConfiguration` – An [AuthConfiguration](#aws-glue-api-catalog-connections-connections-type-AuthConfiguration) object. The type of authentication used for the connection. + `ComputeEnvironmentConfigurations` – A map array of key-value pairs. Each key is a UTF-8 string, not less than 1 or more than 128 bytes long. Each value is a A [ComputeEnvironmentConfiguration](#aws-glue-api-catalog-connections-connections-type-ComputeEnvironmentConfiguration) object. The compute environments that are supported by the connection. + `PhysicalConnectionRequirements` – A map array of key-value pairs. Each key is a UTF-8 string, not less than 1 or more than 128 bytes long. Each value is a A [Property](#aws-glue-api-catalog-connections-connections-type-Property) object. Physical requirements for a connection, such as VPC, Subnet and Security Group specifications. + `AthenaConnectionProperties` – A map array of key-value pairs. Each key is a UTF-8 string, not less than 1 or more than 128 bytes long. Each value is a A [Property](#aws-glue-api-catalog-connections-connections-type-Property) object. Connection properties specific to the Athena compute environment. + `PythonConnectionProperties` – A map array of key-value pairs. Each key is a UTF-8 string, not less than 1 or more than 128 bytes long. Each value is a A [Property](#aws-glue-api-catalog-connections-connections-type-Property) object. Connection properties specific to the Python compute environment. + `SparkConnectionProperties` – A map array of key-value pairs. Each key is a UTF-8 string, not less than 1 or more than 128 bytes long. Each value is a A [Property](#aws-glue-api-catalog-connections-connections-type-Property) object. Connection properties specific to the Spark compute environment. **Errors** + `ValidationException` + `InvalidInputException` + `InternalServiceException` ## ListConnectionTypes action (Python: list\$1connection\$1types) ListConnectionTypes (list\$1connection\$1types) The `ListConnectionTypes` API provides a discovery mechanism to learn available connection types in AWS Glue. The response contains a list of connection types with high-level details of what is supported for each connection type. The connection types listed are the set of supported options for the `ConnectionType` value in the `CreateConnection` API. **Request** + `MaxResults` – Number (integer), not less than 1 or more than 1000. The maximum number of results to return. + `NextToken` – UTF-8 string, not less than 1 or more than 2048 bytes long, matching the [Custom string pattern #11](aws-glue-api-common.md#regex_11). A continuation token, if this is a continuation call. **Response** + `ConnectionTypes` – An array of [ConnectionTypeBrief](#aws-glue-api-catalog-connections-connections-type-ConnectionTypeBrief) objects. A list of `ConnectionTypeBrief` objects containing brief information about the supported connection types. + `NextToken` – UTF-8 string, not less than 1 or more than 2048 bytes long, matching the [Custom string pattern #11](aws-glue-api-common.md#regex_11). A continuation token, if the current list segment is not the last. **Errors** + `InternalServiceException` ## ConnectionTypeBrief structure ConnectionTypeBrief Brief information about a supported connection type returned by the `ListConnectionTypes` API. **Fields** + `ConnectionType` – UTF-8 string (valid values: `JDBC` \$1 `SFTP` \$1 `MONGODB` \$1 `KAFKA` \$1 `NETWORK` \$1 `MARKETPLACE` \$1 `CUSTOM` \$1 `SALESFORCE` \$1 `VIEW_VALIDATION_REDSHIFT` \$1 `VIEW_VALIDATION_ATHENA` \$1 `GOOGLEADS` \$1 `GOOGLESHEETS` \$1 `GOOGLEANALYTICS4` \$1 `SERVICENOW` \$1 `MARKETO` \$1 `SAPODATA` \$1 `ZENDESK` \$1 `JIRACLOUD` \$1 `NETSUITEERP` \$1 `HUBSPOT` \$1 `FACEBOOKADS` \$1 `INSTAGRAMADS` \$1 `ZOHOCRM` \$1 `SALESFORCEPARDOT` \$1 `SALESFORCEMARKETINGCLOUD` \$1 `ADOBEANALYTICS` \$1 `SLACK` \$1 `LINKEDIN` \$1 `MIXPANEL` \$1 `ASANA` \$1 `STRIPE` \$1 `SMARTSHEET` \$1 `DATADOG` \$1 `WOOCOMMERCE` \$1 `INTERCOM` \$1 `SNAPCHATADS` \$1 `PAYPAL` \$1 `QUICKBOOKS` \$1 `FACEBOOKPAGEINSIGHTS` \$1 `FRESHDESK` \$1 `TWILIO` \$1 `DOCUSIGNMONITOR` \$1 `FRESHSALES` \$1 `ZOOM` \$1 `GOOGLESEARCHCONSOLE` \$1 `SALESFORCECOMMERCECLOUD` \$1 `SAPCONCUR` \$1 `DYNATRACE` \$1 `MICROSOFTDYNAMIC365FINANCEANDOPS` \$1 `MICROSOFTTEAMS` \$1 `BLACKBAUDRAISEREDGENXT` \$1 `MAILCHIMP` \$1 `GITLAB` \$1 `PENDO` \$1 `PRODUCTBOARD` \$1 `CIRCLECI` \$1 `PIPEDIVE` \$1 `SENDGRID` \$1 `AZURECOSMOS` \$1 `AZURESQL` \$1 `BIGQUERY` \$1 `BLACKBAUD` \$1 `CLOUDERAHIVE` \$1 `CLOUDERAIMPALA` \$1 `CLOUDWATCH` \$1 `CLOUDWATCHMETRICS` \$1 `CMDB` \$1 `DATALAKEGEN2` \$1 `DB2` \$1 `DB2AS400` \$1 `DOCUMENTDB` \$1 `DOMO` \$1 `DYNAMODB` \$1 `GOOGLECLOUDSTORAGE` \$1 `HBASE` \$1 `KUSTOMER` \$1 `MICROSOFTDYNAMICS365CRM` \$1 `MONDAY` \$1 `MYSQL` \$1 `OKTA` \$1 `OPENSEARCH` \$1 `ORACLE` \$1 `PIPEDRIVE` \$1 `POSTGRESQL` \$1 `SAPHANA` \$1 `SQLSERVER` \$1 `SYNAPSE` \$1 `TERADATA` \$1 `TERADATANOS` \$1 `TIMESTREAM` \$1 `TPCDS` \$1 `VERTICA`). The name of the connection type. + `DisplayName` – UTF-8 string, not less than 1 or more than 128 bytes long. The human-readable name for the connection type that is displayed in the AWS Glue console. + `Vendor` – UTF-8 string, not less than 1 or more than 128 bytes long. The name of the vendor or provider that created or maintains this connection type. + `Description` – UTF-8 string, not more than 1024 bytes long. A description of the connection type. + `Categories` – . A list of categories that this connection type belongs to. Categories help users filter and find appropriate connection types based on their use cases. + `Capabilities` – A [Capabilities](#aws-glue-api-catalog-connections-connections-type-Capabilities) object. The supported authentication types, data interface types (compute environments), and data operations of the connector. + `LogoUrl` – UTF-8 string. The URL of the logo associated with a connection type. + `ConnectionTypeVariants` – An array of [ConnectionTypeVariant](#aws-glue-api-catalog-connections-connections-type-ConnectionTypeVariant) objects. A list of variants available for this connection type. Different variants may provide specialized configurations for specific use cases or implementations of the same general connection type. ## ConnectionTypeVariant structure ConnectionTypeVariant Represents a variant of a connection type in AWS Glue Data Catalog. Connection type variants provide specific configurations and behaviors for different implementations of the same general connection type. **Fields** + `ConnectionTypeVariantName` – UTF-8 string, not less than 1 or more than 128 bytes long. The unique identifier for the connection type variant. This name is used internally to identify the specific variant of a connection type. + `DisplayName` – UTF-8 string, not less than 1 or more than 128 bytes long. The human-readable name for the connection type variant that is displayed in the AWS Glue console. + `Description` – UTF-8 string, not more than 1024 bytes long. A detailed description of the connection type variant, including its purpose, use cases, and any specific configuration requirements. + `LogoUrl` – UTF-8 string. The URL of the logo associated with a connection type variant. ## datatypes  —  datatypes  — + [Validation structure](#aws-glue-api-catalog-connections-connections-type-Validation) + [AuthConfiguration structure](#aws-glue-api-catalog-connections-connections-type-AuthConfiguration) + [Capabilities structure](#aws-glue-api-catalog-connections-connections-type-Capabilities) + [Property structure](#aws-glue-api-catalog-connections-connections-type-Property) + [AllowedValue structure](#aws-glue-api-catalog-connections-connections-type-AllowedValue) + [ComputeEnvironmentConfiguration structure](#aws-glue-api-catalog-connections-connections-type-ComputeEnvironmentConfiguration) ## Validation structure Validation Defines how a validation is performed on a connection property. **Fields** + `ValidationType` – *Required:* UTF-8 string (valid values: `REGEX` \$1 `RANGE`). The type of validation to be performed, such as `REGEX`. + `Patterns` – . A list of patterns that apply to the validation. + `Description` – *Required:* UTF-8 string, not less than 1 or more than 1024 bytes long. A description for the validation. + `MaxLength` – Number (integer). A maximum length for a string connection property. + `Maximum` – Number (integer). A maximum value when specifying a `RANGE` type of validation. + `Minimum` – Number (integer). A minimum value when specifying a `RANGE` type of validation. ## AuthConfiguration structure AuthConfiguration The authentication configuration for a connection returned by the `DescribeConnectionType` API. **Fields** + `AuthenticationType` – *Required:* A [Property](#aws-glue-api-catalog-connections-connections-type-Property) object. The type of authentication for a connection. + `SecretArn` – A [Property](#aws-glue-api-catalog-connections-connections-type-Property) object. The Amazon Resource Name (ARN) for the Secrets Manager. + `OAuth2Properties` – A map array of key-value pairs. Each key is a UTF-8 string, not less than 1 or more than 128 bytes long. Each value is a A [Property](#aws-glue-api-catalog-connections-connections-type-Property) object. A map of key-value pairs for the OAuth2 properties. Each value is a a `Property` object. + `BasicAuthenticationProperties` – A map array of key-value pairs. Each key is a UTF-8 string, not less than 1 or more than 128 bytes long. Each value is a A [Property](#aws-glue-api-catalog-connections-connections-type-Property) object. A map of key-value pairs for the OAuth2 properties. Each value is a a `Property` object. + `CustomAuthenticationProperties` – A map array of key-value pairs. Each key is a UTF-8 string, not less than 1 or more than 128 bytes long. Each value is a A [Property](#aws-glue-api-catalog-connections-connections-type-Property) object. A map of key-value pairs for the custom authentication properties. Each value is a a `Property` object. ## Capabilities structure Capabilities Specifies the supported authentication types returned by the `DescribeConnectionType` API. **Fields** + `SupportedAuthenticationTypes` – *Required:* An array of UTF-8 strings. A list of supported authentication types. + `SupportedDataOperations` – *Required:* An array of UTF-8 strings. A list of supported data operations. + `SupportedComputeEnvironments` – *Required:* An array of UTF-8 strings. A list of supported compute environments. ## Property structure Property An object that defines a connection type for a compute environment. **Fields** + `Name` – *Required:* UTF-8 string, not less than 1 or more than 128 bytes long. The name of the property. + `Description` – *Required:* UTF-8 string, not more than 1024 bytes long. A description of the property. + `Required` – *Required:* Boolean. Indicates whether the property is required. + `PropertyTypes` – *Required:* An array of UTF-8 strings. Describes the type of property. + `AllowedValues` – An array of [AllowedValue](#aws-glue-api-catalog-connections-connections-type-AllowedValue) objects. A list of `AllowedValue` objects representing the values allowed for the property. + `DataOperationScopes` – An array of UTF-8 strings. Indicates which data operations are applicable to the property. ## AllowedValue structure AllowedValue An object representing a value allowed for a property. **Fields** + `Description` – UTF-8 string, not more than 1024 bytes long. A description of the allowed value. + `Value` – *Required:* UTF-8 string, not less than 1 or more than 128 bytes long. The value allowed for the property. ## ComputeEnvironmentConfiguration structure ComputeEnvironmentConfiguration An object containing configuration for a compute environment (such as Spark, Python or Athena) returned by the `DescribeConnectionType` API. **Fields** + `Name` – *Required:* UTF-8 string, not less than 1 or more than 128 bytes long. A name for the compute environment configuration. + `Description` – *Required:* UTF-8 string, not more than 1024 bytes long. A description of the compute environment. + `ComputeEnvironment` – *Required:* UTF-8 string (valid values: `SPARK` \$1 `ATHENA` \$1 `PYTHON`). The type of compute environment. + `SupportedAuthenticationTypes` – *Required:* An array of UTF-8 strings. The supported authentication types for the compute environment. + `ConnectionOptions` – *Required:* A map array of key-value pairs. Each key is a UTF-8 string, not less than 1 or more than 128 bytes long. Each value is a A [Property](#aws-glue-api-catalog-connections-connections-type-Property) object. The parameters used as connection options for the compute environment. + `ConnectionPropertyNameOverrides` – *Required:* A map array of key-value pairs. Each key is a UTF-8 string, not less than 1 or more than 128 bytes long. Each value is a UTF-8 string, not less than 1 or more than 128 bytes long. The connection property name overrides for the compute environment. + `ConnectionOptionNameOverrides` – *Required:* A map array of key-value pairs. Each key is a UTF-8 string, not less than 1 or more than 128 bytes long. Each value is a UTF-8 string, not less than 1 or more than 128 bytes long. The connection option name overrides for the compute environment. + `ConnectionPropertiesRequiredOverrides` – *Required:* . The connection properties that are required as overrides for the compute environment. + `PhysicalConnectionPropertiesRequired` – Boolean. Indicates whether `PhysicalConnectionProperties` are required for the compute environment. # Connection Metadata and Preview API Connection metadata The following connection APIs describe operations for describing connection metadata. ## Data types  —  data types  — + [Entity structure](#aws-glue-api-catalog-connections-connections-metadata-Entity) + [Field structure](#aws-glue-api-catalog-connections-connections-metadata-Field) ## Entity structure Entity An entity supported by a given `ConnectionType`. **Fields** + `EntityName` – UTF-8 string. The name of the entity. + `Label` – UTF-8 string. Label used for the entity. + `IsParentEntity` – Boolean. A Boolean value which helps to determine whether there are sub objects that can be listed. + `Description` – UTF-8 string. A description of the entity. + `Category` – UTF-8 string. The type of entities that are present in the response. This value depends on the source connection. For example this is `SObjects` for Salesforce and `databases` or `schemas` or `tables` for sources like Amazon Redshift. + `CustomProperties` – An optional map of keys which may be returned for an entity by a connector. ## Field structure Field The `Field` object has information about the different properties associated with a field in the connector. **Fields** + `FieldName` – UTF-8 string. A unique identifier for the field. + `Label` – UTF-8 string. A readable label used for the field. + `Description` – UTF-8 string. A description of the field. + `FieldType` – UTF-8 string (valid values: `INT` \$1 `SMALLINT` \$1 `BIGINT` \$1 `FLOAT` \$1 `LONG` \$1 `DATE` \$1 `BOOLEAN` \$1 `MAP` \$1 `ARRAY` \$1 `STRING` \$1 `TIMESTAMP` \$1 `DECIMAL` \$1 `BYTE` \$1 `SHORT` \$1 `DOUBLE` \$1 `STRUCT`). The type of data in the field. + `IsPrimaryKey` – Boolean. Indicates whether this field can used as a primary key for the given entity. + `IsNullable` – Boolean. Indicates whether this field can be nullable or not. + `IsRetrievable` – Boolean. Indicates whether this field can be added in Select clause of SQL query or whether it is retrievable or not. + `IsFilterable` – Boolean. Indicates whether this field can used in a filter clause (`WHERE` clause) of a SQL statement when querying data. + `IsPartitionable` – Boolean. Indicates whether a given field can be used in partitioning the query made to SaaS. + `IsCreateable` – Boolean. Indicates whether this field can be created as part of a destination write. + `IsUpdateable` – Boolean. Indicates whether this field can be updated as part of a destination write. + `IsUpsertable` – Boolean. Indicates whether this field can be upserted as part of a destination write. + `IsDefaultOnCreate` – Boolean. Indicates whether this field is populated automatically when the object is created, such as a created at timestamp. + `SupportedValues` – . A list of supported values for the field. + `SupportedFilterOperators` – An array of UTF-8 strings. Indicates the support filter operators for this field. + `CustomProperties` – Optional map of keys which may be returned. ## Operations  —  operations  — + [ListEntities action (Python: list\$1entities)](#aws-glue-api-catalog-connections-connections-metadata-ListEntities) + [DescribeEntity action (Python: describe\$1entity)](#aws-glue-api-catalog-connections-connections-metadata-DescribeEntity) + [GetEntityRecords action (Python: get\$1entity\$1records)](#aws-glue-api-catalog-connections-connections-metadata-GetEntityRecords) ## ListEntities action (Python: list\$1entities) ListEntities (list\$1entities) Returns the available entities supported by the connection type. **Request** + `ConnectionName` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). A name for the connection that has required credentials to query any connection type. + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The catalog ID of the catalog that contains the connection. This can be null, By default, the AWS Account ID is the catalog ID. + `ParentEntityName` – UTF-8 string. Name of the parent entity for which you want to list the children. This parameter takes a fully-qualified path of the entity in order to list the child entities. + `NextToken` – UTF-8 string, not less than 1 or more than 2048 bytes long, matching the [Custom string pattern #11](aws-glue-api-common.md#regex_11). A continuation token, included if this is a continuation call. + `DataStoreApiVersion` – UTF-8 string, not less than 1 or more than 256 bytes long, matching the [Custom string pattern #23](aws-glue-api-common.md#regex_23). The API version of the SaaS connector. **Response** + `Entities` – An array of [Entity](#aws-glue-api-catalog-connections-connections-metadata-Entity) objects. A list of `Entity` objects. + `NextToken` – UTF-8 string, not less than 1 or more than 2048 bytes long, matching the [Custom string pattern #11](aws-glue-api-common.md#regex_11). A continuation token, present if the current segment is not the last. **Errors** + `EntityNotFoundException` + `OperationTimeoutException` + `InvalidInputException` + `GlueEncryptionException` + `ValidationException` + `FederationSourceException` + `AccessDeniedException` ## DescribeEntity action (Python: describe\$1entity) DescribeEntity (describe\$1entity) Provides details regarding the entity used with the connection type, with a description of the data model for each field in the selected entity. The response includes all the fields which make up the entity. **Request** + `ConnectionName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the connection that contains the connection type credentials. + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The catalog ID of the catalog that contains the connection. This can be null, By default, the AWS Account ID is the catalog ID. + `EntityName` – *Required:* UTF-8 string. The name of the entity that you want to describe from the connection type. + `NextToken` – UTF-8 string, not less than 1 or more than 2048 bytes long, matching the [Custom string pattern #11](aws-glue-api-common.md#regex_11). A continuation token, included if this is a continuation call. + `DataStoreApiVersion` – UTF-8 string, not less than 1 or more than 256 bytes long, matching the [Custom string pattern #23](aws-glue-api-common.md#regex_23). The version of the API used for the data store. **Response** + `Fields` – An array of [Field](#aws-glue-api-catalog-connections-connections-metadata-Field) objects. Describes the fields for that connector entity. This is the list of `Field` objects. `Field` is very similar to column in a database. The `Field` object has information about different properties associated with fields in the connector. + `NextToken` – UTF-8 string, not less than 1 or more than 2048 bytes long, matching the [Custom string pattern #11](aws-glue-api-common.md#regex_11). A continuation token, present if the current segment is not the last. **Errors** + `EntityNotFoundException` + `OperationTimeoutException` + `InvalidInputException` + `GlueEncryptionException` + `ValidationException` + `FederationSourceException` + `AccessDeniedException` ## GetEntityRecords action (Python: get\$1entity\$1records) GetEntityRecords (get\$1entity\$1records) This API is used to query preview data from a given connection type or from a native Amazon S3 based AWS Glue Data Catalog. Returns records as an array of JSON blobs. Each record is formatted using Jackson JsonNode based on the field type defined by the `DescribeEntity` API. Spark connectors generate schemas according to the same data type mapping as in the `DescribeEntity` API. Spark connectors convert data to the appropriate data types matching the schema when returning rows. **Request** + `ConnectionName` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the connection that contains the connection type credentials. + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The catalog ID of the catalog that contains the connection. This can be null, By default, the AWS Account ID is the catalog ID. + `EntityName` – *Required:* UTF-8 string. Name of the entity that we want to query the preview data from the given connection type. + `NextToken` – UTF-8 string, not less than 1 or more than 2048 bytes long, matching the [Custom string pattern #11](aws-glue-api-common.md#regex_11). A continuation token, included if this is a continuation call. + `DataStoreApiVersion` – UTF-8 string, not less than 1 or more than 256 bytes long, matching the [Custom string pattern #23](aws-glue-api-common.md#regex_23). The API version of the SaaS connector. + `ConnectionOptions` – A map array of key-value pairs, not more than 100 pairs. Each key is a UTF-8 string, not less than 1 or more than 256 bytes long, matching the [Custom string pattern #18](aws-glue-api-common.md#regex_18). Each value is a UTF-8 string, not less than 1 or more than 256 bytes long, matching the [Custom string pattern #17](aws-glue-api-common.md#regex_17). Connector options that are required to query the data. + `FilterPredicate` – UTF-8 string, not less than 1 or more than 100000 bytes long. A filter predicate that you can apply in the query request. + `Limit` – *Required:* Number (long), not less than 1 or more than 1000. Limits the number of records fetched with the request. + `SelectedFields` – An array of UTF-8 strings, not less than 1 or more than 1000 strings. List of fields that we want to fetch as part of preview data. **Response** + `Records` – An array of a structures. A list of the requested objects. + `NextToken` – UTF-8 string, not less than 1 or more than 2048 bytes long, matching the [Custom string pattern #11](aws-glue-api-common.md#regex_11). A continuation token, present if the current segment is not the last. **Errors** + `EntityNotFoundException` + `OperationTimeoutException` + `InvalidInputException` + `GlueEncryptionException` + `ValidationException` + `FederationSourceException` + `AccessDeniedException` # User-defined Function API User-defined Functions The User-defined Function API describes AWS Glue data types and operations used in working with functions. ## Data types  —  data types  — + [UserDefinedFunction structure](#aws-glue-api-catalog-functions-UserDefinedFunction) + [UserDefinedFunctionInput structure](#aws-glue-api-catalog-functions-UserDefinedFunctionInput) ## UserDefinedFunction structure UserDefinedFunction Represents the equivalent of a Hive user-defined function (`UDF`) definition. **Fields** + `FunctionName` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the function. + `DatabaseName` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the catalog database that contains the function. + `ClassName` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The Java class that contains the function code. + `OwnerName` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The owner of the function. + `OwnerType` – UTF-8 string (valid values: `USER` \$1 `ROLE` \$1 `GROUP`). The owner type. + `CreateTime` – Timestamp. The time at which the function was created. + `ResourceUris` – An array of [ResourceUri](aws-glue-api-common.md#aws-glue-api-common-ResourceUri) objects, not more than 1000 structures. The resource URIs for the function. + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog in which the function resides. ## UserDefinedFunctionInput structure UserDefinedFunctionInput A structure used to create or update a user-defined function. **Fields** + `FunctionName` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the function. + `ClassName` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The Java class that contains the function code. + `OwnerName` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The owner of the function. + `OwnerType` – UTF-8 string (valid values: `USER` \$1 `ROLE` \$1 `GROUP`). The owner type. + `ResourceUris` – An array of [ResourceUri](aws-glue-api-common.md#aws-glue-api-common-ResourceUri) objects, not more than 1000 structures. The resource URIs for the function. ## Operations  —  operations  — + [CreateUserDefinedFunction action (Python: create\$1user\$1defined\$1function)](#aws-glue-api-catalog-functions-CreateUserDefinedFunction) + [UpdateUserDefinedFunction action (Python: update\$1user\$1defined\$1function)](#aws-glue-api-catalog-functions-UpdateUserDefinedFunction) + [DeleteUserDefinedFunction action (Python: delete\$1user\$1defined\$1function)](#aws-glue-api-catalog-functions-DeleteUserDefinedFunction) + [GetUserDefinedFunction action (Python: get\$1user\$1defined\$1function)](#aws-glue-api-catalog-functions-GetUserDefinedFunction) + [GetUserDefinedFunctions action (Python: get\$1user\$1defined\$1functions)](#aws-glue-api-catalog-functions-GetUserDefinedFunctions) ## CreateUserDefinedFunction action (Python: create\$1user\$1defined\$1function) CreateUserDefinedFunction (create\$1user\$1defined\$1function) Creates a new function definition in the Data Catalog. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog in which to create the function. If none is provided, the AWS account ID is used by default. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the catalog database in which to create the function. + `FunctionInput` – *Required:* An [UserDefinedFunctionInput](#aws-glue-api-catalog-functions-UserDefinedFunctionInput) object. A `FunctionInput` object that defines the function to create in the Data Catalog. **Response** + *No Response parameters.* **Errors** + `AlreadyExistsException` + `InvalidInputException` + `InternalServiceException` + `EntityNotFoundException` + `OperationTimeoutException` + `ResourceNumberLimitExceededException` + `GlueEncryptionException` ## UpdateUserDefinedFunction action (Python: update\$1user\$1defined\$1function) UpdateUserDefinedFunction (update\$1user\$1defined\$1function) Updates an existing function definition in the Data Catalog. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog where the function to be updated is located. If none is provided, the AWS account ID is used by default. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the catalog database where the function to be updated is located. + `FunctionName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the function. + `FunctionInput` – *Required:* An [UserDefinedFunctionInput](#aws-glue-api-catalog-functions-UserDefinedFunctionInput) object. A `FunctionInput` object that redefines the function in the Data Catalog. **Response** + *No Response parameters.* **Errors** + `EntityNotFoundException` + `InvalidInputException` + `InternalServiceException` + `OperationTimeoutException` + `GlueEncryptionException` ## DeleteUserDefinedFunction action (Python: delete\$1user\$1defined\$1function) DeleteUserDefinedFunction (delete\$1user\$1defined\$1function) Deletes an existing function definition from the Data Catalog. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog where the function to be deleted is located. If none is supplied, the AWS account ID is used by default. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the catalog database where the function is located. + `FunctionName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the function definition to be deleted. **Response** + *No Response parameters.* **Errors** + `EntityNotFoundException` + `InvalidInputException` + `InternalServiceException` + `OperationTimeoutException` ## GetUserDefinedFunction action (Python: get\$1user\$1defined\$1function) GetUserDefinedFunction (get\$1user\$1defined\$1function) Retrieves a specified function definition from the Data Catalog. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog where the function to be retrieved is located. If none is provided, the AWS account ID is used by default. + `DatabaseName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the catalog database where the function is located. + `FunctionName` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the function. **Response** + `UserDefinedFunction` – An [UserDefinedFunction](#aws-glue-api-catalog-functions-UserDefinedFunction) object. The requested function definition. **Errors** + `EntityNotFoundException` + `InvalidInputException` + `InternalServiceException` + `OperationTimeoutException` + `GlueEncryptionException` ## GetUserDefinedFunctions action (Python: get\$1user\$1defined\$1functions) GetUserDefinedFunctions (get\$1user\$1defined\$1functions) Retrieves multiple function definitions from the Data Catalog. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the Data Catalog where the functions to be retrieved are located. If none is provided, the AWS account ID is used by default. + `DatabaseName` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the catalog database where the functions are located. If none is provided, functions from all the databases across the catalog will be returned. + `Pattern` – *Required:* UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). An optional function-name pattern string that filters the function definitions returned. + `NextToken` – UTF-8 string. A continuation token, if this is a continuation call. + `MaxResults` – Number (integer), not less than 1 or more than 100. The maximum number of functions to return in one response. **Response** + `UserDefinedFunctions` – An array of [UserDefinedFunction](#aws-glue-api-catalog-functions-UserDefinedFunction) objects. A list of requested function definitions. + `NextToken` – UTF-8 string. A continuation token, if the list of functions returned does not include the last requested function. **Errors** + `EntityNotFoundException` + `InvalidInputException` + `OperationTimeoutException` + `InternalServiceException` + `GlueEncryptionException` # Importing an Athena catalog to AWS Glue Importing an Athena catalog The Migration API describes AWS Glue data types and operations having to do with migrating an Athena Data catalog to AWS Glue. ## Data types  —  data types  — + [CatalogImportStatus structure](#aws-glue-api-catalog-migration-CatalogImportStatus) ## CatalogImportStatus structure CatalogImportStatus A structure containing migration status information. **Fields** + `ImportCompleted` – Boolean. `True` if the migration has completed, or `False` otherwise. + `ImportTime` – Timestamp. The time that the migration was started. + `ImportedBy` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The name of the person who initiated the migration. ## Operations  —  operations  — + [ImportCatalogToGlue action (Python: import\$1catalog\$1to\$1glue)](#aws-glue-api-catalog-migration-ImportCatalogToGlue) + [GetCatalogImportStatus action (Python: get\$1catalog\$1import\$1status)](#aws-glue-api-catalog-migration-GetCatalogImportStatus) ## ImportCatalogToGlue action (Python: import\$1catalog\$1to\$1glue) ImportCatalogToGlue (import\$1catalog\$1to\$1glue) Imports an existing Amazon Athena Data Catalog to AWS Glue. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the catalog to import. Currently, this should be the AWS account ID. **Response** + *No Response parameters.* **Errors** + `InternalServiceException` + `OperationTimeoutException` ## GetCatalogImportStatus action (Python: get\$1catalog\$1import\$1status) GetCatalogImportStatus (get\$1catalog\$1import\$1status) Retrieves the status of a migration operation. **Request** + `CatalogId` – Catalog id string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](aws-glue-api-common.md#aws-glue-api-regex-oneLine). The ID of the catalog to migrate. Currently, this should be the AWS account ID. **Response** + `ImportStatus` – A [CatalogImportStatus](#aws-glue-api-catalog-migration-CatalogImportStatus) object. The status of the specified catalog migration. **Errors** + `InternalServiceException` + `OperationTimeoutException`