/AWS1/CL_GLUCRAWLER

Specifies a crawler program that examines a data source and uses classifiers to try to determine its schema. If successful, the crawler records metadata concerning the data source in the Glue Data Catalog.

CONSTRUCTOR

IMPORTING

Optional arguments:

iv_name TYPE /AWS1/GLUNAMESTRING /AWS1/GLUNAMESTRING

The name of the crawler.

iv_role TYPE /AWS1/GLUROLE /AWS1/GLUROLE

The Amazon Resource Name (ARN) of an IAM role that's used to access customer resources, such as Amazon Simple Storage Service (Amazon S3) data.

io_targets TYPE REF TO /AWS1/CL_GLUCRAWLERTARGETS /AWS1/CL_GLUCRAWLERTARGETS

A collection of targets to crawl.

iv_databasename TYPE /AWS1/GLUDATABASENAME /AWS1/GLUDATABASENAME

The name of the database in which the crawler's output is stored.

iv_description TYPE /AWS1/GLUDESCRIPTIONSTRING /AWS1/GLUDESCRIPTIONSTRING

A description of the crawler.

it_classifiers TYPE /AWS1/CL_GLUCLIFIERNAMELIST_W=>TT_CLASSIFIERNAMELIST TT_CLASSIFIERNAMELIST

A list of UTF-8 strings that specify the custom classifiers that are associated with the crawler.

io_recrawlpolicy TYPE REF TO /AWS1/CL_GLURECRAWLPOLICY /AWS1/CL_GLURECRAWLPOLICY

A policy that specifies whether to crawl the entire dataset again, or to crawl only folders that were added since the last crawler run.

io_schemachangepolicy TYPE REF TO /AWS1/CL_GLUSCHEMACHANGEPOLICY /AWS1/CL_GLUSCHEMACHANGEPOLICY

The policy that specifies update and delete behaviors for the crawler.

io_lineageconfiguration TYPE REF TO /AWS1/CL_GLULINEAGECONF /AWS1/CL_GLULINEAGECONF

A configuration that specifies whether data lineage is enabled for the crawler.

iv_state TYPE /AWS1/GLUCRAWLERSTATE /AWS1/GLUCRAWLERSTATE

Indicates whether the crawler is running, or whether a run is pending.

iv_tableprefix TYPE /AWS1/GLUTABLEPREFIX /AWS1/GLUTABLEPREFIX

The prefix added to the names of tables that are created.

io_schedule TYPE REF TO /AWS1/CL_GLUSCHEDULE /AWS1/CL_GLUSCHEDULE

For scheduled crawlers, the schedule when the crawler runs.

iv_crawlelapsedtime TYPE /AWS1/GLUMILLISECONDSCOUNT /AWS1/GLUMILLISECONDSCOUNT

If the crawler is running, contains the total time elapsed since the last crawl began.

iv_creationtime TYPE /AWS1/GLUTIMESTAMP /AWS1/GLUTIMESTAMP

The time that the crawler was created.

iv_lastupdated TYPE /AWS1/GLUTIMESTAMP /AWS1/GLUTIMESTAMP

The time that the crawler was last updated.

io_lastcrawl TYPE REF TO /AWS1/CL_GLULASTCRAWLINFO /AWS1/CL_GLULASTCRAWLINFO

The status of the last crawl, and potentially error information if an error occurred.

iv_version TYPE /AWS1/GLUVERSIONID /AWS1/GLUVERSIONID

The version of the crawler.

iv_configuration TYPE /AWS1/GLUCRAWLERCONFIGURATION /AWS1/GLUCRAWLERCONFIGURATION

Crawler configuration information. This versioned JSON string allows users to specify aspects of a crawler's behavior. For more information, see Setting crawler configuration options.

iv_crawlersecurityconf TYPE /AWS1/GLUCRAWLERSECURITYCONF /AWS1/GLUCRAWLERSECURITYCONF

The name of the SecurityConfiguration structure to be used by this crawler.

io_lakeformationconf TYPE REF TO /AWS1/CL_GLULAKEFORMATIONCONF /AWS1/CL_GLULAKEFORMATIONCONF

Specifies whether the crawler should use Lake Formation credentials for the crawler instead of the IAM role credentials.

iv_catalogid TYPE /AWS1/GLUCATALOGIDSTRING /AWS1/GLUCATALOGIDSTRING

The ID of the Data Catalog in which the crawler's output is stored.

---

Queryable Attributes

Name

The name of the crawler.

Accessible with the following methods

Method	Description
GET_NAME()	Getter for NAME, with configurable default
ASK_NAME()	Getter for NAME w/ exceptions if field has no value
HAS_NAME()	Determine if NAME has a value

Role

The Amazon Resource Name (ARN) of an IAM role that's used to access customer resources, such as Amazon Simple Storage Service (Amazon S3) data.

Accessible with the following methods

Method	Description
GET_ROLE()	Getter for ROLE, with configurable default
ASK_ROLE()	Getter for ROLE w/ exceptions if field has no value
HAS_ROLE()	Determine if ROLE has a value

Targets

A collection of targets to crawl.

Accessible with the following methods

Method	Description
GET_TARGETS()	Getter for TARGETS

DatabaseName

The name of the database in which the crawler's output is stored.

Accessible with the following methods

Method	Description
GET_DATABASENAME()	Getter for DATABASENAME, with configurable default
ASK_DATABASENAME()	Getter for DATABASENAME w/ exceptions if field has no value
HAS_DATABASENAME()	Determine if DATABASENAME has a value

Description

A description of the crawler.

Accessible with the following methods

Method	Description
GET_DESCRIPTION()	Getter for DESCRIPTION, with configurable default
ASK_DESCRIPTION()	Getter for DESCRIPTION w/ exceptions if field has no value
HAS_DESCRIPTION()	Determine if DESCRIPTION has a value

Classifiers

A list of UTF-8 strings that specify the custom classifiers that are associated with the crawler.

Accessible with the following methods

Method	Description
GET_CLASSIFIERS()	Getter for CLASSIFIERS, with configurable default
ASK_CLASSIFIERS()	Getter for CLASSIFIERS w/ exceptions if field has no value
HAS_CLASSIFIERS()	Determine if CLASSIFIERS has a value

RecrawlPolicy

A policy that specifies whether to crawl the entire dataset again, or to crawl only folders that were added since the last crawler run.

Accessible with the following methods

Method	Description
GET_RECRAWLPOLICY()	Getter for RECRAWLPOLICY

SchemaChangePolicy

The policy that specifies update and delete behaviors for the crawler.

Accessible with the following methods

Method	Description
GET_SCHEMACHANGEPOLICY()	Getter for SCHEMACHANGEPOLICY

LineageConfiguration

A configuration that specifies whether data lineage is enabled for the crawler.

Accessible with the following methods

Method	Description
GET_LINEAGECONFIGURATION()	Getter for LINEAGECONFIGURATION

State

Indicates whether the crawler is running, or whether a run is pending.

Accessible with the following methods

Method	Description
GET_STATE()	Getter for STATE, with configurable default
ASK_STATE()	Getter for STATE w/ exceptions if field has no value
HAS_STATE()	Determine if STATE has a value

TablePrefix

The prefix added to the names of tables that are created.

Accessible with the following methods

Method	Description
GET_TABLEPREFIX()	Getter for TABLEPREFIX, with configurable default
ASK_TABLEPREFIX()	Getter for TABLEPREFIX w/ exceptions if field has no value
HAS_TABLEPREFIX()	Determine if TABLEPREFIX has a value

Schedule

For scheduled crawlers, the schedule when the crawler runs.

Accessible with the following methods

Method	Description
GET_SCHEDULE()	Getter for SCHEDULE

CrawlElapsedTime

If the crawler is running, contains the total time elapsed since the last crawl began.

Accessible with the following methods

Method	Description
GET_CRAWLELAPSEDTIME()	Getter for CRAWLELAPSEDTIME

CreationTime

The time that the crawler was created.

Accessible with the following methods

Method	Description
GET_CREATIONTIME()	Getter for CREATIONTIME, with configurable default
ASK_CREATIONTIME()	Getter for CREATIONTIME w/ exceptions if field has no value
HAS_CREATIONTIME()	Determine if CREATIONTIME has a value

LastUpdated

The time that the crawler was last updated.

Accessible with the following methods

Method	Description
GET_LASTUPDATED()	Getter for LASTUPDATED, with configurable default
ASK_LASTUPDATED()	Getter for LASTUPDATED w/ exceptions if field has no value
HAS_LASTUPDATED()	Determine if LASTUPDATED has a value

LastCrawl

The status of the last crawl, and potentially error information if an error occurred.

Accessible with the following methods

Method	Description
GET_LASTCRAWL()	Getter for LASTCRAWL

Version

The version of the crawler.

Accessible with the following methods

Method	Description
GET_VERSION()	Getter for VERSION

Configuration

Crawler configuration information. This versioned JSON string allows users to specify aspects of a crawler's behavior. For more information, see Setting crawler configuration options.

Accessible with the following methods

Method	Description
GET_CONFIGURATION()	Getter for CONFIGURATION, with configurable default
ASK_CONFIGURATION()	Getter for CONFIGURATION w/ exceptions if field has no value
HAS_CONFIGURATION()	Determine if CONFIGURATION has a value

CrawlerSecurityConfiguration

The name of the SecurityConfiguration structure to be used by this crawler.

Accessible with the following methods

Method	Description
GET_CRAWLERSECURITYCONF()	Getter for CRAWLERSECURITYCONFIGURATION, with configurable d
ASK_CRAWLERSECURITYCONF()	Getter for CRAWLERSECURITYCONFIGURATION w/ exceptions if fie
HAS_CRAWLERSECURITYCONF()	Determine if CRAWLERSECURITYCONFIGURATION has a value

LakeFormationConfiguration

Specifies whether the crawler should use Lake Formation credentials for the crawler instead of the IAM role credentials.

Accessible with the following methods

Method	Description
GET_LAKEFORMATIONCONF()	Getter for LAKEFORMATIONCONFIGURATION

CatalogId

The ID of the Data Catalog in which the crawler's output is stored.

Accessible with the following methods

Method	Description
GET_CATALOGID()	Getter for CATALOGID, with configurable default
ASK_CATALOGID()	Getter for CATALOGID w/ exceptions if field has no value
HAS_CATALOGID()	Determine if CATALOGID has a value

Public Local Types In This Class

Internal table types, representing arrays and maps of this class, are defined as local types:

TT_CRAWLERLIST

TYPES TT_CRAWLERLIST TYPE STANDARD TABLE OF REF TO /AWS1/CL_GLUCRAWLER WITH DEFAULT KEY
.