/AWS1/CL_GLUSPARKSQL

Specifies a transform where you enter a SQL query using Spark SQL syntax to transform the data. The output is a single DynamicFrame.

CONSTRUCTOR

IMPORTING

Required arguments:

iv_name TYPE /AWS1/GLUNODENAME /AWS1/GLUNODENAME

The name of the transform node.

it_inputs TYPE /AWS1/CL_GLUMANYINPUTS_W=>TT_MANYINPUTS TT_MANYINPUTS

The data inputs identified by their node names. You can associate a table name with each input node to use in the SQL query. The name you choose must meet the Spark SQL naming restrictions.

iv_sqlquery TYPE /AWS1/GLUSQLQUERY /AWS1/GLUSQLQUERY

A SQL query that must use Spark SQL syntax and return a single data set.

it_sqlaliases TYPE /AWS1/CL_GLUSQLALIAS=>TT_SQLALIASES TT_SQLALIASES

A list of aliases. An alias allows you to specify what name to use in the SQL for a given input. For example, you have a datasource named "MyDataSource". If you specify From as MyDataSource, and Alias as SqlName, then in your SQL you can do:

select * from SqlName

and that gets data from MyDataSource.

Optional arguments:

it_outputschemas TYPE /AWS1/CL_GLUGLUESCHEMA=>TT_GLUESCHEMAS TT_GLUESCHEMAS

Specifies the data schema for the SparkSQL transform.

---

Queryable Attributes

Name

The name of the transform node.

Accessible with the following methods

Method	Description
GET_NAME()	Getter for NAME, with configurable default
ASK_NAME()	Getter for NAME w/ exceptions if field has no value
HAS_NAME()	Determine if NAME has a value

Inputs

The data inputs identified by their node names. You can associate a table name with each input node to use in the SQL query. The name you choose must meet the Spark SQL naming restrictions.

Accessible with the following methods

Method	Description
GET_INPUTS()	Getter for INPUTS, with configurable default
ASK_INPUTS()	Getter for INPUTS w/ exceptions if field has no value
HAS_INPUTS()	Determine if INPUTS has a value

SqlQuery

A SQL query that must use Spark SQL syntax and return a single data set.

Accessible with the following methods

Method	Description
GET_SQLQUERY()	Getter for SQLQUERY, with configurable default
ASK_SQLQUERY()	Getter for SQLQUERY w/ exceptions if field has no value
HAS_SQLQUERY()	Determine if SQLQUERY has a value

SqlAliases

A list of aliases. An alias allows you to specify what name to use in the SQL for a given input. For example, you have a datasource named "MyDataSource". If you specify From as MyDataSource, and Alias as SqlName, then in your SQL you can do:

select * from SqlName

and that gets data from MyDataSource.

Accessible with the following methods

Method	Description
GET_SQLALIASES()	Getter for SQLALIASES, with configurable default
ASK_SQLALIASES()	Getter for SQLALIASES w/ exceptions if field has no value
HAS_SQLALIASES()	Determine if SQLALIASES has a value

OutputSchemas

Specifies the data schema for the SparkSQL transform.

Accessible with the following methods

Method	Description
GET_OUTPUTSCHEMAS()	Getter for OUTPUTSCHEMAS, with configurable default
ASK_OUTPUTSCHEMAS()	Getter for OUTPUTSCHEMAS w/ exceptions if field has no value
HAS_OUTPUTSCHEMAS()	Determine if OUTPUTSCHEMAS has a value