ContinuousLoggingProps Struct
Properties for enabling Continuous Logging for Glue Jobs.
Example
ts
import * as cdk from 'aws-cdk-lib';
import * as iam from 'aws-cdk-lib/aws-iam';
declare const stack: cdk.Stack;
declare const role: iam.IRole;
declare const script: glue.Code;
new glue.PySparkEtlJob(stack, 'PySparkETLJob', {
jobName: 'PySparkETLJobCustomName',
description: 'This is a description',
role,
script,
glueVersion: glue.GlueVersion.V6_0,
continuousLogging: { enabled: false },
workerConfiguration: {
workerType: glue.WorkerType.G_2X,
numberOfWorkers: 2,
},
maxConcurrentRuns: 100,
timeout: cdk.Duration.hours(2),
connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],
securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),
tags: {
FirstTagName: 'FirstTagValue',
SecondTagName: 'SecondTagValue',
XTagName: 'XTagValue',
},
maxRetries: 2,
});
Properties
conversionPatternoptionalstringconversion_patternoptionalstrconversionPatternoptionalStringConversionPatternoptionalstringConversionPatternoptional*string#Apply the provided conversion pattern.
Default: %d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n
This is a Log4j Conversion Pattern to customize driver and executor logs.
Enable continuous logging.
logGroupoptionalILogGrouplog_groupoptionalILogGrouplogGroupoptionalILogGroupLogGroupoptionalILogGroupLogGroupoptionalILogGroup#Specify a custom CloudWatch log group name.
Default: a log group is created with name /aws-glue/jobs/logs-v2/.
logStreamPrefixoptionalstringlog_stream_prefixoptionalstrlogStreamPrefixoptionalStringLogStreamPrefixoptionalstringLogStreamPrefixoptional*string#Specify a custom CloudWatch log stream prefix.
Default: the job run ID.
Filter out non-useful Apache Spark driver/executor and Apache Hadoop YARN heartbeat log messages.