# Migrate polling pipelines to use event-based change detection
Migrate polling pipelines to use event-based change detection
AWS CodePipeline supports full, end-to-end continuous delivery, which includes starting your pipeline whenever there is a code change. There are two supported ways to start your pipeline upon a code change: event-based change detection and polling. We recommend using event-based change detection for pipelines.
Use the procedures included here to migrate (update) your polling pipelines to the event-based change detection method for your pipeline.
The recommended event-based change detection method for pipelines is determined by the pipeline source, such as CodeCommit. In that case, for example, the polling pipeline would need to migrate to event-based change detection with EventBridge.
## How to migrate polling pipelines
To migrate polling pipelines, determine your polling pipelines and then determine the recommended event-based change detection method:
+ Use the steps in [Viewing polling pipelines in your account](#update-change-detection-view-polling) to determine your polling pipelines.
+ In the table, find your pipeline source type and then choose the procedure with the implementation you want to use to migrate your polling pipeline. Each section contains multiple methods for migration, such as using the CLI or CloudFormation.
| How to migrate pipelines to the recommended change detection method | Pipeline source | Recommended event-based detection method | Migration procedures |
| --- | --- | --- | --- |
| AWS CodeCommit | EventBridge (recommended). | See [Migrate polling pipelines with a CodeCommit source](#update-change-detection-codecommit). |
| Amazon S3 | EventBridge and bucket enabled for event notifications (recommended). | See [Migrate polling pipelines with an S3 source enabled for events](#update-change-detection-S3-event). |
| Amazon S3 | EventBridge and an AWS CloudTrail trail. | See [Migrate polling pipelines with an S3 source and CloudTrail trail](#update-change-detection-S3). |
| GitHub (via GitHub App) | Connections (recommended) | See [Migrate polling pipelines for a GitHub (via OAuth app) source action to connections](#update-change-detection-github-connection). |
| GitHub (via OAuth app) | Webhooks | See [Migrate polling pipelines for a GitHub (via OAuth app) source action to webhooks](#update-change-detection-github-webhooks). |
**Important**
For applicable pipeline action configuration updates, such as pipelines with a GitHub (viaOAuth app) action, you must explicitly set the `PollForSourceChanges` parameter to *false* within your Source action’s configuration to stop a pipeline from polling. As a result, it is possible to erroneously configure a pipeline with both event-based change detection *and* polling by, for example, configuring an EventBridge rule and also omitting the `PollForSourceChanges` parameter. This results in duplicate pipeline executions, and the pipeline is counted toward the limit on total number of polling pipelines, which by default is much lower than event-based pipelines. For more information, see [Quotas in AWS CodePipeline](limits.md).
## Viewing polling pipelines in your account
As a first step, use one of the following scripts to determine which pipelines in your account are configured for polling. These are the pipelines to migrate to event-based change detection.
### Viewing polling pipelines in your account (script)
Follow these steps to use a script to determine pipelines in your account that are using polling.
1. Open a terminal window, and then do one of the following:
+ Run the following command to create a new script named **PollingPipelinesExtractor.sh**.
```
vi PollingPipelinesExtractor.sh
```
+ To use a python script, run the following command to create a new python script named **PollingPipelinesExtractor.py**.
```
vi PollingPipelinesExtractor.py
```
1. Copy and paste the following code into the **PollingPipelinesExtractor** script. Do one of the following:
+ Copy and paste the following code into the **PollingPipelinesExtractor.sh** script.
```
#!/bin/bash
set +x
POLLING_PIPELINES=()
LAST_EXECUTED_DATES=()
NEXT_TOKEN=null
HAS_NEXT_TOKEN=true
if [[ $# -eq 0 ]] ; then
echo 'Please provide region name'
exit 0
fi
REGION=$1
while [ "$HAS_NEXT_TOKEN" != "false" ]; do
if [ "$NEXT_TOKEN" != "null" ];
then
LIST_PIPELINES_RESPONSE=$(aws codepipeline list-pipelines --region $REGION --next-token $NEXT_TOKEN)
else
LIST_PIPELINES_RESPONSE=$(aws codepipeline list-pipelines --region $REGION)
fi
LIST_PIPELINES=$(jq -r '.pipelines[].name' <<< "$LIST_PIPELINES_RESPONSE")
NEXT_TOKEN=$(jq -r '.nextToken' <<< "$LIST_PIPELINES_RESPONSE")
if [ "$NEXT_TOKEN" == "null" ];
then
HAS_NEXT_TOKEN=false
fi
for pipline_name in $LIST_PIPELINES
do
PIPELINE=$(aws codepipeline get-pipeline --name $pipline_name --region $REGION)
HAS_POLLABLE_ACTIONS=$(jq '.pipeline.stages[].actions[] | select(.actionTypeId.category == "Source") | select(.actionTypeId.owner == ("ThirdParty","AWS")) | select(.actionTypeId.provider == ("GitHub","S3","CodeCommit")) | select(.configuration.PollForSourceChanges == ("true",null))' <<< "$PIPELINE")
if [ ! -z "$HAS_POLLABLE_ACTIONS" ];
then
POLLING_PIPELINES+=("$pipline_name")
PIPELINE_EXECUTIONS=$(aws codepipeline list-pipeline-executions --pipeline-name $pipline_name --region $REGION)
LAST_EXECUTION=$(jq -r '.pipelineExecutionSummaries[0]' <<< "$PIPELINE_EXECUTIONS")
if [ "$LAST_EXECUTION" != "null" ];
then
LAST_EXECUTED_TIMESTAMP=$(jq -r '.startTime' <<< "$LAST_EXECUTION")
LAST_EXECUTED_DATE="$(date -r ${LAST_EXECUTED_TIMESTAMP%.*})"
else
LAST_EXECUTED_DATE="Not executed in last year"
fi
LAST_EXECUTED_DATES+=("$LAST_EXECUTED_DATE")
fi
done
done
fileName=$REGION-$(date +%s)
printf "| %-30s | %-30s |\n" "Polling Pipeline Name" "Last Executed Time"
printf "| %-30s | %-30s |\n" "_____________________" "__________________"
for i in "${!POLLING_PIPELINES[@]}"; do
printf "| %-30s | %-30s |\n" "${POLLING_PIPELINES[i]}" "${LAST_EXECUTED_DATES[i]}"
printf "${POLLING_PIPELINES[i]}," >> $fileName.csv
done
printf "\nSaving Polling Pipeline Names to file $fileName.csv."
```
+ Copy and paste the following code into the **PollingPipelinesExtractor.py** script.
```
import boto3
import sys
import time
import math
hasNextToken = True
nextToken = ""
pollablePipelines = []
lastExecutedTimes = []
if len(sys.argv) == 1:
raise Exception("Please provide region name.")
session = boto3.Session(profile_name='default', region_name=sys.argv[1])
codepipeline = session.client('codepipeline')
def is_pollable_action(action):
actionTypeId = action['actionTypeId']
configuration = action['configuration']
return actionTypeId['owner'] in {"AWS", "ThirdParty"} and actionTypeId['provider'] in {"GitHub", "CodeCommit", "S3"} and ('PollForSourceChanges' not in configuration or configuration['PollForSourceChanges'] == 'true')
def has_pollable_actions(pipeline):
hasPollableAction = False
pipelineDefinition = codepipeline.get_pipeline(name=pipeline['name'])['pipeline']
for action in pipelineDefinition['stages'][0]['actions']:
hasPollableAction = is_pollable_action(action)
if hasPollableAction:
break
return hasPollableAction
def get_last_executed_time(pipelineName):
pipelineExecutions=codepipeline.list_pipeline_executions(pipelineName=pipelineName)['pipelineExecutionSummaries']
if pipelineExecutions:
return pipelineExecutions[0]['startTime'].strftime("%A %m/%d/%Y, %H:%M:%S")
else:
return "Not executed in last year"
while hasNextToken:
if nextToken=="":
list_pipelines_response = codepipeline.list_pipelines()
else:
list_pipelines_response = codepipeline.list_pipelines(nextToken=nextToken)
if 'nextToken' in list_pipelines_response:
nextToken = list_pipelines_response['nextToken']
else:
hasNextToken= False
for pipeline in list_pipelines_response['pipelines']:
if has_pollable_actions(pipeline):
pollablePipelines.append(pipeline['name'])
lastExecutedTimes.append(get_last_executed_time(pipeline['name']))
fileName="{region}-{timeNow}.csv".format(region=sys.argv[1],timeNow=math.trunc(time.time()))
file = open(fileName, 'w')
print ("{:<30} {:<30} {:<30}".format('Polling Pipeline Name', '|','Last Executed Time'))
print ("{:<30} {:<30} {:<30}".format('_____________________', '|','__________________'))
for i in range(len(pollablePipelines)):
print("{:<30} {:<30} {:<30}".format(pollablePipelines[i], '|', lastExecutedTimes[i]))
file.write("{pipeline},".format(pipeline=pollablePipelines[i]))
file.close()
print("\nSaving Polling Pipeline Names to file {fileName}".format(fileName=fileName))
```
1. For each Region where you have pipelines, you must run the script for that Region. To run the script, do one of the following:
+ Run the following command to run the script named **PollingPipelinesExtractor.sh**. In this example, the Region is us-west-2.
```
./PollingPipelinesExtractor.sh us-west-2
```
+ For the python script, run the following command to run the python script named**PollingPipelinesExtractor.py**. In this example, the Region is us-west-2.
```
python3 PollingPipelinesExtractor.py us-west-2
```
In the following sample output from the script, the Region us-west-2 returned a list of polling pipelines and shows the last execution time for each pipeline.
```
% ./pollingPipelineExtractor.sh us-west-2
| Polling Pipeline Name | Last Executed Time |
| _____________________ | __________________ |
| myCodeBuildPipeline | Wed Mar 8 09:35:49 PST 2023 |
| myCodeCommitPipeline | Mon Apr 24 22:32:32 PDT 2023 |
| TestPipeline | Not executed in last year |
Saving list of polling pipeline names to us-west-2-1682496174.csv...%
```
Analyze the script output and, for each pipeline in the list, update the polling source to the recommended event-based change detection method.
**Note**
Your polling pipelines are determined by the pipeline's action configuration for the `PollForSourceChanges` parameter. If the pipeline source configuration has the `PollForSourceChanges` parameter ommitted, then CodePipeline defaults to polling your repository for source changes. This behavior is the same as if `PollForSourceChanges` is included and set to true. For more information, see the configuration parameters for your pipeline's source action, such as the Amazon S3 source action configuration parameters in [Amazon S3 source action reference](action-reference-S3.md).
Note that this script also generates a .csv file containing the list of polling pipelines in your account and saves the .csv file to the current working folder.
## Migrate polling pipelines with a CodeCommit source
You can migrate your polling pipeline to use EventBridge to detect changes in your CodeCommit source repository or your Amazon S3 source bucket.
**CodeCommit** -- For a pipeline with a CodeCommit source, modify the pipeline so that change detection is automated through EventBridge. Choose from the following methods to implement the migration:
+ **Console: **[Migrate polling pipelines (CodeCommit or Amazon S3 source) (console)](#update-change-detection-console-codecommit-S3)
+ **CLI: **[Migrate polling pipelines (CodeCommit source) (CLI)](#update-change-detection-cli-codecommit)
+ **CloudFormation: ** [Migrate polling pipelines (CodeCommit source) (CloudFormation template)](#update-change-detection-cfn-codecommit)
### Migrate polling pipelines (CodeCommit or Amazon S3 source) (console)
You can use the CodePipeline console to update your pipeline to use EventBridge to detect changes in your CodeCommit source repository or your Amazon S3 source bucket.
**Note**
When you use the console to edit a pipeline that has a CodeCommit source repository or an Amazon S3 source bucket, the rule and IAM role are created for you. If you use the AWS CLI to edit the pipeline, you must create the EventBridge rule and IAM role yourself. For more information, see [CodeCommit source actions and EventBridge](triggering.md).
Use these steps to edit a pipeline that is using periodic checks. If you want to create a pipeline, see [Create a pipeline, stages, and actions](pipelines-create.md).
**To edit the pipeline source stage**
1. Sign in to the AWS Management Console and open the CodePipeline console at [http://console.aws.amazon.com/codesuite/codepipeline/home](http://console.aws.amazon.com/codesuite/codepipeline/home).
The names of all pipelines associated with your AWS account are displayed.
1. In **Name**, choose the name of the pipeline you want to edit. This opens a detailed view of the pipeline, including the state of each of the actions in each stage of the pipeline.
1. On the pipeline details page, choose **Edit**.
1. In **Edit** stage, choose the edit icon on the source action.
1. Expand **Change Detection Options** and choose **Use CloudWatch Events to automatically start my pipeline when a change occurs (recommended)**.
A message appears showing the EventBridge rule to be created for this pipeline. Choose **Update**.
If you are updating a pipeline that has an Amazon S3 source, you see the following message. Choose **Update**.
1. When you have finished editing your pipeline, choose **Save pipeline changes** to return to the summary page.
A message displays the name of the EventBridge rule to be created for your pipeline. Choose **Save and continue**.
1. To test your action, release a change by using the AWS CLI to commit a change to the source specified in the source stage of the pipeline.
### Migrate polling pipelines (CodeCommit source) (CLI)
Follow these steps to edit a pipeline that is using polling (periodic checks) to use an EventBridge rule to start the pipeline. If you want to create a pipeline, see [Create a pipeline, stages, and actions](pipelines-create.md).
To build an event-driven pipeline with CodeCommit, you edit the `PollForSourceChanges` parameter of your pipeline and then create the following resources:
+ EventBridge event
+ IAM role to allow this event to start your pipeline
**To edit your pipeline's PollForSourceChanges parameter**
**Important**
When you create a pipeline with this method, the `PollForSourceChanges` parameter defaults to true if it is not explicitly set to false. When you add event-based change detection, you must add the parameter to your output and set it to false to disable polling. Otherwise, your pipeline starts twice for a single source change. For details, see [Valid settings for the `PollForSourceChanges` parameter](PollForSourceChanges-defaults.md).
1. Run the **get-pipeline** command to copy the pipeline structure into a JSON file. For example, for a pipeline named `MyFirstPipeline`, run the following command:
```
aws codepipeline get-pipeline --name MyFirstPipeline >pipeline.json
```
This command returns nothing, but the file you created should appear in the directory where you ran the command.
1. Open the JSON file in any plain-text editor and edit the source stage by changing the `PollForSourceChanges` parameter to `false`, as shown in this example.
**Why am I making this change?** Changing this parameter to `false` turns off periodic checks so you can use event-based change detection only.
```
"configuration": {
"PollForSourceChanges": "false",
"BranchName": "main",
"RepositoryName": "MyTestRepo"
},
```
1. If you are working with the pipeline structure retrieved using the **get-pipeline** command, remove the `metadata` lines from the JSON file. Otherwise, the **update-pipeline** command cannot use it. Remove the `"metadata": { }` lines and the `"created"`, `"pipelineARN"`, and `"updated"` fields.
For example, remove the following lines from the structure:
```
"metadata": {
"pipelineArn": "arn:aws:codepipeline:region:account-ID:pipeline-name",
"created": "date",
"updated": "date"
},
```
Save the file.
1. To apply your changes, run the **update-pipeline** command, specifying the pipeline JSON file:
**Important**
Be sure to include `file://` before the file name. It is required in this command.
```
aws codepipeline update-pipeline --cli-input-json file://pipeline.json
```
This command returns the entire structure of the edited pipeline.
**Note**
The **update-pipeline** command stops the pipeline. If a revision is being run through the pipeline when you run the **update-pipeline** command, that run is stopped. You must manually start the pipeline to run that revision through the updated pipeline. Use the **`start-pipeline-execution`** command to manually start your pipeline.
**To create an EventBridge rule with CodeCommit as the event source and CodePipeline as the target**
1. Add permissions for EventBridge to use CodePipeline to invoke the rule. For more information, see [Using resource-based policies for Amazon EventBridge](http://docs.aws.amazon.com/eventbridge/latest/userguide/eb-use-resource-based.html).
1. Use the following sample to create the trust policy that allows EventBridge to assume the service role. Name the trust policy `trustpolicyforEB.json`.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "events.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
```
------
1. Use the following command to create the `Role-for-MyRule` role and attach the trust policy.
```
aws iam create-role --role-name Role-for-MyRule --assume-role-policy-document file://trustpolicyforEB.json
```
1. Create the permissions policy JSON, as shown in this sample, for the pipeline named `MyFirstPipeline`. Name the permissions policy `permissionspolicyforEB.json`.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"codepipeline:StartPipelineExecution"
],
"Resource": [
"arn:aws:codepipeline:us-west-2:111122223333:MyFirstPipeline"
]
}
]
}
```
------
1. Use the following command to attach the `CodePipeline-Permissions-Policy-for-EB` permissions policy to the `Role-for-MyRule` role.
**Why am I making this change?** Adding this policy to the role creates permissions for EventBridge.
```
aws iam put-role-policy --role-name Role-for-MyRule --policy-name CodePipeline-Permissions-Policy-For-EB --policy-document file://permissionspolicyforEB.json
```
1. Call the **put-rule** command and include the `--name`, `--event-pattern` , and`--role-arn` parameters.
**Why am I making this change?** This command enables CloudFormation to create the event.
The following sample command creates a rule called `MyCodeCommitRepoRule`.
```
aws events put-rule --name "MyCodeCommitRepoRule" --event-pattern "{\"source\":[\"aws.codecommit\"],\"detail-type\":[\"CodeCommit Repository State Change\"],\"resources\":[\"repository-ARN\"],\"detail\":{\"referenceType\":[\"branch\"],\"referenceName\":[\"main\"]}}" --role-arn "arn:aws:iam::ACCOUNT_ID:role/Role-for-MyRule"
```
1. To add CodePipeline as a target, call the **put-targets** command and include the following parameters:
+ The `--rule` parameter is used with the `rule_name` you created by using **put-rule**.
+ The `--targets` parameter is used with the list `Id` of the target in the list of targets and the `ARN` of the target pipeline.
The following sample command specifies that for the rule called `MyCodeCommitRepoRule`, the target `Id` is composed of the number one, indicating that in a list of targets for the rule, this is target 1. The sample command also specifies an example `ARN` for the pipeline. The pipeline starts when something changes in the repository.
```
aws events put-targets --rule MyCodeCommitRepoRule --targets Id=1,Arn=arn:aws:codepipeline:us-west-2:80398EXAMPLE:TestPipeline
```
1. (Optional) To configure an input transformer with source overrides for a specific image ID, use the following JSON in your CLI command. The following example configures an override where:
+ The `actionName`, `Source` in this example, is the dynamic value, defined at pipeline creation, not derived from the source event.
+ The `revisionType`, `COMMIT_ID` in this example, is the dynamic value, defined at pipeline creation, not derived from the source event.
+ The `revisionValue`, <*revisionValue*> in this example, is derived from the source event variable.
```
{
"Rule": "my-rule",
"Targets": [
{
"Id": "MyTargetId",
"Arn": "pipeline-ARN",
"InputTransformer": {
"sourceRevisions": {
"actionName": "Source",
"revisionType": "COMMIT_ID",
"revisionValue": ""
},
"variables": [
{
"name": "Branch_Name",
"value": "value"
}
]
}
}
]
}
```
### Migrate polling pipelines (CodeCommit source) (CloudFormation template)
To build an event-driven pipeline with AWS CodeCommit, you edit the `PollForSourceChanges` parameter of your pipeline and then add the following resources to your template:
+ An EventBridge rule
+ An IAM role for your EventBridge rule
If you use CloudFormation to create and manage your pipelines, your template includes content like the following.
**Note**
The `Configuration` property in the source stage called `PollForSourceChanges`. If that property isn't included in your template, then `PollForSourceChanges` is set to `true` by default.
------
#### [ YAML ]
```
Resources:
AppPipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
Name: codecommit-polling-pipeline
RoleArn:
!GetAtt CodePipelineServiceRole.Arn
Stages:
-
Name: Source
Actions:
-
Name: SourceAction
ActionTypeId:
Category: Source
Owner: AWS
Version: 1
Provider: CodeCommit
OutputArtifacts:
- Name: SourceOutput
Configuration:
BranchName: !Ref BranchName
RepositoryName: !Ref RepositoryName
PollForSourceChanges: true
RunOrder: 1
```
------
#### [ JSON ]
```
"Stages": [
{
"Name": "Source",
"Actions": [{
"Name": "SourceAction",
"ActionTypeId": {
"Category": "Source",
"Owner": "AWS",
"Version": 1,
"Provider": "CodeCommit"
},
"OutputArtifacts": [{
"Name": "SourceOutput"
}],
"Configuration": {
"BranchName": {
"Ref": "BranchName"
},
"RepositoryName": {
"Ref": "RepositoryName"
},
"PollForSourceChanges": true
},
"RunOrder": 1
}]
},
```
------
**To update your pipeline CloudFormation template and create EventBridge rule**
1. In the template, under `Resources`, use the `AWS::IAM::Role` CloudFormation resource to configure the IAM role that allows your event to start your pipeline. This entry creates a role that uses two policies:
+ The first policy allows the role to be assumed.
+ The second policy provides permissions to start the pipeline.
**Why am I making this change?** Adding the `AWS::IAM::Role` resource enables CloudFormation to create permissions for EventBridge. This resource is added to your CloudFormation stack.
------
#### [ YAML ]
```
EventRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Principal:
Service:
- events.amazonaws.com
Action: sts:AssumeRole
Path: /
Policies:
-
PolicyName: eb-pipeline-execution
PolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Action: codepipeline:StartPipelineExecution
Resource: !Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ]
```
------
#### [ JSON ]
```
"EventRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"events.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
},
"Path": "/",
"Policies": [
{
"PolicyName": "eb-pipeline-execution",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "codepipeline:StartPipelineExecution",
"Resource": {
"Fn::Join": [
"",
[
"arn:aws:codepipeline:",
{
"Ref": "AWS::Region"
},
":",
{
"Ref": "AWS::AccountId"
},
":",
{
"Ref": "AppPipeline"
}
]
...
```
------
1. In the template, under `Resources`, use the `AWS::Events::Rule` CloudFormation resource to add an EventBridge rule. This event pattern creates an event that monitors push changes to your repository. When EventBridge detects a repository state change, the rule invokes `StartPipelineExecution` on your target pipeline.
**Why am I making this change? ** Adding the `AWS::Events::Rule` resource enables CloudFormation to create the event. This resource is added to your CloudFormation stack.
------
#### [ YAML ]
```
EventRule:
Type: AWS::Events::Rule
Properties:
EventPattern:
source:
- aws.codecommit
detail-type:
- 'CodeCommit Repository State Change'
resources:
- !Join [ '', [ 'arn:aws:codecommit:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref RepositoryName ] ]
detail:
event:
- referenceCreated
- referenceUpdated
referenceType:
- branch
referenceName:
- main
Targets:
-
Arn:
!Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ]
RoleArn: !GetAtt EventRole.Arn
Id: codepipeline-AppPipeline
```
------
#### [ JSON ]
```
"EventRule": {
"Type": "AWS::Events::Rule",
"Properties": {
"EventPattern": {
"source": [
"aws.codecommit"
],
"detail-type": [
"CodeCommit Repository State Change"
],
"resources": [
{
"Fn::Join": [
"",
[
"arn:aws:codecommit:",
{
"Ref": "AWS::Region"
},
":",
{
"Ref": "AWS::AccountId"
},
":",
{
"Ref": "RepositoryName"
}
]
]
}
],
"detail": {
"event": [
"referenceCreated",
"referenceUpdated"
],
"referenceType": [
"branch"
],
"referenceName": [
"main"
]
}
},
"Targets": [
{
"Arn": {
"Fn::Join": [
"",
[
"arn:aws:codepipeline:",
{
"Ref": "AWS::Region"
},
":",
{
"Ref": "AWS::AccountId"
},
":",
{
"Ref": "AppPipeline"
}
]
]
},
"RoleArn": {
"Fn::GetAtt": [
"EventRole",
"Arn"
]
},
"Id": "codepipeline-AppPipeline"
}
]
}
},
```
------
1. (Optional) To configure an input transformer with source overrides for a specific image ID, use the following YAML snippet. The following example configures an override where:
+ The `actionName`, `Source` in this example, is the dynamic value, defined at pipeline creation, not derived from the source event.
+ The `revisionType`, `COMMIT_ID` in this example, is the dynamic value, defined at pipeline creation, not derived from the source event.
+ The `revisionValue`, <*revisionValue*> in this example, is derived from the source event variable.
+ The output variables for `BranchName` and `Value` are specified.
```
Rule: my-rule
Targets:
- Id: MyTargetId
Arn: pipeline-ARN
InputTransformer:
sourceRevisions:
actionName: Source
revisionType: COMMIT_ID
revisionValue:
variables:
- name: BranchName
value: value
```
1. Save the updated template to your local computer, and then open the CloudFormation console.
1. Choose your stack, and then choose **Create Change Set for Current Stack**.
1. Upload the template, and then view the changes listed in CloudFormation. These are the changes to be made to the stack. You should see your new resources in the list.
1. Choose **Execute**.
**To edit your pipeline's PollForSourceChanges parameter**
**Important**
In many cases, the `PollForSourceChanges` parameter defaults to true when you create a pipeline. When you add event-based change detection, you must add the parameter to your output and set it to false to disable polling. Otherwise, your pipeline starts twice for a single source change. For details, see [Valid settings for the `PollForSourceChanges` parameter](PollForSourceChanges-defaults.md).
+ In the template, change `PollForSourceChanges` to `false`. If you did not include `PollForSourceChanges` in your pipeline definition, add it and set it to `false`.
**Why am I making this change?** Changing this parameter to `false` turns off periodic checks so you can use event-based change detection only.
------
#### [ YAML ]
```
Name: Source
Actions:
-
Name: SourceAction
ActionTypeId:
Category: Source
Owner: AWS
Version: 1
Provider: CodeCommit
OutputArtifacts:
- Name: SourceOutput
Configuration:
BranchName: !Ref BranchName
RepositoryName: !Ref RepositoryName
PollForSourceChanges: false
RunOrder: 1
```
------
#### [ JSON ]
```
{
"Name": "Source",
"Actions": [
{
"Name": "SourceAction",
"ActionTypeId": {
"Category": "Source",
"Owner": "AWS",
"Version": 1,
"Provider": "CodeCommit"
},
"OutputArtifacts": [
{
"Name": "SourceOutput"
}
],
"Configuration": {
"BranchName": {
"Ref": "BranchName"
},
"RepositoryName": {
"Ref": "RepositoryName"
},
"PollForSourceChanges": false
},
"RunOrder": 1
}
]
},
```
------
**Example**
When you create these resources with CloudFormation, your pipeline is triggered when files in your repository are created or updated. Here is the final template snippet:
```
Resources:
EventRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Principal:
Service:
- events.amazonaws.com
Action: sts:AssumeRole
Path: /
Policies:
-
PolicyName: eb-pipeline-execution
PolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Action: codepipeline:StartPipelineExecution
Resource: !Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ]
EventRule:
Type: AWS::Events::Rule
Properties:
EventPattern:
source:
- aws.codecommit
detail-type:
- 'CodeCommit Repository State Change'
resources:
- !Join [ '', [ 'arn:aws:codecommit:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref RepositoryName ] ]
detail:
event:
- referenceCreated
- referenceUpdated
referenceType:
- branch
referenceName:
- main
Targets:
-
Arn:
!Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ]
RoleArn: !GetAtt EventRole.Arn
Id: codepipeline-AppPipeline
AppPipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
Name: codecommit-events-pipeline
RoleArn:
!GetAtt CodePipelineServiceRole.Arn
Stages:
-
Name: Source
Actions:
-
Name: SourceAction
ActionTypeId:
Category: Source
Owner: AWS
Version: 1
Provider: CodeCommit
OutputArtifacts:
- Name: SourceOutput
Configuration:
BranchName: !Ref BranchName
RepositoryName: !Ref RepositoryName
PollForSourceChanges: false
RunOrder: 1
...
```
```
"Resources": {
...
"EventRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"events.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
},
"Path": "/",
"Policies": [
{
"PolicyName": "eb-pipeline-execution",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "codepipeline:StartPipelineExecution",
"Resource": {
"Fn::Join": [
"",
[
"arn:aws:codepipeline:",
{
"Ref": "AWS::Region"
},
":",
{
"Ref": "AWS::AccountId"
},
":",
{
"Ref": "AppPipeline"
}
]
]
}
}
]
}
}
]
}
},
"EventRule": {
"Type": "AWS::Events::Rule",
"Properties": {
"EventPattern": {
"source": [
"aws.codecommit"
],
"detail-type": [
"CodeCommit Repository State Change"
],
"resources": [
{
"Fn::Join": [
"",
[
"arn:aws:codecommit:",
{
"Ref": "AWS::Region"
},
":",
{
"Ref": "AWS::AccountId"
},
":",
{
"Ref": "RepositoryName"
}
]
]
}
],
"detail": {
"event": [
"referenceCreated",
"referenceUpdated"
],
"referenceType": [
"branch"
],
"referenceName": [
"main"
]
}
},
"Targets": [
{
"Arn": {
"Fn::Join": [
"",
[
"arn:aws:codepipeline:",
{
"Ref": "AWS::Region"
},
":",
{
"Ref": "AWS::AccountId"
},
":",
{
"Ref": "AppPipeline"
}
]
]
},
"RoleArn": {
"Fn::GetAtt": [
"EventRole",
"Arn"
]
},
"Id": "codepipeline-AppPipeline"
}
]
}
},
"AppPipeline": {
"Type": "AWS::CodePipeline::Pipeline",
"Properties": {
"Name": "codecommit-events-pipeline",
"RoleArn": {
"Fn::GetAtt": [
"CodePipelineServiceRole",
"Arn"
]
},
"Stages": [
{
"Name": "Source",
"Actions": [
{
"Name": "SourceAction",
"ActionTypeId": {
"Category": "Source",
"Owner": "AWS",
"Version": 1,
"Provider": "CodeCommit"
},
"OutputArtifacts": [
{
"Name": "SourceOutput"
}
],
"Configuration": {
"BranchName": {
"Ref": "BranchName"
},
"RepositoryName": {
"Ref": "RepositoryName"
},
"PollForSourceChanges": false
},
"RunOrder": 1
}
]
},
...
```
## Migrate polling pipelines with an S3 source enabled for events
For a pipeline with an Amazon S3 source, modify the pipeline so that change detection is automated through EventBridge and with a source bucket that is enabled for event notifications. This is the recommend method if you are using the CLI or CloudFormation to migrate your pipeline.
**Note**
This includes using a bucket that is enabled for event notifications, where you do not need to create a separate CloudTrail trail. If you are using the console, then an event rule and CloudTrail trail are set up for you. For those steps, see [Migrate polling pipelines with an S3 source and CloudTrail trail](#update-change-detection-S3).
+ **CLI: **[Migrate polling pipelines with an S3 source and CloudTrail trail (CLI)](#update-change-detection-cli-S3)
+ **CloudFormation: **[Migrate polling pipelines with an S3 source and CloudTrail trail (CloudFormation template)](#update-change-detection-cfn-s3)
### Migrate polling pipelines with an S3 source enabled for events (CLI)
Follow these steps to edit a pipeline that is using polling (periodic checks) to use an event in EventBridge instead. If you want to create a pipeline, see [Create a pipeline, stages, and actions](pipelines-create.md).
To build an event-driven pipeline with Amazon S3, you edit the `PollForSourceChanges` parameter of your pipeline and then create the following resources:
+ EventBridge event rule
+ IAM role to allow the EventBridge event to start your pipeline
**To create an EventBridge rule with Amazon S3 as the event source and CodePipeline as the target and apply the permissions policy**
1. Grant permissions for EventBridge to use CodePipeline to invoke the rule. For more information, see [Using resource-based policies for Amazon EventBridge](http://docs.aws.amazon.com/eventbridge/latest/userguide/eb-use-resource-based.html).
1. Use the following sample to create the trust policy to allow EventBridge to assume the service role. Name it `trustpolicyforEB.json`.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "events.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
```
------
1. Use the following command to create the `Role-for-MyRule` role and attach the trust policy.
**Why am I making this change?** Adding this trust policy to the role creates permissions for EventBridge.
```
aws iam create-role --role-name Role-for-MyRule --assume-role-policy-document file://trustpolicyforEB.json
```
1. Create the permissions policy JSON, as shown here for the pipeline named `MyFirstPipeline`. Name the permissions policy `permissionspolicyforEB.json`.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"codepipeline:StartPipelineExecution"
],
"Resource": [
"arn:aws:codepipeline:us-west-2:111122223333:MyFirstPipeline"
]
}
]
}
```
------
1. Use the following command to attach the new `CodePipeline-Permissions-Policy-for-EB` permissions policy to the `Role-for-MyRule` role you created.
```
aws iam put-role-policy --role-name Role-for-MyRule --policy-name CodePipeline-Permissions-Policy-For-EB --policy-document file://permissionspolicyforEB.json
```
1. Call the **put-rule** command and include the `--name`, `--event-pattern`, and `--role-arn` parameters.
The following sample command creates a rule named `EnabledS3SourceRule`.
```
aws events put-rule --name "EnabledS3SourceRule" --event-pattern "{\"source\":[\"aws.s3\"],\"detail-type\":[\"Object Created\"],\"detail\":{\"bucket\":{\"name\":[\"amzn-s3-demo-source-bucket\"]}}}" --role-arn "arn:aws:iam::ACCOUNT_ID:role/Role-for-MyRule"
```
1. To add CodePipeline as a target, call the **put-targets** command and include the `--rule` and `--targets` parameters.
The following command specifies that for the rule named `EnabledS3SourceRule`, the target `Id` is composed of the number one, indicating that in a list of targets for the rule, this is target 1. The command also specifies an example `ARN` for the pipeline. The pipeline starts when something changes in the repository.
```
aws events put-targets --rule EnabledS3SourceRule --targets Id=codepipeline-AppPipeline,Arn=arn:aws:codepipeline:us-west-2:80398EXAMPLE:TestPipeline
```
**To edit your pipeline's PollForSourceChanges parameter**
**Important**
When you create a pipeline with this method, the `PollForSourceChanges` parameter defaults to true if it is not explicitly set to false. When you add event-based change detection, you must add the parameter to your output and set it to false to disable polling. Otherwise, your pipeline starts twice for a single source change. For details, see [Valid settings for the `PollForSourceChanges` parameter](PollForSourceChanges-defaults.md).
1. Run the **get-pipeline** command to copy the pipeline structure into a JSON file. For example, for a pipeline named `MyFirstPipeline`, run the following command:
```
aws codepipeline get-pipeline --name MyFirstPipeline >pipeline.json
```
This command returns nothing, but the file you created should appear in the directory where you ran the command.
1. Open the JSON file in any plain-text editor and edit the source stage by changing the `PollForSourceChanges` parameter for a bucket named `amzn-s3-demo-source-bucket` to `false`, as shown in this example.
**Why am I making this change?** Setting this parameter to `false` turns off periodic checks so you can use event-based change detection only.
```
"configuration": {
"S3Bucket": "amzn-s3-demo-source-bucket",
"PollForSourceChanges": "false",
"S3ObjectKey": "index.zip"
},
```
1. If you are working with the pipeline structure retrieved using the **get-pipeline** command, you must remove the `metadata` lines from the JSON file. Otherwise, the **update-pipeline** command cannot use it. Remove the `"metadata": { }` lines and the `"created"`, `"pipelineARN"`, and `"updated"` fields.
For example, remove the following lines from the structure:
```
"metadata": {
"pipelineArn": "arn:aws:codepipeline:region:account-ID:pipeline-name",
"created": "date",
"updated": "date"
},
```
Save the file.
1. To apply your changes, run the **update-pipeline** command, specifying the pipeline JSON file:
**Important**
Be sure to include `file://` before the file name. It is required in this command.
```
aws codepipeline update-pipeline --cli-input-json file://pipeline.json
```
This command returns the entire structure of the edited pipeline.
**Note**
The **update-pipeline** command stops the pipeline. If a revision is being run through the pipeline when you run the **update-pipeline** command, that run is stopped. You must manually start the pipeline to run that revision through the updated pipeline. Use the **start-pipeline-execution** command to manually start your pipeline.
### Migrate polling pipelines with an S3 source enabled for events (CloudFormation template)
This procedure is for a pipeline where the source bucket has events enabled.
Use these steps to edit your pipeline with an Amazon S3 source from polling to event-based change detection.
To build an event-driven pipeline with Amazon S3, you edit the `PollForSourceChanges` parameter of your pipeline and then add the following resources to your template:
+ EventBridge rule and IAM role to allow this event to start your pipeline.
If you use CloudFormation to create and manage your pipelines, your template includes content like the following.
**Note**
The `Configuration` property in the source stage called `PollForSourceChanges`. If your template doesn't include that property, then `PollForSourceChanges` is set to `true` by default.
------
#### [ YAML ]
```
AppPipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
RoleArn: !GetAtt CodePipelineServiceRole.Arn
Stages:
-
Name: Source
Actions:
-
Name: SourceAction
ActionTypeId:
Category: Source
Owner: AWS
Version: 1
Provider: S3
OutputArtifacts:
-
Name: SourceOutput
Configuration:
S3Bucket: !Ref SourceBucket
S3ObjectKey: !Ref S3SourceObjectKey
PollForSourceChanges: true
RunOrder: 1
...
```
------
#### [ JSON ]
```
"AppPipeline": {
"Type": "AWS::CodePipeline::Pipeline",
"Properties": {
"RoleArn": {
"Fn::GetAtt": ["CodePipelineServiceRole", "Arn"]
},
"Stages": [
{
"Name": "Source",
"Actions": [
{
"Name": "SourceAction",
"ActionTypeId": {
"Category": "Source",
"Owner": "AWS",
"Version": 1,
"Provider": "S3"
},
"OutputArtifacts": [
{
"Name": "SourceOutput"
}
],
"Configuration": {
"S3Bucket": {
"Ref": "SourceBucket"
},
"S3ObjectKey": {
"Ref": "SourceObjectKey"
},
"PollForSourceChanges": true
},
"RunOrder": 1
}
]
},
...
```
------
**To create an EventBridge rule with Amazon S3 as the event source and CodePipeline as the target and apply the permissions policy**
1. In the template, under `Resources`, use the `AWS::IAM::Role` CloudFormation resource to configure the IAM role that allows your event to start your pipeline. This entry creates a role that uses two policies:
+ The first policy allows the role to be assumed.
+ The second policy provides permissions to start the pipeline.
**Why am I making this change?** Adding `AWS::IAM::Role` resource enables CloudFormation to create permissions for EventBridge. This resource is added to your CloudFormation stack.
------
#### [ YAML ]
```
EventRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Principal:
Service:
- events.amazonaws.com
Action: sts:AssumeRole
Path: /
Policies:
-
PolicyName: eb-pipeline-execution
PolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Action: codepipeline:StartPipelineExecution
Resource: !Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ]
...
```
------
#### [ JSON ]
```
"EventRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"events.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
},
"Path": "/",
"Policies": [
{
"PolicyName": "eb-pipeline-execution",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "codepipeline:StartPipelineExecution",
"Resource": {
"Fn::Join": [
"",
[
"arn:aws:codepipeline:",
{
"Ref": "AWS::Region"
},
":",
{
"Ref": "AWS::AccountId"
},
":",
{
"Ref": "AppPipeline"
}
]
]
...
```
------
1. Use the `AWS::Events::Rule` CloudFormation resource to add an EventBridge rule. This event pattern creates an event that monitors creation or deletion of objects in your Amazon S3 source bucket. In addition, include a target of your pipeline. When an object is created, this rule invokes `StartPipelineExecution` on your target pipeline.
**Why am I making this change?** Adding the `AWS::Events::Rule` resource enables CloudFormation to create the event. This resource is added to your CloudFormation stack.
------
#### [ YAML ]
```
EventRule:
Type: AWS::Events::Rule
Properties:
EventBusName: default
EventPattern:
source:
- aws.s3
detail-type:
- Object Created
detail:
bucket:
name:
- !Ref SourceBucket
Name: EnabledS3SourceRule
State: ENABLED
Targets:
-
Arn:
!Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ]
RoleArn: !GetAtt EventRole.Arn
Id: codepipeline-AppPipeline
...
```
------
#### [ JSON ]
```
"EventRule": {
"Type": "AWS::Events::Rule",
"Properties": {
"EventBusName": "default",
"EventPattern": {
"source": [
"aws.s3"
],
"detail-type": [
"Object Created"
],
"detail": {
"bucket": {
"name": [
"s3-pipeline-source-fra-bucket"
]
}
}
},
"Name": "EnabledS3SourceRule",
"State": "ENABLED",
"Targets": [
{
"Arn": {
"Fn::Join": [
"",
[
"arn:aws:codepipeline:",
{
"Ref": "AWS::Region"
},
":",
{
"Ref": "AWS::AccountId"
},
":",
{
"Ref": "AppPipeline"
}
]
]
},
"RoleArn": {
"Fn::GetAtt": [
"EventRole",
"Arn"
]
},
"Id": "codepipeline-AppPipeline"
}
]
}
}
},
...
```
------
1. Save your updated template to your local computer, and open the CloudFormation console.
1. Choose your stack, and then choose **Create Change Set for Current Stack**.
1. Upload your updated template, and then view the changes listed in CloudFormation. These are the changes that will be made to the stack. You should see your new resources in the list.
1. Choose **Execute**.
**To edit your pipeline's PollForSourceChanges parameter**
**Important**
When you create a pipeline with this method, the `PollForSourceChanges` parameter defaults to true if it is not explicitly set to false. When you add event-based change detection, you must add the parameter to your output and set it to false to disable polling. Otherwise, your pipeline starts twice for a single source change. For details, see [Valid settings for the `PollForSourceChanges` parameter](PollForSourceChanges-defaults.md).
+ In the template, change `PollForSourceChanges` to `false`. If you did not include `PollForSourceChanges` in your pipeline definition, add it and set it to `false`.
**Why am I making this change?** Changing `PollForSourceChanges` to `false` turns off periodic checks so you can use event-based change detection only.
------
#### [ YAML ]
```
Name: Source
Actions:
-
Name: SourceAction
ActionTypeId:
Category: Source
Owner: AWS
Version: 1
Provider: S3
OutputArtifacts:
- Name: SourceOutput
Configuration:
S3Bucket: !Ref SourceBucket
S3ObjectKey: !Ref SourceObjectKey
PollForSourceChanges: false
RunOrder: 1
```
------
#### [ JSON ]
```
{
"Name": "SourceAction",
"ActionTypeId": {
"Category": "Source",
"Owner": "AWS",
"Version": 1,
"Provider": "S3"
},
"OutputArtifacts": [
{
"Name": "SourceOutput"
}
],
"Configuration": {
"S3Bucket": {
"Ref": "SourceBucket"
},
"S3ObjectKey": {
"Ref": "SourceObjectKey"
},
"PollForSourceChanges": false
},
"RunOrder": 1
}
```
------
**Example**
When you use CloudFormation to create these resources, your pipeline is triggered when files in your repository are created or updated.
Do not stop here. Although your pipeline is created, you must create a second CloudFormation template for your Amazon S3 pipeline. If you do not create the second template, your pipeline does not have any change detection functionality.
```
Parameters:
SourceObjectKey:
Description: 'S3 source artifact'
Type: String
Default: SampleApp_Linux.zip
ApplicationName:
Description: 'CodeDeploy application name'
Type: String
Default: DemoApplication
BetaFleet:
Description: 'Fleet configured in CodeDeploy'
Type: String
Default: DemoFleet
Resources:
SourceBucket:
Type: AWS::S3::Bucket
Properties:
NotificationConfiguration:
EventBridgeConfiguration:
EventBridgeEnabled: true
VersioningConfiguration:
Status: Enabled
CodePipelineArtifactStoreBucket:
Type: AWS::S3::Bucket
CodePipelineArtifactStoreBucketPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket: !Ref CodePipelineArtifactStoreBucket
PolicyDocument:
Version: 2012-10-17
Statement:
-
Sid: DenyUnEncryptedObjectUploads
Effect: Deny
Principal: '*'
Action: s3:PutObject
Resource: !Join [ '', [ !GetAtt CodePipelineArtifactStoreBucket.Arn, '/*' ] ]
Condition:
StringNotEquals:
s3:x-amz-server-side-encryption: aws:kms
-
Sid: DenyInsecureConnections
Effect: Deny
Principal: '*'
Action: s3:*
Resource: !Sub ${CodePipelineArtifactStoreBucket.Arn}/*
Condition:
Bool:
aws:SecureTransport: false
CodePipelineServiceRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Principal:
Service:
- codepipeline.amazonaws.com
Action: sts:AssumeRole
Path: /
Policies:
-
PolicyName: AWS-CodePipeline-Service-3
PolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Action:
- codecommit:CancelUploadArchive
- codecommit:GetBranch
- codecommit:GetCommit
- codecommit:GetUploadArchiveStatus
- codecommit:UploadArchive
Resource: 'resource_ARN'
-
Effect: Allow
Action:
- codedeploy:CreateDeployment
- codedeploy:GetApplicationRevision
- codedeploy:GetDeployment
- codedeploy:GetDeploymentConfig
- codedeploy:RegisterApplicationRevision
Resource: 'resource_ARN'
-
Effect: Allow
Action:
- codebuild:BatchGetBuilds
- codebuild:StartBuild
Resource: 'resource_ARN'
-
Effect: Allow
Action:
- devicefarm:ListProjects
- devicefarm:ListDevicePools
- devicefarm:GetRun
- devicefarm:GetUpload
- devicefarm:CreateUpload
- devicefarm:ScheduleRun
Resource: 'resource_ARN'
-
Effect: Allow
Action:
- lambda:InvokeFunction
- lambda:ListFunctions
Resource: 'resource_ARN'
-
Effect: Allow
Action:
- iam:PassRole
Resource: 'resource_ARN'
-
Effect: Allow
Action:
- elasticbeanstalk:*
- ec2:*
- elasticloadbalancing:*
- autoscaling:*
- cloudwatch:*
- s3:*
- sns:*
- cloudformation:*
- rds:*
- sqs:*
- ecs:*
Resource: 'resource_ARN'
AppPipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
Name: s3-events-pipeline
RoleArn:
!GetAtt CodePipelineServiceRole.Arn
Stages:
-
Name: Source
Actions:
-
Name: SourceAction
ActionTypeId:
Category: Source
Owner: AWS
Version: 1
Provider: S3
OutputArtifacts:
- Name: SourceOutput
Configuration:
S3Bucket: !Ref SourceBucket
S3ObjectKey: !Ref SourceObjectKey
PollForSourceChanges: false
RunOrder: 1
-
Name: Beta
Actions:
-
Name: BetaAction
InputArtifacts:
- Name: SourceOutput
ActionTypeId:
Category: Deploy
Owner: AWS
Version: 1
Provider: CodeDeploy
Configuration:
ApplicationName: !Ref ApplicationName
DeploymentGroupName: !Ref BetaFleet
RunOrder: 1
ArtifactStore:
Type: S3
Location: !Ref CodePipelineArtifactStoreBucket
EventRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Principal:
Service:
- events.amazonaws.com
Action: sts:AssumeRole
Path: /
Policies:
-
PolicyName: eb-pipeline-execution
PolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Action: codepipeline:StartPipelineExecution
Resource: !Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ]
EventRule:
Type: AWS::Events::Rule
Properties:
EventBusName: default
EventPattern:
source:
- aws.s3
detail-type:
- Object Created
detail:
bucket:
name:
- !Ref SourceBucket
Name: EnabledS3SourceRule
State: ENABLED
Targets:
-
Arn:
!Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ]
RoleArn: !GetAtt EventRole.Arn
Id: codepipeline-AppPipeline
```
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Action": [
"appconfig:StartDeployment",
"appconfig:StopDeployment",
"appconfig:GetDeployment"
],
"Resource": [
"arn:aws:appconfig:*:111122223333:application/[[Application]]",
"arn:aws:appconfig:*:111122223333:application/[[Application]]/*",
"arn:aws:appconfig:*:111122223333:deploymentstrategy/*"
],
"Effect": "Allow"
}
]
}
```
## Migrate polling pipelines with an S3 source and CloudTrail trail
For a pipeline with an Amazon S3 source, modify the pipeline so that change detection is automated through EventBridge. Choose from the following methods to implement the migration:
+ **Console: **[Migrate polling pipelines (CodeCommit or Amazon S3 source) (console)](#update-change-detection-console-codecommit-S3)
+ **CLI: **[Migrate polling pipelines with an S3 source and CloudTrail trail (CLI)](#update-change-detection-cli-S3)
+ **CloudFormation: **[Migrate polling pipelines with an S3 source and CloudTrail trail (CloudFormation template)](#update-change-detection-cfn-s3)
### Migrate polling pipelines with an S3 source and CloudTrail trail (CLI)
Follow these steps to edit a pipeline that is using polling (periodic checks) to use an event in EventBridge instead. If you want to create a pipeline, see [Create a pipeline, stages, and actions](pipelines-create.md).
To build an event-driven pipeline with Amazon S3, you edit the `PollForSourceChanges` parameter of your pipeline and then create the following resources:
+ AWS CloudTrail trail, bucket, and bucket policy that Amazon S3 can use to log the events.
+ EventBridge event
+ IAM role to allow the EventBridge event to start your pipeline
**To create an AWS CloudTrail trail and enable logging**
To use the AWS CLI to create a trail, call the **create-trail** command, specifying:
+ The trail name.
+ The bucket to which you have already applied the bucket policy for AWS CloudTrail.
For more information, see [Creating a trail with the AWS command line interface](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-and-update-a-trail-by-using-the-aws-cli.html).
1. Call the **create-trail** command and include the `--name` and `--s3-bucket-name` parameters.
**Why am I making this change?** This creates the CloudTrail trail required for your S3 source bucket.
The following command uses `--name` and `--s3-bucket-name` to create a trail named `my-trail` and a bucket named `amzn-s3-demo-source-bucket`.
```
aws cloudtrail create-trail --name my-trail --s3-bucket-name amzn-s3-demo-source-bucket
```
1. Call the **start-logging** command and include the `--name` parameter.
**Why am I making this change? ** This command starts the CloudTrail logging for your source bucket and sends events to EventBridge.
Example:
The following command uses `--name` to start logging on a trail named `my-trail`.
```
aws cloudtrail start-logging --name my-trail
```
1. Call the **put-event-selectors** command and include the `--trail-name` and `--event-selectors` parameters. Use event selectors to specify that you want your trail to log data events for your source bucket and send the events to the EventBridge rule.
**Why am I making this change? ** This command filters events.
Example:
The following command uses `--trail-name` and `--event-selectors` to specify data events for a source bucket and prefix named `amzn-s3-demo-source-bucket/myFolder`.
```
aws cloudtrail put-event-selectors --trail-name my-trail --event-selectors '[{ "ReadWriteType": "WriteOnly", "IncludeManagementEvents":false, "DataResources": [{ "Type": "AWS::S3::Object", "Values": ["arn:aws:s3:::amzn-s3-demo-source-bucket/myFolder/file.zip"] }] }]'
```
**To create an EventBridge rule with Amazon S3 as the event source and CodePipeline as the target and apply the permissions policy**
1. Grant permissions for EventBridge to use CodePipeline to invoke the rule. For more information, see [Using resource-based policies for Amazon EventBridge](http://docs.aws.amazon.com/eventbridge/latest/userguide/eb-use-resource-based.html).
1. Use the following sample to create the trust policy to allow EventBridge to assume the service role. Name it `trustpolicyforEB.json`.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "events.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
```
------
1. Use the following command to create the `Role-for-MyRule` role and attach the trust policy.
**Why am I making this change?** Adding this trust policy to the role creates permissions for EventBridge.
```
aws iam create-role --role-name Role-for-MyRule --assume-role-policy-document file://trustpolicyforEB.json
```
1. Create the permissions policy JSON, as shown here for the pipeline named `MyFirstPipeline`. Name the permissions policy `permissionspolicyforEB.json`.
------
#### [ JSON ]
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"codepipeline:StartPipelineExecution"
],
"Resource": [
"arn:aws:codepipeline:us-west-2:111122223333:MyFirstPipeline"
]
}
]
}
```
------
1. Use the following command to attach the new `CodePipeline-Permissions-Policy-for-EB` permissions policy to the `Role-for-MyRule` role you created.
```
aws iam put-role-policy --role-name Role-for-MyRule --policy-name CodePipeline-Permissions-Policy-For-EB --policy-document file://permissionspolicyforEB.json
```
1. Call the **put-rule** command and include the `--name`, `--event-pattern`, and `--role-arn` parameters.
The following sample command creates a rule named `MyS3SourceRule`.
```
aws events put-rule --name "MyS3SourceRule" --event-pattern "{\"source\":[\"aws.s3\"],\"detail-type\":[\"AWS API Call via CloudTrail\"],\"detail\":{\"eventSource\":[\"s3.amazonaws.com\"],\"eventName\":[\"CopyObject\",\"PutObject\",\"CompleteMultipartUpload\"],\"requestParameters\":{\"bucketName\":[\"amzn-s3-demo-source-bucket\"],\"key\":[\"my-key\"]}}}
--role-arn "arn:aws:iam::ACCOUNT_ID:role/Role-for-MyRule"
```
1. To add CodePipeline as a target, call the **put-targets** command and include the `--rule` and `--targets` parameters.
The following command specifies that for the rule named `MyS3SourceRule`, the target `Id` is composed of the number one, indicating that in a list of targets for the rule, this is target 1. The command also specifies an example `ARN` for the pipeline. The pipeline starts when something changes in the repository.
```
aws events put-targets --rule MyS3SourceRule --targets Id=1,Arn=arn:aws:codepipeline:us-west-2:80398EXAMPLE:TestPipeline
```
1. (Optional) To configure an input transformer with source overrides for a specific image ID, use the following JSON in your CLI command. The following example configures an override where:
+ The `actionName`, `Source` in this example, is the dynamic value, defined at pipeline creation, not derived from the source event.
+ The `revisionType`, `S3_OBJECT_VERSION_ID` in this example, is the dynamic value, defined at pipeline creation, not derived from the source event.
+ The `revisionValue`, <*revisionValue*> in this example, is derived from the source event variable.
```
{
"Rule": "my-rule",
"Targets": [
{
"Id": "MyTargetId",
"Arn": "ARN",
"InputTransformer": {
"InputPathsMap": {
"revisionValue": "$.detail.object.version-id"
},
"InputTemplate": {
"sourceRevisions": {
"actionName": "Source",
"revisionType": "S3_OBJECT_VERSION_ID",
"revisionValue": ""
}
}
}
}
]
}
```
**To edit your pipeline's PollForSourceChanges parameter**
**Important**
When you create a pipeline with this method, the `PollForSourceChanges` parameter defaults to true if it is not explicitly set to false. When you add event-based change detection, you must add the parameter to your output and set it to false to disable polling. Otherwise, your pipeline starts twice for a single source change. For details, see [Valid settings for the `PollForSourceChanges` parameter](PollForSourceChanges-defaults.md).
1. Run the **get-pipeline** command to copy the pipeline structure into a JSON file. For example, for a pipeline named `MyFirstPipeline`, run the following command:
```
aws codepipeline get-pipeline --name MyFirstPipeline >pipeline.json
```
This command returns nothing, but the file you created should appear in the directory where you ran the command.
1. Open the JSON file in any plain-text editor and edit the source stage by changing the `PollForSourceChanges` parameter for a bucket named `amzn-s3-demo-source-bucket` to `false`, as shown in this example.
**Why am I making this change?** Setting this parameter to `false` turns off periodic checks so you can use event-based change detection only.
```
"configuration": {
"S3Bucket": "amzn-s3-demo-source-bucket",
"PollForSourceChanges": "false",
"S3ObjectKey": "index.zip"
},
```
1. If you are working with the pipeline structure retrieved using the **get-pipeline** command, you must remove the `metadata` lines from the JSON file. Otherwise, the **update-pipeline** command cannot use it. Remove the `"metadata": { }` lines and the `"created"`, `"pipelineARN"`, and `"updated"` fields.
For example, remove the following lines from the structure:
```
"metadata": {
"pipelineArn": "arn:aws:codepipeline:region:account-ID:pipeline-name",
"created": "date",
"updated": "date"
},
```
Save the file.
1. To apply your changes, run the **update-pipeline** command, specifying the pipeline JSON file:
**Important**
Be sure to include `file://` before the file name. It is required in this command.
```
aws codepipeline update-pipeline --cli-input-json file://pipeline.json
```
This command returns the entire structure of the edited pipeline.
**Note**
The **update-pipeline** command stops the pipeline. If a revision is being run through the pipeline when you run the **update-pipeline** command, that run is stopped. You must manually start the pipeline to run that revision through the updated pipeline. Use the **start-pipeline-execution** command to manually start your pipeline.
### Migrate polling pipelines with an S3 source and CloudTrail trail (CloudFormation template)
Use these steps to edit your pipeline with an Amazon S3 source from polling to event-based change detection.
To build an event-driven pipeline with Amazon S3, you edit the `PollForSourceChanges` parameter of your pipeline and then add the following resources to your template:
+ EventBridge requires that all Amazon S3 events must be logged. You must create an AWS CloudTrail trail, bucket, and bucket policy that Amazon S3 can use to log the events that occur. For more information, see [Logging data events for trails](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/logging-data-events-with-cloudtrail.html) and [Logging management events for trails](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/logging-management-events-with-cloudtrail.html).
+ EventBridge rule and IAM role to allow this event to start our pipeline.
If you use CloudFormation to create and manage your pipelines, your template includes content like the following.
**Note**
The `Configuration` property in the source stage called `PollForSourceChanges`. If your template doesn't include that property, then `PollForSourceChanges` is set to `true` by default.
------
#### [ YAML ]
```
AppPipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
RoleArn: !GetAtt CodePipelineServiceRole.Arn
Stages:
-
Name: Source
Actions:
-
Name: SourceAction
ActionTypeId:
Category: Source
Owner: AWS
Version: 1
Provider: S3
OutputArtifacts:
-
Name: SourceOutput
Configuration:
S3Bucket: !Ref SourceBucket
S3ObjectKey: !Ref S3SourceObjectKey
PollForSourceChanges: true
RunOrder: 1
...
```
------
#### [ JSON ]
```
"AppPipeline": {
"Type": "AWS::CodePipeline::Pipeline",
"Properties": {
"RoleArn": {
"Fn::GetAtt": ["CodePipelineServiceRole", "Arn"]
},
"Stages": [
{
"Name": "Source",
"Actions": [
{
"Name": "SourceAction",
"ActionTypeId": {
"Category": "Source",
"Owner": "AWS",
"Version": 1,
"Provider": "S3"
},
"OutputArtifacts": [
{
"Name": "SourceOutput"
}
],
"Configuration": {
"S3Bucket": {
"Ref": "SourceBucket"
},
"S3ObjectKey": {
"Ref": "SourceObjectKey"
},
"PollForSourceChanges": true
},
"RunOrder": 1
}
]
},
...
```
------
**To create an EventBridge rule with Amazon S3 as the event source and CodePipeline as the target and apply the permissions policy**
1. In the template, under `Resources`, use the `AWS::IAM::Role` CloudFormation resource to configure the IAM role that allows your event to start your pipeline. This entry creates a role that uses two policies:
+ The first policy allows the role to be assumed.
+ The second policy provides permissions to start the pipeline.
**Why am I making this change?** Adding `AWS::IAM::Role` resource enables CloudFormation to create permissions for EventBridge. This resource is added to your CloudFormation stack.
------
#### [ YAML ]
```
EventRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Principal:
Service:
- events.amazonaws.com
Action: sts:AssumeRole
Path: /
Policies:
-
PolicyName: eb-pipeline-execution
PolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Action: codepipeline:StartPipelineExecution
Resource: !Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ]
...
```
------
#### [ JSON ]
```
"EventRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"events.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
},
"Path": "/",
"Policies": [
{
"PolicyName": "eb-pipeline-execution",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "codepipeline:StartPipelineExecution",
"Resource": {
"Fn::Join": [
"",
[
"arn:aws:codepipeline:",
{
"Ref": "AWS::Region"
},
":",
{
"Ref": "AWS::AccountId"
},
":",
{
"Ref": "AppPipeline"
}
]
]
...
```
------
1. Use the `AWS::Events::Rule` CloudFormation resource to add an EventBridge rule. This event pattern creates an event that monitors `CopyObject`, `PutObject` and `CompleteMultipartUpload` on your Amazon S3 source bucket. In addition, include a target of your pipeline. When `CopyObject`, `PutObject`, or `CompleteMultipartUpload` occurs, this rule invokes `StartPipelineExecution` on your target pipeline.
**Why am I making this change?** Adding the `AWS::Events::Rule` resource enables CloudFormation to create the event. This resource is added to your CloudFormation stack.
------
#### [ YAML ]
```
EventRule:
Type: AWS::Events::Rule
Properties:
EventPattern:
source:
- aws.s3
detail-type:
- 'AWS API Call via CloudTrail'
detail:
eventSource:
- s3.amazonaws.com
eventName:
- CopyObject
- PutObject
- CompleteMultipartUpload
requestParameters:
bucketName:
- !Ref SourceBucket
key:
- !Ref SourceObjectKey
Targets:
-
Arn:
!Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ]
RoleArn: !GetAtt EventRole.Arn
Id: codepipeline-AppPipeline
...
```
------
#### [ JSON ]
```
"EventRule": {
"Type": "AWS::Events::Rule",
"Properties": {
"EventPattern": {
"source": [
"aws.s3"
],
"detail-type": [
"AWS API Call via CloudTrail"
],
"detail": {
"eventSource": [
"s3.amazonaws.com"
],
"eventName": [
"CopyObject",
"PutObject",
"CompleteMultipartUpload"
],
"requestParameters": {
"bucketName": [
{
"Ref": "SourceBucket"
}
],
"key": [
{
"Ref": "SourceObjectKey"
}
]
}
}
},
"Targets": [
{
"Arn": {
"Fn::Join": [
"",
[
"arn:aws:codepipeline:",
{
"Ref": "AWS::Region"
},
":",
{
"Ref": "AWS::AccountId"
},
":",
{
"Ref": "AppPipeline"
}
]
]
},
"RoleArn": {
"Fn::GetAtt": [
"EventRole",
"Arn"
]
},
"Id": "codepipeline-AppPipeline"
}
]
}
}
},
...
```
------
1. Add this snippet to your first template to allow cross-stack functionality:
------
#### [ YAML ]
```
Outputs:
SourceBucketARN:
Description: "S3 bucket ARN that Cloudtrail will use"
Value: !GetAtt SourceBucket.Arn
Export:
Name: SourceBucketARN
```
------
#### [ JSON ]
```
"Outputs" : {
"SourceBucketARN" : {
"Description" : "S3 bucket ARN that Cloudtrail will use",
"Value" : { "Fn::GetAtt": ["SourceBucket", "Arn"] },
"Export" : {
"Name" : "SourceBucketARN"
}
}
...
```
------
1. (Optional) To configure an input transformer with source overrides for a specific image ID, use the following YAML snippet. The following example configures an override where:
+ The `actionName`, `Source` in this example, is the dynamic value, defined at pipeline creation, not derived from the source event.
+ The `revisionType`, `S3_OBJECT_VERSION_ID` in this example, is the dynamic value, defined at pipeline creation, not derived from the source event.
+ The `revisionValue`, <*revisionValue*> in this example, is derived from the source event variable.
```
---
Rule: my-rule
Targets:
- Id: MyTargetId
Arn: pipeline-ARN
InputTransformer:
InputPathsMap:
revisionValue: "$.detail.object.version-id"
InputTemplate:
sourceRevisions:
actionName: Source
revisionType: S3_OBJECT_VERSION_ID
revisionValue: ''
```
1. Save your updated template to your local computer, and open the CloudFormation console.
1. Choose your stack, and then choose **Create Change Set for Current Stack**.
1. Upload your updated template, and then view the changes listed in CloudFormation. These are the changes that will be made to the stack. You should see your new resources in the list.
1. Choose **Execute**.
**To edit your pipeline's PollForSourceChanges parameter**
**Important**
When you create a pipeline with this method, the `PollForSourceChanges` parameter defaults to true if it is not explicitly set to false. When you add event-based change detection, you must add the parameter to your output and set it to false to disable polling. Otherwise, your pipeline starts twice for a single source change. For details, see [Valid settings for the `PollForSourceChanges` parameter](PollForSourceChanges-defaults.md).
+ In the template, change `PollForSourceChanges` to `false`. If you did not include `PollForSourceChanges` in your pipeline definition, add it and set it to `false`.
**Why am I making this change?** Changing `PollForSourceChanges` to `false` turns off periodic checks so you can use event-based change detection only.
------
#### [ YAML ]
```
Name: Source
Actions:
-
Name: SourceAction
ActionTypeId:
Category: Source
Owner: AWS
Version: 1
Provider: S3
OutputArtifacts:
- Name: SourceOutput
Configuration:
S3Bucket: !Ref SourceBucket
S3ObjectKey: !Ref SourceObjectKey
PollForSourceChanges: false
RunOrder: 1
```
------
#### [ JSON ]
```
{
"Name": "SourceAction",
"ActionTypeId": {
"Category": "Source",
"Owner": "AWS",
"Version": 1,
"Provider": "S3"
},
"OutputArtifacts": [
{
"Name": "SourceOutput"
}
],
"Configuration": {
"S3Bucket": {
"Ref": "SourceBucket"
},
"S3ObjectKey": {
"Ref": "SourceObjectKey"
},
"PollForSourceChanges": false
},
"RunOrder": 1
}
```
------
**To create a second template for your Amazon S3 pipeline's CloudTrail resources**
+ In a separate template, under `Resources`, use the `AWS::S3::Bucket`, `AWS::S3::BucketPolicy`, and `AWS::CloudTrail::Trail` CloudFormation resources to provide a simple bucket definition and trail for CloudTrail.
**Why am I making this change? ** Given the current limit of five trails per account, the CloudTrail trail must be created and managed separately. (See [Limits in AWS CloudTrail](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/WhatIsCloudTrail-Limits.html).) However, you can include many Amazon S3 buckets on a single trail, so you can create the trail once and then add Amazon S3 buckets for other pipelines as necessary. Paste the following into your second sample template file.
------
#### [ YAML ]
```
###################################################################################
# Prerequisites:
# - S3 SourceBucket and SourceObjectKey must exist
###################################################################################
Parameters:
SourceObjectKey:
Description: 'S3 source artifact'
Type: String
Default: SampleApp_Linux.zip
Resources:
AWSCloudTrailBucketPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket: !Ref AWSCloudTrailBucket
PolicyDocument:
Version: 2012-10-17
Statement:
-
Sid: AWSCloudTrailAclCheck
Effect: Allow
Principal:
Service:
- cloudtrail.amazonaws.com
Action: s3:GetBucketAcl
Resource: !GetAtt AWSCloudTrailBucket.Arn
-
Sid: AWSCloudTrailWrite
Effect: Allow
Principal:
Service:
- cloudtrail.amazonaws.com
Action: s3:PutObject
Resource: !Join [ '', [ !GetAtt AWSCloudTrailBucket.Arn, '/AWSLogs/', !Ref 'AWS::AccountId', '/*' ] ]
Condition:
StringEquals:
s3:x-amz-acl: bucket-owner-full-control
AWSCloudTrailBucket:
Type: AWS::S3::Bucket
DeletionPolicy: Retain
AwsCloudTrail:
DependsOn:
- AWSCloudTrailBucketPolicy
Type: AWS::CloudTrail::Trail
Properties:
S3BucketName: !Ref AWSCloudTrailBucket
EventSelectors:
-
DataResources:
-
Type: AWS::S3::Object
Values:
- !Join [ '', [ !ImportValue SourceBucketARN, '/', !Ref SourceObjectKey ] ]
ReadWriteType: WriteOnly
IncludeManagementEvents: false
IncludeGlobalServiceEvents: true
IsLogging: true
IsMultiRegionTrail: true
...
```
------
#### [ JSON ]
```
{
"Parameters": {
"SourceObjectKey": {
"Description": "S3 source artifact",
"Type": "String",
"Default": "SampleApp_Linux.zip"
}
},
"Resources": {
"AWSCloudTrailBucket": {
"Type": "AWS::S3::Bucket",
"DeletionPolicy": "Retain"
},
"AWSCloudTrailBucketPolicy": {
"Type": "AWS::S3::BucketPolicy",
"Properties": {
"Bucket": {
"Ref": "AWSCloudTrailBucket"
},
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AWSCloudTrailAclCheck",
"Effect": "Allow",
"Principal": {
"Service": [
"cloudtrail.amazonaws.com"
]
},
"Action": "s3:GetBucketAcl",
"Resource": {
"Fn::GetAtt": [
"AWSCloudTrailBucket",
"Arn"
]
}
},
{
"Sid": "AWSCloudTrailWrite",
"Effect": "Allow",
"Principal": {
"Service": [
"cloudtrail.amazonaws.com"
]
},
"Action": "s3:PutObject",
"Resource": {
"Fn::Join": [
"",
[
{
"Fn::GetAtt": [
"AWSCloudTrailBucket",
"Arn"
]
},
"/AWSLogs/",
{
"Ref": "AWS::AccountId"
},
"/*"
]
]
},
"Condition": {
"StringEquals": {
"s3:x-amz-acl": "bucket-owner-full-control"
}
}
}
]
}
}
},
"AwsCloudTrail": {
"DependsOn": [
"AWSCloudTrailBucketPolicy"
],
"Type": "AWS::CloudTrail::Trail",
"Properties": {
"S3BucketName": {
"Ref": "AWSCloudTrailBucket"
},
"EventSelectors": [
{
"DataResources": [
{
"Type": "AWS::S3::Object",
"Values": [
{
"Fn::Join": [
"",
[
{
"Fn::ImportValue": "SourceBucketARN"
},
"/",
{
"Ref": "SourceObjectKey"
}
]
]
}
]
}
],
"ReadWriteType": "WriteOnly",
"IncludeManagementEvents": false
}
],
"IncludeGlobalServiceEvents": true,
"IsLogging": true,
"IsMultiRegionTrail": true
}
}
}
}
...
```
------
**Example**
When you use CloudFormation to create these resources, your pipeline is triggered when files in your repository are created or updated.
Do not stop here. Although your pipeline is created, you must create a second CloudFormation template for your Amazon S3 pipeline. If you do not create the second template, your pipeline does not have any change detection functionality.
```
Resources:
SourceBucket:
Type: AWS::S3::Bucket
Properties:
VersioningConfiguration:
Status: Enabled
CodePipelineArtifactStoreBucket:
Type: AWS::S3::Bucket
CodePipelineArtifactStoreBucketPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket: !Ref CodePipelineArtifactStoreBucket
PolicyDocument:
Version: 2012-10-17
Statement:
-
Sid: DenyUnEncryptedObjectUploads
Effect: Deny
Principal: '*'
Action: s3:PutObject
Resource: !Join [ '', [ !GetAtt CodePipelineArtifactStoreBucket.Arn, '/*' ] ]
Condition:
StringNotEquals:
s3:x-amz-server-side-encryption: aws:kms
-
Sid: DenyInsecureConnections
Effect: Deny
Principal: '*'
Action: s3:*
Resource: !Join [ '', [ !GetAtt CodePipelineArtifactStoreBucket.Arn, '/*' ] ]
Condition:
Bool:
aws:SecureTransport: false
CodePipelineServiceRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Principal:
Service:
- codepipeline.amazonaws.com
Action: sts:AssumeRole
Path: /
Policies:
-
PolicyName: AWS-CodePipeline-Service-3
PolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Action:
- codecommit:CancelUploadArchive
- codecommit:GetBranch
- codecommit:GetCommit
- codecommit:GetUploadArchiveStatus
- codecommit:UploadArchive
Resource: 'resource_ARN'
-
Effect: Allow
Action:
- codedeploy:CreateDeployment
- codedeploy:GetApplicationRevision
- codedeploy:GetDeployment
- codedeploy:GetDeploymentConfig
- codedeploy:RegisterApplicationRevision
Resource: 'resource_ARN'
-
Effect: Allow
Action:
- codebuild:BatchGetBuilds
- codebuild:StartBuild
Resource: 'resource_ARN'
-
Effect: Allow
Action:
- devicefarm:ListProjects
- devicefarm:ListDevicePools
- devicefarm:GetRun
- devicefarm:GetUpload
- devicefarm:CreateUpload
- devicefarm:ScheduleRun
Resource: 'resource_ARN'
-
Effect: Allow
Action:
- lambda:InvokeFunction
- lambda:ListFunctions
Resource: 'resource_ARN'
-
Effect: Allow
Action:
- iam:PassRole
Resource: 'resource_ARN'
-
Effect: Allow
Action:
- elasticbeanstalk:*
- ec2:*
- elasticloadbalancing:*
- autoscaling:*
- cloudwatch:*
- s3:*
- sns:*
- cloudformation:*
- rds:*
- sqs:*
- ecs:*
Resource: 'resource_ARN'
AppPipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
Name: s3-events-pipeline
RoleArn:
!GetAtt CodePipelineServiceRole.Arn
Stages:
-
Name: Source
Actions:
-
Name: SourceAction
ActionTypeId:
Category: Source
Owner: AWS
Version: 1
Provider: S3
OutputArtifacts:
- Name: SourceOutput
Configuration:
S3Bucket: !Ref SourceBucket
S3ObjectKey: !Ref SourceObjectKey
PollForSourceChanges: false
RunOrder: 1
-
Name: Beta
Actions:
-
Name: BetaAction
InputArtifacts:
- Name: SourceOutput
ActionTypeId:
Category: Deploy
Owner: AWS
Version: 1
Provider: CodeDeploy
Configuration:
ApplicationName: !Ref ApplicationName
DeploymentGroupName: !Ref BetaFleet
RunOrder: 1
ArtifactStore:
Type: S3
Location: !Ref CodePipelineArtifactStoreBucket
EventRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Principal:
Service:
- events.amazonaws.com
Action: sts:AssumeRole
Path: /
Policies:
-
PolicyName: eb-pipeline-execution
PolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Action: codepipeline:StartPipelineExecution
Resource: !Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ]
EventRule:
Type: AWS::Events::Rule
Properties:
EventPattern:
source:
- aws.s3
detail-type:
- 'AWS API Call via CloudTrail'
detail:
eventSource:
- s3.amazonaws.com
eventName:
- PutObject
- CompleteMultipartUpload
resources:
ARN:
- !Join [ '', [ !GetAtt SourceBucket.Arn, '/', !Ref SourceObjectKey ] ]
Targets:
-
Arn:
!Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ]
RoleArn: !GetAtt EventRole.Arn
Id: codepipeline-AppPipeline
Outputs:
SourceBucketARN:
Description: "S3 bucket ARN that Cloudtrail will use"
Value: !GetAtt SourceBucket.Arn
Export:
Name: SourceBucketARN
```
```
"Resources": {
"SourceBucket": {
"Type": "AWS::S3::Bucket",
"Properties": {
"VersioningConfiguration": {
"Status": "Enabled"
}
}
},
"CodePipelineArtifactStoreBucket": {
"Type": "AWS::S3::Bucket"
},
"CodePipelineArtifactStoreBucketPolicy": {
"Type": "AWS::S3::BucketPolicy",
"Properties": {
"Bucket": {
"Ref": "CodePipelineArtifactStoreBucket"
},
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Sid": "DenyUnEncryptedObjectUploads",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:PutObject",
"Resource": {
"Fn::Join": [
"",
[
{
"Fn::GetAtt": [
"CodePipelineArtifactStoreBucket",
"Arn"
]
},
"/*"
]
]
},
"Condition": {
"StringNotEquals": {
"s3:x-amz-server-side-encryption": "aws:kms"
}
}
},
{
"Sid": "DenyInsecureConnections",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:*",
"Resource": {
"Fn::Join": [
"",
[
{
"Fn::GetAtt": [
"CodePipelineArtifactStoreBucket",
"Arn"
]
},
"/*"
]
]
},
"Condition": {
"Bool": {
"aws:SecureTransport": false
}
}
}
]
}
}
},
"CodePipelineServiceRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"codepipeline.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
},
"Path": "/",
"Policies": [
{
"PolicyName": "AWS-CodePipeline-Service-3",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"codecommit:CancelUploadArchive",
"codecommit:GetBranch",
"codecommit:GetCommit",
"codecommit:GetUploadArchiveStatus",
"codecommit:UploadArchive"
],
"Resource": "resource_ARN"
},
{
"Effect": "Allow",
"Action": [
"codedeploy:CreateDeployment",
"codedeploy:GetApplicationRevision",
"codedeploy:GetDeployment",
"codedeploy:GetDeploymentConfig",
"codedeploy:RegisterApplicationRevision"
],
"Resource": "resource_ARN"
},
{
"Effect": "Allow",
"Action": [
"codebuild:BatchGetBuilds",
"codebuild:StartBuild"
],
"Resource": "resource_ARN"
},
{
"Effect": "Allow",
"Action": [
"devicefarm:ListProjects",
"devicefarm:ListDevicePools",
"devicefarm:GetRun",
"devicefarm:GetUpload",
"devicefarm:CreateUpload",
"devicefarm:ScheduleRun"
],
"Resource": "resource_ARN"
},
{
"Effect": "Allow",
"Action": [
"lambda:InvokeFunction",
"lambda:ListFunctions"
],
"Resource": "resource_ARN"
},
{
"Effect": "Allow",
"Action": [
"iam:PassRole"
],
"Resource": "resource_ARN"
},
{
"Effect": "Allow",
"Action": [
"elasticbeanstalk:*",
"ec2:*",
"elasticloadbalancing:*",
"autoscaling:*",
"cloudwatch:*",
"s3:*",
"sns:*",
"cloudformation:*",
"rds:*",
"sqs:*",
"ecs:*"
],
"Resource": "resource_ARN"
}
]
}
}
]
}
},
"AppPipeline": {
"Type": "AWS::CodePipeline::Pipeline",
"Properties": {
"Name": "s3-events-pipeline",
"RoleArn": {
"Fn::GetAtt": [
"CodePipelineServiceRole",
"Arn"
]
},
"Stages": [
{
"Name": "Source",
"Actions": [
{
"Name": "SourceAction",
"ActionTypeId": {
"Category": "Source",
"Owner": "AWS",
"Version": 1,
"Provider": "S3"
},
"OutputArtifacts": [
{
"Name": "SourceOutput"
}
],
"Configuration": {
"S3Bucket": {
"Ref": "SourceBucket"
},
"S3ObjectKey": {
"Ref": "SourceObjectKey"
},
"PollForSourceChanges": false
},
"RunOrder": 1
}
]
},
{
"Name": "Beta",
"Actions": [
{
"Name": "BetaAction",
"InputArtifacts": [
{
"Name": "SourceOutput"
}
],
"ActionTypeId": {
"Category": "Deploy",
"Owner": "AWS",
"Version": 1,
"Provider": "CodeDeploy"
},
"Configuration": {
"ApplicationName": {
"Ref": "ApplicationName"
},
"DeploymentGroupName": {
"Ref": "BetaFleet"
}
},
"RunOrder": 1
}
]
}
],
"ArtifactStore": {
"Type": "S3",
"Location": {
"Ref": "CodePipelineArtifactStoreBucket"
}
}
}
},
"EventRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"events.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
},
"Path": "/",
"Policies": [
{
"PolicyName": "eb-pipeline-execution",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "codepipeline:StartPipelineExecution",
"Resource": {
"Fn::Join": [
"",
[
"arn:aws:codepipeline:",
{
"Ref": "AWS::Region"
},
":",
{
"Ref": "AWS::AccountId"
},
":",
{
"Ref": "AppPipeline"
}
]
]
}
}
]
}
}
]
}
},
"EventRule": {
"Type": "AWS::Events::Rule",
"Properties": {
"EventPattern": {
"source": [
"aws.s3"
],
"detail-type": [
"AWS API Call via CloudTrail"
],
"detail": {
"eventSource": [
"s3.amazonaws.com"
],
"eventName": [
"PutObject",
"CompleteMultipartUpload"
],
"resources": {
"ARN": [
{
"Fn::Join": [
"",
[
{
"Fn::GetAtt": [
"SourceBucket",
"Arn"
]
},
"/",
{
"Ref": "SourceObjectKey"
}
]
]
}
]
}
}
},
"Targets": [
{
"Arn": {
"Fn::Join": [
"",
[
"arn:aws:codepipeline:",
{
"Ref": "AWS::Region"
},
":",
{
"Ref": "AWS::AccountId"
},
":",
{
"Ref": "AppPipeline"
}
]
]
},
"RoleArn": {
"Fn::GetAtt": [
"EventRole",
"Arn"
]
},
"Id": "codepipeline-AppPipeline"
}
]
}
}
},
"Outputs" : {
"SourceBucketARN" : {
"Description" : "S3 bucket ARN that Cloudtrail will use",
"Value" : { "Fn::GetAtt": ["SourceBucket", "Arn"] },
"Export" : {
"Name" : "SourceBucketARN"
}
}
}
}
...
```
## Migrate polling pipelines for a GitHub (via OAuth app) source action to connections
You can migrate a GitHub (via OAuth app) source action to use connections for your external repository. This is the recommended change detection method for pipelines with a GitHub (via OAuth app) source action.
For a pipeline with a GitHub (via OAuth app) source action, we recommend modifying the pipeline to use a GitHub (via GitHub App) action so that change detection is automated through AWS CodeConnections. For more information about working with connections, see [GitHub connections](connections-github.md).
### Create a connection to GitHub (console)
You can use the console to create a connection to GitHub.
#### Step 1: Replace your GitHub (via OAuth app) action
Use the pipeline edit page to replace your GitHub (via OAuth app) action with a GitHub (via GitHub App) action.
**To replace your GitHub (via OAuth app) action**
1. Sign in to the CodePipeline console.
1. Choose your pipeline, and choose **Edit**. Choose **Edit stage** on your source stage. A message displays that recommends you update your action.
1. In **Action provider**, choose **GitHub (via GitHub App)**.
1. Do one of the following:
+ Under **Connection**, if you have not already created a connection to your provider, choose **Connect to GitHub**. Proceed to Step 2: Create a connection to GitHub.
+ Under **Connection**, if you have already created a connection to your provider, choose the connection. Proceed to Step 3: Save the Source Action for Your Connection.
#### Step 2: Create a connection to GitHub
After you choose to create the connection, the **Connect to GitHub** page is shown.
**To create a connection to GitHub**
1. Under **GitHub connection settings**, your connection name is shown in **Connection name**.
Under **GitHub Apps**, choose an app installation or choose **Install a new app** to create one.
**Note**
You install one app for all of your connections to a particular provider. If you have already installed the GitHub app, choose it and skip this step.
1. If the authorization page for GitHub displays, log in with your credentials and then choose to continue.
1. On the app installation page, a message shows that the AWS CodeStar app is trying to connect to your GitHub account.
**Note**
You only install the app once for each GitHub account. If you previously installed the app, you can choose **Configure** to proceed to a modification page for your app installation, or you can use the back button to return to the console.
1. On the **Install AWS CodeStar** page, choose **Install**.
1. On the **Connect to GitHub** page, the connection ID for your new installation is displayed. Choose **Connect**.
#### Step 3: Save your GitHub source action
Complete your updates on the **Edit action** page to save your new source action.
**To save your GitHub source action**
1. In **Repository**, enter the name of your third-party repository. In **Branch**, enter the branch where you want your pipeline to detect source changes.
**Note**
In **Repository**, type `owner-name/repository-name` as shown in this example:
```
my-account/my-repository
```
1. In **Output artifact format**, choose the format for your artifacts.
+ To store output artifacts from the GitHub action using the default method, choose **CodePipeline default**. The action accesses the files from the GitHub repository and stores the artifacts in a ZIP file in the pipeline artifact store.
+ To store a JSON file that contains a URL reference to the repository so that downstream actions can perform Git commands directly, choose **Full clone**. This option can only be used by CodeBuild downstream actions.
If you choose this option, you will need to update the permissions for your CodeBuild project service role as shown in [Add CodeBuild GitClone permissions for connections to Bitbucket, GitHub, GitHub Enterprise Server, or GitLab.com](troubleshooting.md#codebuild-role-connections). For a tutorial that shows you how to use the **Full clone** option, see [Tutorial: Use full clone with a GitHub pipeline source](tutorials-github-gitclone.md).
1. In **Output artifacts**, you can retain the name of the output artifact for this action, such as `SourceArtifact`. Choose **Done** to close the **Edit action** page.
1. Choose **Done** to close the stage editing page. Choose **Save** to close the pipeline editing page.
### Create a connection to GitHub (CLI)
You can use the AWS Command Line Interface (AWS CLI) to create a connection to GitHub.
To do this, use the **create-connection** command.
**Important**
A connection created through the AWS CLI or AWS CloudFormation is in `PENDING` status by default. After you create a connection with the CLI or CloudFormation, use the console to edit the connection to make its status `AVAILABLE`.
**To create a connection to GitHub**
1. Open a terminal (Linux, macOS, or Unix) or command prompt (Windows). Use the AWS CLI to run the **create-connection** command, specifying the `--provider-type` and `--connection-name` for your connection. In this example, the third-party provider name is `GitHub` and the specified connection name is `MyConnection`.
```
aws codeconnections create-connection --provider-type GitHub --connection-name MyConnection
```
If successful, this command returns the connection ARN information similar to the following.
```
{
"ConnectionArn": "arn:aws:codeconnections:us-west-2:account_id:connection/aEXAMPLE-8aad-4d5d-8878-dfcab0bc441f"
}
```
1. Use the console to complete the connection.
## Migrate polling pipelines for a GitHub (via OAuth app) source action to webhooks
You can migrate your pipeline to use webhooks to detect changes in your GitHub source repository. This migration to webhooks is for the GitHub (via OAuth app) action only.
+ **Console: **[Migrate polling pipelines to webhooks (GitHub (via OAuth app) source actions) (console)](#update-change-detection-console-github)
+ **CLI: **[Migrate polling pipelines to webhooks (GitHub (via OAuth app) source actions) (CLI)](#update-change-detection-cli-github)
+ **CloudFormation: ** [Update pipelines for push events (GitHub (via OAuth app) source actions) (CloudFormation template)](#update-change-detection-cfn-github)
**Important**
When creating CodePipeline webhooks, do not use your own credentials or reuse the same secret token across multiple webhooks. For optimal security, generate a unique secret token for each webhook you create. The secret token is an arbitrary string that you provide, which GitHub uses to compute and sign the webhook payloads sent to CodePipeline, for protecting the integrity and authenticity of the webhook payloads. Using your own credentials or reusing the same token across multiple webhooks can lead to security vulnerabilities.
### Migrate polling pipelines to webhooks (GitHub (via OAuth app) source actions) (console)
For the GitHub (via OAuth app) source action, you can use the CodePipeline console to update your pipeline to use webhooks to detect changes in your GitHub source repository.
Follow these steps to edit a pipeline that is using polling (periodic checks) to use EventBridge instead. If you want to create a pipeline, see [Create a pipeline, stages, and actions](pipelines-create.md).
When you use the console, the `PollForSourceChanges` parameter for your pipelined is changed for you. The GitHub webhook is created and registered for you.
**To edit the pipeline source stage**
1. Sign in to the AWS Management Console and open the CodePipeline console at [http://console.aws.amazon.com/codesuite/codepipeline/home](http://console.aws.amazon.com/codesuite/codepipeline/home).
The names of all pipelines associated with your AWS account are displayed.
1. In **Name**, choose the name of the pipeline you want to edit. This opens a detailed view of the pipeline, including the state of each of the actions in each stage of the pipeline.
1. On the pipeline details page, choose **Edit**.
1. In **Edit stage**, choose the edit icon on the source action.
1. Expand **Change detection options** and choose **Use Amazon CloudWatch Events to automatically start my pipeline when a change occurs (recommended)**.
A message is displayed to advise that CodePipeline creates a webhook in GitHub to detect source changes: AWS CodePipeline will create a webhook for you. You can opt-out in the options below. Choose **Update**. In addition to the webhook, CodePipeline creates the following:
+ A secret, randomly generated and used to authorize the connection to GitHub.
+ The webhook URL, generated using the public endpoint for the Region.
CodePipeline registers the webhook with GitHub. This subscribes the URL to receive repository events.
1. When you have finished editing your pipeline, choose **Save pipeline changes** to return to the summary page.
A message displays the name of the webhook to be created for your pipeline. Choose **Save and continue**.
1. To test your action, release a change by using the AWS CLI to commit a change to the source specified in the source stage of the pipeline.
### Migrate polling pipelines to webhooks (GitHub (via OAuth app) source actions) (CLI)
Follow these steps to edit a pipeline that is using periodic checks to use a webhook instead. If you want to create a pipeline, see [Create a pipeline, stages, and actions](pipelines-create.md).
To build an event-driven pipeline, you edit the `PollForSourceChanges` parameter of your pipeline and then create the following resources manually:
+ GitHub webhook and authorization parameters
**To create and register your webhook**
**Note**
When you use the CLI or CloudFormation to create a pipeline and add a webhook, you must disable periodic checks. To disable periodic checks, you must explicitly add the `PollForSourceChanges` parameter and set it to false, as detailed in the final procedure below. Otherwise, the default for a CLI or CloudFormation pipeline is that `PollForSourceChanges` defaults to true and does not display in the pipeline structure output. For more information about PollForSourceChanges defaults, see [Valid settings for the `PollForSourceChanges` parameter](PollForSourceChanges-defaults.md).
1. In a text editor, create and save a JSON file for the webhook you want to create. Use this sample file for a webhook named `my-webhook`:
```
{
"webhook": {
"name": "my-webhook",
"targetPipeline": "pipeline_name",
"targetAction": "source_action_name",
"filters": [{
"jsonPath": "$.ref",
"matchEquals": "refs/heads/{Branch}"
}],
"authentication": "GITHUB_HMAC",
"authenticationConfiguration": {
"SecretToken": "secret"
}
}
}
```
1. Call the **put-webhook** command and include the `--cli-input` and `--region` parameters.
The following sample command creates a webhook with the `webhook_json` JSON file.
```
aws codepipeline put-webhook --cli-input-json file://webhook_json.json --region "eu-central-1"
```
1. In the output shown in this example, the URL and ARN are returned for a webhook named `my-webhook`.
```
{
"webhook": {
"url": "https://webhooks.domain.com/trigger111111111EXAMPLE11111111111111111",
"definition": {
"authenticationConfiguration": {
"SecretToken": "secret"
},
"name": "my-webhook",
"authentication": "GITHUB_HMAC",
"targetPipeline": "pipeline_name",
"targetAction": "Source",
"filters": [
{
"jsonPath": "$.ref",
"matchEquals": "refs/heads/{Branch}"
}
]
},
"arn": "arn:aws:codepipeline:eu-central-1:ACCOUNT_ID:webhook:my-webhook"
},
"tags": [{
"key": "Project",
"value": "ProjectA"
}]
}
```
This example adds tagging to the webhook by including the `Project` tag key and `ProjectA` value on the webhook. For more information about tagging resources in CodePipeline, see [Tagging resources](tag-resources.md).
1. Call the **register-webhook-with-third-party** command and include the `--webhook-name` parameter.
The following sample command registers a webhook named `my-webhook`.
```
aws codepipeline register-webhook-with-third-party --webhook-name my-webhook
```
**To edit your pipeline's PollForSourceChanges parameter**
**Important**
When you create a pipeline with this method, the `PollForSourceChanges` parameter defaults to true if it is not explicitly set to false. When you add event-based change detection, you must add the parameter to your output and set it to false to disable polling. Otherwise, your pipeline starts twice for a single source change. For details, see [Valid settings for the `PollForSourceChanges` parameter](PollForSourceChanges-defaults.md).
1. Run the **get-pipeline** command to copy the pipeline structure into a JSON file. For example, for a pipeline named `MyFirstPipeline`, you would type the following command:
```
aws codepipeline get-pipeline --name MyFirstPipeline >pipeline.json
```
This command returns nothing, but the file you created should appear in the directory where you ran the command.
1. Open the JSON file in any plain-text editor and edit the source stage by changing or adding the `PollForSourceChanges` parameter. In this example, for a repository named `UserGitHubRepo`, the parameter is set to `false` .
**Why am I making this change? ** Changing this parameter turns off periodic checks so you can use event-based change detection only.
```
"configuration": {
"Owner": "name",
"Repo": "UserGitHubRepo",
"PollForSourceChanges": "false",
"Branch": "main",
"OAuthToken": "****"
},
```
1. If you are working with the pipeline structure retrieved using the **get-pipeline** command, you must edit the structure in the JSON file by removing the `metadata` lines from the file. Otherwise, the **update-pipeline** command cannot use it. Remove the `"metadata"` section from the pipeline structure in the JSON file, including the : `{ }` and the `"created"`, `"pipelineARN"`, and `"updated"` fields.
For example, remove the following lines from the structure:
```
"metadata": {
"pipelineArn": "arn:aws:codepipeline:region:account-ID:pipeline-name",
"created": "date",
"updated": "date"
},
```
Save the file.
1. To apply your changes, run the **update-pipeline** command, specifying the pipeline JSON file, similar to the following:
**Important**
Be sure to include `file://` before the file name. It is required in this command.
```
aws codepipeline update-pipeline --cli-input-json file://pipeline.json
```
This command returns the entire structure of the edited pipeline.
**Note**
The **update-pipeline** command stops the pipeline. If a revision is being run through the pipeline when you run the **update-pipeline** command, that run is stopped. You must manually start the pipeline to run that revision through the updated pipeline. Use the **start-pipeline-execution** command to manually start your pipeline.
### Update pipelines for push events (GitHub (via OAuth app) source actions) (CloudFormation template)
Follow these steps to update your pipeline (with a GitHub source) from periodic checks (polling) to event-based change detection using webhooks.
To build an event-driven pipeline with AWS CodeCommit, you edit the `PollForSourceChanges` parameter of your pipeline and then add a GitHub webhook resource to your template.
If you use CloudFormation to create and manage your pipelines, your template has content like the following.
**Note**
Note the `PollForSourceChanges` configuration property in the source stage. If your template doesn't include that property, then `PollForSourceChanges` is set to `true` by default.
------
#### [ YAML ]
```
Resources:
AppPipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
Name: github-polling-pipeline
RoleArn:
!GetAtt CodePipelineServiceRole.Arn
Stages:
-
Name: Source
Actions:
-
Name: SourceAction
ActionTypeId:
Category: Source
Owner: ThirdParty
Version: 1
Provider: GitHub
OutputArtifacts:
- Name: SourceOutput
Configuration:
Owner: !Ref GitHubOwner
Repo: !Ref RepositoryName
Branch: !Ref BranchName
OAuthToken: {{resolve:secretsmanager:MyGitHubSecret:SecretString:token}} PollForSourceChanges: true
RunOrder: 1
...
```
------
#### [ JSON ]
```
"AppPipeline": {
"Type": "AWS::CodePipeline::Pipeline",
"Properties": {
"Name": "github-polling-pipeline",
"RoleArn": {
"Fn::GetAtt": [
"CodePipelineServiceRole",
"Arn"
]
},
"Stages": [
{
"Name": "Source",
"Actions": [
{
"Name": "SourceAction",
"ActionTypeId": {
"Category": "Source",
"Owner": "ThirdParty",
"Version": 1,
"Provider": "GitHub"
},
"OutputArtifacts": [
{
"Name": "SourceOutput"
}
],
"Configuration": {
"Owner": {
"Ref": "GitHubOwner"
},
"Repo": {
"Ref": "RepositoryName"
},
"Branch": {
"Ref": "BranchName"
},
"OAuthToken": "{{resolve:secretsmanager:MyGitHubSecret:SecretString:token}}",
"PollForSourceChanges": true
},
"RunOrder": 1
}
]
},
...
```
------
**To add parameters and create a webhook in your template**
We strongly recommend that you use AWS Secrets Manager to store your credentials. If you use Secrets Manager, you must have already configured and stored your secret parameters in Secrets Manager. This example uses dynamic references to Secrets Manager for the GitHub credentials for your webhook. For more information, see [ Using Dynamic References to Specify Template Values](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/dynamic-references.html#dynamic-references-secretsmanager).
**Important**
When passing secret parameters, do not enter the value directly into the template. The value is rendered as plaintext and is therefore readable. For security reasons, do not use plaintext in your CloudFormation template to store your credentials.
When you use the CLI or CloudFormation to create a pipeline and add a webhook, you must disable periodic checks.
**Note**
To disable periodic checks, you must explicitly add the `PollForSourceChanges` parameter and set it to false, as detailed in the final procedure below. Otherwise, the default for a CLI or CloudFormation pipeline is that `PollForSourceChanges` defaults to true and does not display in the pipeline structure output. For more information about PollForSourceChanges defaults, see [Valid settings for the `PollForSourceChanges` parameter](PollForSourceChanges-defaults.md).
1. In the template, under `Resources`, add your parameters:
------
#### [ YAML ]
```
Parameters:
GitHubOwner:
Type: String
...
```
------
#### [ JSON ]
```
{
"Parameters": {
"BranchName": {
"Description": "GitHub branch name",
"Type": "String",
"Default": "main"
},
"GitHubOwner": {
"Type": "String"
},
...
```
------
1. Use the `AWS::CodePipeline::Webhook` CloudFormation resource to add a webhook.
**Note**
The `TargetAction` you specify must match the `Name` property of the source action defined in the pipeline.
If `RegisterWithThirdParty` is set to `true`, make sure the user associated to the `OAuthToken` can set the required scopes in GitHub. The token and webhook require the following GitHub scopes:
+ `repo` - used for full control to read and pull artifacts from public and private repositories into a pipeline.
+ `admin:repo_hook` - used for full control of repository hooks.
Otherwise, GitHub returns a 404. For more information about the 404 returned, see [https://help.github.com/articles/about-webhooks](https://help.github.com/articles/about-webhooks).
------
#### [ YAML ]
```
AppPipelineWebhook:
Type: AWS::CodePipeline::Webhook
Properties:
Authentication: GITHUB_HMAC
AuthenticationConfiguration:
SecretToken: {{resolve:secretsmanager:MyGitHubSecret:SecretString:token}}
Filters:
-
JsonPath: "$.ref"
MatchEquals: refs/heads/{Branch}
TargetPipeline: !Ref AppPipeline
TargetAction: SourceAction
Name: AppPipelineWebhook
TargetPipelineVersion: !GetAtt AppPipeline.Version
RegisterWithThirdParty: true
...
```
------
#### [ JSON ]
```
"AppPipelineWebhook": {
"Type": "AWS::CodePipeline::Webhook",
"Properties": {
"Authentication": "GITHUB_HMAC",
"AuthenticationConfiguration": {
"SecretToken": "{{resolve:secretsmanager:MyGitHubSecret:SecretString:token}}"
},
"Filters": [{
"JsonPath": "$.ref",
"MatchEquals": "refs/heads/{Branch}"
}],
"TargetPipeline": {
"Ref": "AppPipeline"
},
"TargetAction": "SourceAction",
"Name": "AppPipelineWebhook",
"TargetPipelineVersion": {
"Fn::GetAtt": [
"AppPipeline",
"Version"
]
},
"RegisterWithThirdParty": true
}
},
...
```
------
1. Save the updated template to your local computer, and then open the CloudFormation console.
1. Choose your stack, and then choose **Create Change Set for Current Stack**.
1. Upload the template, and then view the changes listed in CloudFormation. These are the changes to be made to the stack. You should see your new resources in the list.
1. Choose **Execute**.
**To edit your pipeline's PollForSourceChanges parameter**
**Important**
When you create a pipeline with this method, the `PollForSourceChanges` parameter defaults to true if it is not explicitly set to false. When you add event-based change detection, you must add the parameter to your output and set it to false to disable polling. Otherwise, your pipeline starts twice for a single source change. For details, see [Valid settings for the `PollForSourceChanges` parameter](PollForSourceChanges-defaults.md).
+ In the template, change `PollForSourceChanges` to `false`. If you did not include `PollForSourceChanges` in your pipeline definition, add it and set it to false.
**Why am I making this change? ** Changing this parameter to `false` turns off periodic checks so you can use event-based change detection only.
------
#### [ YAML ]
```
Name: Source
Actions:
-
Name: SourceAction
ActionTypeId:
Category: Source
Owner: ThirdParty
Version: 1
Provider: GitHub
OutputArtifacts:
- Name: SourceOutput
Configuration:
Owner: !Ref GitHubOwner
Repo: !Ref RepositoryName
Branch: !Ref BranchName
OAuthToken: {{resolve:secretsmanager:MyGitHubSecret:SecretString:token}}
PollForSourceChanges: false
RunOrder: 1
```
------
#### [ JSON ]
```
{
"Name": "Source",
"Actions": [{
"Name": "SourceAction",
"ActionTypeId": {
"Category": "Source",
"Owner": "ThirdParty",
"Version": 1,
"Provider": "GitHub"
},
"OutputArtifacts": [{
"Name": "SourceOutput"
}],
"Configuration": {
"Owner": {
"Ref": "GitHubOwner"
},
"Repo": {
"Ref": "RepositoryName"
},
"Branch": {
"Ref": "BranchName"
},
"OAuthToken": "{{resolve:secretsmanager:MyGitHubSecret:SecretString:token}}",
PollForSourceChanges: false
},
"RunOrder": 1
}]
```
------
**Example**
When you create these resources with CloudFormation, the webhook defined is created in the specified GitHub repository. Your pipeline is triggered on commit.
```
Parameters:
GitHubOwner:
Type: String
Resources:
AppPipelineWebhook:
Type: AWS::CodePipeline::Webhook
Properties:
Authentication: GITHUB_HMAC
AuthenticationConfiguration:
SecretToken: {{resolve:secretsmanager:MyGitHubSecret:SecretString:token}}
Filters:
-
JsonPath: "$.ref"
MatchEquals: refs/heads/{Branch}
TargetPipeline: !Ref AppPipeline
TargetAction: SourceAction
Name: AppPipelineWebhook
TargetPipelineVersion: !GetAtt AppPipeline.Version
RegisterWithThirdParty: true
AppPipeline:
Type: AWS::CodePipeline::Pipeline
Properties:
Name: github-events-pipeline
RoleArn:
!GetAtt CodePipelineServiceRole.Arn
Stages:
-
Name: Source
Actions:
-
Name: SourceAction
ActionTypeId:
Category: Source
Owner: ThirdParty
Version: 1
Provider: GitHub
OutputArtifacts:
- Name: SourceOutput
Configuration:
Owner: !Ref GitHubOwner
Repo: !Ref RepositoryName
Branch: !Ref BranchName
OAuthToken: {{resolve:secretsmanager:MyGitHubSecret:SecretString:token}}
PollForSourceChanges: false
RunOrder: 1
...
```
```
{
"Parameters": {
"BranchName": {
"Description": "GitHub branch name",
"Type": "String",
"Default": "main"
},
"RepositoryName": {
"Description": "GitHub repository name",
"Type": "String",
"Default": "test"
},
"GitHubOwner": {
"Type": "String"
},
"ApplicationName": {
"Description": "CodeDeploy application name",
"Type": "String",
"Default": "DemoApplication"
},
"BetaFleet": {
"Description": "Fleet configured in CodeDeploy",
"Type": "String",
"Default": "DemoFleet"
}
},
"Resources": {
...
},
"AppPipelineWebhook": {
"Type": "AWS::CodePipeline::Webhook",
"Properties": {
"Authentication": "GITHUB_HMAC",
"AuthenticationConfiguration": {
"SecretToken": {
"{{resolve:secretsmanager:MyGitHubSecret:SecretString:token}}"
}
},
"Filters": [
{
"JsonPath": "$.ref",
"MatchEquals": "refs/heads/{Branch}"
}
],
"TargetPipeline": {
"Ref": "AppPipeline"
},
"TargetAction": "SourceAction",
"Name": "AppPipelineWebhook",
"TargetPipelineVersion": {
"Fn::GetAtt": [
"AppPipeline",
"Version"
]
},
"RegisterWithThirdParty": true
}
},
"AppPipeline": {
"Type": "AWS::CodePipeline::Pipeline",
"Properties": {
"Name": "github-events-pipeline",
"RoleArn": {
"Fn::GetAtt": [
"CodePipelineServiceRole",
"Arn"
]
},
"Stages": [
{
"Name": "Source",
"Actions": [
{
"Name": "SourceAction",
"ActionTypeId": {
"Category": "Source",
"Owner": "ThirdParty",
"Version": 1,
"Provider": "GitHub"
},
"OutputArtifacts": [
{
"Name": "SourceOutput"
}
],
"Configuration": {
"Owner": {
"Ref": "GitHubOwner"
},
"Repo": {
"Ref": "RepositoryName"
},
"Branch": {
"Ref": "BranchName"
},
"OAuthToken": "{{resolve:secretsmanager:MyGitHubSecret:SecretString:token}}",
"PollForSourceChanges": false
},
"RunOrder": 1
...
```