Automate Amazon VPC IPAM IPv4 CIDR allocations for new AWS accounts by using AFT - AWS Prescriptive Guidance

Automate Amazon VPC IPAM IPv4 CIDR allocations for new AWS accounts by using AFT

Kien Pham and Alex Pazik, Amazon Web Services

Summary

This pattern shows how to automate Amazon VPC IP Address Manager (IPAM) IPv4 CIDR allocations for new AWS accounts by using AWS Control Tower Account Factory for Terraform (AFT). This is done using an account-level customization that allocates a IPv4 CIDR block from IPAM to a new virtual private cloud (VPC) using the aft-account-customizations module.

With IPAM, you can organize, assign, monitor, and audit IP addresses at scale, allowing you to easily plan, track, and monitor IP addresses for your AWS workloads. You can create an IPAM and IPAM pool to use to allocate an IPv4 CIDR block to a new VPC during the account vending process.

Prerequisites and limitations

Prerequisites

  • An active AWS account with AWS Control Tower enabled in a supported AWS Region and AFT deployed

  • A supported version control system (VCS) provider such as BitBucket, GitHub, and GitHub Enterprise

  • Terraform Command Line Interface (CLI) installed

  • A runtime environment where you can run the Terraform module that installs AFT

  • AWS Command Line Interface (AWS CLI) installed and configured

Limitations

Product versions

Architecture

The following diagram shows the workflow and components of this pattern.

Workflow to create Amazon VPC IPAM IPv4 CIDR allocation.

The workflow consists of the following main tasks:

  1. Trigger changes – The changes to Terraform and IPAM customization are committed to the GitHub repository and pushed. This task triggers the AWS CodeBuild pipeline automatically.

  2. Automate build – Within CodeBuild, multiple build projects trigger AWS Step Functions.

  3. Apply customization – Step Functions coordinates with CodeBuild to plan and apply Terraform changes. This task uses the AFT Terraform module to coordinate the IPAM pool IP assignment to the AWS vended account.

Tools

AWS services

  • AWS CodeBuild is a fully managed build service that helps you compile source code, run unit tests, and produce artifacts that are ready to deploy.

  • AWS CodePipeline helps you quickly model and configure the different stages of a software release and automate the steps required to release software changes continuously.

  • AWS Control Tower orchestrates the capabilities of several other AWS services, including AWS Organizations, AWS Service Catalog, and AWS IAM Identity Center. It can help you set up and govern an AWS multi-account environment, following prescriptive best practices.

  • Amazon DynamoDB is a fully managed NoSQL database service that provides fast, predictable, and scalable performance.

  • AWS Lambda is a compute service that helps you run code without needing to provision or manage servers. It runs your code only when needed and scales automatically, so you pay only for the compute time that you use.

  • AWS SDK for Python (Boto3) is a software development kit that helps you integrate your Python application, library, or script with AWS services.

  • AWS Service Catalog helps you centrally manage catalogs of IT services that are approved for AWS. End users can quickly deploy only the approved IT services they need, following the constraints set by your organization.

  • AWS Step Functions is a serverless orchestration service that helps you combine AWS Lambda functions and other AWS services to build business-critical applications.

  • Amazon Virtual Private Cloud (Amazon VPC) helps you launch AWS resources into a virtual network that you’ve defined. This virtual network resembles a traditional network that you’d operate in your own data center, with the benefits of using the scalable infrastructure of AWS. Amazon VPC IP Address Manager (IPAM) is a VPC feature that makes it easier for you to plan, track, and monitor IP addresses for your AWS workloads.

Other tools

  • GitHub is a developer platform that developers can use to create, store, manage, and share their code.

  • HashiCorp Terraform is an infrastructure as code (IaC) tool that helps you create and manage cloud and on-premises resources. This includes low-level components such as compute instances, storage, and networking and high-level components such as DNS entries and software a a service (SaaS) features.

  • Python is a general-purpose computer programming language. You can use it to build applications, automate tasks, and develop services on the AWS Cloud.

Code repository

Best practices

When you deploy AFT, we recommend that you follow best practices to help ensure a secure, efficient, and successful implementation. Key guidelines and recommendations for implementing and operating AFT include the following:

  • Thorough review of inputs – Carefully review and understand each input. Correct input configuration is crucial for the setup and functioning of AFT.

  • Regular template updates – Keep templates updated with the latest AWS features and Terraform versions. Regular updates help you take advantage of new functionality and maintain security.

  • Versioning – Pin your AFT module version and use a separate AFT deployment for testing if possible.

  • Scope – Use AFT only to deploy infrastructure guardrails and customizations. Do not use it to deploy your application.

  • Linting and validation – The AFT pipeline requires a linted and validated Terraform configuration. Run lint, validate, and test before pushing the configuration to AFT repositories.

  • Terraform modules – Build reusable Terraform code as modules, and always specify the Terraform and AWS provider versions to match your organization's requirements.

Epics

TaskDescriptionSkills required

Deploy AWS Control Tower.

Set up and configure AWS Control Tower in your AWS environment to ensure centralized management and governance of your AWS accounts. For more information, see Getting started with AWS Control Tower in the AWS Control Tower documentation.

Cloud administrator

Deploy AWS Control Tower Account Factory for Terraform (AFT).

Set up AFT in a new, dedicated AFT management account. For more information, see Configure and launch your AWS Control Tower Account Factory for Terraform in the AWS Control Tower documentation.

Cloud administrator

Complete AFT post-deployment.

After the AFT infrastructure deployment is complete, complete the steps in Post-deployment steps in the AWS Control Tower documentation.

Cloud administrator
TaskDescriptionSkills required

Delegate an IPAM administrator.

To delegate an IPAM administrator account in your AWS organization, use the following steps:

  1. Using the AWS Organizations management account, open the IPAM console at https://console.aws.amazon.com/ipam/.

  2. In the AWS Management Console, choose the AWS Region in which you want to work with IPAM.

  3. In the navigation pane, choose Organization settings.

  4. Choose Delegate. The Delegate option is only available if you signed in to the console as the AWS Organizations management account.

  5. Enter the AWS account ID for an IPAM account. The IPAM administrator must be in an AWS Organizations member account.

  6. Choose Save changes.

Alternatively, you can use the AWS CLI and run the following command:

aws ec2 enable-ipam-organization-admin-account \ --delegated-admin-account-id 012345678901

For more information, see Integrate IPAM with accounts in an AWS organization in the Amazon VPC documentation and enable-ipam-organization-admin-account in the AWS CLI Command Reference.

Important

To continue using IPAM, you must sign in to the delegated administrator account. The SSO profile or AWS environment variables specified in the next step must allow you to sign in to that account and grant permissions to create an IPAM top-level and regional pool.

AWS administrator

Create an IPAM top-level and regional pool.

This pattern’s GitHub repository contains a Terraform template that you can use to create your IPAM top-level pool and regional pool. Then you can share the pools with an organization, organizational unit (OU), AWS account, or other resource by using AWS Resource Access Manager (AWS RAM).

Use the following steps:

  1. Run the following commands:

    # Navigate to the IPAM module cd ipam/terraform # Initiate the IPAM module terraform init
  2. Open the main.tf file, modify the code as described in the following comments, and save the file.

    terraform { required_providers { aws = { source = "hashicorp/aws" version = ">= 5.11.0, < 6.0.0" } } } provider "aws" { region = "us-east-1" # Replace with the region to create your top-level pool } module "ipam" { source = "aws-ia/ipam/aws" version = ">= 2.1.0" top_name = "my-top-level-pool" # Replace with your desired top-level pool name top_description = "top-level pool" # Replace with your desired top-level level pool description top_cidr = ["10.0.0.0/16"] # Replace with your desired top-level pool CIDR pool_configurations = { my-regional-pool = { # (Optional) Replace with a different resource name name = "my-regional-pool" # Replace with your desired pool name description = "regional pool" # Replace with your desired pool description cidr = ["10.0.0.0/23"] # Replace with your desired pool CIDR locale = "us-east-1" # Replace with your desired pool locale ram_share_principals = ["arn:aws:organizations::012345678901:ou/ou-ab1c2de345/ou-ab1c2de345"] # Replace with your desired principal ARN to share with via Resource Access Manager (RAM) } } } output "my_regional_pool_id" { description = "The ID of the regional pool" value = module.ipam.pools_level_1["my-regional-pool"].id # Replace with your desired resource name if changed above } # Create the IPAM top-level and resource pool terraform apply

Make a note of the resource pool ID that’s output after creation. You will need the ID when submitting the account request. If you forget the resource pool ID, you can get it later from the AWS Management Console.

Important

Make sure that the created pools’ CIDRs do not overlap with any other pools in your working Region. You can create a pool without a CIDR, but you won’t be able to use the pool for allocations until you’ve provisioned a CIDR for it. You can add CIDRs to a pool at any time by editing the pool.

AWS administrator
TaskDescriptionSkills required

Begin to create account customization.

To begin a new account customization, run the following commands from your terminal:

# Default name for customization repo cd aft-account-customizations # Replace with your actual repo name if different than the default mkdir -p APG-AFT-IPAM/terraform # Replace APG-AFT-IPAM with your desired customization name cd APG-AFT-IPAM/terraform
DevOps engineer

Create aft-providers.jinja file.

Add dynamic code to the aft-providers.jinja file that specifies the Terraform backend and provider to use.

Use the following steps:

  1. Create a new aft-providers.jinja file in the terraform directory.

  2. Open the file, add the following code, and save the file.

    provider "aws" { region = "{{ provider_region }}" assume_role { role_arn = "{{ target_admin_role_arn }}" } default_tags { tags = { managed_by = "AFT" } } }
DevOps engineer

Create backend.jinja file.

Add dynamic code to the backend.jinja file that specifies the Terraform backend and provider to use.

Use the following steps:

  1. Create a new backend.jinja file in the terraform directory.

  2. Open the file, add the following code, and save the file.

    ## Auto generated backend.tf ## ## Updated on: {{ timestamp }} ## {% if tf_distribution_type == "oss" -%} terraform { required_version = ">= {{ tf_version }}" backend "s3" { region = "{{ region }}" bucket = "{{ bucket }}" key = "{{ key }}" dynamodb_table = "{{ dynamodb_table }}" encrypt = "true" kms_key_id = "{{ kms_key_id }}" role_arn = "{{ aft_admin_role_arn }}" } } {% else -%} terraform { backend "remote" { organization = "{{ terraform_org_name }}" workspaces { name = "{{ terraform_workspace_name }}" } } } {% endif %}
DevOps engineer

Create main.tf file.

Create a new main.tf file and add code that defines two data sources that retrieve two values from AWS Systems Manager (aws_ssm) and creates the VPC.

Use the following steps:

  1. Create a new main.tf file in the terraform directory.

  2. Open the file, add the following code, and save the file.

    # Define data sources data "aws_ssm_parameter" "vpc_ipam_id" { name = "/aft/account-request/custom-fields/vpc-ipam-id" # Value is defined in the account-request.tf file } data "aws_ssm_parameter" "vpc_netmask" { name = "/aft/account-request/custom-fields/vpc-ipam-netmask" # Value is defined in the account-request.tf file } # Create new VPC resource "aws_vpc" "vpc1" { ipv4_ipam_pool_id = data.aws_ssm_parameter.vpc_ipam_id.value # Retrieved from SSM - this is how we integrate with IPAM ipv4_netmask_length = data.aws_ssm_parameter.vpc_netmask.value # Retrieved from SSM assign_generated_ipv6_cidr_block = var.enable_ipv6 ? true : null ipv6_cidr_block = var.ipv6_cidr ipv6_ipam_pool_id = var.ipv6_ipam_pool_id ipv6_netmask_length = var.ipv6_netmask_length ipv6_cidr_block_network_border_group = var.ipv6_cidr_block_network_border_group instance_tenancy = var.instance_tenancy enable_dns_hostnames = var.enable_dns_hostnames enable_dns_support = var.enable_dns_support enable_network_address_usage_metrics = var.enable_network_address_usage_metrics tags = var.tags lifecycle { ignore_changes = [ tags, # Any changes made to VPC tags after creation will not be overwritten - remove to revert these changes during future 'terraform apply' operations ] } }
DevOps engineer

Create variables.tf file.

Create a variables.tf file that declares the variables used by the Terraform module.

Use the following steps:

  1. Create a new variables.tf file in the terraform directory.

  2. Open the file, add the following code, and save the file.

    # Copied from AWS VPC module # https://github.com/terraform-aws-modules/terraform-aws-vpc/blob/master/variables.tf variable "name" { description = "Name to be used on all the resources as identifier" type = string default = "" } variable "enable_ipv6" { description = "Requests an Amazon-provided IPv6 CIDR block with a /56 prefix length for the VPC. You cannot specify the range of IP addresses, or the size of the CIDR block" type = bool default = false } variable "ipv6_cidr" { description = "(Optional) IPv6 CIDR block to request from an IPAM Pool. Can be set explicitly or derived from IPAM using `ipv6_netmask_length`" type = string default = null } variable "ipv6_ipam_pool_id" { description = "(Optional) IPAM Pool ID for a IPv6 pool. Conflicts with `assign_generated_ipv6_cidr_block`" type = string default = null } variable "ipv6_netmask_length" { description = "(Optional) Netmask length to request from IPAM Pool. Conflicts with `ipv6_cidr_block`. This can be omitted if IPAM pool as a `allocation_default_netmask_length` set. Valid values: `56`" type = number default = null } variable "ipv6_cidr_block_network_border_group" { description = "By default when an IPv6 CIDR is assigned to a VPC a default ipv6_cidr_block_network_border_group will be set to the region of the VPC. This can be changed to restrict advertisement of public addresses to specific Network Border Groups such as LocalZones" type = string default = null } variable "instance_tenancy" { description = "A tenancy option for instances launched into the VPC" type = string default = "default" } variable "enable_dns_hostnames" { description = "Should be true to enable DNS hostnames in the VPC" type = bool default = true } variable "enable_dns_support" { description = "Should be true to enable DNS support in the VPC" type = bool default = true } variable "enable_network_address_usage_metrics" { description = "Determines whether network address usage metrics are enabled for the VPC" type = bool default = null } variable "tags" { description = "A map of tags to add to all resources" type = map(string) default = {} }
DevOps engineer

Create terraform.tfvars file.

Create a terraform.tfvars file that defines the values of the variables that are passed to the main.tf file.

Use the following steps:

  1. Create a new terraform.tfvars file in the terraform directory.

  2. Open the file, add the following code, and save the file.

    name = "demo-ipam" enable_ipv6 = false enable_dns_hostnames = true enable_dns_support = true
DevOps engineer

Create outputs.tf file.

Create a new outputs.tf file that exposes some values in CodeBuild.

Use the following steps:

  1. Create a new outputs.tf file in the terraform directory.

  2. Open the file, add the following code, and save the file.

    # Output VPC ID and ARN in CodeBuild output "vpc1" { description = "VPC 1 information" value = { id = try(aws_vpc.vpc1.id, null) arn = try(aws_vpc.vpc1.arn, null) } }
DevOps engineer

Commit the customization.

To commit the new customization to the account customizations repository, run the following commands:

# Assumes you are still in the /terraform directory cd .. # Skip if you are in the account customization root directory (APG-AFT-IPAM) git add . git commit -m "APG customization" git push origin
DevOps engineer

Apply the customization.

Add code to the account-requests.tf file that requests a new account with the newly created account customization. The custom fields create Systems Manager parameters in the vended account that are required to create the VPC with the correct IPAM allocated IPv4 CIDR.

Use the following steps:

  1. From your terminal, navigate to the aft-account-request/terraform directory.

  2. Open the account-requests.tf file, add and modify the following code with your values, and save the file.

    module "sandbox_account_01" { source = "./modules/aft-account-request" control_tower_parameters = { AccountEmail = "john.doe@example.com" AccountName = "sandbox-account-01" # Syntax for top-level OU ManagedOrganizationalUnit = "Sandbox" # Syntax for nested OU # ManagedOrganizationalUnit = "Sandbox (ou-xfe5-a8hb8ml8)" SSOUserEmail = "john.doe@example.com" SSOUserFirstName = "John" SSOUserLastName = "Doe" } account_tags = { "ABC:Owner" = "john.doe@example.com" "ABC:Division" = "ENT" "ABC:Environment" = "Dev" "ABC:CostCenter" = "123456" "ABC:Vended" = "true" "ABC:DivCode" = "102" "ABC:BUCode" = "ABC003" "ABC:Project" = "123456" } change_management_parameters = { change_requested_by = "John Doe" change_reason = "testing the account vending process" } custom_fields = { vpc-ipam-id = "ipam-pool-123456ab789041cd2" vpc-ipam-netmask = "24" } account_customizations_name = "APG-AFT-IPAM" # Do not change this value }
    Important

    Make sure the vpc-ipam-netmask value is less than or equal to the IPAM pool allocation netmask, and that there are enough IPs available in the IPAM pool to allocate to the VPC.

  3. To commit the new account to the account requests repository, use the following commands:

    git add . git commit -m 'requested APG account' git push origin
AWS DevOps

Validate the customization.

Sign in to the newly vended account and verify that the customization was successfully applied.

Use the following steps:

  1. Open your AWS IAM Identity Center sign-in page and authenticate as the user you specified in the account-requests.tf file.

  2. From the IAM Identity Center account list, choose the account that you vended earlier and sign in using the AWS AdministratorAccess role.

  3. Navigate to the VPC console, and select Your VPCs.

  4. Select the VPC ID of the VPC that you just created.

  5. Choose the CIDRs tab. You should see an IPAM-allocated IPv4 CIDR.

  6. (Optional) Sign in to your IPAM delegated administrator account and verify that the allocation is shown in the IPAM console.

DevOps engineer

Troubleshooting

IssueSolution

You encounter failures in resource creation or management caused by inadequate permissions.

Review the AWS Identity and Access Management (IAM) roles and policies that are attached to Step Functions, CodeBuild, and other services involved in the deployment. Confirm that they have the necessary permissions. If there are permission issues, adjust the IAM policies to grant the required access.

You reach AWS service quotas during deployment.

Before you deploy the pipeline, check AWS service quotas for resources such as Amazon Simple Storage Service (Amazon S3) buckets, IAM roles, and AWS Lambda functions. If necessary, request increases to the quotas. For more information, see AWS service quotas in the AWS General Reference.

Related resources

AWS service documentation

Other resources