Using Terraform dynamic provider credentials in your AWS landing zones

We often hear from customers about the steps they took to start a new project or migration in Amazon Web Services (AWS). Landing zones are a popular pattern that many customers use. For AWS environments, that often means using AWS Control Tower Account Factory for Terraform (AFT), which sets up a Terraform Cloud workspace and configures the AWS credentials.

AWS Control Tower provides an easy way to set up and govern secure, multi-account AWS environments. Using AWS Control Tower, you can set up a new AWS account with a baseline security features such as audit logging and controls/guardrails. AFT extends AWS Control Tower functionality using a Terraform pipeline to provision and customize AWS Control Tower managed accounts. Using AFT, you can apply account-specific customizations or global customizations to all accounts.

Terraform Cloud dynamic provider credentials are temporary credentials that are automatically generated for each run by HashiCorp Vault or the cloud vendor you’re using. Terraform Cloud kicks off that process by using an OpenID Connect protocol (OIDC) token. This process allows Terraform Cloud to use more secure, short-lived credentials to authenticate with AWS, and assign granular roles for each plan and apply.

Using customizations via AFT, this post demonstrates how to set up a new Terraform Cloud workspace with dynamic provider credentials pre-configured. This new workspace will then accompany every new AWS account provisioned via AFT.

Architecture overview

It’s best to use AFT customization only for guardrails and account governance tools, such as shared VPCs, identity and access management (IAM) roles for break-glass, and Amazon S3 Block Public Access. You should also provide application-level configuration in a separate Terraform Cloud organization. Application-level configuration can be the infrastructure that runs the application (Amazon EC2, ECS, or EKS) or the data store (Amazon S3 bucket, RDS database, etc.). To satisfy both of these recommendations, this tutorial uses two separate Terraform Cloud organizations:

The Infrastructure organization: Used exclusively by the infrastructure team to deploy guardrails that enforce security, compliance, governance, and best practices, such as service control policies (SCPs), IAM policies, approved Amazon machine images (AMIs), encryption, logging, and tagging standards.

The Application organization: Used by application teams to deploy application resources, such as EC2 instances, RDS databases, Amazon Load Balancers (ALBs), and other application-specific resources.

The diagram below illustrates how to use AFT to build and configure both Terraform Cloud organizations:

1. In the Infrastructure organization, AFT creates Terraform workspaces for account and global customizations. The account customization workspace is meant for account-specific guardrails and other account-level governance. The global customization workspace is meant for generic customizations that apply to all accounts.
2. Using AFT’s account customization, create an IAM role, IAM permissions, and a Terraform Cloud OIDC identity provider in the target AWS account.
3. Use the same AFT account customization to authenticate to the Application organization and create an Application workspace.
4. AFT account customization also configures the Application workspace with the required environment variables to enable dynamic provider credentials.
5. With dynamic provider credentials configured on the Application workspace, the workspace users can run the Terraform workflow (plan/apply) without having to set up the AWS credentials. Terraform Cloud will automatically assume a role that uses temporary credentials to access the target AWS account.
6. The Application workspace user can connect the Application workspace with their VCS of choice (GitHub, GitLab, BitBucket, etc.) to enable a GitOps workflow.

Using this architecture, every new AWS account provisioned will include the guardrails and governance you configure in AFT and a dedicated Terraform Cloud Application workspace that automatically generates dynamic provider credentials for each piece of AWS infrastructure that you provision.

Prerequisites

Before you can adopt this architecture, there are several prerequisites that you must complete:

• AWS Control Tower enabled and AFT deployed with Terraform Cloud backend. For more information, refer to the module example and the tutorial.
• Two Terraform cloud organizations, labeled “Infrastructure” and “Application”. You can use different organization names, but they must be separate.
• A Terraform Cloud user token with permissions to create workspaces in the Application organization.

Using the Terraform Cloud/Enterprise provider in AFT

To create a new Application workspace in the Application organization, AFT account customization uses the Terraform Cloud/Enterprise (TFE) provider. This provider accepts a token argument in the provider configuration as shown below:

provider "tfe" {
 hostname = "app.terraform.io"
 token    = var.token
}

To securely retrieve this token during the plan and apply stages, we use the data source aws_secretsmanager_secret_version from the AWS provider and store the Terraform Cloud token in an AWS Secrets Manager secret. (Note: The secret must be stored with the name “/tfc/token”.) Optionally, you can also use Vault. Here is an example of the provider configuration with a token stored in AWS Secrets Manager:

provider "tfe" {
 hostname = "app.terraform.io"
 token    = data.aws_secretsmanager_secret_version.tfe_token_secret.secret_string
}


data "aws_secretsmanager_secret_version" "tfe_token_secret" {
 secret_id = "/tfc/token"
}

To implement this in AFT account customization, we create a new customization name called SANDBOX with the following folder structure:

├── aft-account-customizations
│   └── SANDBOX
│     ├── api_helpers
│     │   ├── post-api-helpers.sh
│     │   ├── pre-api-helpers.sh
│     │   └── python
│     │     └── requirements.txt
│     └── terraform
│         ├── aft-providers.jinja
│         ├── backend.jinja
│         └── tfe.tf

We use the tfe_workspace resource to set up the Application workspace. A snippet of the tfe.tf file is shown below. Each application workspace is configured with a unique name using the format: {aws_account_id}-app-workspace.

To avoid hardcoding the Application workspace organization name, we use the data source aws_ssm_parameter. The value for this SSM parameter is populated during account requests using custom fields. AFT custom fields are used to capture key-value metadata that deploy as SSM parameters in the vended account under the path “/aft/account-request/custom-fields/”:

# Configure the Terraform Cloud / Enterprise provider
provider "tfe" {
 hostname = "app.terraform.io"
 token    = data.aws_secretsmanager_secret_version.tfe_token_secret.secret_string
}

# Retrieve the Terraform Cloud token from AWS Secrets Manager secret
data "aws_secretsmanager_secret_version" "tfe_token_secret" {
 secret_id = "/tfc/token"
 provider  = aws.aft-mgt
}

# Retrieve the application organization name from AWS Parameter Store
data "aws_ssm_parameter" "app_org" {
 name = "/aft/account-request/custom-fields/app_org"
}

# Verify the application organization data source
data "tfe_organization" "app_org" {
 name = data.aws_ssm_parameter.app_org.value
}

# Set the application workspace name
data "aws_caller_identity" "target_account_id" {}

locals {
 app_workspace_name = "${data.aws_caller_identity.target_account_id.account_id}-app-workspace"
}

# Create Application workspace in the Application org
resource "tfe_workspace" "app_workspace" {
 name         = local.app_workspace_name
 organization = data.tfe_organization.app_org.name
 tag_names    = ["app", "${data.aws_caller_identity.target_account_id.account_id}"]
}

Note from the example above: The data source aws_secretsmanager_secret_version uses provider alias aws.aft-mgt. This is intentional, because the Terraform Cloud token is stored as an AWS Secrets Manager secret in the AFT management account. This means you need to modify the AFT account customization’s aft-providers.jinja to include the new alias as shown here:

provider "aws" {
 region = ""
 assume_role {
   role_arn = ""
 }
 default_tags {
   tags = {
     managed_by = "AFT"
   }
 }
}


provider "aws" {
 region = ""
 alias  = "aft-mgt"
 assume_role {
   role_arn = ""
 }
 default_tags {
   tags = {
     managed_by = "AFT"
   }
 }
}

So far, you have configured the AFT account customizations to automatically configure the Application workspace. Next, set up the dynamic provider credentials for this workspace.

Configure dynamic provider credentials

We briefly touched on dynamic provider credentials earlier, and if you need to learn more, check out this tutorial: Authenticate Providers with Dynamic Credentials.

To enable dynamic provider credentials in AWS, configure Terraform Cloud as an IAM OIDC identity provider (IdP) and add an IAM role with a trust policy to the Terraform Cloud as an IdP. To implement this in AFT account customizations, add a new iam.tf file to the SANDBOX customization:

├── aft-account-customizations
│   └── SANDBOX
│     ├── api_helpers
│     │   ├── post-api-helpers.sh
│     │   ├── pre-api-helpers.sh
│     │   └── python
│     │     └── requirements.txt
│     └── terraform
│         ├── aft-providers.jinja
│         ├── backend.jinja
│         ├── iam.tf <- configure OIDC and IAM role
│         └── tfe.tf

Read our terraform-dynamic-credentials-setup-examples repository as an example. Here is the snippet for OIDC IdP setup:

# Data source used to grab the TLS certificate for Terraform Cloud.
data "tls_certificate" "tfc_certificate" {
 url = "https://app.terraform.io"
}

# Creates an OIDC provider which is restricted to
resource "aws_iam_openid_connect_provider" "tfc_provider" {
 url             = data.tls_certificate.tfc_certificate.url
 client_id_list  = ["aws.workload.identity"]
 thumbprint_list = [data.tls_certificate.tfc_certificate.certificates[0].sha1_fingerprint]
}

# Creates a role which can only be used by the specified Terraform cloud workspace.
resource "aws_iam_role" "tfc_role" {
 name = "tfc-role"
 assume_role_policy = 

from HashiCorp Blog https://bit.ly/43OhXWd
via IFTTT