Infrastructure as Code & Automation

Infrastructure as Code & Automation

A pull request changes an instance image, a database setting and a network rule. Which resources will be replaced? Who can approve the deployment? If the application fails afterward, which changes can actually be reversed? Infrastructure as Code (IaC) makes these questions reviewable; it does not answer them automatically.

This lesson develops four applications: a CloudFormation application stack, a reusable Terraform network module, a controlled Systems Manager traffic switch, and a pipeline that applies the reviewed plan. Examples use prepared AWS accounts, IAM roles and network access. They are infrastructure components with explicit prerequisites, not a complete business application.

1. Understand the contract behind the code

IaC describes resources and their relationships in files. Version control preserves the intended configuration; a deployment tool reconciles that intent with resources it manages. Repeating a deployment can converge when inputs and dependencies remain stable. A changing “latest AMI,” an external secret, a provider upgrade or a manual edit can change the result without changing the main configuration file.

• CloudFormation manages AWS stacks from JSON/YAML templates. You design the resources, permissions, update behavior and recovery.
• Terraform manages resources through providers and configuration/state. You manage provider versions, state access, environment boundaries and change review.
• AWS CDK defines constructs in programming languages and synthesizes CloudFormation. You inspect the synthesized resources and manage stack deployments.
• Pulumi defines infrastructure programs using supported languages/providers. You manage program behavior, credentials, state and deployment review.
• Systems Manager Automation orchestrates operational steps in runbooks. You define preconditions, failure paths, permissions and verification.

CDK does not bypass CloudFormation's deployment behavior. Terraform's multi-provider model does not make an AWS resource portable to another cloud. Reverting a Git commit proposes another infrastructure change; it does not restore deleted data. CloudFormation concepts, Terraform purpose, CDK, Pulumi concepts.

2. Model dependencies, identity and change

A CloudFormation logical ID identifies a resource within its template. A physical name is a service-level property with service-specific uniqueness rules. !Sub builds strings; it cannot repair duplicate logical IDs. !Ref returns a parameter's value or a resource-specific return value, which is not universally an ARN or ID. !GetAtt retrieves a documented attribute. Outputs need an explicit Export to support !ImportValue; export names are unique within an account and Region. Imports also constrain changing or deleting the exported value. Ref, GetAtt, cross-stack references.

References create dependencies: an instance referencing a subnet already depends on that subnet and, transitively, its VPC. Add explicit dependencies for relationships the references do not express, such as bootstrap traffic requiring an attached internet gateway and configured routes. Creating a gateway alone does not attach it or make a subnet public. CloudFormation DependsOn, Terraform dependencies.

template/configuration + fixed inputs + provider/tool versions
             |
             v
     change set / execution plan <-- observed managed resources
             |
       review and approval
             |
             v
       execute --> inspect final state --> test application behavior

A change set can reveal additions, deletions and replacements. Current CloudFormation pre-deployment validation catches some syntax, naming, quota and other constraints, but cannot guarantee execution success. Terraform plans also contain unknown values and cannot certify application correctness. Review replacement, data retention, access changes and downtime, not just the number of resources. Change sets, Terraform plan.

3. State is a protected coordination record

Terraform state maps resource addresses to remote objects and records attributes and metadata. It is not a continuously current inventory of the whole AWS account. Planning normally refreshes managed objects through providers; drift can occur between observations. Do not hand-edit state or put it in Git. Import associates an existing object with a resource address; it is not a promise that the surrounding configuration already matches. Removing an address from state does not delete the remote object, and leaving its resource declaration can cause a later plan to create another object. State purpose, import, state removal.

For a team using the S3 backend, prepare a private, versioned bucket outside the application stack's teardown path. Terraform 1.10 and later support native S3 lockfiles with use_lockfile = true. DynamoDB locking is a deprecated alternative; migrate all participating clients deliberately so they still coordinate. S3 storage alone does not enable locking. The lock object needs GetObject, PutObject and DeleteObject permissions; state-object permissions differ. Encryption does not replace access control or recovery testing. S3 backend and permissions.

terraform {
  required_version = ">= 1.10, < 2.0"
  backend "s3" {
    key          = "production/network/terraform.tfstate"
    region       = "us-east-1"
    encrypt      = true
    use_lockfile = true
    # Supply the prepared bucket through a non-secret backend file.
  }
}

Initialize with terraform init -backend-config=production.s3.tfbackend, whose contents include bucket = "YOUR-EXISTING-STATE-BUCKET". Supply short-lived credentials through the normal credential chain, not backend arguments. Backend configuration is distinct from the AWS provider configuration; verify both destinations. Separate production root configurations, state paths and AWS roles. CLI workspaces separate state instances but are not an authorization boundary. Never turn off locking or force-unlock merely because a deployment is slow. Backend configuration, workspaces, force-unlock.

4. Application: deploy an application stack into a prepared network

The following application.yaml manages an HTTPS ALB, an EC2 Auto Scaling group and a conventional Multi-AZ PostgreSQL DB instance. Supply two public ALB subnets and two private application/database subnets spanning two AZs, all in VpcId. Public subnet routes must reach an attached internet gateway. Private instances need the AWS API connectivity used by bootstrap and the application. Security groups, NACLs, DNS and routes must agree.

AppImage must be a tested, Region-local x86-64 image with the application already installed, a local HTTP /health endpoint on port 8080, curl, and /opt/aws/bin/cfn-signal. InstanceProfile is an existing EC2 instance profile with narrowly scoped CloudFormation signaling and application permissions. CertificateArn is a valid ACM certificate in the ALB Region. DBEngineVersion must be available for the selected class/Region. These are deliberate inputs, not invented resource IDs.

AWSTemplateFormatVersion: '2010-09-09'
Description: Application infrastructure in an existing routed network
Parameters:
  VpcId: {Type: 'AWS::EC2::VPC::Id'}
  PublicSubnets: {Type: 'List<AWS::EC2::Subnet::Id>'}
  PrivateSubnets: {Type: 'List<AWS::EC2::Subnet::Id>'}
  AppImage: {Type: 'AWS::EC2::Image::Id'}
  InstanceProfile: {Type: String}
  CertificateArn: {Type: String}
  DBEngineVersion: {Type: String}
Resources:
  EdgeSG:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: HTTPS from viewers
      VpcId: !Ref VpcId
      SecurityGroupIngress:
        - {IpProtocol: tcp, FromPort: 443, ToPort: 443, CidrIp: 0.0.0.0/0}
  AppSG:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: App traffic from this ALB
      VpcId: !Ref VpcId
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 8080
          ToPort: 8080
          SourceSecurityGroupId: !Ref EdgeSG
  DatabaseSG:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: PostgreSQL from this app tier
      VpcId: !Ref VpcId
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 5432
          ToPort: 5432
          SourceSecurityGroupId: !Ref AppSG
  ALB:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties:
      Scheme: internet-facing
      Subnets: !Ref PublicSubnets
      SecurityGroups: [!Ref EdgeSG]
  Targets:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      VpcId: !Ref VpcId
      Port: 8080
      Protocol: HTTP
      TargetType: instance
      HealthCheckPath: /health
      Matcher: {HttpCode: '200'}
  Listener:
    Type: AWS::ElasticLoadBalancingV2::Listener
    Properties:
      LoadBalancerArn: !Ref ALB
      Port: 443
      Protocol: HTTPS
      Certificates: [{CertificateArn: !Ref CertificateArn}]
      DefaultActions: [{Type: forward, TargetGroupArn: !Ref Targets}]
  LaunchTemplate:
    Type: AWS::EC2::LaunchTemplate
    Properties:
      LaunchTemplateData:
        ImageId: !Ref AppImage
        InstanceType: t3.small
        IamInstanceProfile: {Name: !Ref InstanceProfile}
        SecurityGroupIds: [!Ref AppSG]
        MetadataOptions: {HttpTokens: required, HttpEndpoint: enabled}
        UserData:
          Fn::Base64: !Sub |
            #!/bin/bash
            set -u
            result=1
            for attempt in $(seq 1 30); do
              if curl --fail --silent --max-time 5 http://127.0.0.1:8080/health >/dev/null; then
                result=0
                break
              fi
              sleep 5
            done
            /opt/aws/bin/cfn-signal -e "$result" --stack '${AWS::StackName}' --resource AppGroup --region '${AWS::Region}'
  AppGroup:
    Type: AWS::AutoScaling::AutoScalingGroup
    DependsOn: Listener
    CreationPolicy:
      ResourceSignal: {Count: 2, Timeout: PT10M}
    UpdatePolicy:
      AutoScalingRollingUpdate:
        MinInstancesInService: 1
        MaxBatchSize: 1
        PauseTime: PT10M
        WaitOnResourceSignals: true
    Properties:
      MinSize: '2'
      MaxSize: '4'
      DesiredCapacity: '2'
      VPCZoneIdentifier: !Ref PrivateSubnets
      LaunchTemplate:
        LaunchTemplateId: !Ref LaunchTemplate
        Version: !GetAtt LaunchTemplate.LatestVersionNumber
      TargetGroupARNs: [!Ref Targets]
      HealthCheckType: ELB
      HealthCheckGracePeriod: 300
  DBSubnets:
    Type: AWS::RDS::DBSubnetGroup
    Properties:
      DBSubnetGroupDescription: Private database subnets in two AZs
      SubnetIds: !Ref PrivateSubnets
  Database:
    Type: AWS::RDS::DBInstance
    DeletionPolicy: Snapshot
    UpdateReplacePolicy: Snapshot
    Properties:
      Engine: postgres
      EngineVersion: !Ref DBEngineVersion
      DBInstanceClass: db.t3.medium
      AllocatedStorage: '20'
      StorageType: gp3
      StorageEncrypted: true
      MasterUsername: dbadmin
      ManageMasterUserPassword: true
      MultiAZ: true
      PubliclyAccessible: false
      BackupRetentionPeriod: 7
      DBSubnetGroupName: !Ref DBSubnets
      VPCSecurityGroups: [!Ref DatabaseSG]
Outputs:
  LoadBalancerDNS: {Value: !GetAtt ALB.DNSName}
  DatabaseAddress: {Value: !GetAtt Database.Endpoint.Address}
  MasterSecretArn: {Value: !GetAtt Database.MasterUserSecret.SecretArn}

The allowed path is viewer HTTPS → ALB → application port 8080 → PostgreSQL port 5432. Opening port 443 alone does not install a certificate or configure TLS. The template includes a listener and certificate, while application-to-database TLS and database credentials must still be configured in the application. The master secret ARN is a reference, not a password; production applications should use restricted database users. The readiness endpoint should distinguish process startup from full business readiness to avoid a bootstrap dependency cycle.

Creation waits for two local-readiness signals; rolling updates replace one instance at a time while maintaining at least one in-service instance. These settings do not prove external reachability or provide a scaling policy. Test the ALB endpoint and database connectivity afterward. Snapshot policies preserve recovery material on covered deletion/replacement operations; they do not automatically reconnect the application to a restored database. CreationPolicy, UpdatePolicy, DB resource, snapshot replacement policy.

Prepare parameter values in parameters.json without literal secrets. Create and inspect a change set before executing it; CREATE is for a new stack, while an existing stack uses UPDATE. Persist a distinct change-set name for each proposal. Treat a failed/no-change result according to its reported reason.

aws cloudformation create-change-set --stack-name lesson-app \
  --change-set-name reviewed-release-001 --change-set-type CREATE \
  --template-body file://application.yaml --parameters file://parameters.json
aws cloudformation wait change-set-create-complete --stack-name lesson-app \
  --change-set-name reviewed-release-001
aws cloudformation describe-change-set --stack-name lesson-app \
  --change-set-name reviewed-release-001
# After reviewing this exact proposal:
aws cloudformation execute-change-set --stack-name lesson-app \
  --change-set-name reviewed-release-001
aws cloudformation wait stack-create-complete --stack-name lesson-app

5. Application: reuse a module without sharing production state

A module defines a reusable contract; it is not a shared deployment. Put this example in modules/isolated-network/main.tf. It creates a fixed /16 VPC and two isolated subnets. The caller supplies AZ names available in its account/Region. The stable keys a and b identify subnet instances; changing an AZ value can still require replacement.

variable "az_by_key" {
  type = map(string)
  validation {
    condition     = toset(keys(var.az_by_key)) == toset(["a", "b"]) && length(toset(values(var.az_by_key))) == 2
    error_message = "Supply exactly a and b, with two distinct AZ names."
  }
}
resource "aws_vpc" "main" {
  cidr_block           = "10.40.0.0/16"
  enable_dns_support   = true
  enable_dns_hostnames = true
}
locals {
  subnet_number = { a = 1, b = 2 }
}
resource "aws_subnet" "isolated" {
  for_each                = var.az_by_key
  vpc_id                  = aws_vpc.main.id
  availability_zone       = each.value
  cidr_block              = cidrsubnet(aws_vpc.main.cidr_block, 8, local.subnet_number[each.key])
  map_public_ip_on_launch = false
}
resource "aws_route_table" "isolated" {
  vpc_id = aws_vpc.main.id
}
resource "aws_route_table_association" "isolated" {
  for_each       = aws_subnet.isolated
  subnet_id      = each.value.id
  route_table_id = aws_route_table.isolated.id
}
output "subnet_ids" {
  value = { for key, subnet in aws_subnet.isolated : key => subnet.id }
}
output "vpc_id" {
  value = aws_vpc.main.id
}

The root configures the AWS provider, declares its version requirement and calls the module. Add required_providers with source = "hashicorp/aws" in reusable modules too; pass aliased configurations explicitly when needed. In this simple call the child inherits the default AWS provider configuration. Module providers.

module "network" {
  source    = "./modules/isolated-network"
  az_by_key = { a = "us-east-1a", b = "us-east-1b" }
}
output "network_subnets" {
  value = module.network.subnet_ids
}

cidrsubnet("10.40.0.0/16", 8, 1) yields 10.40.1.0/24; index 2 yields 10.40.2.0/24. There is no default route to a NAT gateway or internet gateway. These subnets cannot satisfy the preceding application's connectivity prerequisites without an additional network design. Fixed CIDRs also overlap if the module is reused in connected environments; parameterize address allocation before that reuse.

Deploy development and production from separate roots/state with deliberately selected inputs and roles. Editing a module changes future plans for its callers; it does not instantly update every deployed environment. Commit .terraform.lock.hcl for reproducible provider selection; it is different from a runtime state lock. Pin remote module releases separately, since the provider lock file does not lock module versions. for_each, cidrsubnet, dependency lock file.

6. Application: orchestrate a controlled traffic switch

Systems Manager Run Command executes commands on managed nodes; it requires an agent, role and service connectivity. Automation orchestrates runbook steps. State Manager maintains associations on its schedule, not continuously guaranteed configuration. Parameter Store stores configuration and SecureString values; Session Manager provides managed sessions without requiring an inbound SSH rule. These features have different permissions and availability requirements. Systems Manager capabilities.

Read a particular secret with get_parameter(Name=..., WithDecryption=True) and appropriate SSM/KMS authorization. If reading a hierarchy, paginate and preserve full names to avoid collisions between nested paths. Do not print decrypted values:

def read_parameters(ssm, path):
    values = {}
    for page in ssm.get_paginator("get_parameters_by_path").paginate(
            Path=path, Recursive=True, WithDecryption=True):
        for item in page.get("Parameters", []):
            if item["Name"] in values:
                raise ValueError("Duplicate parameter during paginated read")
            values[item["Name"]] = item["Value"]
    return values

A hierarchy read is not an atomic configuration snapshot. Pin versions or publish a release manifest if values must change together. Granting recursive access to a parent path can expose its descendants; separate secrets by authorization boundary. Parameter retrieval.

For blue/green, prepare two releases and distinct target groups. An ALB instance target is an instance ID, not an Auto Scaling group name. Attach each ASG to its own target group; use a preview listener/rule to make green testable and healthy before switching production. Verify the actual green release, not an unrelated public URL. Keep blue running through a measured observation window. ALB targets, ASG target groups.

This runbook switches only a listener whose sole default action forwards directly to blue. It assumes controlled, serialized deployment ownership, two or more healthy green targets already registered, an Automation role with the required ELB read/modify permissions, and designated IAM approvers. Other listener rules remain unchanged; authentication chains and weighted routing need a richer implementation.

schemaVersion: '0.3'
description: Switch one simple ALB default action and retain blue for recovery
assumeRole: '{{ AutomationAssumeRole }}'
parameters:
  AutomationAssumeRole: {type: String}
  ListenerArn: {type: String}
  BlueTargetGroupArn: {type: String}
  GreenTargetGroupArn: {type: String}
  Approvers: {type: StringList}
mainSteps:
  - name: inspect
    action: aws:executeScript
    timeoutSeconds: 120
    inputs:
      Runtime: python3.11
      Handler: inspect
      InputPayload:
        listener: '{{ ListenerArn }}'
        blue: '{{ BlueTargetGroupArn }}'
        green: '{{ GreenTargetGroupArn }}'
      Script: |
        import boto3
        def inspect(events, context):
            elb = boto3.client("elbv2")
            if events["blue"] == events["green"]:
                raise ValueError("Blue and green must differ")
            listener = elb.describe_listeners(ListenerArns=[events["listener"]])["Listeners"][0]
            actions = listener["DefaultActions"]
            if len(actions) != 1 or actions[0]["Type"] != "forward":
                raise ValueError("Expected one simple forwarding action")
            action = actions[0]
            groups = action.get("ForwardConfig", {}).get("TargetGroups", [])
            if action.get("TargetGroupArn") != events["blue"] or (groups and any(g["TargetGroupArn"] != events["blue"] for g in groups)):
                raise ValueError("Listener is not forwarding only to blue")
            targets = elb.describe_target_health(TargetGroupArn=events["green"])["TargetHealthDescriptions"]
            if len(targets) < 2 or any(t["TargetHealth"]["State"] != "healthy" for t in targets):
                raise ValueError("Green does not meet target-health requirement")
            return {"OriginalActions": actions}
    outputs:
      - {Name: OriginalActions, Selector: '$.Payload.OriginalActions', Type: MapList}
  - name: approveSwitch
    action: aws:approve
    timeoutSeconds: 600
    inputs:
      Message: Review green release tests and the intended listener switch
      MinRequiredApprovals: 1
      Approvers: '{{ Approvers }}'
  - name: switchGreen
    action: aws:executeAwsApi
    onFailure: step:restoreBlue
    onCancel: step:restoreBlue
    inputs:
      Service: elbv2
      Api: ModifyListener
      ListenerArn: '{{ ListenerArn }}'
      DefaultActions:
        - {Type: forward, TargetGroupArn: '{{ GreenTargetGroupArn }}'}
  - name: verifyRelease
    action: aws:approve
    timeoutSeconds: 600
    nextStep: complete
    onFailure: step:restoreBlue
    onCancel: step:restoreBlue
    inputs:
      Message: Check the production path and release metrics; approve to retain green
      MinRequiredApprovals: 1
      Approvers: '{{ Approvers }}'
  - name: restoreBlue
    action: aws:executeAwsApi
    isEnd: true
    inputs:
      Service: elbv2
      Api: ModifyListener
      ListenerArn: '{{ ListenerArn }}'
      DefaultActions: '{{ inspect.OriginalActions }}'
  - name: complete
    action: aws:sleep
    isEnd: true
    inputs:
      Duration: PT1S

Register this as an Automation document and start its exact reviewed version with aws ssm start-automation-execution --document-name NAME --document-version VERSION --parameters file://switch-parameters.json. An approval timeout/rejection after switching routes to restoration. The restored actions come from the inspection step, not a guessed reconstruction. A restore operation can fail, and a request timeout can leave an uncertain outcome: inspect the actual listener and execution history. Serialize all writers to this listener, revalidate readiness near execution, and reconcile any external changes before resuming. The runbook never terminates blue and does not roll back database writes or schema migrations. executeScript, approve, step failure/cancellation, ModifyListener.

7. Application: apply the plan that was approved

Use a separate root for each environment. A saved Terraform plan belongs to its configuration, inputs, state/backend and target account. Initializing another backend does not turn a development plan into a production plan. Match Terraform/provider versions between planning and applying, preserve the dependency lock file, and restrict access to the plan: it can contain plaintext secrets even when terminal output hides them. Automation workflow, saved backend configuration.

The following GitHub Actions workflow operates only on infra/production, on a protected main push. Before enabling it, commit that root's .terraform.lock.hcl, production.tfvars, and non-secret production.s3.tfbackend. Configure repository variables AWS_REGION, TF_VERSION, TF_PLAN_ROLE_ARN and TF_APPLY_ROLE_ARN. The roles target the same intended production account and backend; the plan role needs refresh/read and state-lock access, while the apply role needs the reviewed mutation permissions.

Configure the GitHub production environment with required reviewers, prevented self-review where appropriate, and deployment restrictions for main; declaring environment: production alone does not create approval protection. production-plan is a separate environment for planning. Restrict each AWS OIDC trust policy to the actual repository/environment subject and aud=sts.amazonaws.com. Repository subject formats can include immutable owner/repository IDs; verify the repository's format rather than copying an old name-only policy. GitHub OIDC with AWS, environment protection.

name: Reviewed production infrastructure
on:
  push:
    branches: [main]
permissions:
  contents: read
concurrency:
  group: terraform-production
  cancel-in-progress: false
env:
  TF_IN_AUTOMATION: 'true'
  AWS_REGION: ${{ vars.AWS_REGION }}
jobs:
  plan:
    runs-on: ubuntu-24.04
    environment: production-plan
    permissions: {contents: read, id-token: write}
    defaults:
      run: {working-directory: infra/production, shell: bash}
    steps:
      - uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09
        with: {persist-credentials: false}
      - uses: hashicorp/setup-terraform@b9cd54a3c349d3f38e8881555d616ced269862dd
        with: {terraform_version: '${{ vars.TF_VERSION }}', terraform_wrapper: false}
      - uses: aws-actions/configure-aws-credentials@61815dcd50bd041e203e49132bacad1fd04d2708
        with:
          role-to-assume: ${{ vars.TF_PLAN_ROLE_ARN }}
          aws-region: ${{ env.AWS_REGION }}
      - run: |
          terraform fmt -check -recursive
          terraform init -input=false -lockfile=readonly -backend-config=production.s3.tfbackend
          terraform validate
          terraform plan -input=false -lock-timeout=5m -var-file=production.tfvars -out=tfplan
          terraform show -no-color tfplan > plan.txt
      - uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02
        with:
          name: production-plan-${{ github.run_id }}-${{ github.run_attempt }}
          path: |
            infra/production/tfplan
            infra/production/plan.txt
          if-no-files-found: error
          retention-days: 1
  apply:
    needs: plan
    runs-on: ubuntu-24.04
    environment: production
    permissions: {contents: read, id-token: write}
    defaults:
      run: {working-directory: infra/production, shell: bash}
    steps:
      - uses: actions/checkout@fbc6f3992d24b796d5a048ff273f7fcc4a7b6c09
        with: {persist-credentials: false}
      - uses: hashicorp/setup-terraform@b9cd54a3c349d3f38e8881555d616ced269862dd
        with: {terraform_version: '${{ vars.TF_VERSION }}', terraform_wrapper: false}
      - uses: aws-actions/configure-aws-credentials@61815dcd50bd041e203e49132bacad1fd04d2708
        with:
          role-to-assume: ${{ vars.TF_APPLY_ROLE_ARN }}
          aws-region: ${{ env.AWS_REGION }}
      - uses: actions/download-artifact@634f93cb2916e3fdff6788551b99b062d0335ce0
        with:
          name: production-plan-${{ github.run_id }}-${{ github.run_attempt }}
          path: infra/production
      - run: |
          terraform init -input=false -lockfile=readonly -backend-config=production.s3.tfbackend
          terraform apply -input=false -lock-timeout=5m tfplan

Reviewers inspect plan.txt and the exact commit/run before approving. For a retry, choose Re-run all jobs so planning produces a fresh artifact for the new run attempt, then review and approve it again. Re-running only the failed apply job increments github.run_attempt without re-running its successful plan job, so this attempt-specific artifact name will not exist. After a partial apply, inspect what changed before starting that full retry. Run attempts, failed-job reruns. Applying a saved plan does not prompt again; the environment gate is the approval. Plain interactive terraform apply, without a plan file or auto-approval, normally generates a plan and asks for confirmation—it is not an inherently unreviewed command. Apply behavior.

The workflow's concurrency group prevents overlapping runs in this repository; backend locking also coordinates other cooperating clients. Neither prevents console changes, cross-repository writers or all stale observations. If state changes or the approval becomes stale, create and review a new plan. Keep artifact access restricted and do not publish raw plan JSON or state to PR comments. Add actual application smoke/integration tests after apply and an explicit recovery procedure before production adoption; this workflow does not claim those tests or rollback are already implemented. Validate untrusted pull requests separately without AWS deployment credentials. Update pinned action commits and tool versions through review. GitHub concurrency, artifact handling.

8. Diagnose deployment failures as state transitions

After a failure, first determine what actually changed. Inspect CloudFormation events, Terraform's state and plan, or the Automation execution's steps. Preserve error evidence without leaking secrets. A retry must account for partially created resources and uncertain API outcomes. Force-unlocking state, removing resource addresses or deleting a stack is not a generic repair procedure.

For a concrete rehearsal, change the application image, force its readiness check to fail, and explain which instances remain serving, whether the database was touched, and how recovery is verified. Then propose a database replacement and identify what the snapshot policies preserve and what they do not. The goal is a deployment whose boundaries and failure paths a teammate can understand before it runs.