EC2 at Scale

EC2 at Scale: Managing Large Fleets Effectively

A fleet can have ten running instances and still fail to serve customers: some may be starting, registered to the wrong target group, or unable to reach the database. Scaling reliably means connecting capacity, readiness, placement, and application health rather than counting virtual machines alone.

This lesson develops Auto Scaling Groups, launch templates, purchase mixes, placement, monitoring, lifecycle hooks, and warm pools. It assumes the earlier compute, IAM, and networking lessons. AWS details were checked 15 September 2026. Commands are configuration examples for a disposable lab with existing prerequisites; they are not a complete production deployment or a versioned exam syllabus.

1. Know what an Auto Scaling Group maintains

An Auto Scaling Group (ASG) attempts to maintain its desired capacity, replaces instances it considers unhealthy, and can change capacity using scaling policies. The launch configuration, capacity availability, permissions, and application bootstrap must all work for this to succeed. A configured desired value is not a promise that the same amount of healthy capacity is immediately available. ASG concepts

With instance weighting or another desired-capacity unit, capacity is not necessarily a count of instances. Replacements, rebalancing, and other supported operations can also temporarily create more instances than a simple reading of the maximum suggests. Track the actual fleet and cost, not only its configured bounds. Instance weights

For a multi-AZ web application, use suitable subnets in distinct AZs, a load balancer, and enough capacity to survive the chosen failure scenario. Two instances or two subnet IDs alone do not establish that design. A pair of subnets might occupy the same AZ, and losing one of two correctly distributed instances can still leave insufficient throughput.

Four timing controls with different jobs

ELB health checks can run during the ASG grace period. The period does not switch off the load balancer's checks or force it to send traffic to an unhealthy target. An instance that stops running can still be replaced during grace. The console creation default is 300 seconds; CLI/SDK creation defaults to zero. Choose the value from application startup behavior. Health-check grace

Default instance warmup is not configured automatically for every group. Set it explicitly when appropriate; target-tracking/step-policy fallback behavior can otherwise use default cooldown. Warmup starts after InService, and is not proof that the application has passed a transaction test. Default warmup

Without launch hook: Pending → InService
With launch hook: Pending → Pending:Wait → Pending:Proceed → InService

After InService: grace period and metric warmup serve their separate purposes.

Health checks and metric collection are not one serial “wait, then check” step.

2. Use launch templates as controlled configuration

A launch template contains versioned EC2 launch parameters: image, instance type, instance profile, security groups, storage, user data, and other supported options. Mixed-type ASGs use overrides or attribute-based selection; a single InstanceType field is not a list of alternatives. ASG launch templates

For repeatable releases, test and pin a numeric version such as 3. $Latest means the newest version when a launch occurs. $Default resolves to the configured default, which an administrator can change. Neither alias is immutable. Updating the ASG's template reference does not retrofit already running instances: use a planned instance refresh or another controlled replacement process. Template versioning, instance refresh

In a shell, protect literal dollar-prefixed template versions with single quotes:

aws autoscaling update-auto-scaling-group --region us-east-1 \
  --auto-scaling-group-name web-app-asg \
  --launch-template 'LaunchTemplateName=web-app-template,Version=$Default'

Double quotes allow shell expansion of $Default or $Latest, potentially sending an empty or unintended value. Numeric versions avoid that ambiguity and make release review easier. Pinning the version still requires control over referenced artifacts, such as the AMI and bootstrap downloads.

Example: create the web fleet's template

Prepare a lab-specific web-launch.json, replacing the example AMI and group IDs. The AMI must be compatible with m6i.large, exist in us-east-1, and already contain the configured web application. The existing instance profile must contain its intended role; a role name by itself is not an instance profile.

{
  "ImageId": "ami-0123456789abcdef0",
  "InstanceType": "m6i.large",
  "SecurityGroupIds": ["sg-0123456789abcdef0"],
  "IamInstanceProfile": {"Name": "WebAppInstanceProfile"},
  "Monitoring": {"Enabled": true},
  "MetadataOptions": {"HttpEndpoint": "enabled", "HttpTokens": "required"},
  "TagSpecifications": [{
    "ResourceType": "instance",
    "Tags": [{"Key": "Name", "Value": "WebApp-Lab"}]
  }]
}

aws ec2 create-launch-template --region us-east-1 \
  --launch-template-name web-app-template \
  --version-description 'Tested lab application image' \
  --launch-template-data file://web-launch.json

This assumes a new template name, so the created version is 1. Enable permissions for the relevant EC2/ASG operations and passing the profile's role. Configure the application SG to accept the intended load-balancer traffic, plus necessary egress; the command does not create these rules. Detailed EC2 monitoring can add charges. CreateLaunchTemplate

If you add user data, provide correctly base64-encoded launch-template data and check what it decodes to. Bootstrapping also needs access to its package/configuration sources and permission to retrieve them. A script that prints “installing” does not actually install software. Avoid changing package versions implicitly on every launch when release consistency matters.

3. Configure target tracking with the EC2 Auto Scaling API

For this example, $LAB_SUBNET_A and $LAB_SUBNET_B are existing private subnets in different AZs of the same VPC. $LAB_TARGET_GROUP_ARN identifies an existing, compatible instance target group with an ALB listener and health check for the application. The launch template above uses that VPC's security group.

aws autoscaling create-auto-scaling-group --region us-east-1 \
  --auto-scaling-group-name web-app-asg \
  --launch-template 'LaunchTemplateName=web-app-template,Version=1' \
  --min-size 2 --max-size 10 --desired-capacity 4 \
  --health-check-type ELB --health-check-grace-period 300 \
  --default-instance-warmup 180 \
  --vpc-zone-identifier "$LAB_SUBNET_A,$LAB_SUBNET_B" \
  --target-group-arns "$LAB_TARGET_GROUP_ARN"

aws autoscaling put-scaling-policy --region us-east-1 \
  --auto-scaling-group-name web-app-asg \
  --policy-name cpu-target-tracking --policy-type TargetTrackingScaling \
  --target-tracking-configuration '{
    "PredefinedMetricSpecification": {
      "PredefinedMetricType": "ASGAverageCPUUtilization"
    },
    "TargetValue": 70.0
  }'

The chosen 300-second grace and 180-second warmup are independent example values requiring measurement. EC2 health checks remain relevant when ELB checks are enabled. Confirm the targets actually become healthy and can serve a representative request. The policy seeks a CPU target around 70%; it cannot pin instantaneous CPU to exactly 70% or guarantee sufficient capacity during every burst. ASG health checks, target tracking

API distinction: ScaleInCooldown and ScaleOutCooldown belong to Application Auto Scaling's target-tracking configuration, used for services such as ECS. They are not valid members of EC2 Auto Scaling's TargetTrackingConfiguration. A per-policy EC2 EstimatedInstanceWarmup is a top-level PutScalingPolicy parameter (--estimated-instance-warmup in the CLI), not a member of the target-tracking JSON. The group-level default often keeps multiple policies consistent. EC2 PutScalingPolicy

Choose a metric that changes meaningfully with capacity. CPU can work for CPU-bound requests; it may stay low while a database bottleneck makes response times unacceptable. ALBRequestCountPerTarget represents requests per target over a one-minute interval and needs a resource label identifying both load balancer and target group. A target of 1,000 is not 1,000 requests per second. Predefined metrics

Target tracking manages its own alarms. Step scaling provides explicit adjustments for alarm ranges. Simple scaling uses a single adjustment and cooldown behavior. Scheduled scaling handles known timing, while predictive scaling forecasts recurring load. Evaluate forecasts before relying on them, and retain a response to unexpected demand. Scaling methods

4. Calculate a mixed fleet in the correct units

An ASG can combine On-Demand and Spot instances. RI/Savings Plan discounts can apply to qualifying On-Demand usage; these are not separate instance categories that make the fleet immune to failure. A billing commitment and a guarantee of spare capacity are different things. Mixed ASGs

The following MixedInstancesPolicy object uses compatible x86 types and no weights, so each instance counts as one unit. It assumes an existing, tested batch-processor-template version 1 whose AMI and other settings suit all three types. Save it as batch-mix.json:

{
  "LaunchTemplate": {
    "LaunchTemplateSpecification": {
      "LaunchTemplateName": "batch-processor-template",
      "Version": "1"
    },
    "Overrides": [
      {"InstanceType": "c6i.large"},
      {"InstanceType": "c6a.large"},
      {"InstanceType": "c5.large"}
    ]
  },
  "InstancesDistribution": {
    "OnDemandAllocationStrategy": "prioritized",
    "OnDemandBaseCapacity": 2,
    "OnDemandPercentageAboveBaseCapacity": 20,
    "SpotAllocationStrategy": "price-capacity-optimized"
  }
}

aws autoscaling create-auto-scaling-group --region us-east-1 \
  --auto-scaling-group-name batch-processor-asg \
  --min-size 5 --max-size 50 --desired-capacity 10 \
  --vpc-zone-identifier "$LAB_SUBNET_A,$LAB_SUBNET_B" \
  --mixed-instances-policy file://batch-mix.json

At desired count ten, allocate two to the On-Demand base. Of the remaining eight, 20% is 1.6, rounded up toward On-Demand to two. The target mix is therefore four On-Demand and six Spot, not eight Spot. Actual fulfillment depends on available capacity. This fixed mixture does not automatically replace an unavailable Spot allocation with On-Demand. Distribution and rounding

capacity-optimized selects pools using available capacity, rather than merely the lowest price. price-capacity-optimized, currently recommended by AWS, considers both capacity and price. Neither predicts away every interruption. SpotInstancePools applies to the lowest-price strategy; do not add it to a capacity-optimized example and claim it controls the pool count. Allocation strategies, distribution API

Worked cost and weight examples

For an arithmetic exercise, assume every On-Demand instance costs $0.085/hour and every Spot instance costs $0.0255/hour. These uniform rates are invented inputs for the mix calculation, not live quotes for the three listed types:

from decimal import Decimal as D

mixed = 4 * D('0.085') + 6 * D('0.0255')
all_on_demand = 10 * D('0.085')
saving = (1 - mixed / all_on_demand) * 100
print(mixed, all_on_demand, saving)  # 0.4930 0.850 42.00

The fleet saves 42%, even though each illustrative Spot instance is 70% cheaper. This accounts only for the modeled instance-hour charges; storage, traffic, monitoring, and other resources remain separate.

If you instead assign WeightedCapacity: "2" to a two-unit instance and "4" to a four-unit instance, a desired capacity of ten means ten units. Three instances with weights 4, 4, and 2 can supply them. The base capacity uses the same units, and packing/rounding can overshoot the requested capacity. WeightedCapacity is a string in the API. Choose weights from meaningful workload capacity, not merely because one size name looks twice as large. Weighting rules

5. Select placement for the failure and network model

Placement groups influence EC2 placement. Three commonly used strategies are cluster, spread, and partition; consult the current catalog for additional specialized options.

Cluster placement suits tightly coupled HPC when measured network performance matters. Partition placement supports topology-aware distributed systems such as Cassandra; the application must place replicas across failure domains appropriately. Spread suits small sets of critical instances. A conventional web tier can use multiple AZs without a specialized placement group. Placement strategies

Cluster:    communicating instances within one AZ
Spread:     instance A on rack A; instance B on rack B
Partition:  partition 1's rack set does not overlap partition 2's rack set
            multiple instances can occupy each partition

Supported stopped instances can be added to, removed from, or moved between placement groups. Groups are not merged into a new layout. For an ASG-managed instance, coordinate such maintenance with the group's replacement behavior; stopping a member casually can trigger replacement. Change placement

Example: an EFA-enabled HPC cluster

An Elastic Fabric Adapter (EFA) can benefit supported HPC/ML software, but a launch parameter alone does not install the required drivers and communication libraries. Use a compatible instance type, image, software stack, and security-group rules. Actual throughput depends on the instance, message pattern, and software; a generic “20× bandwidth” table is not a benchmark. EFA setup

This fragment assumes $LAB_HPC_AMI, $LAB_HPC_SUBNET, and $LAB_EFA_SG identify compatible existing resources in the same Region, with the subnet in the chosen AZ. The group name must be unused:

aws ec2 create-placement-group --region us-east-1 \
  --group-name hpc-cluster --strategy cluster

aws ec2 run-instances --region us-east-1 \
  --image-id "$LAB_HPC_AMI" --instance-type c5n.18xlarge --count 16 \
  --placement 'GroupName=hpc-cluster' \
  --network-interfaces "DeviceIndex=0,NetworkCardIndex=0,InterfaceType=efa,SubnetId=$LAB_HPC_SUBNET,Groups=$LAB_EFA_SG"

This launches substantial billable capacity if executed successfully. Confirm quotas, capacity, image access, and application configuration first. Launching the intended cluster together can improve the chance of compatible placement; it is not an absolute rule that later additions are forbidden. Validate the actual MPI/AllReduce workload after launch and clean up disposable instances when finished.

6. Monitor capacity and health as separate signals

ASG group metrics must be enabled. They are available at one-minute granularity, at no additional charge for those group metrics. That is distinct from EC2 detailed monitoring, which changes the granularity of many instance metrics from five minutes to one minute. Memory and filesystem metrics generally require an agent/custom collection. ASG metrics, EC2 monitoring

aws autoscaling enable-metrics-collection --region us-east-1 \
  --auto-scaling-group-name web-app-asg --granularity 1Minute \
  --metrics GroupDesiredCapacity GroupInServiceInstances GroupPendingInstances

Compare desired capacity with in-service and pending counts. An in-service count is a lifecycle count, not a full application-health test. With weights, use the corresponding capacity-unit metrics when making capacity comparisons. A desired-minus-in-service gap can mean launching or other transitions; it is not a direct count of unhealthy application instances.

An ALB's UnHealthyHostCount—including its capital H in Healthy—counts unhealthy targets for the selected dimensions. It is not an ASG metric named UnhealthyHostCount. This PutMetricAlarm request monitors the maximum reported value for one target group/load balancer, exceeding three in two of two evaluated one-minute datapoints. With complete reporting, these represent consecutive minutes:

{
  "AlarmName": "WebTargetGroup-UnhealthyTargets",
  "Namespace": "AWS/ApplicationELB",
  "MetricName": "UnHealthyHostCount",
  "Dimensions": [
    {"Name": "LoadBalancer", "Value": "app/lab-alb/0123456789abcdef"},
    {"Name": "TargetGroup", "Value": "targetgroup/lab-web/0123456789abcdef"}
  ],
  "Statistic": "Maximum",
  "Period": 60,
  "EvaluationPeriods": 2,
  "DatapointsToAlarm": 2,
  "Threshold": 3,
  "ComparisonOperator": "GreaterThanThreshold",
  "TreatMissingData": "missing"
}

Replace the dimension suffixes with the actual ALB and target-group values, not their full ARNs. Add a valid notification action if operators should be notified. Missing data is not proof that all targets are healthy. In the default sliding evaluation window, CloudWatch can use older real samples when recent samples are missing; two breaching evaluated datapoints need not occupy adjacent clock minutes. Missing-data evaluation The metric does not deduplicate instances across different target groups; interpret its scope precisely. ALB metrics, PutMetricAlarm

7. Complete lifecycle work honestly

Lifecycle hooks give automation time to prepare a launching instance or perform termination-related work. Ordinary launch paths include Pending:Wait; termination paths include Terminating:Wait. For a standard ASG/ELB integration, load-balancer deregistration and draining are already part of scale-in. A termination hook is useful for additional cleanup, such as stopping external queue consumption and persisting application state; it should not pretend to create all ALB draining behavior from scratch. Lifecycle order

This hook uses the default EventBridge event path, rather than an incomplete SNS notification setup:

aws autoscaling put-lifecycle-hook --region us-east-1 \
  --lifecycle-hook-name persist-work-hook \
  --auto-scaling-group-name web-app-asg \
  --lifecycle-transition autoscaling:EC2_INSTANCE_TERMINATING \
  --default-result CONTINUE --heartbeat-timeout 300

An EventBridge rule and target, target invocation permission, and the worker's scoped IAM permissions must also be configured. A direct SNS/SQS notification design instead needs its delivery-role configuration. Creating a hook does not create a working cleanup handler. Notification targets

The following reusable helper expects a validated, current lifecycle-event detail object and an application-specific cleanup callable. This is the completion boundary, not a complete Lambda handler. The cleanup function must perform and confirm the work, and call the supplied heartbeat periodically during lengthy processing:

def finish_termination(detail, asg, cleanup):
    if detail['LifecycleTransition'] != 'autoscaling:EC2_INSTANCE_TERMINATING':
        raise ValueError('Expected a termination lifecycle action')
    action = {
        'LifecycleHookName': detail['LifecycleHookName'],
        'AutoScalingGroupName': detail['AutoScalingGroupName'],
        'LifecycleActionToken': detail['LifecycleActionToken'],
        'InstanceId': detail['EC2InstanceId']
    }

    def heartbeat():
        asg.record_lifecycle_action_heartbeat(**action)

    heartbeat()
    if cleanup(detail['EC2InstanceId'], heartbeat) is not True:
        raise RuntimeError('Cleanup not confirmed; lifecycle action not completed')
    asg.complete_lifecycle_action(**action, LifecycleActionResult='CONTINUE')

A cleanup timeout or error must not be reported as successful draining. This helper leaves completion unsent on failure; the surrounding worker must record/alert on the error and handle retry/stale-token behavior. It does not guarantee the instance stays alive: the hook timeout and configured default still apply. With the example's CONTINUE default, expiration allows termination even if cleanup never succeeded. Keep durable work recoverable elsewhere.

By default, a heartbeat timeout is one hour. Heartbeats can extend waiting only within the global ceiling of 48 hours or 100 times the heartbeat timeout, whichever is smaller. With a 300-second heartbeat timeout that ceiling is 30,000 seconds, or 8 hours 20 minutes; a Lambda invocation still has its own execution timeout. Use durable orchestration for longer coordination. Lifecycle limits

For ordinary lifecycle behavior without a separate instance-retention policy, launch ABANDON leads to termination/replacement; it does not return the instance to an earlier healthy state. On termination, both CONTINUE and ABANDON permit termination, with different handling of remaining lifecycle actions. A hook also cannot postpone an EC2 Spot reclamation deadline. Completing actions

8. Warm pools and interruption handling

Warm pools hold prepared instances alongside an ASG. Supported states are Stopped, Running, and Hibernated, with different cost and startup behavior. Stopped/hibernated instances retain storage costs; running ones incur running-instance charges. Scale-in terminates instances by default; returning them to the pool requires an instance reuse policy. Hibernation preserves RAM to disk, while stopping does not. Warm-pool concepts

Use a launch hook to ensure initialization completes before an instance is stopped/hibernated into the pool. A warm pool does not inherently wait for a long user-data script. Measure the resulting readiness time rather than assuming a universal 30–60-second startup.

Warm pools cannot be added to weighted mixed groups, and mixed groups using them must be On-Demand-only. Therefore, do not bolt one onto the preceding Spot batch group. A depleted pool or unavailable capacity can still require a cold launch. Updating a template alone does not update existing warm instances; include them in the replacement plan.

For Spot workers, checkpoint continuously and make jobs restartable. Use best-effort interruption notices and capacity-rebalance signals as additional opportunities to stop taking work, not your only recovery mechanism. A termination/stop warning normally gives two minutes; hibernation starts immediately. Spot interruption handling

This single-check example runs on the EC2 instance with IMDS enabled, using an IMDSv2 token. It disables proxy use for the link-local metadata request. It is a building block for a monitored loop, not the worker's shutdown implementation:

import json
from urllib.request import Request, build_opener, ProxyHandler
from urllib.error import HTTPError

def spot_action():
    base = 'http://169.254.169.254/latest/'
    opener = build_opener(ProxyHandler({}))
    token_request = Request(
        base + 'api/token', method='PUT',
        headers={'X-aws-ec2-metadata-token-ttl-seconds': '60'}
    )
    with opener.open(token_request, timeout=2) as response:
        token = response.read().decode()
    request = Request(
        base + 'meta-data/spot/instance-action',
        headers={'X-aws-ec2-metadata-token': token}
    )
    try:
        with opener.open(request, timeout=2) as response:
            return json.load(response)
    except HTTPError as error:
        if error.code == 404:
            return None
        raise

A 404 means the notice item is absent at that check. Authentication failures or network errors are not silently converted into “no interruption.” A polling supervisor can call this roughly every five seconds, handle errors, and trigger bounded cleanup when a notice appears. Regular checkpoints remain necessary even if notices are missed. IMDSv2

9. Check the capacity estimate against actual constraints

For a measured capacity of 500 requests/second per instance, a peak of 10,000 requests/second, and an assumed 1.3 multiplier, the planning count is:

import math

instances = math.ceil((10000 / 500) * 1.3)
print(instances)  # 26

Twenty-six is correct for that arithmetic. It is not inherently “optimal,” and 1.3 is not a universal production safety factor. If those instances are evenly split across two AZs, losing thirteen leaves only 6,500 requests/second at the same measured per-instance rate. A 30% margin over normal demand has not provided full capacity after that AZ failure.

Test steady load, sudden bursts, failed launches, an unhealthy application, and the chosen failure scenario. Observe requested versus ready capacity, scaling lag, latency, queue age, and the dependencies that can bottleneck the fleet. Match launch versions, timers, purchase mix, and recovery behavior to that evidence, and remove disposable lab resources after verification.