Compute & Runtime Architecture

AWS Compute & Runtime Architecture

An online store has several kinds of work: serving an API, processing orders, resizing images, and transcoding videos. Choosing one compute service for everything can create unnecessary cost or a runtime limit that the application cannot satisfy. Start with what each workload needs, then choose how it will run and recover.

This lesson covers EC2 families and purchasing options, Lambda execution, ECS/EKS orchestration, Fargate capacity, and four architecture examples. It assumes the earlier IAM and VPC lessons. Service details were checked 15 September 2026; the examples teach architecture, rather than a particular certification exam version.

1. Separate the workload, orchestrator, and compute capacity

EC2 provides virtual servers, with bare-metal options for supported instance types. You configure the guest operating system and application, subject to the selected service's capabilities. AWS operates the underlying infrastructure. Containers package applications and dependencies; an orchestrator schedules and maintains those containers. ECS is AWS-native container orchestration, while EKS provides managed Kubernetes. Fargate supplies compute for supported ECS tasks and EKS pods. It is not a third competing orchestrator. EC2 concepts, ECS overview, EKS overview

These are common choices, not a complete list of every managed capacity option. EKS Auto Mode and ECS Managed Instances provide additional ways to reduce host operations. Choosing Kubernetes gives you Kubernetes APIs and ecosystem compatibility; it does not automatically make an application portable when its storage, identity, and integrations depend on AWS. EKS compute options, ECS Managed Instances

Keep durable application state outside replaceable processes. An EC2 application can use persistent EBS volumes, but instance-store data has a different lifetime, and a volume's deletion settings matter. Containers and Lambda can also access persistent services. The useful question is where state survives replacement, rather than whether a service is simply “stateful” or “stateless.” EC2 storage

2. Match EC2 resources and purchasing terms to the work

An instance type includes a family, generation, possible capability suffixes, and size. For example, c6g.large is a compute-optimized, Graviton-based type. Image architecture and native dependencies must match the processor; an x86 image cannot simply be moved unchanged to an Arm instance.

These names are examples, not a latest-generation shopping list. Check the actual type's CPU, memory, network, storage, Region availability, and software support. A T instance is not a permanently discounted substitute for sustained CPU capacity: Standard and Unlimited credit modes behave differently, and surplus-credit charges can matter. Instance naming, burstable performance

Use the measured bottleneck to shortlist families. A cache that runs out of memory may benefit from R-family capacity, but the family name alone does not prove a better total cost. Benchmark representative data, request concurrency, network traffic, and recovery behavior. A larger memory allocation that reduces expensive disk access can be valuable even when its hourly price is higher.

Separate a billing discount from a capacity reservation. On-Demand avoids a long-term usage commitment. Reserved Instances and Savings Plans discount qualifying usage under their terms; Compute Savings Plans and EC2 Instance Savings Plans have different flexibility. Savings Plans do not reserve EC2 capacity. Regional and zonal Reserved Instances also differ: qualifying zonal reservations include an AZ capacity benefit. Ordinary On-Demand launch requests can encounter insufficient capacity. Purchasing options, Savings Plan types, RI scope

Spot uses spare EC2 capacity and advertises savings up to 90% against On-Demand. That is a maximum, not an expected discount for every pool. Make work retryable, checkpoint progress, and diversify compatible instance types and AZs. Interruption notices for stopping/termination normally provide two minutes but are best effort; hibernation begins immediately. An On-Demand baseline does not guarantee that the remaining capacity will survive a failure. Spot guidance, interruption notices

3. Understand the Lambda execution boundary

For the standard function invocations discussed here, Lambda manages execution environments and invokes your handler. A cold start includes initialization work; subsequent invocations may reuse an environment, but reuse is not guaranteed. Latency depends on the runtime, package, initialization, and dependencies. There is no universal “1–5 seconds” cold-start range or a guarantee that a warm request meets a particular end-to-end latency target. Execution environment

Event or request
  → Lambda invocation, possibly preceded by initialization
  → handler reads/writes durable services as needed
  → response or completion according to the invocation model

Logs are emitted during execution; they are not a mandatory final workflow step.

Synchronous callers receive the invocation result. Asynchronous delivery and polled sources such as SQS have different retry and batching behavior. Design handlers so redelivery does not duplicate a charge, order, or other side effect. One message need not mean one invocation: an SQS event-source mapping can deliver a batch. Invocation models, SQS integration

New accounts can have reduced quotas. Lambda's documented MB convention uses binary units. A durable workflow's elapsed lifetime is a different concept from a single invocation's execution timeout; breaking work into resumable steps does not give one uninterrupted CPU-bound invocation more than 900 seconds. Lambda quotas

Lambda request and duration charges are separate. Duration is rounded up to the nearest 1 millisecond, not 100 milliseconds, and depends on configured memory. Initialization and optional features can also be billed. Provisioned concurrency incurs charges while capacity is configured, including when idle. Lambda pricing

Reserved concurrency sets aside concurrency for a function and caps its concurrent executions; it does not pre-initialize environments. Provisioned concurrency prepares environments for a published version or alias. This example assumes the live alias already points to a published version and the caller has permission:

import boto3

lambda_client = boto3.client('lambda', region_name='us-east-1')
response = lambda_client.put_provisioned_concurrency_config(
    FunctionName='latency-sensitive-api',
    Qualifier='live',
    ProvisionedConcurrentExecutions=5
)

Wait until allocation is ready, and direct invocations to that version/alias. $LATEST is unsupported. Five provisioned environments do not cap the function at five executions: excess traffic can use on-demand environments when concurrency limits permit, with possible cold starts. Provisioned concurrency

SnapStart restores initialized snapshots for supported Java 11+, Python 3.12+, and .NET 8+ managed runtimes. It is not Java-only, instantaneous startup, or a feature to combine with provisioned concurrency on the same version. Check supported configurations and Regions, and handle snapshot-sensitive state such as uniqueness and connections. SnapStart

4. Configure containers and their scaling separately

For ECS, a task definition describes containers and resources. A task is a running instantiation; a service maintains a desired task count. Fargate is an ECS launch type/capacity option. EKS uses Fargate profiles and pod scheduling terminology, rather than an ECS launchType field. ECS task definitions, EKS Fargate

The following Linux task definition selects Fargate compatibility and uses the task's own network interface through awsvpc:

{
  "family": "web-app",
  "requiresCompatibilities": ["FARGATE"],
  "networkMode": "awsvpc",
  "cpu": "256",
  "memory": "512",
  "runtimePlatform": {
    "operatingSystemFamily": "LINUX",
    "cpuArchitecture": "X86_64"
  },
  "containerDefinitions": [{
    "name": "web",
    "image": "public.ecr.aws/docker/library/nginx:stable",
    "essential": true,
    "portMappings": [{"containerPort": 80, "protocol": "tcp"}]
  }]
}

This is a definition, not a running deployment. requiresCompatibilities requests compatibility validation; it neither launches a task nor converts an existing service. Running it also needs an ECS cluster, supported subnet/security-group choices, and a path to download its image. Add an execution role for features that require it, such as private ECR pulls or CloudWatch Logs, and a separate task role for application AWS permissions. Pin a tested image digest for reproducible releases; the illustrative stable tag can change. Fargate parameters, execution role

Task CPU uses units: 1,024 units = 1 vCPU. Numeric task memory strings use MiB. Valid Linux examples include 256 CPU/512 MiB and 512 CPU/1,024 MiB. A 1-vCPU Fargate task requires at least 2,048 MiB; 1,024 CPU/1,024 MiB is invalid. Container limits cannot make an invalid task-level combination valid. Check peaks and sidecars before reducing memory.

Fargate supplies host capacity for requested tasks. It does not, by itself, decide how many API replicas or queue workers your application needs. For an existing ECS service, register a scalable target and then attach an Application Auto Scaling policy:

import boto3

scaling = boto3.client('application-autoscaling', region_name='us-east-1')
scaling.register_scalable_target(
    ServiceNamespace='ecs',
    ResourceId='service/store-cluster/api-service',
    ScalableDimension='ecs:service:DesiredCount',
    MinCapacity=2,
    MaxCapacity=10
)
scaling.put_scaling_policy(
    PolicyName='api-cpu-target',
    ServiceNamespace='ecs',
    ResourceId='service/store-cluster/api-service',
    ScalableDimension='ecs:service:DesiredCount',
    PolicyType='TargetTrackingScaling',
    TargetTrackingScalingPolicyConfiguration={
        'TargetValue': 70.0,
        'PredefinedMetricSpecification': {
            'PredefinedMetricType': 'ECSServiceAverageCPUUtilization'
        },
        'ScaleInCooldown': 300,
        'ScaleOutCooldown': 60
    }
)

These cooldown fields belong to Application Auto Scaling. Do not copy them into the separate EC2 Auto Scaling target-tracking API. CPU target tracking is useful when CPU reflects per-task load; a queue workload may need a backlog-per-worker metric instead. ECS service auto scaling, Application Auto Scaling API

5. Example: an online store and its compute bill

One plausible design uses S3/CloudFront for static assets, a Fargate service for the API, and short Lambda invocations for order events and images. Static hosting still uses AWS infrastructure; the team simply does not provision an application server for those files.

Browser → CloudFront → S3 static content
Browser → API Gateway HTTP API → VPC link → ALB listener → ECS API tasks
                                                            ├→ database
                                                            └→ SQS orders
                                                               → Lambda poller/handler
                                                               → notification service

Separate image path: S3 incoming/ → Lambda validation/resizing → S3 processed/

The private API integration needs the appropriate VPC link and load-balancer listener; API Gateway does not directly route to an “ECS cluster” as a generic endpoint. Alternatives can expose the ALB through a different front-door design. Keep the diagram and actual integration consistent. HTTP API private integrations

For images, filter notifications to the incoming prefix or use separate output storage so resizing does not trigger itself forever. Validate content and resource usage, and make retries idempotent. For orders, configure batch/error handling, visibility timeout, retry limits, and dead-letter processing as appropriate. Returning successfully before a required side effect is durable can lose work. S3 Lambda notifications

Cost exercise: use illustrative Linux/x86 Fargate rates of $0.04048 per vCPU-hour and $0.004445 per GB-hour, with 720 billable hours per task in a 30-day month. These are calculation inputs, not a current quote for every Region or OS. Fargate bills requested task resources from the start of container-image download until task termination, rounded up to the nearest second. Charges are not reduced in proportion to observed CPU utilization. Linux has a one-minute billing minimum; Windows differs. Fargate pricing

from decimal import Decimal as D

def fargate_cost(tasks, vcpu, memory_gb, hours):
    return D(tasks) * (D(vcpu) * D('0.04048') +
                       D(memory_gb) * D('0.004445')) * D(hours)

print(fargate_cost('1', '0.25', '0.5', '720'))  # 8.88660000
print(fargate_cost('2', '0.5', '1', '720'))     # 35.5464000
print(fargate_cost('1', '1', '4', '24'))        # 1.398240
print(fargate_cost('1', '1', '2', '24'))        # 1.184880

Thus two 0.5-vCPU/1-GB API tasks, each billed for 720 hours, cost about $35.55/month for these CPU/memory line items. Additional tasks during scaling add cost. Reducing one 1-vCPU task from 4 GB to the valid 2-GB minimum saves about 15.26% of those modeled daily charges; reducing it to 1 GB is not a valid alternative.

For Lambda, 100 orders/day for 30 days gives 3,000 invocations. At an illustrative $0.20 per million requests, the request charge alone is $0.0006, before free allowances. Duration, configured memory, retries, and initialization must also be counted. Request counts alone cannot establish a full compute total. Include the ALB, API Gateway, network paths, logs, storage, and database separately before claiming a total application budget.

6. Example: a video job longer than one Lambda invocation

A 45-minute uninterrupted transcoding job exceeds the standard Lambda invocation timeout. An ECS Fargate task, EC2 worker, or suitable AWS Batch job can run that work; AWS Elemental MediaConvert is a purpose-built transcoding service to consider. Fargate does not impose Lambda's 15-minute invocation limit, but tasks can stop through failure, maintenance, or orchestration decisions. “No time limit” should not be read as guaranteed uninterrupted execution. Fargate task retirement, MediaConvert

A Step Functions Standard workflow can validate an input, run an ECS task with the .sync integration, and notify on completion. Express workflows do not support this Run-a-Job integration. The following state-machine fragment assumes existing resources and permissions; replace the illustrative identifiers. The transcoder task definition must contain a container named transcoder that consumes these environment variables:

{
  "StartAt": "Transcode",
  "States": {
    "Transcode": {
      "Type": "Task",
      "Resource": "arn:aws:states:::ecs:runTask.sync",
      "Parameters": {
        "Cluster": "video-processing-cluster",
        "TaskDefinition": "ffmpeg-transcoder:1",
        "LaunchType": "FARGATE",
        "NetworkConfiguration": {
          "AwsvpcConfiguration": {
            "Subnets": ["subnet-0123456789abcdef0"],
            "SecurityGroups": ["sg-0123456789abcdef0"],
            "AssignPublicIp": "DISABLED"
          }
        },
        "Overrides": {
          "ContainerOverrides": [{
            "Name": "transcoder",
            "Environment": [
              {"Name": "INPUT_BUCKET", "Value.$": "$.inputBucket"},
              {"Name": "INPUT_KEY", "Value.$": "$.inputKey"},
              {"Name": "OUTPUT_BUCKET", "Value.$": "$.outputBucket"}
            ]
          }]
        }
      },
      "TimeoutSeconds": 7200,
      "End": true
    }
  }
}

The state timeout is two hours, allowing more room than a one-hour timeout for a job that might take an hour plus startup. It is still a deadline. The private subnet needs access to the image registry, S3, and any configured logging or secret services. The workflow role needs the documented ECS/EventBridge permissions and permission to pass the task roles. ECS workflow integration, integration patterns

The container image must actually include compatible FFmpeg and application dependencies; installing boto3 does not install the AWS CLI. Download input to a unique temporary directory, preserve the video's intended aspect ratio, check transcoder exit status, and upload output under a stable job identifier. Size storage for input plus outputs and clean up temporary files on failure. A retry can repeat an upload, so design completion records and output keys accordingly.

Step Functions coordinates tasks; the shown state machine has no queue-depth scaling policy. Limit concurrent jobs deliberately and add error paths and retries that are safe for the operation. A successful RunTask submission also does not prove a completed transcode: standalone callers must inspect failures and subsequently observe task exit/completion.

7. Example: a news site's changing EC2 capacity

Suppose load tests establish 1,000 requests/second per instance at the required latency, and the capacity exercise adds 20% headroom. Round up, then compare with the ASG maximum:

import math

def capacity_plan(rps, per_instance=1000, maximum=50):
    required = max(2, math.ceil(rps / per_instance * 1.2))
    return {'required': required, 'within_maximum': required <= maximum}

for rps in (800, 5000, 45000, 8000, 1200, 50000):
    print(rps, capacity_plan(rps))

The required counts are 2, 6, 54, 10, 2, and 60. A maximum of 50 cannot satisfy the 45,000- or 50,000-request cases with that headroom. Increasing the maximum is only one step: quotas, available capacity, boot time, and downstream throughput must support the plan. This arithmetic is an estimate under stated assumptions, not the actual target-tracking algorithm.

For an unweighted mixed ASG, a base of two On-Demand instances plus 20% On-Demand above that base targets four On-Demand and six Spot instances at desired capacity ten, after rounding toward On-Demand. The overall fleet is therefore 60% Spot, not 80%. At desired capacity fifty the target is twelve On-Demand and thirty-eight Spot. A fixed purchase mix does not automatically switch unavailable Spot capacity to On-Demand. Mixed-group distribution

An ALBRequestCountPerTarget target uses average requests per target over one minute, not requests per second. It also needs the correct load-balancer/target-group resource label. The next lesson develops the actual EC2 scaling API, warmup, and fleet health. EC2 target tracking

8. Example: GPU inference and a compatible alternative

If the image classifier requires a GPU, use a compatible GPU-backed EC2/container deployment or SageMaker real-time endpoint configuration. Lambda may validate the request and call the endpoint, but that does not put a GPU inside Lambda. SageMaker Serverless Inference does not support GPUs; it cannot satisfy this requirement merely because traffic is sporadic.

A CPU-compatible model is a different case. Benchmarking might show that SageMaker Serverless Inference meets its latency and memory requirements. Its selectable memory values run from 1,024 through 6,144 MB in 1,024-MB steps, not just 2, 4, and 6 GB. Cold starts and concurrency limits remain relevant. Serverless Inference capabilities

This helper demonstrates a request to an existing endpoint whose model contract accepts application/x-image and returns a JSON list of objects containing numeric score fields:

import json
import boto3

runtime = boto3.client('sagemaker-runtime', region_name='us-east-1')

def classify_image(image_bytes, endpoint_name):
    response = runtime.invoke_endpoint(
        EndpointName=endpoint_name,
        ContentType='application/x-image',
        Body=image_bytes
    )
    stream = response['Body']
    try:
        predictions = json.loads(stream.read())
    finally:
        stream.close()
    return sorted(predictions, key=lambda item: item['score'], reverse=True)[:3]

invoke_endpoint is the SDK method; response shape and content type depend on the deployed model. The caller needs permission to invoke that endpoint and a suitable network path. This code neither deploys the model nor determines whether its hosting hardware is CPU or GPU. InvokeEndpoint

Compare compatible architectures using measured inference duration, throughput, cold-start tolerance, and provisioned/idle charges. The original kind of comparison—an unmeasured CPU serverless estimate against a continuously running GPU—cannot establish equivalent performance or a defensible savings percentage.

For each workload, record its duration, hardware requirements, durable state, scaling signal, recovery plan, and cost assumptions. Those facts let you explain a compute choice—and recognize when changed requirements call for a different one.