Container Orchestration

Container Orchestration on AWS

A container image packages an application. It does not decide where replicas run, replace failed workloads, expose a stable endpoint, or coordinate a deployment. An orchestrator supplies those control loops. Your application still needs suitable resource requests, health signals, dependency handling, and a deployment strategy that fits its state and traffic.

This lesson covers ECS, EKS, Fargate, networking, discovery, scaling, and four practical examples. The examples use Linux containers. Resource IDs, account numbers, images, and ARNs must be replaced with your own existing resources; a syntactically valid definition cannot create missing networks, permissions, application endpoints, or container images.

1. Choose orchestration and compute separately

Amazon ECS is AWS-native container orchestration. Amazon EKS provides managed Kubernetes. AWS Fargate supplies compute for supported ECS tasks and EKS pods; it is not a third orchestration API equivalent to ECS and EKS.

EKS always manages the Kubernetes control plane, but host responsibility depends on the compute option. Managed node groups do not remove every upgrade decision; Auto Mode and Fargate manage more of the infrastructure lifecycle. AWS-managed hosts do not make the application, IAM policy, or data model AWS's responsibility. ECS introduction, ECS Managed Instances, EKS compute choices.

A practical choice might be ECS for a team already using AWS APIs and EKS for a team needing Kubernetes interfaces and ecosystem tools. Portability still depends on the application's AWS integrations. Fargate reduces host operations; EC2 options offer different hardware, placement, and host-control choices. Test the workload's compatibility and total cost instead of assuming a universal winner.

ECS objects and Kubernetes analogies

Regional task-definition family → revision → task configuration
                                               ↓
ECS cluster → service → desired count → running tasks → application containers
                          ↓
                 Selected compatible compute capacity

A task definition is registered in an account/Region and can be used by multiple clusters. It is not stored inside one cluster. An ECS container instance is a registered compute host running the ECS agent; a Fargate task is not an EC2 container instance you register or manage.

A standalone task does not become a continuously maintained service simply because it has a health check. In Kubernetes, a Service provides stable networking; a Deployment/ReplicaSet maintains replicas. These are functional comparisons, not interchangeable objects. ECS task definitions, Kubernetes workloads, Kubernetes Services.

2. Task definitions, resources, and networking

This small Fargate task definition requests 0.25 vCPU and 512 MiB. requiresCompatibilities asks ECS to validate compatibility; registering it does not launch a task. Use a tested image digest for reproducible releases. The tag below illustrates the field, and is mutable.

{
  "family": "web-intro",
  "networkMode": "awsvpc",
  "requiresCompatibilities": ["FARGATE"],
  "cpu": "256",
  "memory": "512",
  "runtimePlatform": {"operatingSystemFamily": "LINUX", "cpuArchitecture": "X86_64"},
  "containerDefinitions": [{
    "name": "web", "image": "nginx:stable", "essential": true,
    "portMappings": [{"containerPort": 80, "protocol": "tcp"}]
  }]
}

At task level, numeric CPU strings use 1,024 CPU units per vCPU, and numeric memory strings use MiB. Fargate requires supported CPU/memory combinations: for example, 512 CPU units with 1,024 MiB is valid, while 1,024 CPU units with 1,024 MiB is not. At container level, resource fields use numbers rather than these task-level strings. A container's hard memory limit can cause it to be killed when exceeded; memoryReservation is a soft reservation where supported. Allocate resources from measurements, not a rule that every container must use exactly 75% of task memory. Task definition parameters.

By default, with essential: true, that container exiting stops the task's other containers. An enabled restart policy can restart eligible container exits without replacing the task. Container restart policies. An ECS service can replace the task. A nonessential sidecar stopping alone does not necessarily stop the task; whether continuing without that sidecar is acceptable is an application decision.

The task definition contains networkMode. The service or RunTask request supplies networkConfiguration with actual subnets and security groups. They are not both task-definition fields. For awsvpc tasks behind an ALB, use a target group with target type ip, and allow the ALB security group to reach the container port. For private tasks, provide required NAT or VPC endpoints/routes for image pulls, logs, secrets, and application dependencies. A private IP alone does not supply internet access. On EC2, instance ENI/task density limits and supported ENI trunking affect placement. Task networking, ALB integration.

Fargate provisions capacity for scheduled tasks/pods, but task count scaling must be configured separately. Startup includes scheduling, networking, image download, and application initialization; it is not instantaneous. Fargate charges for requested resources during the billable lifecycle, not CPU cycles actually consumed. Billing starts with image download and ends when the task/pod terminates, rounded to seconds, with a one-minute minimum for Linux and five minutes for Windows. Extra storage and other services can add charges. Fargate pricing.

3. Service discovery: find endpoints, then connect

AWS Cloud Map maintains a registry that can support API discovery and, for suitable namespace types, DNS. DNS gives a client endpoints; it does not proxy each application request or guarantee instant removal from client caches. An ALB is a separate request-routing component. ECS Service Connect is another option for service-to-service connectivity and discovery through managed proxies. Cloud Map integration, Service Connect.

Creating a private DNS namespace is asynchronous. Wait for the returned operation to succeed before using its namespace ID. The VPC must have suitable DNS settings. This bounded polling example reports pending/failure states instead of assuming the initial response contains a ready namespace:

import time
import boto3

sd = boto3.client('servicediscovery', region_name='us-east-1')
operation_id = sd.create_private_dns_namespace(
    Name='internal.example.com', Vpc='vpc-0123456789abcdef0')['OperationId']
namespace_id = None
for attempt in range(60):
    operation = sd.get_operation(OperationId=operation_id)['Operation']
    if operation['Status'] == 'SUCCESS':
        namespace_id = operation['Targets']['NAMESPACE']
        break
    if operation['Status'] == 'FAIL':
        raise RuntimeError(operation.get('ErrorMessage', 'Namespace creation failed'))
    time.sleep(2)
if namespace_id is None:
    raise TimeoutError('Still pending; inspect the existing operation before retrying creation')
registry = sd.create_service(
    Name='api', NamespaceId=namespace_id,
    DnsConfig={'DnsRecords': [{'Type': 'A', 'TTL': 60}],
               'RoutingPolicy': 'MULTIVALUE'},
    HealthCheckCustomConfig={'FailureThreshold': 1})['Service']

Configure the ECS service with serviceRegistries=[{'registryArn': registry['Arn']}] in addition to its normal task, count, network, and compute settings. ECS registers task endpoints and updates custom health status using container health information. An A record gives IPs, not ports: clients of this example's port-3000 API would use http://api.internal.example.com:3000. IPv4 awsvpc tasks fit this A-record example; other modes/address families have different record requirements.

MULTIVALUE can return up to eight healthy endpoints, not every endpoint in a large service. If all are unhealthy, DNS can return unhealthy records; clients must handle connection failures. TTL 60 is DNS cache lifetime, not a health-check interval. Cloud Map custom health checks do not probe the application themselves. FailureThreshold is deprecated and fixed at 1; health updates take approximately 30 seconds rather than meaning “one failed probe immediately removes the task.” CreatePrivateDnsNamespace, DNS configuration, custom health semantics.

4. Scaling tasks and scaling hosts are different loops

Service Auto Scaling changes an ECS service's desired task count. EC2 capacity-provider managed scaling changes the associated Auto Scaling group's host capacity to accommodate tasks. Fargate capacity providers select Fargate/Spot capacity; they do not manage an EC2 ASG for you.

Measured service load → scaling policy adjusts desired task count
                                      ↓
                           ECS tries to place tasks
                            /                   \
             Fargate supplies capacity    EC2 capacity provider/ASG supplies hosts
                            \                   /
                         Tasks start and become healthy
                                      ↓
                         Metrics reflect new capacity

Register a scalable target before adding policies. A CPU target such as 70% means the service average, not a hard cap for each task. CloudWatch alarms, cooldowns, missing data, deployment state, and capacity limits influence the response. Scale-out cooldown does not block every possible larger scale-out request. Choose timings from startup behavior; 60 seconds out and 300 seconds in are example settings, not required values.

A capacity-provider strategy's base is a task placement base on one provider. It does not create a minimum number of EC2 instances and does not increase the service's desired count. Weights distribute tasks after satisfying the base. If desired count is below the base, the strategy does not invent extra tasks.

For an existing cluster, preserve its full intended provider association list when using put_cluster_capacity_providers: the request specifies the resulting list. This example assumes a Fargate-only teaching cluster:

import boto3

ecs = boto3.client('ecs', region_name='us-east-1')
ecs.put_cluster_capacity_providers(
    cluster='production',
    capacityProviders=['FARGATE', 'FARGATE_SPOT'],
    defaultCapacityProviderStrategy=[
        {'capacityProvider': 'FARGATE', 'base': 2, 'weight': 1},
        {'capacityProvider': 'FARGATE_SPOT', 'weight': 3}
    ])

At desired count ten, the two-task base leaves eight tasks; a 1:3 weight targets another two regular Fargate tasks and six Spot tasks, giving four and six overall when capacity is available. Spot is interruptible and not guaranteed available. Do not assume unavailable Spot automatically falls back to regular Fargate. A cluster can have EC2-ASG and Fargate providers, but a single strategy cannot mix those two provider categories. Capacity-provider rules, Fargate capacity providers.

For EC2 capacity, managed scaling uses CapacityProviderReservation, not “average CPU across the cluster.” targetCapacity: 80 targets spare placement capacity; it does not keep CPU at exactly 80%. Hosts must register with the cluster, task sizes must fit supported instances, and ASG limits must allow growth. Managed termination protection requires the corresponding ASG instance scale-in protection setup. Managed scaling behavior, termination protection.

5. Example: an ECS Fargate API with rolling deployment

Assume an existing production cluster, private subnets in two AZs, a dedicated api-rolling-alb ALB/listener and api-rolling IP target group, and an application image that listens on 3000 and implements /health. The image must include curl for this health check. Create the log group and secret first. All account IDs below use the required 12-digit format but remain placeholders.

Save the following as api-task.json, substitute your resources, and register it. An execution role lets ECS perform operations such as image pulls, log delivery, and secret injection; a task role gives your application AWS permissions. Both need appropriate trust policies, and the deploying principal needs applicable iam:PassRole permissions. The execution role needs secretsmanager:GetSecretValue for the named secret and, when applicable, permission to decrypt with its customer-managed KMS key.

{
  "family": "api-service",
  "networkMode": "awsvpc",
  "requiresCompatibilities": ["FARGATE"],
  "cpu": "512", "memory": "1024",
  "executionRoleArn": "arn:aws:iam::123456789012:role/ecsTaskExecutionRole",
  "taskRoleArn": "arn:aws:iam::123456789012:role/apiTaskRole",
  "containerDefinitions": [{
    "name": "api",
    "image": "123456789012.dkr.ecr.us-east-1.amazonaws.com/api:v1.2.3",
    "essential": true,
    "portMappings": [{"containerPort": 3000, "protocol": "tcp"}],
    "environment": [{"name": "NODE_ENV", "value": "production"}],
    "secrets": [{"name": "DB_PASSWORD", "valueFrom": "arn:aws:secretsmanager:us-east-1:123456789012:secret:db-password-AbCdEf"}],
    "logConfiguration": {
      "logDriver": "awslogs",
      "options": {"awslogs-group": "/ecs/api-service", "awslogs-region": "us-east-1", "awslogs-stream-prefix": "api"}
    },
    "healthCheck": {
      "command": ["CMD-SHELL", "curl -f http://localhost:3000/health || exit 1"],
      "interval": 30, "timeout": 5, "retries": 3, "startPeriod": 60
    }
  }]
}

aws logs create-log-group --region us-east-1 --log-group-name /ecs/api-service
aws logs put-retention-policy --region us-east-1 --log-group-name /ecs/api-service --retention-in-days 7
aws ecs register-task-definition --region us-east-1 --cli-input-json file://api-task.json

Use the task-definition ARN/revision returned by registration; api-service:3 below assumes that is your tested revision. Configure ALB health checks to match the application endpoint. This service explicitly uses the ECS controller; with that controller the deployment strategy defaults to rolling. Because the call names launchType='FARGATE', the cluster's default capacity-provider strategy from section 4 does not apply to it: all three tasks run on regular Fargate and none on Spot. ECS uses that default only for a service created with neither launchType nor capacityProviderStrategy. The call as written:

import boto3

ecs = boto3.client('ecs', region_name='us-east-1')
ecs.create_service(
    cluster='production', serviceName='api-service',
    taskDefinition='api-service:3', desiredCount=3, launchType='FARGATE',
    networkConfiguration={'awsvpcConfiguration': {
        'subnets': ['subnet-0123456789abcdef0', 'subnet-0123456789abcdef1'],
        'securityGroups': ['sg-0123456789abcdef0'], 'assignPublicIp': 'DISABLED'}},
    loadBalancers=[{
        'targetGroupArn': 'arn:aws:elasticloadbalancing:us-east-1:123456789012:targetgroup/api-rolling/0123456789abcdef',
        'containerName': 'api', 'containerPort': 3000}],
    deploymentController={'type': 'ECS'},
    deploymentConfiguration={
        'minimumHealthyPercent': 100, 'maximumPercent': 200,
        'deploymentCircuitBreaker': {'enable': True, 'rollback': True}},
    healthCheckGracePeriodSeconds=60)

With desired count three, these deployment percentages permit up to six running/pending tasks and establish a lower deployment bound of three healthy tasks. The minimum uses rounding up and the maximum rounding down for fractional results. They guide scheduler decisions; they cannot guarantee three remain healthy through every failure. A replica service's usual rolling defaults are already 100/200. Those defaults do not mean all old tasks are stopped simultaneously.

The container startPeriod handles health-check startup accounting; the service grace period temporarily ignores specified unhealthy signals after task startup. Neither forces an ALB to route to unhealthy targets. A circuit-breaker rollback needs a previous completed deployment to roll back to; a first deployment has no successful predecessor. Check capacity headroom, readiness, connection draining, and database migration compatibility before calling a rollout zero-downtime. Rolling behavior, circuit breaker, container health checks, execution role.

Add target tracking and a schedule

The following policies belong to the rolling service above. The ALB resource label concatenates the actual load-balancer and target-group ARN suffixes. ALBRequestCountPerTarget is requests per target per minute, not requests per second. Its target of 1,000 is an example workload threshold.

import boto3

scaling = boto3.client('application-autoscaling', region_name='us-east-1')
resource_id = 'service/production/api-service'
scaling.register_scalable_target(
    ServiceNamespace='ecs', ResourceId=resource_id,
    ScalableDimension='ecs:service:DesiredCount', MinCapacity=3, MaxCapacity=20)
metrics = [
    ('cpu', 70.0, {'PredefinedMetricType': 'ECSServiceAverageCPUUtilization'}),
    ('memory', 80.0, {'PredefinedMetricType': 'ECSServiceAverageMemoryUtilization'}),
    ('requests', 1000.0, {
        'PredefinedMetricType': 'ALBRequestCountPerTarget',
        'ResourceLabel': 'app/api-rolling-alb/0123456789abcdef/targetgroup/api-rolling/0123456789abcdef'})
]
for name, target, metric in metrics:
    scaling.put_scaling_policy(
        PolicyName=name + '-tracking', ServiceNamespace='ecs', ResourceId=resource_id,
        ScalableDimension='ecs:service:DesiredCount', PolicyType='TargetTrackingScaling',
        TargetTrackingScalingPolicyConfiguration={
            'TargetValue': target, 'PredefinedMetricSpecification': metric,
            'ScaleOutCooldown': 60, 'ScaleInCooldown': 300})

scaling.put_scheduled_action(
    ServiceNamespace='ecs', ScheduledActionName='morning-capacity', ResourceId=resource_id,
    ScalableDimension='ecs:service:DesiredCount', Schedule='cron(0 8 * * ? *)',
    Timezone='Etc/UTC', ScalableTargetAction={'MinCapacity': 10, 'MaxCapacity': 20})
scaling.put_scheduled_action(
    ServiceNamespace='ecs', ScheduledActionName='evening-capacity', ResourceId=resource_id,
    ScalableDimension='ecs:service:DesiredCount', Schedule='cron(0 20 * * ? *)',
    Timezone='Etc/UTC', ScalableTargetAction={'MinCapacity': 3, 'MaxCapacity': 10})

Multiple target-tracking policies scale out when any policy requires it; scale-in needs all policies with scale-in enabled to agree. Missing metric data is not a signal of zero load. A memory metric that does not fall as replicas increase can be a poor scaling signal, so verify the relationship. These policies operate within the currently configured minimum/maximum.

Scheduled actions update bounds. At 08:00, a count below ten is raised toward ten; at 20:00, a count above ten is reduced toward ten. A count already within the new range is not forced to its minimum. The code runs daily, including weekends. The ECS documentation lists ALBRequestCountPerTarget target tracking as unsupported for blue/green deployments; do not copy that policy blindly into the next example. ECS target tracking, Application Auto Scaling target tracking, scheduled scaling.

6. Example: CodeDeploy blue/green deployment

A blue/green deployment starts a replacement task set, validates it, shifts traffic, and later removes the original set. Keeping blue available can speed rollback, but rollback is not instantaneous or unconditional. Application/database compatibility and deployment state matter.

This example uses the CodeDeploy controller. Assume a separate existing service named api-bluegreen configured with deploymentController={'type':'CODE_DEPLOY'}, the same compatible task/network settings as above, and an initial blue IP target group. Do not attach the circuit-breaker configuration to it: the deployment circuit breaker is only for services that use the ECS controller. Provide api-blue and api-green target groups and production/test listeners on the same api-alb ALB, plus a CodeDeploy service role. Keep these target groups and listener routes separate from the rolling service’s api-rolling-alb resources. Current ECS also supports native blue/green deployments; AWS recommends considering that option for new setups. Its configuration is distinct from CodeDeploy's. CodeDeploy ECS deployment model, native ECS blue/green.

Save an AppSpec referencing an already registered new task revision as appspec.yaml. It is a CodeDeploy deployment input, not a Kubernetes manifest:

version: 0.0
Resources:
  - TargetService:
      Type: AWS::ECS::Service
      Properties:
        TaskDefinition: arn:aws:ecs:us-east-1:123456789012:task-definition/api-service:4
        LoadBalancerInfo:
          ContainerName: api
          ContainerPort: 3000
        PlatformVersion: LATEST

Optional AppSpec hooks can run validation Lambda functions. Those functions need to report success/failure through CodeDeploy's lifecycle-event API; merely returning an HTTP-style 200 is insufficient. Include changed network settings when the new deployment needs them. AppSpec ECS resources, lifecycle hooks.

from pathlib import Path
import boto3

cd = boto3.client('codedeploy', region_name='us-east-1')
cd.create_application(applicationName='api-app', computePlatform='ECS')
cd.create_deployment_group(
    applicationName='api-app', deploymentGroupName='api-prod-dg',
    deploymentConfigName='CodeDeployDefault.ECSCanary10Percent5Minutes',
    serviceRoleArn='arn:aws:iam::123456789012:role/CodeDeployRole',
    deploymentStyle={'deploymentType': 'BLUE_GREEN',
                     'deploymentOption': 'WITH_TRAFFIC_CONTROL'},
    ecsServices=[{'serviceName': 'api-bluegreen', 'clusterName': 'production'}],
    loadBalancerInfo={'targetGroupPairInfoList': [{
        'targetGroups': [{'name': 'api-blue'}, {'name': 'api-green'}],
        'prodTrafficRoute': {'listenerArns': [
            'arn:aws:elasticloadbalancing:us-east-1:123456789012:listener/app/api-alb/0123456789abcdef/0123456789abcdef']},
        'testTrafficRoute': {'listenerArns': [
            'arn:aws:elasticloadbalancing:us-east-1:123456789012:listener/app/api-alb/0123456789abcdef/abcdef0123456789']}
    }]},
    blueGreenDeploymentConfiguration={
        'terminateBlueInstancesOnDeploymentSuccess': {
            'action': 'TERMINATE', 'terminationWaitTimeInMinutes': 5},
        'deploymentReadyOption': {'actionOnTimeout': 'CONTINUE_DEPLOYMENT'}},
    autoRollbackConfiguration={'enabled': True, 'events': ['DEPLOYMENT_FAILURE']})
cd.create_deployment(
    applicationName='api-app', deploymentGroupName='api-prod-dg',
    revision={'revisionType': 'AppSpecContent',
              'appSpecContent': {'content': Path('appspec.yaml').read_text()}})

Assuming successful progression, the canary configuration sends 10% of traffic to green, waits five minutes, then shifts the remaining 90%. The termination setting waits another five minutes after successful deployment before terminating blue. This is not a generic “10% every five minutes” ramp. Add appropriate validation and CloudWatch alarm/rollback settings if application-level regressions should halt deployment; failure-based rollback alone cannot recognize every bad business outcome. For CodeDeploy with an NLB, the supported predefined deployment configuration is all-at-once rather than this ALB canary. CreateDeploymentGroup API, deployment configurations.

7. Example: a logging sidecar that receives application logs

Starting a Fluentd container beside an application does not automatically connect their logs. Either configure a real shared volume/file path in both containers and a reader, or route stdout/stderr to a log router. The example below uses FireLens with Fluent Bit on Linux Fargate. It has a real application-to-router path, instead of an unused shared volume.

Create /ecs/web and /ecs/router log groups. The task role needs permissions for Fluent Bit's CloudWatch Logs output, while the execution role supplies applicable image-pull and awslogs permissions. Replace the application image and pin both images to tested digests for release. The stable router tag is an illustrative mutable tag.

{
  "family": "web-with-logging",
  "networkMode": "awsvpc", "requiresCompatibilities": ["FARGATE"],
  "cpu": "1024", "memory": "2048",
  "executionRoleArn": "arn:aws:iam::123456789012:role/ecsTaskExecutionRole",
  "taskRoleArn": "arn:aws:iam::123456789012:role/FireLensLogsRole",
  "containerDefinitions": [
    {
      "name": "web", "image": "123456789012.dkr.ecr.us-east-1.amazonaws.com/web:v1",
      "essential": true, "cpu": 768, "memory": 1536,
      "portMappings": [{"containerPort": 80}],
      "dependsOn": [{"containerName": "log-router", "condition": "START"}],
      "logConfiguration": {
        "logDriver": "awsfirelens",
        "options": {
          "Name": "cloudwatch_logs", "region": "us-east-1",
          "log_group_name": "/ecs/web", "log_stream_prefix": "web-",
          "auto_create_group": "false"
        }
      }
    },
    {
      "name": "log-router",
      "image": "public.ecr.aws/aws-observability/aws-for-fluent-bit:stable",
      "essential": true, "cpu": 256, "memory": 512,
      "firelensConfiguration": {"type": "fluentbit"},
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {"awslogs-group": "/ecs/router", "awslogs-region": "us-east-1", "awslogs-stream-prefix": "router"}
      }
    }
  ]
}

The resource allocations sum to one vCPU and 2,048 MiB; test memory headroom and log buffering under load. START waits for the router container to start, not for a health check or verified downstream log delivery. HEALTHY is a separate dependency condition and requires a container health check. In this example, with no restart policy enabled, the essential router exiting stops the task; it does not guarantee lossless logs. Choose buffering/backpressure behavior and monitor delivery failures. Keep port 24224 closed to inbound traffic from outside the task. FireLens behavior and security, FireLens definition example, container dependencies.

8. Example: EKS Deployment, Service, and HPA

This example assumes an existing EKS cluster with working nodes, cluster access, Metrics Server, and the AWS Load Balancer Controller installed with appropriate permissions and subnet discovery. It uses that controller's NLB integration; EKS Auto Mode uses a different load-balancer class. Create the production namespace, api-service-account, api-config ConfigMap (database.host key), and api-secrets Secret (db.password key) before applying the manifest.

The application image in this example listens on 8080 and implements /health and /ready; this is a different image contract from the port-3000 ECS API example. Configure workload IAM through a supported mechanism such as EKS Pod Identity or IAM roles for service accounts when it needs AWS access; naming a Kubernetes service account alone grants no AWS permissions. Supported identity mechanisms depend on compute choice. Kubernetes Secrets are not made confidential merely by base64 encoding; configure access and storage protections. EKS load balancing, EKS workload identity, Kubernetes Secret behavior.

Save as deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-deployment
  namespace: production
spec:
  replicas: 3
  selector:
    matchLabels: {app: api}
  strategy:
    type: RollingUpdate
    rollingUpdate: {maxSurge: 1, maxUnavailable: 0}
  template:
    metadata:
      labels: {app: api}
    spec:
      serviceAccountName: api-service-account
      containers:
        - name: api
          image: 123456789012.dkr.ecr.us-east-1.amazonaws.com/api-eks:v1.2.3
          ports:
            - {containerPort: 8080, name: http}
          resources:
            requests: {memory: 256Mi, cpu: 250m}
            limits: {memory: 512Mi, cpu: 500m}
          env:
            - name: DB_HOST
              valueFrom:
                configMapKeyRef: {name: api-config, key: database.host}
            - name: DB_PASSWORD
              valueFrom:
                secretKeyRef: {name: api-secrets, key: db.password}
          livenessProbe:
            httpGet: {path: /health, port: http}
            initialDelaySeconds: 30
            periodSeconds: 10
            timeoutSeconds: 5
            failureThreshold: 3
          readinessProbe:
            httpGet: {path: /ready, port: http}
            initialDelaySeconds: 10
            periodSeconds: 5
            timeoutSeconds: 3
            failureThreshold: 2
---
apiVersion: v1
kind: Service
metadata:
  name: api-service
  namespace: production
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
    service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
spec:
  type: LoadBalancer
  loadBalancerClass: service.k8s.aws/nlb
  selector: {app: api}
  ports:
    - {protocol: TCP, port: 80, targetPort: http}
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api-hpa
  namespace: production
spec:
  scaleTargetRef: {apiVersion: apps/v1, kind: Deployment, name: api-deployment}
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target: {type: Utilization, averageUtilization: 70}
    - type: Resource
      resource:
        name: memory
        target: {type: Utilization, averageUtilization: 80}
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - {type: Percent, value: 50, periodSeconds: 60}
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
        - {type: Percent, value: 100, periodSeconds: 30}
        - {type: Pods, value: 4, periodSeconds: 30}
      selectPolicy: Max

250m requests one quarter of a CPU; the 70% CPU target is relative to that request, not the 500m limit or the entire node. Readiness failure removes a pod from normal ready service endpoints; liveness failure can restart its container. Add a startup probe when slow initialization would otherwise fail liveness. Neither probe guarantees application correctness. Probes, resource management.

The HPA needs a metrics pipeline; merely creating the object does not install Metrics Server. With multiple metrics it chooses the largest recommended replica count, subject to limits, stabilization, and error handling. HPA scales replicas, not node capacity; configure compatible node scaling/capacity management separately. At three replicas, a 100% increase permits adding three while the four-pod policy permits four, so selectPolicy: Max permits the larger change when scaling is warranted. It does not schedule four new pods unconditionally every 30 seconds. Avoid repeatedly applying a stale spec.replicas: 3 over the HPA's live scaling decisions. HPA algorithm, Metrics Server on EKS.

The Service explicitly requests an internet-facing NLB with IP targets through the AWS Load Balancer Controller. Use an internal scheme for a private API, and configure authentication and TLS for a real public application. A generic type: LoadBalancer does not universally mean an ALB; the controller and class determine the implementation.

After verifying the selected account, Region, cluster, and prerequisites:

aws eks update-kubeconfig --name production-cluster --region us-east-1
kubectl apply -f deployment.yaml
kubectl rollout status deployment/api-deployment -n production
kubectl get pods,services,hpa -n production
kubectl describe hpa api-hpa -n production
kubectl logs deployment/api-deployment -n production

9. Diagnose the control loop that failed

If tasks remain pending, check compatible capacity, task size, subnet IPs, image access, and scheduling constraints. If tasks start then stop, inspect exit codes, memory limits, dependencies, and secret/log permissions. If a service has tasks but no successful requests, check the application port, readiness, security groups, target type, and load-balancer health checks. An ALB with only unhealthy registered targets can fail open, so health checks are not a substitute for resilient clients or application recovery. ALB health checks.

Injected secrets are snapshots taken at task/container startup; rotating a secret does not automatically update environment variables in existing containers. Start replacement tasks or use an application retrieval/refresh design. enableExecuteCommand=True alone is also not a complete ECS Exec setup: task permissions, caller permissions, supported platform, networking, and the client plugin must be configured. Secret injection, ECS Exec.

Monitor service events, running versus desired tasks, request latency/error rates, resource saturation, and scaling history. Test failed health checks, unavailable capacity, a bad image revision, dependency outages, and rollback before relying on the automation. Practice: with desired count five and deployment percentages 100/200, calculate the scheduler's bounds; then explain why an image that always fails its health checks cannot be rescued by those percentages alone.