DynamoDB Design

Access Patterns First: Deciding Whether DynamoDB Fits

In a relational database you usually start with entities (customers, orders, products), normalize them, and trust SQL to answer whatever question shows up later. DynamoDB reverses that order. Its speed comes from routing every read by a key, so the access patterns, meaning the specific questions your application must answer, have to be known before you design anything. Get them wrong and you pay for it in full-table reads, throttling, or a rewrite.

This section gives you the core vocabulary, a trace of why Query beats Scan, a fit test for choosing DynamoDB at all, and an early cost lens. Our running example is an e-commerce backend.

Start with the questions, not the entities

Write the questions as a list:

1. Get a customer's profile.
2. List a customer's recent orders, newest first.
3. Get one order together with its line items.
4. Find all orders shipped today, across all customers.

Try it first: before reading on, decide which of these four could be answered by looking data up under a customer's key. Then check.

The vocabulary

A table is a collection of items; apart from the key, it has no fixed columns. An item is a row-like record: a set of named attributes (values), up to 400 KB. Every item is identified by its primary key, made of:

• the partition key: a value DynamoDB hashes to decide which storage partition (a unit of storage and throughput that DynamoDB manages for you) holds the item;
• optionally a sort key: a value that orders items sharing the same partition key, so ranges of them can be read together.

Here is a slice of our table, using generic attribute names pk and sk:

Two read operations matter now. Query reads the items under one partition key, optionally narrowed by a sort-key condition, and touches only those items. Scan reads every item in the table and bills for everything it reads. Patterns 1 and 2 are a GetItem (fetch one item by its full primary key) and a Query. Pattern 4 would be a Scan unless we add an index.

Worked trace: the same request, two ways

Request: the 10 most recent orders for customer 123, who has 25 orders of about 1 KB each.

import boto3
from boto3.dynamodb.conditions import Key, Attr

table = boto3.resource('dynamodb').Table('Shop')

# Query: go straight to one partition key, read a sort-key range
resp = table.query(
    KeyConditionExpression=Key('pk').eq('CUSTOMER#123') & Key('sk').begins_with('ORDER#'),
    ScanIndexForward=False,   # newest first
    Limit=10,
    ReturnConsumedCapacity='TOTAL',
)
print(resp['Count'], resp['ConsumedCapacity']['CapacityUnits'])

The Scan version reads everything, then discards what does not match:

kwargs = {
    'FilterExpression': Attr('pk').eq('CUSTOMER#123') & Attr('sk').begins_with('ORDER#'),
    'ReturnConsumedCapacity': 'TOTAL',
}
items, scanned, units = [], 0, 0.0
while True:                                   # a Scan returns at most 1 MB per call
    page = table.scan(**kwargs)
    items += page['Items']
    scanned += page['ScannedCount']           # items examined
    units += page['ConsumedCapacity']['CapacityUnits']
    if 'LastEvaluatedKey' not in page:        # no more pages
        break
    kwargs['ExclusiveStartKey'] = page['LastEvaluatedKey']

Count is what survived the filter; ScannedCount is what DynamoDB read. The filter shrinks the response, not the bill. Capacity is charged on items read. For strongly consistent reads at 4 KB per unit, with 1 KB items:

The Query figure is 10 items × 1 KB = 10 KB ÷ 4 KB = 2.5, rounded up to 3 (Limit=10 stops the read after 10 items). The Scan figure is total table size in KB ÷ 4. Eventually consistent reads, the default that both snippets above use, cost half in both columns. Latency follows the same shape: the Query stays one request at single-digit milliseconds, while the Scan needs roughly one page per MB, so at 1 million 1 KB items that is about a thousand sequential calls. (Parallel Scan segments cut wall-clock time, but you pay the same units.)

The cost lens, early

One read unit covers a read of up to 4 KB (strongly consistent; eventually consistent costs half). One write unit covers a write of up to 1 KB. Sizes round up, so a 2.5 KB order costs 3 write units, and reading it costs 1 strongly consistent read unit.

There are two capacity modes. In on-demand mode you pay per request unit consumed and provision nothing (a table can still throttle if traffic jumps past double its previous peak within 30 minutes). In provisioned mode you reserve units per second and pay for the reservation whether used or not. Simple estimate: a steady 100 writes/second of 1 KB items is 100 × 86,400 = 8.64 million write units per day. Provisioned at 100 write units fits that flat load. If traffic peaks at ten times the average, provisioned either throttles or you pay for the peak all day, which is when on-demand wins. Multiply units by your region's current price from the AWS pricing page; you can switch modes later, with limits on how often.

Deciding fit

Counterexample. A finance dashboard lets analysts group revenue by any mix of region, product line, channel, and quarter, chosen at runtime. Each new combination would need its own index or a full Scan, and joins across tables must be done in application code. Aurora, or DynamoDB data exported to S3 and queried with Athena, fits far better. Many systems combine the two: DynamoDB serves the transactional path, an export feeds reporting. This table is a heuristic, not a verdict; borderline workloads need the full design exercise to settle.

Exercise: label eight access patterns

A fictional workout app, FitTrack, stores users, workouts, and exercise sets. For each pattern, label it Key (Query-able with a key design), Index (needs a secondary index), or Other store (relational or analytics).

1. Get a user's profile by userId.
2. List a user's last 20 workouts, newest first.
3. Get one workout with all its sets.
4. List a user's workouts between two dates.
5. Log in by email address.
6. List all workouts logged at gym G this week.
7. Average calories burned per age bracket, per month.
8. Support staff filter workouts by any combination of age, device, gym, calories, and date.

Checklist: Can you write your app's questions as a list before choosing keys? Can you say why a filter does not cut a Scan's cost? Can you name one workload that should not go in DynamoDB?

Designing Keys: Partition Keys, Sort Keys, and Hot Partitions

A team launches an orders table with status as the partition key, because the warehouse wants "all PENDING orders." Capacity is provisioned generously, yet during a sale writes start failing with throttling errors while the table's capacity graph looks nearly idle. Nothing is broken in the service. The key choice sent all the traffic to one place. This section teaches how to choose keys so that doesn't happen, and so your main queries become a single Query call.

Why cardinality and evenness matter

DynamoDB hashes each partition key value to pick a partition, one of the physical storage units that hold your data. Each partition can serve only a limited rate: roughly 3,000 read units and 1,000 write units per second. Table-level capacity is spread across partitions, so a single busy partition can be throttled (requests rejected with ProvisionedThroughputExceededException, or on-demand throttling) even while the table as a whole has headroom.

A good partition key has high cardinality (many distinct values) and receives evenly spread requests. Both properties matter, since a million customers with one celebrity still produces a hot key.

🔮 Predict: with PK = status and 5,000 order writes/sec, of which 80% are PENDING, roughly how many writes/sec go to one partition key value? About 4,000, which is four times the ~1,000/sec partition limit. Most of those writes can be throttled even if the table is provisioned for 10,000.

⚠️ Adaptive capacity softens this, not removes it. DynamoDB can shift unused capacity toward busier partitions and can isolate frequently accessed items, which rescues moderately skewed workloads. On a table with no Local Secondary Index it can also, over time, split one busy partition key's items across several partitions by sort-key range. That helps only when the writes are spread over that range, not when every new item sorts last, as with a timestamp sort key. And a single item still lives on one partition, so one item cannot be written faster than the per-partition limit.

Worked example: the popular-product counter

A product page increments views on item PRODUCT#42 during a flash sale at about 3,000 updates/sec. Each update of a small item costs one write unit, so you need ~3,000 WCU against one item and the limit is ~1,000. No amount of table capacity or adaptive capacity fixes that. The key has to change.

Mitigation: write sharding

Write sharding spreads one logical key over N physical keys by adding a suffix. Writers pick a random shard; readers fan out and combine.

import random
import boto3
from boto3.dynamodb.conditions import Key

table = boto3.resource("dynamodb").Table("AppTable")
SHARDS = 10

def record_view(product_id: str) -> None:
    shard = random.randrange(SHARDS)          # 0..9
    table.update_item(
        Key={"PK": f"PRODUCT#{product_id}#SHARD#{shard}", "SK": "COUNTER"},
        UpdateExpression="ADD #v :one",       # atomic increment; creates item if missing
        ExpressionAttributeNames={"#v": "views"},
        ExpressionAttributeValues={":one": 1},
    )

def total_views(product_id: str) -> int:
    # Read-side fan-out: 10 GetItem calls, then merge by summing
    total = 0
    for shard in range(SHARDS):
        resp = table.get_item(
            Key={"PK": f"PRODUCT#{product_id}#SHARD#{shard}", "SK": "COUNTER"}
        )
        total += int(resp.get("Item", {}).get("views", 0))
    return total

Now 3,000 writes/sec average ~300/sec per shard, comfortably under the limit. The price: every read costs 10 reads instead of 1, and the total is a merge you must compute (or cache). Sharding trades read simplicity for write throughput, so apply it only to keys you can show are hot, and choose N from the peak rate divided by a safe per-shard rate.

Sort key design: one Query per question

The sort key orders items within one partition key and lets a Query select a slice. DynamoDB compares sort keys as strings (or numbers), so a composite sort key that concatenates fields, such as ORDER#2025-03-14#ID123, gives you ordering by date for free. Use ISO dates and zero-padded numbers so alphabetical order equals chronological order.

Suppose partition PK = CUSTOMER#123 holds these items, in sort-key order:

SK
ORDER#2025-02-27#ID101
ORDER#2025-03-02#ID102
ORDER#2025-03-14#ID123
ORDER#2025-03-14#ID124
ORDER#2025-04-01#ID130
PROFILE

from boto3.dynamodb.conditions import Key

pk = Key("PK").eq("CUSTOMER#123")

# 1. All March 2025 orders (prefix match)
march = table.query(
    KeyConditionExpression=pk & Key("SK").begins_with("ORDER#2025-03")
)
# -> ID102, ID123, ID124

# 2. Orders in a range (both bounds inclusive)
range_ = table.query(
    KeyConditionExpression=pk & Key("SK").between(
        "ORDER#2025-03-02", "ORDER#2025-03-14#ID123"
    )
)
# -> ID102, ID123  (ID124 sorts after the upper bound, so it is excluded)

# 3. Latest 2 orders: newest first, stop after 2 items
latest = table.query(
    KeyConditionExpression=pk & Key("SK").begins_with("ORDER#"),
    ScanIndexForward=False,   # descending sort-key order
    Limit=2,
)
# -> ID130, ID124

Trace of query 3: the begins_with("ORDER#") condition excludes PROFILE; descending order starts at ORDER#2025-04-01#ID130, then ORDER#2025-03-14#ID124, and Limit=2 stops there. Only two items are read and billed. The upper bound in query 2 shows a classic trap: because the ID is part of the string, a bound ending at a date alone would exclude every order from that date, so end ranges with a suffix that sorts after any ID (or use the next day's prefix).

Mitigation: time-bucketed keys

For time-series data, put a time bucket (such as a day or month) into the partition key alongside a high-cardinality entity, for example DEVICE#7#2025-03. This caps how much any one partition key accumulates. The cost is that a query spanning several buckets becomes several Query calls whose results you concatenate.

Item size and unbounded lists

An item can be at most 400 KB. A tempting design stores orders: [...] as a list inside the customer item. It fails in two ways: the list eventually hits 400 KB and writes start failing, and every append is billed by the size of the whole item, so writes get steadily more expensive. Reads pull the whole list even if you want only the latest order. The fix is the sort-key pattern above: one item per order in the customer's partition. Split any collection that grows without a bound into separate items. (If the table has Local Secondary Indexes, which can only be defined when the table is created, all items sharing one partition key are also capped at 10 GB; that arrives in the section on secondary indexes.)

Guided attempt: fix the IoT design

Flawed design: table Telemetry with PK = date (2025-03-14) and SK = deviceId#timestamp. It must support (a) latest readings for device X and (b) readings for device X in a time range.

Your task: redesign the keys. Then name the key condition for (a) and (b), and say what happens when one device produces years of data. Try it before opening the answer.

Single-Table Design: Modeling Relationships with Item Collections

Suppose the app needs a customer page: profile plus the five most recent orders. With one table per entity you read the profile, query orders, and then perhaps make more calls for details. DynamoDB has no joins, so each extra call is another network round trip. Single-table design avoids this by putting related records where one Query can fetch them together.

Item collections: the unit of co-location

An item collection is every item that shares the same partition key value. DynamoDB stores them together, ordered by sort key, so a Query on that partition key returns the whole collection (or a slice of it) in one call. If a customer and their orders share a partition key, one Query reads both.

To make that work, the table uses generic key names, PK and SK, because different entity types fill them with different things. Each value carries an entity prefix (CUSTOMER#, ORDER#, ITEM#) so that types stay distinguishable and never collide.

There are two collections here, because there are two questions: what belongs to a customer, and what belongs to an order. The order appears twice on purpose. The summary row (date, total, status) lives under the customer, and the full record lives under the order's own key. That duplication is a cost we will meet again below.

Worked trace: two Queries replace the joins

Get an order with its line items. One Query on ORDER#456:

import boto3
from boto3.dynamodb.conditions import Key

table = boto3.resource("dynamodb").Table("AppTable")

resp = table.query(KeyConditionExpression=Key("PK").eq("ORDER#456"))
items = resp["Items"]
order = next(i for i in items if i["SK"] == "META")
lines = [i for i in items if i["SK"].startswith("ITEM#")]

Items come back sorted by SK in character order, so the result is ITEM#789, ITEM#790, META. The SK value tells you the entity type, and the code splits the list by that prefix. (A Query returns at most 1 MB per call, so a very large collection needs pagination with LastEvaluatedKey.)

Customer with recent orders. Read the collection backwards and cap it:

resp = table.query(
    KeyConditionExpression=Key("PK").eq("CUSTOMER#123"),
    ScanIndexForward=False,  # descending SK order
    Limit=6,                 # profile + up to 5 newest orders
)
for item in resp["Items"]:
    kind = item["SK"].split("#")[0]  # PROFILE or ORDER

Descending SK order puts PROFILE first ("P" sorts after "O"), then ORDER#2026-03-20#457, then ORDER#2026-03-14#456. The date inside the sort key makes "newest first" free. One request returns the profile and the orders.

The access-pattern matrix

The design artifact is a table that lists each pattern with the exact key condition and index that serves it. If a row has no answer, the design is not finished.

Try it: fill in the two empty rows for the new features "find orders containing product X" and "add several addresses per customer". State any new item types and any cost.

Many-to-many: the adjacency-list pattern

An adjacency list stores each relationship as its own item, one per direction. Take students and courses:

The first row answers "courses for student s1" (PK=STUDENT#s1, SK begins_with COURSE#). The mirrored second row answers "students in course c9". Each Query is a single call, and the denormalized names mean no follow-up reads. Users and groups work the same way.

The costs are concrete:

• Two writes per enrollment, and two deletes per withdrawal. If only one side lands, the two directions disagree. TransactWriteItems (all-or-nothing across items, covered in the final section) fixes this at a higher capacity price.
• Copied attributes go stale. Renaming a course means updating every COURSE#c9 mirror row under every student.
• An alternative: keep one item and add a GSI with the key attributes swapped (SK as the index partition key, PK as the sort key). It avoids duplicate writes, but the reverse direction is then eventually consistent and pays index write costs.

Honest trade-offs

⚠️ Single-table design is a bet on stable access patterns, and it has real costs:

• Console readability. Mixed item types with generic names are hard to scan by eye.
• Painful schema change. A new pattern may need new key attributes on existing items, which means a backfill of every old item.
• Shared settings. Capacity mode, backups and the stream are per table, so every entity type shares them. TTL names a single attribute per table, and only items carrying it expire. Stream consumers receive every entity type and must filter.
• Team cost. If nobody else can read the key scheme, maintenance suffers.

Separate tables are often the better choice when entities are independent (nothing is fetched together) or differ greatly in scale and lifecycle. For example, a very high-volume event log that expires after weeks should not share throughput, backup and stream settings with a small customer table. The same goes when different teams own the data, or when patterns are still changing weekly. Co-locate data that is read together; separate data that merely belongs to the same app.

Independent variation: cancel an order and release reserved items

Assume stock lives in items with PK=PRODUCT#sku, SK=STOCK, holding available and reserved counts. Orders in the customer collection carry a status copy. Task: list which items change when order 456 is cancelled, what condition each write needs, and which access pattern breaks if you skip one of the writes.

Secondary Indexes: GSIs, LSIs, and Sparse Indexes for New Questions

The warehouse team asks: "Show me every order shipped today, oldest first." Our base table uses PK = CUSTOMER#<id> and SK = ORDER#<date>#<orderId>. That layout answers "this customer's orders" and nothing else. Today the only way to answer the warehouse's question is a Scan, which reads and bills for every item in the table. Changing the base key would break the customer queries. The better move is to give the same data a second key.

The Global Secondary Index

A Global Secondary Index (GSI) is an automatically maintained copy of chosen attributes, stored under a different partition key and sort key. You Query it by passing IndexName, just as you would query a table. "Global" means its keys are independent of the base table's partition key, so one index spans all base partitions. You can add a GSI to an existing table, and DynamoDB backfills it.

To feed the index, order items carry generic key attributes (GSI1PK, GSI1SK) that we fill in at write time. A first, naive design might use GSI1PK = status and GSI1SK = shippedAt. Before we see why that fails, three mechanics shape every GSI decision.

1. Reads are eventually consistent. The index is updated after the base write, and ConsistentRead=True is not supported on a GSI. A warehouse dashboard can tolerate a short lag. A post-checkout confirmation page should read the base table, with ConsistentRead=True.

2. Projection decides what the index copies. A projection is the set of attributes stored in the index entry. Base table keys and index keys are always included.

3. Writes fan out. Every base write that changes an index key or a projected attribute also writes to that index, and you pay write units for both. With ALL, every update to any attribute hits the index. An update that changes an index key costs a delete at the old position plus a put at the new one. So a mutable key like status makes each transition cost two index writes. Key a time-ordered index on a fact that never changes, such as the shipped date, and the entry is written once.

Pitfall trace: the hot index partition

With the naive GSI1PK = SHIPPED, trace a busy day:

2,000 orders/sec ship
  ↓ every index entry has GSI1PK = "SHIPPED"
one index partition receives 2,000 writes/sec
  ↓ a single partition tops out near 1,000 write units/sec
the GSI throttles
  ↓ DynamoDB must apply each base write to the index too
base-table writes get throttled (the index back-pressures the table)

The order-taking path fails because of a reporting index. This is the hot-key problem from "Designing Keys", relocated to the index. The fix is the same: a composite key that adds date, plus a shard suffix. The shard is computed from the order ID when the item is written.

import boto3
from boto3.dynamodb.conditions import Key

table = boto3.resource('dynamodb').Table('shop')
SHARDS = 4  # at write time: shard = zlib.crc32(order_id.encode()) % SHARDS

def shipped_on(day):
    """Fan out over the shard keys, then merge by sort key."""
    items = []
    for shard in range(SHARDS):
        resp = table.query(
            IndexName='GSI1',
            KeyConditionExpression=Key('GSI1PK').eq(f'SHIPPED#{day}#{shard}'),
        )  # pagination via LastEvaluatedKey omitted for brevity
        items.extend(resp['Items'])
    return sorted(items, key=lambda i: i['GSI1SK'])  # GSI1SK = shippedAt#orderId

Each shard now takes about a quarter of the traffic, and the cost is four Query calls plus a merge on every read. Because GSI1PK embeds the ship date, the entry never moves after the order ships. It costs one index write per order.

Sparse indexes and overloading

An item appears in a GSI only if it has all of that index's key attributes. That makes sparse indexes free filters. Give an order an openForPickup attribute (value WAREHOUSE#7) when it becomes ready, and REMOVE it when the customer collects it. A GSI whose partition key is openForPickup contains only orders currently waiting, so "what is waiting at warehouse 7?" reads a tiny index instead of filtering millions of orders.

GSI overloading means using generic names (GSI1PK, GSI1SK) so different entity types fill them with different meanings. The customer profile item sets GSI1PK = EMAIL#a@b.com (plus a fixed GSI1SK such as PROFILE, because an item without both index keys is left out of the index) for login lookup, while shipped orders set GSI1PK = SHIPPED#2026-03-14#2. One index, one write path per item type, and one index to monitor. The limit is that an item holds one value per attribute, so it can sit in only one "meaning" of that index at a time.

Local Secondary Indexes and the decision rule

A Local Secondary Index (LSI) keeps the base table's partition key but offers an alternate sort key, for example a customer's orders sorted by total instead of date. It differs from a GSI in ways that matter:

Because an LSI shares the base table's partition, it also counts toward the item-collection size limit. A customer or tenant that grows without bound can eventually hit that cap, and its writes then fail.

⚠️ Decision rule: default to a GSI. Choose an LSI only if all four hold: you are creating the table now, the alternate sort key is within the same partition key, you need strongly consistent reads of that ordering, and every collection will stay far under 10 GB.

Exercise: five new patterns on the order model

Assumptions: the table is already in production. An order item is about 1 KB. Its lifecycle is PLACED → SHIPPED → DELIVERED, which is one create and two updates (about 3 base write units). A standard write unit covers up to 1 KB, and each index entry rounds up to at least 1 write unit. For each pattern, choose a base-table key change, a GSI with a projection, an LSI, or a sparse index. Then estimate the extra write units per order.

1. A customer's 10 newest orders.
2. All orders shipped on a given day, for the warehouse (about 500 per second at peak).
3. Orders waiting for pickup at one warehouse.
4. A customer's orders sorted by total, largest first.
5. Look up one order by orderId alone, a rare support-tool action.

Try next: sketch which index you would add for "orders by warehouse and day", then name its projection and its write cost before reading further in the lesson.

Writes, Consistency, Cost, and a Full Design Review

A flash sale starts and 40 people click "buy" on a product with one unit left. Keys, collections and indexes decide how fast you can read data. They do nothing to stop all 40 orders from succeeding. This section covers three things: keeping writes correct inside the database, pricing a design with real arithmetic, and reviewing a whole design before it ships.

Conditional Writes: Correctness Inside the Database

A conditional write is a write that DynamoDB applies only if a ConditionExpression (a predicate evaluated against the item's current state) is true. The check and the write happen as one step on that item, so no other request can slip in between them.

The buggy pattern is read-then-write: read stock, check it in application code, then write stock - 1. Two requests can both read 1 and both write 0, selling two units from one. The fix moves the check into the write:

import boto3
from botocore.exceptions import ClientError

table = boto3.resource("dynamodb").Table("shop")

def reserve(product_id, qty):
    try:
        table.update_item(
            Key={"PK": f"PRODUCT#{product_id}", "SK": "STOCK"},
            UpdateExpression="SET stock = stock - :q",
            ConditionExpression="stock >= :q",
            ExpressionAttributeValues={":q": qty},
        )
        return True
    except ClientError as e:
        if e.response["Error"]["Code"] == "ConditionalCheckFailedException":
            return False  # sold out: a normal business outcome, not a crash
        raise

Trace (stock starts at 1, requests A and B arrive together, each wanting 1). DynamoDB applies writes to a single item one at a time:

1. A is applied first: condition 1 >= 1 is true, so stock becomes 0 and A returns success.
2. B is applied next: condition 0 >= 1 is false, so nothing is written and B receives ConditionalCheckFailedException.

The order could be reversed, but exactly one request wins either way, and stock never goes negative. A failed condition on a write is still billed as a write.

Optimistic Locking and Idempotency

Optimistic locking assumes conflicts are rare, so nobody takes a lock. Each item carries a version number. You read the item, work out its new contents in your own code, and write them back with one UpdateItem that both checks the version you read and increments it. This example reuses table and ClientError from the block above:

def update_state(key, change, attempts=5):
    for _ in range(attempts):
        item = table.get_item(Key=key, ConsistentRead=True)["Item"]
        try:
            table.update_item(
                Key=key,
                UpdateExpression="SET #st = :new, version = version + :one",
                ConditionExpression="version = :seen",
                ExpressionAttributeNames={"#st": "state"},  # state is a reserved word
                ExpressionAttributeValues={
                    ":new": change(item["state"]),
                    ":seen": item["version"],
                    ":one": 1,  # a bare 1 in the expression is a syntax error
                },
            )
            return True
        except ClientError as e:
            if e.response["Error"]["Code"] != "ConditionalCheckFailedException":
                raise
            # the version moved: loop to re-read the item and try again
    return False  # still conflicting: report it to the caller

Trace. Writers A and B both read version 7, so both send the condition version = :seen with 7. DynamoDB applies A first, and the item moves to version 8. B's condition is then checked against 8, fails with ConditionalCheckFailedException, and nothing is written. B must not resend the same write, because its new state was computed from data that is now stale. It re-reads, recomputes and tries again, which is what the loop does, up to attempts times. When many writers keep hitting the same item, some calls use up their attempts and return False, so the caller must handle that result; a short random pause before each retry makes it much rarer. When the change cannot be recomputed (a person edited an old copy of a form), skip the retry and surface the conflict to the caller.

This is safe without a transaction because the check and the write are one request on one item, the same guarantee reserve relied on. It holds only if every writer of that item uses the version condition; one unconditional write bypasses it. Create the item with a put_item that sets version to 1 under ConditionExpression="attribute_not_exists(PK)", so two creators cannot both win. Use optimistic locking for read-modify-write cycles that cannot be expressed as a single SET stock = stock - :q.

An idempotent write is one that is safe to repeat. Retries and duplicate messages are normal, so Put with ConditionExpression="attribute_not_exists(PK)" makes "create order 123" succeed once; later repeats fail the condition and change nothing. Choose a client-generated ID (an order ID, not a timestamp) as the key so repeats collide.

The locking loop above hides a repeat too. If your write was applied but its response was lost, the SDK sends the request again on its own; that second copy fails the version condition, so the loop re-reads and applies change a second time, and an increment or an append lands twice. So change must be safe to apply twice: have it look at the fresh state and return it unchanged when its work is already there (move PLACED to SHIPPED only if the state is still PLACED; give each appended entry an ID and skip one that is present), and when a change leaves no such trace, as a counter does not, send the update as a one-action TransactWriteItems, whose ClientRequestToken (next section) makes the resend do nothing, at double the write units.

Transactions: When One Item Is Not Enough

Creating an order and decrementing stock touch two items, and a single conditional write cannot span them. TransactWriteItems applies up to 100 actions all-or-nothing (and across tables in one region). The combined request is limited to 4 MB, and two actions may not target the same item.

client = boto3.client("dynamodb")  # low-level client: values carry type tags
client.transact_write_items(
    ClientRequestToken=order_id,  # makes retries within ~10 minutes idempotent
    TransactItems=[
        {"Put": {"TableName": "shop",
                 "Item": {"PK": {"S": f"ORDER#{order_id}"}, "SK": {"S": "META"},
                          "status": {"S": "PLACED"}},
                 "ConditionExpression": "attribute_not_exists(PK)"}},
        {"Update": {"TableName": "shop",
                    "Key": {"PK": {"S": "PRODUCT#42"}, "SK": {"S": "STOCK"}},
                    "UpdateExpression": "SET stock = stock - :q",
                    "ConditionExpression": "stock >= :q",
                    "ExpressionAttributeValues": {":q": {"N": "1"}}}},
    ],
)

If stock is insufficient, the whole call raises TransactionCanceledException and no order exists. The price is that transactional writes consume twice the write units of standard writes. Decision rule: if the invariant lives in one item (stock never below zero, create-once), use a conditional single-item write. Reach for a transaction only when two or more items must change together.

Cost Walkthrough With Real Arithmetic

The unit rules, building on the 1 KB write and 4 KB read units from earlier: sizes round up per operation, an eventually consistent read costs half a strongly consistent one, and a transactional write costs double.

Take a 2.5 KB item with two GSIs: GSI1 projects ALL (2.5 KB) and GSI2 projects KEYS_ONLY (about 0.2 KB).

The lesson: a GSI roughly multiplies write cost by how much you project into it. Changing an indexed key attribute can also cost an extra index write, to remove the old entry. Over-projecting is therefore expensive.

Scan Design vs Query-and-GSI Design at 10 Million Items

Assume 1 KB items (about 10 GB total) and "orders shipped today" shown on a dashboard that refreshes every minute (1,440 times a day). For prices I use, as illustrative figures, the on-demand rates for US East (N. Virginia) at the time of writing: $0.625 per million write request units and $0.125 per million read request units. Prices differ by Region and change over time, so check the current pricing page for yours.

• Scan design: a full eventually consistent scan reads about 10 GB / 8 KB, roughly 1.3 million RRU, or about $0.16 per scan. At 1,440 scans a day that is about $235 a day, or roughly $7,000 a month, and it gets slower as the table grows. A filter would not help, because it trims what is returned, not what is billed.
• GSI design: a sparse GSI keyed on ship date returns about 2,000 items (about 2 MB). That costs about 256 RRU (2,048 KB / 4 KB × 0.5), or about $0.00003 a query: roughly $1.40 a month. The index adds one extra write unit per order. For 1 million orders that is about $0.63.

The index costs pennies in writes to save thousands in reads. Not every index pays off this way, though, which is why the checklist below exists.

Guided Attempt

An item is 3.2 KB. It has one GSI that projects KEYS_ONLY (about 0.3 KB). Compute the write units per put and the RRU for one eventually consistent read.

TTL and Streams

TTL (time to live) deletes items automatically once an attribute you designate, holding a Unix epoch time in seconds stored as a Number, is in the past. Expiry deletions do not consume write capacity, which makes TTL cheap for sessions, temporary tokens, and old log-like items. Deletion is not instant (expired items are typically removed within a few days), so queries that must never see expired data should still filter on the timestamp.

DynamoDB Streams is an ordered change log of item-level inserts, updates and deletes, retained for 24 hours. Consumers such as Lambda read it to update search indexes, send notifications, or feed analytics. The roadmap's event-driven topics (EventBridge, queues and streams) cover how to build on it.

The Design-Review Checklist

For every GSI, ask: which access pattern in the matrix needs it, and what is its extra write cost?

Independent Transfer Task: Multi-Tenant Support Tickets

Brief. A SaaS help desk serves many tenants (customer companies), some with millions of tickets. Patterns: (1) get a ticket by ID with its comments, (2) list a tenant's newest tickets, (3) list an assignee's tickets by status, (4) list a tenant's overdue open tickets, (5) add a comment. Assume 1M new tickets a month, 3M ticket updates, 5M comments, 1 KB tickets and 0.5 KB comments.

Produce: the key schema, an access-pattern matrix, a GSI list, a hot-key risk assessment, a monthly cost estimate (using the illustrative prices above), and a short Architecture Decision Record (ADR, a one-page record of a decision, its alternatives and consequences).

Success criteria: every pattern maps to a Query with no Scan; every GSI is justified by a pattern and a write cost; the largest tenant's hot-key risk is named with a mitigation; the cost arithmetic is shown; the ADR names an alternative and a downside.

Closing Self-Check

• Can I list every access pattern before choosing a single key?
• Can I spot a hot partition from a key's cardinality and a tenant's traffic share?
• Can I lay out an item collection so one Query returns a ticket and its comments?
• Can I justify each GSI by the pattern it serves and its write cost?
• Can I argue for or against single-table design in an ADR, naming the alternative I rejected?