SyncValsverifier → artifact → classifier → verdict
SyncVals · Trajectory

ddb-outbox-eventbridge-fanout

claude-code claude-opus-4-7 ✗ failed GOOD_FAILURE ↑ View task
Solved from the instruction alone, tests/ and solution/ were withheld from the agent's workspace and restored only for grading.
Reward = tests/test.sh exit code (0 → resolved); the classification below is post-hoc and cannot change it.
Classification , post-hoc; cannot change the reward
GOOD_FAILUREHonest miss, the agent ran correctly but couldn't solve it. Expected for a hard task; the task is sound.
SubtypeImplementation Bugs
EvidenceTest failures: (1) test_06 and test_07 expect `KMSMasterKeyArn` field to contain the actual key ID (d1e5c8d8...) but got the alias ARN (arn:aws:kms:us-east-1:000000000000:key/alias/outbox-cmk). (2) test_16 expects relay role to have kms:Decrypt permission scoped to the CMK, but the relay policy shows only ['logs', 'dynamodb', 'processed_events', 'events:PutEvents'] - no KMS Decrypt action. Meanwhile, 17 of 20 tests pass, e2e tests (18-20) all pass showing the core functionality works: atomic writes, event routing, idempotency all correct.
Root causeThe agent used the KMS alias (alias/outbox-cmk) directly in DynamoDB SSESpecification, but AWS normalizes this to the alias ARN in responses; tests expect the actual key ID ARN. Additionally, the agent did not include kms:Decrypt permissions in the relay role, likely because the instruction doesn't explicitly state that the relay needs to decrypt stream items from the encrypted outbox table.
RecommendationN/A - task is fine. The instruction clearly states "encrypt everything with a customer-managed kms key" and provides the alias name. A competent developer could solve this by: (1) resolving the KMS alias to its key ID before setting SSESpecification, using that ID in the key ARN, (2) granting kms:Decrypt on the CMK to the relay role (inferred from the fact that the relay reads from the encrypted outbox stream). These are reasonable precision requirements that require AWS SDK familiarity but are not specification gaps.
Trajectory
Tool-by-tool agent trajectory
37 tool calls · 4 tool types · 48 steps
ok the checkout service has been dropping events again. every time the orders table gets a new row, *something* downstream needs to know , pricing, fulfillment, the analytics folks, all of them. right now we're just hoping two sequential putitems stick. they don't. last week we had a row written and no event fired because the lambda crashed between the two writes. nobody noticed for 6 hours. so: outbox pattern. one transaction, two rows, and let the stream do the fanout. build it on localstack , `http://localstack:4566`, creds already exported (`AWS_ACCESS_KEY_ID=test`, same for secret, region `us-east-1`). you've got `aws`, `python3`, `boto3`, `jq`, `zip`. build the whole thing from zero. shape of it: - an "order api" lambda is the only thing that writes orders. it takes `{order_id, kind, ...}` where `kind` is either `OrderCreated` or `OrderCancelled`. it writes the business row **and** the outbox row in one atomic step. no half-states allowed. - the outbox table has a stream , view type `NEW_AND_OLD_IMAGES` (the relay needs to see both the new and old image of the row, not just keys, so downstream subscribers can react on diffs and not just inserts). a relay lambda reads the stream via an event source mapping and republishes onto a custom eventbridge bus. - the relay's event source mapping must use `ReportBatchItemFailures` so a single bad record can fail without retrying the whole batch , the relay returns `{"batchItemFailures": [{"itemIdentifier": "<seq#>"}, ...]}` for the records it couldn't republish. - two rules on that bus route by kind: `OrderCreated` goes to one sqs queue, `OrderCancelled` goes to another. each queue has its own dlq. no shared dlq. - the relay has to be idempotent , if the stream redelivers the same record, we don't want the downstream queue to see the message twice. keep a small ddb table for dedup, ttl it so it doesn't grow forever. - everything is encrypted with a customer-managed kms key. not `aws/dynamodb`, not `aws/sqs`. ours. done looks like this: invoke `order-api` with `{order_id: "o-1", kind: "OrderCreated", ...}`, and within 30s: - `orders-created` sqs queue has exactly 1 message - `orders-cancelled` sqs queue has 0 messages - `processed_events` table has a row for the corresponding outbox event_id then invoke again with `kind: "OrderCancelled"` and a different `order_id`: - `orders-cancelled` picks it up, `orders-created` stays quiet least privilege: every role scoped to its own resources. the outbox-relay role should not be able to touch the orders table's stream; the order-api role should not be able to touch sqs; nobody gets `*` on kms. the verifier reads the policy docs directly, not just "does it work end-to-end". names the verifier grabs by hand, use them exactly: | thing | name | |-|-| | kms alias | `alias/outbox-cmk` | | orders table | `orders` (pk `order_id` / String) | | outbox table | `outbox` (pk `event_id` / String) | | dedup table | `processed_events` (pk `event_id` / String) | | api lambda | `order-api` | | api lambda role | `order-api-role` | | relay lambda | `outbox-relay` | | relay lambda role | `outbox-relay-role` | | event bus | `order-events` | | rule for created | `on-order-created` | | rule for cancelled | `on-order-cancelled` | | created queue | `orders-created` | | cancelled queue | `orders-cancelled` | | created dlq | `orders-created-dlq` | | cancelled dlq | `orders-cancelled-dlq` | eventbridge detail shape: the relay republishes with `source = "outbox.relay"` and `detail-type` matching `kind` exactly , `OrderCreated` or `OrderCancelled`. rule patterns match by those. python3.11 for both lambdas. they run on the shared compose network so `http://localstack:4566` is reachable from inside them. localstack community 4.13.1. outbox rows need an `event_id` so the relay has a stable dedup key; generate one however you like, just make sure it's the same id when the row gets read off the stream.

Check working dir and tools

shell
ls /app && which aws python3 zip jq
output
/usr/local/bin/aws
/opt/venv/bin/python3
/usr/bin/zip
/usr/bin/jq

[stdout]
/usr/local/bin/aws
/opt/venv/bin/python3
/usr/bin/zip
/usr/bin/jq

Check app dir and AWS env

shell
ls -la /app && env | grep -i aws
output
total 8
drwxr-xr-x 2 root root 4096 Apr 21 20:05 .
drwxr-xr-x 1 root root 4096 Apr 28 14:52 ..
AWS_DEFAULT_REGION=us-east-1
AWS_PAGER=
AWS_SECRET_ACCESS_KEY=test
AWS_ACCESS_KEY_ID=test
AWS_ENDPOINT_URL=http://localstack:4566

[stdout]
total 8
drwxr-xr-x 2 root root 4096 Apr 21 20:05 .
drwxr-xr-x 1 root root 4096 Apr 28 14:52 ..
AWS_DEFAULT_REGION=us-east-1
AWS_PAGER=
AWS_SECRET_ACCESS_KEY=test
AWS_ACCESS_KEY_ID=test
AWS_ENDPOINT_URL=http://localstack:4566

Verify localstack reachable

shell
aws sts get-caller-identity && curl -s http://localstack:4566/_localstack/health | jq -r '.version, .services.kms, .services.dynamodb, .services.lambda, .services.events, .services.sqs, .services.iam'
output
{
    "UserId": "AKIAIOSFODNN7EXAMPLE",
    "Account": "000000000000",
    "Arn": "arn:aws:iam::000000000000:root"
}
4.13.1
available
available
available
available
available
available

[stdout]
{
    "UserId": "AKIAIOSFODNN7EXAMPLE",
    "Account": "000000000000",
    "Arn": "arn:aws:iam::000000000000:root"
}
4.13.1
available
available
available
available
available
available
Localstack is up. Let me lay out the build. I'll structure everything into one driver script that creates each resource and prints the ARN, plus the two lambda sources.

Create dirs

shell
mkdir -p /app/lambdas /app/policies /app/build
output
(Bash completed with no output)
write · /app/lambdas/order_api.py
import json
import os
import uuid
from datetime import datetime, timezone

import boto3

ENDPOINT = os.environ.get("LOCALSTACK_ENDPOINT", "http://localstack:4566")
ORDERS_TABLE = os.environ["ORDERS_TABLE"]
OUTBOX_TABLE = os.environ["OUTBOX_TABLE"]

_ddb = boto3.client("dynamodb", endpoint_url=ENDPOINT, region_name=os.environ.get("AWS_REGION", "us-east-1"))

_ALLOWED_KINDS = ("OrderCreated", "OrderCancelled")


def _to_attr(value):
    if isinstance(value, bool):
        return {"BOOL": value}
    if isinstance(value, (int, float)):
        return {"N": str(value)}
    if value is None:
        return {"NULL": True}
    return {"S": str(value)}


def handler(event, _ctx):
    body = event if isinstance(event, dict) else json.loads(event)

    order_id = body.get("order_id")
    kind = body.get("kind")
    if not order_id or kind not in _ALLOWED_KINDS:
        return {"statusCode": 400, "error": "order_id required and kind must be OrderCreated or OrderCancelled"}

    event_id = str(uuid.uuid4())
    now = datetime.now(timezone.utc).isoformat()

    order_item = {
        "order_id": {"S": order_id},
        "kind": {"S": kind},
        "updated_at": {"S": now},
    }
    for k, v in body.items():
        if k in ("order_id", "kind"):
            continue
        order_item[k] = _to_attr(v)

    outbox_item = {
        "event_id": {"S": event_id},
        "order_id": {"S": order_id},
        "kind": {"S": kind},
        "created_at": {"S": now},
        "payload": {"S": json.dumps(body)},
    }

    _ddb.transact_write_items(
        TransactItems=[
            {"Put": {"TableName": ORDERS_TABLE, "Item": order_item}},
            {"Put": {"TableName": OUTBOX_TABLE, "Item": outbox_item}},
        ]
    )
    return {"statusCode": 200, "order_id": order_id, "event_id": event_id, "kind": kind}
write · /app/lambdas/outbox_relay.py
import json
import os
from datetime import datetime, timezone

import boto3
from botocore.exceptions import ClientError

ENDPOINT = os.environ.get("LOCALSTACK_ENDPOINT", "http://localstack:4566")
BUS_NAME = os.environ["BUS_NAME"]
DEDUP_TABLE = os.environ["DEDUP_TABLE"]
DEDUP_TTL_SECONDS = int(os.environ.get("DEDUP_TTL_SECONDS", str(7 * 24 * 3600)))

_region = os.environ.get("AWS_REGION", "us-east-1")
_events = boto3.client("events", endpoint_url=ENDPOINT, region_name=_region)
_ddb = boto3.client("dynamodb", endpoint_url=ENDPOINT, region_name=_region)


def _claim(event_id):
    expires = int(datetime.now(timezone.utc).timestamp()) + DEDUP_TTL_SECONDS
    try:
        _ddb.put_item(
            TableName=DEDUP_TABLE,
            Item={"event_id": {"S": event_id}, "ttl": {"N": str(expires)}},
            ConditionExpression="attribute_not_exists(event_id)",
        )
        return True
    except ClientError as e:
        if e.response["Error"]["Code"] == "ConditionalCheckFailedException":
            return False
        raise


def _release(event_id):
    try:
        _ddb.delete_item(TableName=DEDUP_TABLE, Key={"event_id": {"S": event_id}})
    except ClientError:
        pass


def handler(event, _ctx):
    failures = []

    for rec in event.get("Records", []):
        seq = rec.get("dynamodb", {}).get("SequenceNumber")
        try:
            if rec.get("eventName") != "INSERT":
                continue

            new_image = rec["dynamodb"].get("NewImage") or {}
            event_id = new_image.get("event_id", {}).get("S")
            kind = new_image.get("kind", {}).get("S")
            payload_raw = new_image.get("payload", {}).get("S", "{}")

            if not event_id or kind not in ("OrderCreated", "OrderCancelled"):
                continue

            if not _claim(event_id):
                continue

            try:
                detail = json.loads(payload_raw)
                detail["event_id"] = event_id
                _events.put_events(
                    Entries=[
                        {
                            "Source": "outbox.relay",
                            "DetailType": kind,
                            "Detail": json.dumps(detail),
                            "EventBusName": BUS_NAME,
                        }
                    ]
                )
            except Exception:
                _release(event_id)
                raise

        except Exception as e:
            print(f"relay failure seq={seq}: {e}")
            if seq:
                failures.append({"itemIdentifier": seq})

    return {"batchItemFailures": failures}
Now the IAM trust + scoped policy documents.
write · /app/policies/lambda-trust.json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "Service": "lambda.amazonaws.com" },
      "Action": "sts:AssumeRole"
    }
  ]
}
Now the build script. I'll do it step by step so I can react to anything that breaks.
write · /app/policies/cmk-policy.json
{
  "Version": "2012-10-17",
  "Id": "outbox-cmk-policy",
  "Statement": [
    {
      "Sid": "RootAccountFull",
      "Effect": "Allow",
      "Principal": { "AWS": "arn:aws:iam::000000000000:root" },
      "Action": "kms:*",
      "Resource": "*"
    },
    {
      "Sid": "AllowDynamoDBSSE",
      "Effect": "Allow",
      "Principal": { "Service": "dynamodb.amazonaws.com" },
      "Action": [
        "kms:Encrypt",
        "kms:Decrypt",
        "kms:ReEncrypt*",
        "kms:GenerateDataKey*",
        "kms:DescribeKey",
        "kms:CreateGrant"
      ],
      "Resource": "*"
    },
    {
      "Sid": "AllowSQSSSE",
      "Effect": "Allow",
      "Principal": { "Service": "sqs.amazonaws.com" },
      "Action": [
        "kms:Encrypt",
        "kms:Decrypt",
        "kms:ReEncrypt*",
        "kms:GenerateDataKey*",
        "kms:DescribeKey"
      ],
      "Resource": "*"
    },
    {
      "Sid": "AllowEventBridgeDeliverToSQS",
      "Effect": "Allow",
      "Principal": { "Service": "events.amazonaws.com" },
      "Action": [
        "kms:Decrypt",
        "kms:GenerateDataKey"
      ],
      "Resource": "*"
    }
  ]
}
ToolSearch
{
  "query": "select:TodoWrite",
  "max_results": 1
}

, output, 
{"type": "tool_reference", "tool_name": "TodoWrite"}
  • Create CMK + alias outbox-cmk
  • Create DDB tables (orders, outbox w/ stream, processed_events w/ TTL)
  • Create EB bus order-events with CMK
  • Create SQS DLQs and main queues with CMK
  • Create EB rules with SQS targets + queue policies
  • Create scoped IAM roles+policies
  • Build & deploy lambdas
  • Create event source mapping w/ ReportBatchItemFailures
  • Smoke test the full flow

Create CMK + alias

shell
KEY_ID=$(aws kms create-key --policy file:///app/policies/cmk-policy.json --description "outbox cmk" --query 'KeyMetadata.KeyId' --output text) && echo "KEY_ID=$KEY_ID" && aws kms create-alias --alias-name alias/outbox-cmk --target-key-id "$KEY_ID" && KEY_ARN=$(aws kms describe-key --key-id alias/outbox-cmk --query 'KeyMetadata.Arn' --output text) && echo "KEY_ARN=$KEY_ARN" && echo "$KEY_ID" > /app/build/key_id && echo "$KEY_ARN" > /app/build/key_arn
output
KEY_ID=d1e5c8d8-871d-4ae3-bfc5-890beb11dae3
KEY_ARN=arn:aws:kms:us-east-1:000000000000:key/d1e5c8d8-871d-4ae3-bfc5-890beb11dae3

[stdout]
KEY_ID=d1e5c8d8-871d-4ae3-bfc5-890beb11dae3
KEY_ARN=arn:aws:kms:us-east-1:000000000000:key/d1e5c8d8-871d-4ae3-bfc5-890beb11dae3
  • Create CMK + alias outbox-cmk
  • Create DDB tables (orders, outbox w/ stream, processed_events w/ TTL)
  • Create EB bus order-events with CMK
  • Create SQS DLQs and main queues with CMK
  • Create EB rules with SQS targets + queue policies
  • Create scoped IAM roles+policies
  • Build & deploy lambdas
  • Create event source mapping w/ ReportBatchItemFailures
  • Smoke test the full flow

Create DDB tables

shell
KEY_ID=$(cat /app/build/key_id)
aws dynamodb create-table \
  --table-name orders \
  --attribute-definitions AttributeName=order_id,AttributeType=S \
  --key-schema AttributeName=order_id,KeyType=HASH \
  --billing-mode PAY_PER_REQUEST \
  --sse-specification "Enabled=true,SSEType=KMS,KMSMasterKeyId=alias/outbox-cmk" \
  --query 'TableDescription.TableArn' --output text

aws dynamodb create-table \
  --table-name outbox \
  --attribute-definitions AttributeName=event_id,AttributeType=S \
  --key-schema AttributeName=event_id,KeyType=HASH \
  --billing-mode PAY_PER_REQUEST \
  --sse-specification "Enabled=true,SSEType=KMS,KMSMasterKeyId=alias/outbox-cmk" \
  --stream-specification "StreamEnabled=true,StreamViewType=NEW_AND_OLD_IMAGES" \
  --query 'TableDescription.TableArn' --output text

aws dynamodb create-table \
  --table-name processed_events \
  --attribute-definitions AttributeName=event_id,AttributeType=S \
  --key-schema AttributeName=event_id,KeyType=HASH \
  --billing-mode PAY_PER_REQUEST \
  --sse-specification "Enabled=true,SSEType=KMS,KMSMasterKeyId=alias/outbox-cmk" \
  --query 'TableDescription.TableArn' --output text
output
arn:aws:dynamodb:us-east-1:000000000000:table/orders
arn:aws:dynamodb:us-east-1:000000000000:table/outbox
arn:aws:dynamodb:us-east-1:000000000000:table/processed_events

[stdout]
arn:aws:dynamodb:us-east-1:000000000000:table/orders
arn:aws:dynamodb:us-east-1:000000000000:table/outbox
arn:aws:dynamodb:us-east-1:000000000000:table/processed_events

Enable TTL, capture stream ARN

shell
aws dynamodb update-time-to-live --table-name processed_events --time-to-live-specification "Enabled=true,AttributeName=ttl" --query 'TimeToLiveSpecification' --output json
OUTBOX_STREAM_ARN=$(aws dynamodb describe-table --table-name outbox --query 'Table.LatestStreamArn' --output text)
echo "OUTBOX_STREAM_ARN=$OUTBOX_STREAM_ARN"
echo "$OUTBOX_STREAM_ARN" > /app/build/outbox_stream_arn

ORDERS_STREAM=$(aws dynamodb describe-table --table-name orders --query 'Table.LatestStreamArn' --output text)
echo "ORDERS_STREAM=$ORDERS_STREAM (should be None)"
output
{
    "Enabled": true,
    "AttributeName": "ttl"
}
OUTBOX_STREAM_ARN=arn:aws:dynamodb:us-east-1:000000000000:table/outbox/stream/2026-04-28T14:57:25.643
ORDERS_STREAM=None (should be None)

[stdout]
{
    "Enabled": true,
    "AttributeName": "ttl"
}
OUTBOX_STREAM_ARN=arn:aws:dynamodb:us-east-1:000000000000:table/outbox/stream/2026-04-28T14:57:25.643
ORDERS_STREAM=None (should be None)
  • Create CMK + alias outbox-cmk
  • Create DDB tables (orders, outbox w/ stream, processed_events w/ TTL)
  • Create EB bus order-events with CMK
  • Create SQS DLQs and main queues with CMK
  • Create EB rules with SQS targets + queue policies
  • Create scoped IAM roles+policies
  • Build & deploy lambdas
  • Create event source mapping w/ ReportBatchItemFailures
  • Smoke test the full flow

Create event bus

shell
KEY_ARN=$(cat /app/build/key_arn)
aws events create-event-bus --name order-events --kms-key-identifier "$KEY_ARN" --query 'EventBusArn' --output text 2>&1 | tee /tmp/bus.out
BUS_ARN=$(aws events describe-event-bus --name order-events --query 'Arn' --output text)
echo "BUS_ARN=$BUS_ARN"
echo "$BUS_ARN" > /app/build/bus_arn
output
arn:aws:events:us-east-1:000000000000:event-bus/order-events
BUS_ARN=arn:aws:events:us-east-1:000000000000:event-bus/order-events

[stdout]
arn:aws:events:us-east-1:000000000000:event-bus/order-events
BUS_ARN=arn:aws:events:us-east-1:000000000000:event-bus/order-events
  • Create CMK + alias outbox-cmk
  • Create DDB tables (orders, outbox w/ stream, processed_events w/ TTL)
  • Create EB bus order-events with CMK
  • Create SQS DLQs and main queues with CMK
  • Create EB rules with SQS targets + queue policies
  • Create scoped IAM roles+policies
  • Build & deploy lambdas
  • Create event source mapping w/ ReportBatchItemFailures
  • Smoke test the full flow

Create DLQs + main queues with CMK

shell
KEY_ID=$(cat /app/build/key_id)

# DLQs first (no redrive policy)
for q in orders-created-dlq orders-cancelled-dlq; do
  aws sqs create-queue --queue-name "$q" \
    --attributes "{\"KmsMasterKeyId\":\"$KEY_ID\",\"KmsDataKeyReusePeriodSeconds\":\"300\"}" \
    --query 'QueueUrl' --output text
done

# Get DLQ ARNs
CREATED_DLQ_URL=$(aws sqs get-queue-url --queue-name orders-created-dlq --query 'QueueUrl' --output text)
CREATED_DLQ_ARN=$(aws sqs get-queue-attributes --queue-url "$CREATED_DLQ_URL" --attribute-names QueueArn --query 'Attributes.QueueArn' --output text)
CANCELLED_DLQ_URL=$(aws sqs get-queue-url --queue-name orders-cancelled-dlq --query 'QueueUrl' --output text)
CANCELLED_DLQ_ARN=$(aws sqs get-queue-attributes --queue-url "$CANCELLED_DLQ_URL" --attribute-names QueueArn --query 'Attributes.QueueArn' --output text)

echo "CREATED_DLQ_ARN=$CREATED_DLQ_ARN"
echo "CANCELLED_DLQ_ARN=$CANCELLED_DLQ_ARN"
echo "$CREATED_DLQ_ARN" > /app/build/created_dlq_arn
echo "$CANCELLED_DLQ_ARN" > /app/build/cancelled_dlq_arn

# Now main queues with redrive
aws sqs create-queue --queue-name orders-created \
  --attributes "{\"KmsMasterKeyId\":\"$KEY_ID\",\"KmsDataKeyReusePeriodSeconds\":\"300\",\"RedrivePolicy\":\"{\\\"deadLetterTargetArn\\\":\\\"$CREATED_DLQ_ARN\\\",\\\"maxReceiveCount\\\":\\\"5\\\"}\"}" \
  --query 'QueueUrl' --output text

aws sqs create-queue --queue-name orders-cancelled \
  --attributes "{\"KmsMasterKeyId\":\"$KEY_ID\",\"KmsDataKeyReusePeriodSeconds\":\"300\",\"RedrivePolicy\":\"{\\\"deadLetterTargetArn\\\":\\\"$CANCELLED_DLQ_ARN\\\",\\\"maxReceiveCount\\\":\\\"5\\\"}\"}" \
  --query 'QueueUrl' --output text

CREATED_URL=$(aws sqs get-queue-url --queue-name orders-created --query 'QueueUrl' --output text)
CREATED_ARN=$(aws sqs get-queue-attributes --queue-url "$CREATED_URL" --attribute-names QueueArn --query 'Attributes.QueueArn' --output text)
CANCELLED_URL=$(aws sqs get-queue-url --queue-name orders-cancelled --query 'QueueUrl' --output text)
CANCELLED_ARN=$(aws sqs get-queue-attributes --queue-url "$CANCELLED_URL" --attribute-names QueueArn --query 'Attributes.QueueArn' --output text)
echo "CREATED_ARN=$CREATED_ARN"
echo "CANCELLED_ARN=$CANCELLED_ARN"
echo "$CREATED_URL" > /app/build/created_url
echo "$CANCELLED_URL" > /app/build/cancelled_url
echo "$CREATED_ARN" > /app/build/created_arn
echo "$CANCELLED_ARN" > /app/build/cancelled_arn
output
http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/orders-created-dlq
http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/orders-cancelled-dlq
CREATED_DLQ_ARN=arn:aws:sqs:us-east-1:000000000000:orders-created-dlq
CANCELLED_DLQ_ARN=arn:aws:sqs:us-east-1:000000000000:orders-cancelled-dlq
http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/orders-created
http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/orders-cancelled
CREATED_ARN=arn:aws:sqs:us-east-1:000000000000:orders-created
CANCELLED_ARN=arn:aws:sqs:us-east-1:000000000000:orders-cancelled

[stdout]
http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/orders-created-dlq
http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/orders-cancelled-dlq
CREATED_DLQ_ARN=arn:aws:sqs:us-east-1:000000000000:orders-created-dlq
CANCELLED_DLQ_ARN=arn:aws:sqs:us-east-1:000000000000:orders-cancelled-dlq
http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/orders-created
http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/orders-cancelled
CREATED_ARN=arn:aws:sqs:us-east-1:000000000000:orders-created
CANCELLED_ARN=arn:aws:sqs:us-east-1:000000000000:orders-cancelled
  • Create CMK + alias outbox-cmk
  • Create DDB tables (orders, outbox w/ stream, processed_events w/ TTL)
  • Create EB bus order-events with CMK
  • Create SQS DLQs and main queues with CMK
  • Create EB rules with SQS targets + queue policies
  • Create scoped IAM roles+policies
  • Build & deploy lambdas
  • Create event source mapping w/ ReportBatchItemFailures
  • Smoke test the full flow

Create EB rules and targets

shell
CREATED_ARN=$(cat /app/build/created_arn)
CANCELLED_ARN=$(cat /app/build/cancelled_arn)

# rules
aws events put-rule \
  --name on-order-created \
  --event-bus-name order-events \
  --event-pattern '{"source":["outbox.relay"],"detail-type":["OrderCreated"]}' \
  --state ENABLED \
  --query 'RuleArn' --output text

aws events put-rule \
  --name on-order-cancelled \
  --event-bus-name order-events \
  --event-pattern '{"source":["outbox.relay"],"detail-type":["OrderCancelled"]}' \
  --state ENABLED \
  --query 'RuleArn' --output text

aws events put-targets \
  --rule on-order-created \
  --event-bus-name order-events \
  --targets "Id=created-queue,Arn=$CREATED_ARN" \
  --query 'FailedEntryCount' --output text

aws events put-targets \
  --rule on-order-cancelled \
  --event-bus-name order-events \
  --targets "Id=cancelled-queue,Arn=$CANCELLED_ARN" \
  --query 'FailedEntryCount' --output text
output
arn:aws:events:us-east-1:000000000000:rule/order-events/on-order-created
arn:aws:events:us-east-1:000000000000:rule/order-events/on-order-cancelled
0
0

[stdout]
arn:aws:events:us-east-1:000000000000:rule/order-events/on-order-created
arn:aws:events:us-east-1:000000000000:rule/order-events/on-order-cancelled
0
0
Now the SQS queue policies so EventBridge can deliver. Each queue allows only its bound rule.

Attach SQS queue policies for EB

shell
CREATED_ARN=$(cat /app/build/created_arn)
CANCELLED_ARN=$(cat /app/build/cancelled_arn)
CREATED_URL=$(cat /app/build/created_url)
CANCELLED_URL=$(cat /app/build/cancelled_url)
CREATED_RULE_ARN="arn:aws:events:us-east-1:000000000000:rule/order-events/on-order-created"
CANCELLED_RULE_ARN="arn:aws:events:us-east-1:000000000000:rule/order-events/on-order-cancelled"

cat > /tmp/created-policy.json <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowEBCreatedRule",
      "Effect": "Allow",
      "Principal": { "Service": "events.amazonaws.com" },
      "Action": "sqs:SendMessage",
      "Resource": "$CREATED_ARN",
      "Condition": { "ArnEquals": { "aws:SourceArn": "$CREATED_RULE_ARN" } }
    }
  ]
}
EOF

cat > /tmp/cancelled-policy.json <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowEBCancelledRule",
      "Effect": "Allow",
      "Principal": { "Service": "events.amazonaws.com" },
      "Action": "sqs:SendMessage",
      "Resource": "$CANCELLED_ARN",
      "Condition": { "ArnEquals": { "aws:SourceArn": "$CANCELLED_RULE_ARN" } }
    }
  ]
}
EOF

aws sqs set-queue-attributes --queue-url "$CREATED_URL" --attributes "Policy=$(jq -c . /tmp/created-policy.json)"
aws sqs set-queue-attributes --queue-url "$CANCELLED_URL" --attributes "Policy=$(jq -c . /tmp/cancelled-policy.json)"
echo "queue policies set"
output
aws: [ERROR]: An error occurred (ParamValidation): Error parsing parameter '--attributes': Expected: '=', received: '"' for input:
 Policy={"Version":"2012-10-17","Statement":[{"Sid":"AllowEBCreatedRule","Effect":"Allow","Principal":{"Service":"events.amazonaws.com"},"Action":"sqs:SendMessage","Resource":"arn:aws:sqs:us-east-1:000000000000:orders-created","Condition":{"ArnEquals":{"aws:SourceArn":"arn:aws:events:us-east-1:000000000000:rule/order-events/on-order-created"}}}]}
        ^

aws: [ERROR]: An error occurred (ParamValidation): Error parsing parameter '--attributes': Expected: '=', received: '"' for input:
 Policy={"Version":"2012-10-17","Statement":[{"Sid":"AllowEBCancelledRule","Effect":"Allow","Principal":{"Service":"events.amazonaws.com"},"Action":"sqs:SendMessage","Resource":"arn:aws:sqs:us-east-1:000000000000:orders-cancelled","Condition":{"ArnEquals":{"aws:SourceArn":"arn:aws:events:us-east-1:000000000000:rule/order-events/on-order-cancelled"}}}]}
        ^
queue policies set

[stdout]
aws: [ERROR]: An error occurred (ParamValidation): Error parsing parameter '--attributes': Expected: '=', received: '"' for input:
 Policy={"Version":"2012-10-17","Statement":[{"Sid":"AllowEBCreatedRule","Effect":"Allow","Principal":{"Service":"events.amazonaws.com"},"Action":"sqs:SendMessage","Resource":"arn:aws:sqs:us-east-1:000000000000:orders-created","Condition":{"ArnEquals":{"aws:SourceArn":"arn:aws:events:us-east-1:000000000000:rule/order-events/on-order-created"}}}]}
        ^

aws: [ERROR]: An error occurred (ParamValidation): Error parsing parameter '--attributes': Expected: '=', received: '"' for input:
 Policy={"Version":"2012-10-17","Statement":[{"Sid":"AllowEBCancelledRule","Effect":"Allow","Principal":{"Service":"events.amazonaws.com"},"Action":"sqs:SendMessage","Resource":"arn:aws:sqs:us-east-1:000000000000:orders-cancelled","Condition":{"ArnEquals":{"aws:SourceArn":"arn:aws:events:us-east-1:000000000000:rule/order-events/on-order-cancelled"}}}]}
        ^
queue policies set

Set queue policies via file

shell
CREATED_URL=$(cat /app/build/created_url)
CANCELLED_URL=$(cat /app/build/cancelled_url)

POL_CREATED=$(jq -c . /tmp/created-policy.json)
POL_CANCELLED=$(jq -c . /tmp/cancelled-policy.json)

jq -n --arg p "$POL_CREATED" '{Policy:$p}' > /tmp/sqs-attrs-created.json
jq -n --arg p "$POL_CANCELLED" '{Policy:$p}' > /tmp/sqs-attrs-cancelled.json

aws sqs set-queue-attributes --queue-url "$CREATED_URL" --attributes file:///tmp/sqs-attrs-created.json
aws sqs set-queue-attributes --queue-url "$CANCELLED_URL" --attributes file:///tmp/sqs-attrs-cancelled.json

echo "verifying:"
aws sqs get-queue-attributes --queue-url "$CREATED_URL" --attribute-names Policy --query 'Attributes.Policy' --output text | jq -c .
aws sqs get-queue-attributes --queue-url "$CANCELLED_URL" --attribute-names Policy --query 'Attributes.Policy' --output text | jq -c .
output
verifying:
{"Version":"2012-10-17","Statement":[{"Sid":"AllowEBCreatedRule","Effect":"Allow","Principal":{"Service":"events.amazonaws.com"},"Action":"sqs:SendMessage","Resource":"arn:aws:sqs:us-east-1:000000000000:orders-created","Condition":{"ArnEquals":{"aws:SourceArn":"arn:aws:events:us-east-1:000000000000:rule/order-events/on-order-created"}}}]}
{"Version":"2012-10-17","Statement":[{"Sid":"AllowEBCancelledRule","Effect":"Allow","Principal":{"Service":"events.amazonaws.com"},"Action":"sqs:SendMessage","Resource":"arn:aws:sqs:us-east-1:000000000000:orders-cancelled","Condition":{"ArnEquals":{"aws:SourceArn":"arn:aws:events:us-east-1:000000000000:rule/order-events/on-order-cancelled"}}}]}

[stdout]
verifying:
{"Version":"2012-10-17","Statement":[{"Sid":"AllowEBCreatedRule","Effect":"Allow","Principal":{"Service":"events.amazonaws.com"},"Action":"sqs:SendMessage","Resource":"arn:aws:sqs:us-east-1:000000000000:orders-created","Condition":{"ArnEquals":{"aws:SourceArn":"arn:aws:events:us-east-1:000000000000:rule/order-events/on-order-created"}}}]}
{"Version":"2012-10-17","Statement":[{"Sid":"AllowEBCancelledRule","Effect":"Allow","Principal":{"Service":"events.amazonaws.com"},"Action":"sqs:SendMessage","Resource":"arn:aws:sqs:us-east-1:000000000000:orders-cancelled","Condition":{"ArnEquals":{"aws:SourceArn":"arn:aws:events:us-east-1:000000000000:rule/order-events/on-order-cancelled"}}}]}
  • Create CMK + alias outbox-cmk
  • Create DDB tables (orders, outbox w/ stream, processed_events w/ TTL)
  • Create EB bus order-events with CMK
  • Create SQS DLQs and main queues with CMK
  • Create EB rules with SQS targets + queue policies
  • Create scoped IAM roles+policies
  • Build & deploy lambdas
  • Create event source mapping w/ ReportBatchItemFailures
  • Smoke test the full flow
Now scoped IAM policies. The `order-api-role` only writes to `orders` and `outbox` tables (TransactWriteItems). The `outbox-relay-role` only reads the `outbox` stream, only writes to `processed_events`, and only puts events on `order-events`.
write · /app/policies/order-api-policy.json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Logs",
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "arn:aws:logs:us-east-1:000000000000:log-group:/aws/lambda/order-api*"
    },
    {
      "Sid": "OrdersAndOutboxTransactWrite",
      "Effect": "Allow",
      "Action": [
        "dynamodb:PutItem",
        "dynamodb:UpdateItem",
        "dynamodb:DescribeTable"
      ],
      "Resource": [
        "arn:aws:dynamodb:us-east-1:000000000000:table/orders",
        "arn:aws:dynamodb:us-east-1:000000000000:table/outbox"
      ]
    },
    {
      "Sid": "TransactWrite",
      "Effect": "Allow",
      "Action": "dynamodb:TransactWriteItems",
      "Resource": [
        "arn:aws:dynamodb:us-east-1:000000000000:table/orders",
        "arn:aws:dynamodb:us-east-1:000000000000:table/outbox"
      ]
    }
  ]
}

Write outbox-relay policy

shell
OUTBOX_STREAM_ARN=$(cat /app/build/outbox_stream_arn)
BUS_ARN=$(cat /app/build/bus_arn)

cat > /app/policies/outbox-relay-policy.json <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Logs",
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "arn:aws:logs:us-east-1:000000000000:log-group:/aws/lambda/outbox-relay*"
    },
    {
      "Sid": "ReadOutboxStreamOnly",
      "Effect": "Allow",
      "Action": [
        "dynamodb:DescribeStream",
        "dynamodb:GetRecords",
        "dynamodb:GetShardIterator",
        "dynamodb:ListStreams"
      ],
      "Resource": "$OUTBOX_STREAM_ARN"
    },
    {
      "Sid": "DedupTableWrite",
      "Effect": "Allow",
      "Action": [
        "dynamodb:PutItem",
        "dynamodb:GetItem",
        "dynamodb:DeleteItem"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:000000000000:table/processed_events"
    },
    {
      "Sid": "PublishToOrderEventsBus",
      "Effect": "Allow",
      "Action": "events:PutEvents",
      "Resource": "$BUS_ARN"
    }
  ]
}
EOF
cat /app/policies/outbox-relay-policy.json | jq .
output
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Logs",
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "arn:aws:logs:us-east-1:000000000000:log-group:/aws/lambda/outbox-relay*"
    },
    {
      "Sid": "ReadOutboxStreamOnly",
      "Effect": "Allow",
      "Action": [
        "dynamodb:DescribeStream",
        "dynamodb:GetRecords",
        "dynamodb:GetShardIterator",
        "dynamodb:ListStreams"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:000000000000:table/outbox/stream/2026-04-28T14:57:25.643"
    },
    {
      "Sid": "DedupTableWrite",
      "Effect": "Allow",
      "Action": [
        "dynamodb:PutItem",
        "dynamodb:GetItem",
        "dynamodb:DeleteItem"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:000000000000:table/processed_events"
    },
    {
      "Sid": "PublishToOrderEventsBus",
      "Effect": "Allow",
      "Action": "events:PutEvents",
      "Resource": "arn:aws:events:us-east-1:000000000000:event-bus/order-events"
    }
  ]
}

[stdout]
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Logs",
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "arn:aws:logs:us-east-1:000000000000:log-group:/aws/lambda/outbox-relay*"
    },
    {
      "Sid": "ReadOutboxStreamOnly",
      "Effect": "Allow",
      "Action": [
        "dynamodb:DescribeStream",
        "dynamodb:GetRecords",
        "dynamodb:GetShardIterator",
        "dynamodb:ListStreams"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:000000000000:table/outbox/stream/2026-04-28T14:57:25.643"
    },
    {
      "Sid": "DedupTableWrite",
      "Effect": "Allow",
      "Action": [
        "dynamodb:PutItem",
        "dynamodb:GetItem",
        "dynamodb:DeleteItem"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:000000000000:table/processed_events"
    },
    {
      "Sid": "PublishToOrderEventsBus",
      "Effect": "Allow",
      "Action": "events:PutEvents",
      "Resource": "arn:aws:events:us-east-1:000000000000:event-bus/order-events"
    }
  ]
}

Create roles + attach policies

shell
# Roles
aws iam create-role --role-name order-api-role \
  --assume-role-policy-document file:///app/policies/lambda-trust.json \
  --query 'Role.Arn' --output text

aws iam create-role --role-name outbox-relay-role \
  --assume-role-policy-document file:///app/policies/lambda-trust.json \
  --query 'Role.Arn' --output text

# Inline policies
aws iam put-role-policy --role-name order-api-role \
  --policy-name order-api-inline \
  --policy-document file:///app/policies/order-api-policy.json

aws iam put-role-policy --role-name outbox-relay-role \
  --policy-name outbox-relay-inline \
  --policy-document file:///app/policies/outbox-relay-policy.json

echo "policies attached:"
aws iam list-role-policies --role-name order-api-role --query 'PolicyNames' --output text
aws iam list-role-policies --role-name outbox-relay-role --query 'PolicyNames' --output text
output
arn:aws:iam::000000000000:role/order-api-role
arn:aws:iam::000000000000:role/outbox-relay-role
policies attached:
order-api-inline
outbox-relay-inline

[stdout]
arn:aws:iam::000000000000:role/order-api-role
arn:aws:iam::000000000000:role/outbox-relay-role
policies attached:
order-api-inline
outbox-relay-inline
  • Create CMK + alias outbox-cmk
  • Create DDB tables (orders, outbox w/ stream, processed_events w/ TTL)
  • Create EB bus order-events with CMK
  • Create SQS DLQs and main queues with CMK
  • Create EB rules with SQS targets + queue policies
  • Create scoped IAM roles+policies
  • Build & deploy lambdas
  • Create event source mapping w/ ReportBatchItemFailures
  • Smoke test the full flow

Zip lambda code

shell
cd /app/lambdas && zip -j /app/build/order-api.zip order_api.py && zip -j /app/build/outbox-relay.zip outbox_relay.py
ls -la /app/build/*.zip
output
adding: order_api.py (deflated 59%)
  adding: outbox_relay.py (deflated 62%)
-rw-r--r-- 1 root root  924 Apr 28 14:59 /app/build/order-api.zip
-rw-r--r-- 1 root root 1175 Apr 28 14:59 /app/build/outbox-relay.zip

[stdout]
  adding: order_api.py (deflated 59%)
  adding: outbox_relay.py (deflated 62%)
-rw-r--r-- 1 root root  924 Apr 28 14:59 /app/build/order-api.zip
-rw-r--r-- 1 root root 1175 Apr 28 14:59 /app/build/outbox-relay.zip

Create lambdas, wait for active

shell
API_ROLE_ARN=$(aws iam get-role --role-name order-api-role --query 'Role.Arn' --output text)
RELAY_ROLE_ARN=$(aws iam get-role --role-name outbox-relay-role --query 'Role.Arn' --output text)

aws lambda create-function \
  --function-name order-api \
  --runtime python3.11 \
  --handler order_api.handler \
  --role "$API_ROLE_ARN" \
  --zip-file fileb:///app/build/order-api.zip \
  --timeout 15 \
  --environment "Variables={LOCALSTACK_ENDPOINT=http://localstack:4566,ORDERS_TABLE=orders,OUTBOX_TABLE=outbox}" \
  --query 'FunctionArn' --output text

aws lambda create-function \
  --function-name outbox-relay \
  --runtime python3.11 \
  --handler outbox_relay.handler \
  --role "$RELAY_ROLE_ARN" \
  --zip-file fileb:///app/build/outbox-relay.zip \
  --timeout 30 \
  --environment "Variables={LOCALSTACK_ENDPOINT=http://localstack:4566,BUS_NAME=order-events,DEDUP_TABLE=processed_events,DEDUP_TTL_SECONDS=604800}" \
  --query 'FunctionArn' --output text

# wait until both Active
for fn in order-api outbox-relay; do
  for i in 1 2 3 4 5 6 7 8 9 10; do
    STATE=$(aws lambda get-function --function-name $fn --query 'Configuration.State' --output text)
    [ "$STATE" = "Active" ] && break
    echo "$fn state=$STATE; waiting..."
    sleep 1
  done
  echo "$fn final state: $(aws lambda get-function --function-name $fn --query 'Configuration.State' --output text)"
done
output
arn:aws:lambda:us-east-1:000000000000:function:order-api
arn:aws:lambda:us-east-1:000000000000:function:outbox-relay
order-api state=Pending; waiting...
order-api state=Pending; waiting...
order-api final state: Active
outbox-relay final state: Active

[stdout]
arn:aws:lambda:us-east-1:000000000000:function:order-api
arn:aws:lambda:us-east-1:000000000000:function:outbox-relay
order-api state=Pending; waiting...
order-api state=Pending; waiting...
order-api final state: Active
outbox-relay final state: Active
  • Create CMK + alias outbox-cmk
  • Create DDB tables (orders, outbox w/ stream, processed_events w/ TTL)
  • Create EB bus order-events with CMK
  • Create SQS DLQs and main queues with CMK
  • Create EB rules with SQS targets + queue policies
  • Create scoped IAM roles+policies
  • Build & deploy lambdas
  • Create event source mapping w/ ReportBatchItemFailures
  • Smoke test the full flow

Create event source mapping

shell
OUTBOX_STREAM_ARN=$(cat /app/build/outbox_stream_arn)

aws lambda create-event-source-mapping \
  --function-name outbox-relay \
  --event-source-arn "$OUTBOX_STREAM_ARN" \
  --starting-position LATEST \
  --batch-size 10 \
  --maximum-batching-window-in-seconds 1 \
  --function-response-types ReportBatchItemFailures \
  --query '{UUID:UUID,State:State,FunctionResponseTypes:FunctionResponseTypes}' --output json

# wait for ESM to enable
for i in 1 2 3 4 5 6 7 8 9 10; do
  STATE=$(aws lambda list-event-source-mappings --function-name outbox-relay --query 'EventSourceMappings[0].State' --output text)
  echo "ESM state=$STATE"
  [ "$STATE" = "Enabled" ] && break
  sleep 1
done
output
{
    "UUID": "d42eb0b6-f5a4-4fea-a171-daca91a01889",
    "State": "Creating",
    "FunctionResponseTypes": [
        "ReportBatchItemFailures"
    ]
}
ESM state=Enabled

[stdout]
{
    "UUID": "d42eb0b6-f5a4-4fea-a171-daca91a01889",
    "State": "Creating",
    "FunctionResponseTypes": [
        "ReportBatchItemFailures"
    ]
}
ESM state=Enabled
  • Create CMK + alias outbox-cmk
  • Create DDB tables (orders, outbox w/ stream, processed_events w/ TTL)
  • Create EB bus order-events with CMK
  • Create SQS DLQs and main queues with CMK
  • Create EB rules with SQS targets + queue policies
  • Create scoped IAM roles+policies
  • Build & deploy lambdas
  • Create event source mapping w/ ReportBatchItemFailures
  • Smoke test the full flow
Now the smoke test. Invoke order-api with a Created event, wait, check `orders-created`, check `processed_events`. Then the same with Cancelled.

Smoke test: OrderCreated

shell
CREATED_URL=$(cat /app/build/created_url)
CANCELLED_URL=$(cat /app/build/cancelled_url)

# Drain queues just in case
aws sqs purge-queue --queue-url "$CREATED_URL" 2>/dev/null
aws sqs purge-queue --queue-url "$CANCELLED_URL" 2>/dev/null

# Invoke 1
aws lambda invoke --function-name order-api \
  --cli-binary-format raw-in-base64-out \
  --payload '{"order_id":"o-1","kind":"OrderCreated","total":42.50,"customer":"alice"}' \
  /tmp/r1.json >/dev/null
echo "invoke 1 result:" && cat /tmp/r1.json && echo

EVENT_ID_1=$(jq -r '.event_id' /tmp/r1.json)
echo "EVENT_ID_1=$EVENT_ID_1"

# Wait up to 30s for the relay to publish
for i in $(seq 1 30); do
  N_CREATED=$(aws sqs get-queue-attributes --queue-url "$CREATED_URL" --attribute-names ApproximateNumberOfMessages --query 'Attributes.ApproximateNumberOfMessages' --output text)
  N_CANCELLED=$(aws sqs get-queue-attributes --queue-url "$CANCELLED_URL" --attribute-names ApproximateNumberOfMessages --query 'Attributes.ApproximateNumberOfMessages' --output text)
  echo "t=${i}s created=$N_CREATED cancelled=$N_CANCELLED"
  if [ "$N_CREATED" = "1" ]; then break; fi
  sleep 1
done
output
invoke 1 result:
{"statusCode": 200, "order_id": "o-1", "event_id": "934063af-8c42-42e8-98d4-dae523ca0f3a", "kind": "OrderCreated"}
EVENT_ID_1=934063af-8c42-42e8-98d4-dae523ca0f3a
t=1s created=0 cancelled=0
t=2s created=1 cancelled=0

[stdout]
invoke 1 result:
{"statusCode": 200, "order_id": "o-1", "event_id": "934063af-8c42-42e8-98d4-dae523ca0f3a", "kind": "OrderCreated"}
EVENT_ID_1=934063af-8c42-42e8-98d4-dae523ca0f3a
t=1s created=0 cancelled=0
t=2s created=1 cancelled=0
`OrderCreated` lands in 2 seconds. Verify the message body and dedup row.

Verify created queue + dedup row

shell
CREATED_URL=$(cat /app/build/created_url)
CANCELLED_URL=$(cat /app/build/cancelled_url)
EVENT_ID_1="934063af-8c42-42e8-98d4-dae523ca0f3a"

echo "--- orders-created msg ---"
aws sqs receive-message --queue-url "$CREATED_URL" --max-number-of-messages 10 --wait-time-seconds 1 --query 'Messages[*].Body' --output text | jq .

echo "--- orders-cancelled count (expect 0) ---"
aws sqs get-queue-attributes --queue-url "$CANCELLED_URL" --attribute-names ApproximateNumberOfMessages --query 'Attributes.ApproximateNumberOfMessages' --output text

echo "--- processed_events row for $EVENT_ID_1 ---"
aws dynamodb get-item --table-name processed_events --key "{\"event_id\":{\"S\":\"$EVENT_ID_1\"}}" --query 'Item' --output json
output
--- orders-created msg ---
{
  "version": "0",
  "id": "4afb2ed0-c4b7-4e62-9d74-969eeb7ae932",
  "detail-type": "OrderCreated",
  "source": "outbox.relay",
  "account": "000000000000",
  "time": "2026-04-28T15:00:24Z",
  "region": "us-east-1",
  "resources": [],
  "detail": {
    "order_id": "o-1",
    "kind": "OrderCreated",
    "total": 42.5,
    "customer": "alice",
    "event_id": "934063af-8c42-42e8-98d4-dae523ca0f3a"
  }
}
--- orders-cancelled count (expect 0) ---
0
--- processed_events row for 934063af-8c42-42e8-98d4-dae523ca0f3a ---
{
    "ttl": {
        "N": "1777993223"
    },
    "event_id": {
        "S": "934063af-8c42-42e8-98d4-dae523ca0f3a"
    }
}

[stdout]
--- orders-created msg ---
{
  "version": "0",
  "id": "4afb2ed0-c4b7-4e62-9d74-969eeb7ae932",
  "detail-type": "OrderCreated",
  "source": "outbox.relay",
  "account": "000000000000",
  "time": "2026-04-28T15:00:24Z",
  "region": "us-east-1",
  "resources": [],
  "detail": {
    "order_id": "o-1",
    "kind": "OrderCreated",
    "total": 42.5,
    "customer": "alice",
    "event_id": "934063af-8c42-42e8-98d4-dae523ca0f3a"
  }
}
--- orders-cancelled count (expect 0) ---
0
--- processed_events row for 934063af-8c42-42e8-98d4-dae523ca0f3a ---
{
    "ttl": {
        "N": "1777993223"
    },
    "event_id": {
        "S": "934063af-8c42-42e8-98d4-dae523ca0f3a"
    }
}
Created path works. Now test the Cancelled path with a different `order_id`.

Smoke test: OrderCancelled

shell
CREATED_URL=$(cat /app/build/created_url)
CANCELLED_URL=$(cat /app/build/cancelled_url)

# drain (we already received above; let it settle)
aws sqs purge-queue --queue-url "$CREATED_URL" 2>/dev/null
aws sqs purge-queue --queue-url "$CANCELLED_URL" 2>/dev/null
sleep 2

aws lambda invoke --function-name order-api \
  --cli-binary-format raw-in-base64-out \
  --payload '{"order_id":"o-2","kind":"OrderCancelled","reason":"customer_request"}' \
  /tmp/r2.json >/dev/null
cat /tmp/r2.json && echo
EVENT_ID_2=$(jq -r '.event_id' /tmp/r2.json)
echo "EVENT_ID_2=$EVENT_ID_2"

for i in $(seq 1 30); do
  N_CREATED=$(aws sqs get-queue-attributes --queue-url "$CREATED_URL" --attribute-names ApproximateNumberOfMessages --query 'Attributes.ApproximateNumberOfMessages' --output text)
  N_CANCELLED=$(aws sqs get-queue-attributes --queue-url "$CANCELLED_URL" --attribute-names ApproximateNumberOfMessages --query 'Attributes.ApproximateNumberOfMessages' --output text)
  echo "t=${i}s created=$N_CREATED cancelled=$N_CANCELLED"
  if [ "$N_CANCELLED" = "1" ]; then break; fi
  sleep 1
done

echo "--- orders-cancelled msg ---"
aws sqs receive-message --queue-url "$CANCELLED_URL" --max-number-of-messages 10 --wait-time-seconds 1 --query 'Messages[*].Body' --output text | jq .

echo "--- dedup row for cancelled ---"
aws dynamodb get-item --table-name processed_events --key "{\"event_id\":{\"S\":\"$EVENT_ID_2\"}}" --query 'Item' --output json
output
{"statusCode": 200, "order_id": "o-2", "event_id": "754585b8-7dfc-4ec8-a1ce-9e658b3c79c6", "kind": "OrderCancelled"}
EVENT_ID_2=754585b8-7dfc-4ec8-a1ce-9e658b3c79c6
t=1s created=0 cancelled=0
t=2s created=0 cancelled=1
--- orders-cancelled msg ---
{
  "version": "0",
  "id": "8816074f-d77b-41c4-8b7d-1aaf051a9cbd",
  "detail-type": "OrderCancelled",
  "source": "outbox.relay",
  "account": "000000000000",
  "time": "2026-04-28T15:00:48Z",
  "region": "us-east-1",
  "resources": [],
  "detail": {
    "order_id": "o-2",
    "kind": "OrderCancelled",
    "reason": "customer_request",
    "event_id": "754585b8-7dfc-4ec8-a1ce-9e658b3c79c6"
  }
}
--- dedup row for cancelled ---
{
    "ttl": {
        "N": "1777993247"
    },
    "event_id": {
        "S": "754585b8-7dfc-4ec8-a1ce-9e658b3c79c6"
    }
}

[stdout]
{"statusCode": 200, "order_id": "o-2", "event_id": "754585b8-7dfc-4ec8-a1ce-9e658b3c79c6", "kind": "OrderCancelled"}
EVENT_ID_2=754585b8-7dfc-4ec8-a1ce-9e658b3c79c6
t=1s created=0 cancelled=0
t=2s created=0 cancelled=1
--- orders-cancelled msg ---
{
  "version": "0",
  "id": "8816074f-d77b-41c4-8b7d-1aaf051a9cbd",
  "detail-type": "OrderCancelled",
  "source": "outbox.relay",
  "account": "000000000000",
  "time": "2026-04-28T15:00:48Z",
  "region": "us-east-1",
  "resources": [],
  "detail": {
    "order_id": "o-2",
    "kind": "OrderCancelled",
    "reason": "customer_request",
    "event_id": "754585b8-7dfc-4ec8-a1ce-9e658b3c79c6"
  }
}
--- dedup row for cancelled ---
{
    "ttl": {
        "N": "1777993247"
    },
    "event_id": {
        "S": "754585b8-7dfc-4ec8-a1ce-9e658b3c79c6"
    }
}
Both paths work cleanly: Created → only `orders-created`, Cancelled → only `orders-cancelled`, both `processed_events` rows present. Let me confirm a few non-obvious things the verifier will look at: the orders table has the row, the IAM policies are scoped (no `*` on KMS), the relay role can't touch the orders table.

Final verification dump

shell
echo "--- orders rows (atomic write proof) ---"
aws dynamodb scan --table-name orders --query 'Items' --output json
echo
echo "--- outbox rows ---"
aws dynamodb scan --table-name outbox --query 'Items' --output json
echo
echo "--- order-api role policy ---"
aws iam get-role-policy --role-name order-api-role --policy-name order-api-inline --query 'PolicyDocument' --output json
echo
echo "--- outbox-relay role policy ---"
aws iam get-role-policy --role-name outbox-relay-role --policy-name outbox-relay-inline --query 'PolicyDocument' --output json
echo
echo "--- ESM ReportBatchItemFailures ---"
aws lambda list-event-source-mappings --function-name outbox-relay --query 'EventSourceMappings[0].{State:State,FunctionResponseTypes:FunctionResponseTypes,EventSourceArn:EventSourceArn}' --output json
echo
echo "--- outbox stream view type ---"
aws dynamodb describe-table --table-name outbox --query 'Table.StreamSpecification' --output json
echo
echo "--- processed_events TTL ---"
aws dynamodb describe-time-to-live --table-name processed_events --query 'TimeToLiveDescription' --output json
echo
echo "--- KMS alias resolves ---"
aws kms describe-key --key-id alias/outbox-cmk --query '{Arn:KeyMetadata.Arn,KeyManager:KeyMetadata.KeyManager}' --output json
output
--- orders rows (atomic write proof) ---
[
    {
        "total": {
            "N": "42.5"
        },
        "updated_at": {
            "S": "2026-04-28T15:00:21.743895+00:00"
        },
        "order_id": {
            "S": "o-1"
        },
        "kind": {
            "S": "OrderCreated"
        },
        "customer": {
            "S": "alice"
        }
    },
    {
        "reason": {
            "S": "customer_request"
        },
        "updated_at": {
            "S": "2026-04-28T15:00:45.776803+00:00"
        },
        "order_id": {
            "S": "o-2"
        },
        "kind": {
            "S": "OrderCancelled"
        }
    }
]

--- outbox rows ---
[
    {
        "created_at": {
            "S": "2026-04-28T15:00:45.776803+00:00"
        },
        "event_id": {
            "S": "754585b8-7dfc-4ec8-a1ce-9e658b3c79c6"
        },
        "order_id": {
            "S": "o-2"
        },
        "payload": {
            "S": "{\"order_id\": \"o-2\", \"kind\": \"OrderCancelled\", \"reason\": \"customer_request\"}"
        },
        "kind": {
            "S": "OrderCancelled"
        }
    },
    {
        "created_at": {
            "S": "2026-04-28T15:00:21.743895+00:00"
        },
        "event_id": {
            "S": "934063af-8c42-42e8-98d4-dae523ca0f3a"
        },
        "order_id": {
            "S": "o-1"
        },
        "payload": {
            "S": "{\"order_id\": \"o-1\", \"kind\": \"OrderCreated\", \"total\": 42.5, \"customer\": \"alice\"}"
        },
        "kind": {
            "S": "OrderCreated"
        }
    }
]

--- order-api role policy ---
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Logs",
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "arn:aws:logs:us-east-1:000000000000:log-group:/aws/lambda/order-api*"
        },
        {
            "Sid": "OrdersAndOutboxTransactWrite",
            "Effect": "Allow",
            "Action": [
                "dynamodb:PutItem",
                "dynamodb:UpdateItem",
                "dynamodb:DescribeTable"
            ],
            "Resource": [
                "arn:aws:dynamodb:us-east-1:000000000000:table/orders",
                "arn:aws:dynamodb:us-east-1:000000000000:table/outbox"
            ]
        },
        {
            "Sid": "TransactWrite",
            "Effect": "Allow",
            "Action": "dynamodb:TransactWriteItems",
            "Resource": [
                "arn:aws:dynamodb:us-east-1:000000000000:table/orders",
                "arn:aws:dynamodb:us-east-1:000000000000:table/outbox"
            ]
        }
    ]
}

--- outbox-relay role policy ---
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Logs",
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "arn:aws:logs:us-east-1:000000000000:log-group:/aws/lambda/outbox-relay*"
        },
        {
            "Sid": "ReadOutboxStreamOnly",
            "Effect": "Allow",
            "Action": [
                "dynamodb:DescribeStream",
                "dynamodb:GetRecords",
                "dynamodb:GetShardIterator",
                "dynamodb:ListStreams"
            ],
            "Resource": "arn:aws:dynamodb:us-east-1:000000000000:table/outbox/stream/2026-04-28T14:57:25.643"
        },
        {
            "Sid": "DedupTableWrite",
            "Effect": "Allow",
            "Action": [
                "dynamodb:PutItem",
                "dynamodb:GetItem",
                "dynamodb:DeleteItem"
            ],
            "Resource": "arn:aws:dynamodb:us-east-1:000000000000:table/processed_events"
        },
        {
            "Sid": "PublishToOrderEventsBus",
            "Effect": "Allow",
            "Action": "events:PutEvents",
            "Resource": "arn:aws:events:us-east-1:000000000000:event-bus/order-events"
        }
    ]
}

--- ESM ReportBatchItemFailures ---
{
    "State": "Enabled",
    "FunctionResponseTypes": [
        "ReportBatchItemFailures"
    ],
    "EventSourceArn": "arn:aws:dynamodb:us-east-1:000000000000:table/outbox/stream/2026-04-28T14:57:25.643"
}

--- outbox stream view type ---
{
    "StreamEnabled": true,
    "StreamViewType": "NEW_AND_OLD_IMAGES"
}

--- processed_events TTL ---
{
    "TimeToLiveStatus": "ENABLED",
    "AttributeName": "ttl"
}

--- KMS alias resolves ---
{
    "Arn": "arn:aws:kms:us-east-1:000000000000:key/d1e5c8d8-871d-4ae3-bfc5-890beb11dae3",
    "KeyManager": "CUSTOMER"
}

[stdout]
--- orders rows (atomic write proof) ---
[
    {
        "total": {
            "N": "42.5"
        },
        "updated_at": {
            "S": "2026-04-28T15:00:21.743895+00:00"
        },
        "order_id": {
            "S": "o-1"
        },
        "kind": {
            "S": "OrderCreated"
        },
        "customer": {
            "S": "alice"
        }
    },
    {
        "reason": {
            "S": "customer_request"
        },
        "updated_at": {
            "S": "2026-04-28T15:00:45.776803+00:00"
        },
        "order_id": {
            "S": "o-2"
        },
        "kind": {
            "S": "OrderCancelled"
        }
    }
]

--- outbox rows ---
[
    {
        "created_at": {
            "S": "2026-04-28T15:00:45.776803+00:00"
        },
        "event_id": {
            "S": "754585b8-7dfc-4ec8-a1ce-9e658b3c79c6"
        },
        "order_id": {
            "S": "o-2"
        },
        "payload": {
            "S": "{\"order_id\": \"o-2\", \"kind\": \"OrderCancelled\", \"reason\": \"customer_request\"}"
        },
        "kind": {
            "S": "OrderCancelled"
        }
    },
    {
        "created_at": {
            "S": "2026-04-28T15:00:21.743895+00:00"
        },
        "event_id": {
            "S": "934063af-8c42-42e8-98d4-dae523ca0f3a"
        },
        "order_id": {
            "S": "o-1"
        },
        "payload": {
            "S": "{\"order_id\": \"o-1\", \"kind\": \"OrderCreated\", \"total\": 42.5, \"customer\": \"alice\"}"
        },
        "kind": {
            "S": "OrderCreated"
        }
    }
]

--- order-api role policy ---
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Logs",
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "arn:aws:logs:us-east-1:000000000000:log-group:/aws/lambda/order-api*"
        },
        {
            "Sid": "OrdersAndOutboxTransactWrite",
            "Effect": "Allow",
            "Action": [
                "dynamodb:PutItem",
                "dynamodb:UpdateItem",
                "dynamodb:DescribeTable"
            ],
            "Resource": [
                "arn:aws:dynamodb:us-east-1:000000000000:table/orders",
                "arn:aws:dynamodb:us-east-1:000000000000:table/outbox"
            ]
        },
        {
            "Sid": "TransactWrite",
            "Effect": "Allow",
            "Action": "dynamodb:TransactWriteItems",
            "Resource": [
                "arn:aws:dynamodb:us-east-1:000000000000:table/orders",
                "arn:aws:dynamodb:us-east-1:000000000000:table/outbox"
            ]
        }
    ]
}

--- outbox-relay role policy ---
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Logs",
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "arn:aws:logs:us-east-1:000000000000:log-group:/aws/lambda/outbox-relay*"
        },
        {
            "Sid": "ReadOutboxStreamOnly",
            "Effect": "Allow",
            "Action": [
                "dynamodb:DescribeStream",
                "dynamodb:GetRecords",
                "dynamodb:GetShardIterator",
                "dynamodb:ListStreams"
            ],
            "Resource": "arn:aws:dynamodb:us-east-1:000000000000:table/outbox/stream/2026-04-28T14:57:25.643"
        },
        {
            "Sid": "DedupTableWrite",
            "Effect": "Allow",
            "Action": [
                "dynamodb:PutItem",
                "dynamodb:GetItem",
                "dynamodb:DeleteItem"
            ],
            "Resource": "arn:aws:dynamodb:us-east-1:000000000000:table/processed_events"
        },
        {
            "Sid": "PublishToOrderEventsBus",
            "Effect": "Allow",
            "Action": "events:PutEvents",
            "Resource": "arn:aws:events:us-east-1:000000000000:event-bus/order-events"
        }
    ]
}

--- ESM ReportBatchItemFailures ---
{
    "State": "Enabled",
    "FunctionResponseTypes": [
        "ReportBatchItemFailures"
    ],
    "EventSourceArn": "arn:aws:dynamodb:us-east-1:000000000000:table/outbox/stream/2026-04-28T14:57:25.643"
}

--- outbox stream view type ---
{
    "StreamEnabled": true,
    "StreamViewType": "NEW_AND_OLD_IMAGES"
}

--- processed_events TTL ---
{
    "TimeToLiveStatus": "ENABLED",
    "AttributeName": "ttl"
}

--- KMS alias resolves ---
{
    "Arn": "arn:aws:kms:us-east-1:000000000000:key/d1e5c8d8-871d-4ae3-bfc5-890beb11dae3",
    "KeyManager": "CUSTOMER"
}
  • Create CMK + alias outbox-cmk
  • Create DDB tables (orders, outbox w/ stream, processed_events w/ TTL)
  • Create EB bus order-events with CMK
  • Create SQS DLQs and main queues with CMK
  • Create EB rules with SQS targets + queue policies
  • Create scoped IAM roles+policies
  • Build & deploy lambdas
  • Create event source mapping w/ ReportBatchItemFailures
  • Smoke test the full flow
Done. The whole thing is up and routes correctly: - **`order-api`** (one transaction, two rows): `TransactWriteItems` writes both `orders` and `outbox` atomically. Uses uuid4 as the `event_id`, embedded in the outbox row so the relay sees the same id. - **Outbox stream → relay**: `outbox` has `NEW_AND_OLD_IMAGES`. `outbox-relay`'s ESM is wired with `FunctionResponseTypes=ReportBatchItemFailures` and the handler returns `{batchItemFailures: [{itemIdentifier: <SequenceNumber>}, ...]}` per failed record. - **Idempotency**: relay claims `event_id` via conditional `PutItem` on `processed_events` (with a 7-day `ttl` attribute, TTL enabled). On EB publish failure it deletes the claim so the retry can republish. - **EB bus → routing**: rules `on-order-created` / `on-order-cancelled` filter on `source = "outbox.relay"` and `detail-type = "OrderCreated"|"OrderCancelled"`. Each queue has its own DLQ (`maxReceiveCount=5`); no shared DLQ. - **CMK everywhere**: `alias/outbox-cmk` is on the three DDB tables, all four SQS queues, and the EB bus. Key policy delegates to root + `dynamodb`/`sqs`/`events` services so EB can deliver into encrypted queues. - **Least privilege**: - `order-api-role`: `TransactWriteItems` + `PutItem`/`UpdateItem`/`DescribeTable` only on `orders` and `outbox`. No SQS, no streams, no `events:*`, no `kms:*`. - `outbox-relay-role`: stream perms scoped to the **outbox** stream ARN only (orders has no stream and is unreachable), `Put/Get/DeleteItem` only on `processed_events`, `events:PutEvents` only on the `order-events` bus. No `kms:*`. Smoke test: - `o-1`/`OrderCreated` → 1 message in `orders-created`, 0 in `orders-cancelled`, dedup row `934063af…` present, ~2s end-to-end. - `o-2`/`OrderCancelled` → 1 message in `orders-cancelled`, 0 in `orders-created`, dedup row `754585b8…` present, ~2s end-to-end. Sources at `/app/lambdas/`, IAM/KMS docs at `/app/policies/`, deployment artifacts (zips + saved ARNs) at `/app/build/`.
Final diff , what the agent changed

No file changes, the agent left the workspace untouched.

Verifier output , the proof behind the reward
tests/test.sh
============================= test session starts ==============================
platform linux -- Python 3.12.3, pytest-8.4.1, pluggy-1.6.0 -- /root/.cache/uv/archive-v0/A8zCpMMJJkqPh2g2RPN5f/bin/python
cachedir: .pytest_cache
rootdir: /tests
plugins: json-ctrf-0.3.5
collecting ... collected 20 items

../tests/test_outputs.py::test_01_cmk_alias_exists_and_enabled PASSED    [  5%]
../tests/test_outputs.py::test_02_orders_table_exists PASSED             [ 10%]
../tests/test_outputs.py::test_03_outbox_table_exists PASSED             [ 15%]
../tests/test_outputs.py::test_04_processed_events_table_exists PASSED   [ 20%]
../tests/test_outputs.py::test_05_custom_event_bus_exists PASSED         [ 25%]
../tests/test_outputs.py::test_06_orders_table_encrypted_with_cmk FAILED [ 30%]
../tests/test_outputs.py::test_07_outbox_table_encrypted_with_cmk FAILED [ 35%]
../tests/test_outputs.py::test_08_outbox_stream_is_new_and_old_images PASSED [ 40%]
../tests/test_outputs.py::test_09_processed_events_ttl_enabled PASSED    [ 45%]
../tests/test_outputs.py::test_10_both_rule_patterns_use_array_wrap_and_disjoint_types PASSED [ 50%]
../tests/test_outputs.py::test_11_each_rule_targets_only_its_intended_queue PASSED [ 55%]
../tests/test_outputs.py::test_12_both_main_queues_encrypted_with_cmk PASSED [ 60%]
../tests/test_outputs.py::test_13_main_queues_have_redrive_to_distinct_dlqs PASSED [ 65%]
../tests/test_outputs.py::test_14_queue_policies_scope_events_with_source_arn PASSED [ 70%]
../tests/test_outputs.py::test_15_relay_esm_has_batch_item_failures_on_outbox_stream PASSED [ 75%]
../tests/test_outputs.py::test_16_relay_role_is_scoped_not_wildcard FAILED [ 80%]
../tests/test_outputs.py::test_17_order_api_role_scoped_to_orders_and_outbox_only PASSED [ 85%]
../tests/test_outputs.py::test_18_e2e_order_created_routes_to_created_queue_only PASSED [ 90%]
../tests/test_outputs.py::test_19_e2e_order_cancelled_routes_to_cancelled_queue_only PASSED [ 95%]
../tests/test_outputs.py::test_20_e2e_idempotent_duplicate_does_not_double_fanout PASSED [100%]

=================================== FAILURES ===================================
___________________ test_06_orders_table_encrypted_with_cmk ____________________

    def test_06_orders_table_encrypted_with_cmk():
        ddb = _client("dynamodb")
        cmk_arn, cmk_id = _cmk_arn()
        t = ddb.describe_table(TableName=ORDERS_TABLE)["Table"]
        sse = t.get("SSEDescription") or {}
        assert sse.get("Status") == "ENABLED", "SSE must be enabled"
        assert sse.get("SSEType") == "KMS", "SSE must be KMS, not default"
        used = sse.get("KMSMasterKeyArn", "")
>       assert cmk_id in used or cmk_arn == used, (
            f"orders table must use CMK {cmk_id}, got {used}"
        )
E       AssertionError: orders table must use CMK d1e5c8d8-871d-4ae3-bfc5-890beb11dae3, got arn:aws:kms:us-east-1:000000000000:key/alias/outbox-cmk
E       assert ('d1e5c8d8-871d-4ae3-bfc5-890beb11dae3' in 'arn:aws:kms:us-east-1:000000000000:key/alias/outbox-cmk' or 'arn:aws:kms:...-890beb11dae3' == 'arn:aws:kms:...as/outbox-cmk'
E         
E         - arn:aws:kms:us-east-1:000000000000:key/alias/outbox-cmk
E         + arn:aws:kms:us-east-1:000000000000:key/d1e5c8d8-871d-4ae3-bfc5-890beb11dae3)

/tests/test_outputs.py:228: AssertionError
___________________ test_07_outbox_table_encrypted_with_cmk ____________________

    def test_07_outbox_table_encrypted_with_cmk():
        ddb = _client("dynamodb")
        cmk_arn, cmk_id = _cmk_arn()
        t = ddb.describe_table(TableName=OUTBOX_TABLE)["Table"]
        sse = t.get("SSEDescription") or {}
        assert sse.get("Status") == "ENABLED"
        assert sse.get("SSEType") == "KMS"
        used = sse.get("KMSMasterKeyArn", "")
>       assert cmk_id in used or cmk_arn == used
E       AssertionError: assert ('d1e5c8d8-871d-4ae3-bfc5-890beb11dae3' in 'arn:aws:kms:us-east-1:000000000000:key/alias/outbox-cmk' or 'arn:aws:kms:...-890beb11dae3' == 'arn:aws:kms:...as/outbox-cmk'
E         
E         - arn:aws:kms:us-east-1:000000000000:key/alias/outbox-cmk
E         + arn:aws:kms:us-east-1:000000000000:key/d1e5c8d8-871d-4ae3-bfc5-890beb11dae3)

/tests/test_outputs.py:241: AssertionError
__________________ test_16_relay_role_is_scoped_not_wildcard ___________________

    def test_16_relay_role_is_scoped_not_wildcard():
        """outbox-relay role must NOT have '*' on Resource for dynamodb stream
        actions or for kms:Decrypt. These actions must be scoped to the outbox
        stream ARN and to the CMK respectively."""
        ddb = _client("dynamodb")
        stream_arn = ddb.describe_table(TableName=OUTBOX_TABLE)["Table"]["LatestStreamArn"]
        cmk_arn, cmk_id = _cmk_arn()
    
        docs = _get_role_policies(RELAY_ROLE)
        stmts = _statements(docs)
    
        for s in stmts:
            if s.get("Effect") != "Allow":
                continue
            actions = _as_list(s.get("Action"))
            resources = _as_list(s.get("Resource"))
            if any(a.startswith("dynamodb:") or a == "*" for a in actions):
                for r in resources:
                    if r == "*":
                        if any(a in ("dynamodb:GetRecords", "dynamodb:GetShardIterator",
                                     "dynamodb:DescribeStream", "dynamodb:ListStreams",
                                     "*") for a in actions):
                            pytest.fail(
                                f"relay role wildcards dynamodb stream actions on '*': "
                                f"actions={actions}"
                            )
    
        def stream_resource_ok(r):
            return isinstance(r, str) and OUTBOX_TABLE in r and ("stream" in r.lower() or "/stream/" in r)
        assert _allows(stmts, "dynamodb:GetRecords", stream_resource_ok) or _allows(
            stmts, "dynamodb:GetShardIterator", stream_resource_ok
        ), "relay role must allow dynamodb stream actions on the outbox stream ARN"
    
        for s in stmts:
            if s.get("Effect") != "Allow":
                continue
            actions = _as_list(s.get("Action"))
            if "kms:Decrypt" in actions or "kms:*" in actions or "*" in actions:
                resources = _as_list(s.get("Resource"))
                for r in resources:
                    if r == "*":
                        pytest.fail(
                            f"relay role wildcards kms:Decrypt on Resource '*' - "
                            f"must scope to CMK {cmk_id}"
                        )
    
        def cmk_resource_ok(r):
            return isinstance(r, str) and (cmk_id in r or KEY_ALIAS in r or r == cmk_arn)
>       assert _allows(stmts, "kms:Decrypt", cmk_resource_ok), (
            f"relay role must allow kms:Decrypt on the CMK ({cmk_id})"
        )
E       AssertionError: relay role must allow kms:Decrypt on the CMK (d1e5c8d8-871d-4ae3-bfc5-890beb11dae3)
E       assert False
E        +  where False = _allows([{'Action': ['logs:CreateLogGroup', 'logs:CreateLogStream', 'logs:PutLogEvents'], 'Effect': 'Allow', 'Resource': 'arn:aws:logs:us-east-1:000000000000:log-group:/aws/lambda/outbox-relay*', 'Sid': 'Logs'}, {'Action': ['dynamodb:DescribeStream', 'dynamodb:GetRecords', 'dynamodb:GetShardIterator', 'dynamodb:ListStreams'], 'Effect': 'Allow', 'Resource': 'arn:aws:dynamodb:us-east-1:000000000000:table/outbox/stream/2026-04-28T14:57:25.643', 'Sid': 'ReadOutboxStreamOnly'}, {'Action': ['dynamodb:PutItem', 'dynamodb:GetItem', 'dynamodb:DeleteItem'], 'Effect': 'Allow', 'Resource': 'arn:aws:dynamodb:us-east-1:000000000000:table/processed_events', 'Sid': 'DedupTableWrite'}, {'Action': 'events:PutEvents', 'Effect': 'Allow', 'Resource': 'arn:aws:events:us-east-1:000000000000:event-bus/order-events', 'Sid': 'PublishToOrderEventsBus'}], 'kms:Decrypt', <function test_16_relay_role_is_scoped_not_wildcard.<locals>.cmk_resource_ok at 0xffff7d5336a0>)

/tests/test_outputs.py:482: AssertionError
=============================== warnings summary ===============================
test_outputs.py: 91 warnings
  /root/.cache/uv/archive-v0/A8zCpMMJJkqPh2g2RPN5f/lib/python3.12/site-packages/botocore/auth.py:424: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
    datetime_now = datetime.datetime.utcnow()

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
==================================== PASSES ====================================
=========================== short test summary info ============================
PASSED ../tests/test_outputs.py::test_01_cmk_alias_exists_and_enabled
PASSED ../tests/test_outputs.py::test_02_orders_table_exists
PASSED ../tests/test_outputs.py::test_03_outbox_table_exists
PASSED ../tests/test_outputs.py::test_04_processed_events_table_exists
PASSED ../tests/test_outputs.py::test_05_custom_event_bus_exists
PASSED ../tests/test_outputs.py::test_08_outbox_stream_is_new_and_old_images
PASSED ../tests/test_outputs.py::test_09_processed_events_ttl_enabled
PASSED ../tests/test_outputs.py::test_10_both_rule_patterns_use_array_wrap_and_disjoint_types
PASSED ../tests/test_outputs.py::test_11_each_rule_targets_only_its_intended_queue
PASSED ../tests/test_outputs.py::test_12_both_main_queues_encrypted_with_cmk
PASSED ../tests/test_outputs.py::test_13_main_queues_have_redrive_to_distinct_dlqs
PASSED ../tests/test_outputs.py::test_14_queue_policies_scope_events_with_source_arn
PASSED ../tests/test_outputs.py::test_15_relay_esm_has_batch_item_failures_on_outbox_stream
PASSED ../tests/test_outputs.py::test_17_order_api_role_scoped_to_orders_and_outbox_only
PASSED ../tests/test_outputs.py::test_18_e2e_order_created_routes_to_created_queue_only
PASSED ../tests/test_outputs.py::test_19_e2e_order_cancelled_routes_to_cancelled_queue_only
PASSED ../tests/test_outputs.py::test_20_e2e_idempotent_duplicate_does_not_double_fanout
FAILED ../tests/test_outputs.py::test_06_orders_table_encrypted_with_cmk - As...
FAILED ../tests/test_outputs.py::test_07_outbox_table_encrypted_with_cmk - As...
FAILED ../tests/test_outputs.py::test_16_relay_role_is_scoped_not_wildcard - ...
================== 3 failed, 17 passed, 91 warnings in 22.42s ==================

Reproduce this trial: git checkout 2f94510 && PYTHONPATH=src python3 scripts/build_site.py , then open trial/trial_964eb40653844d05. Re-running the agent live requires EVAL_PLATFORM_ENABLE_OAUTH_SMOKE=1 and is non-deterministic.

Trial trial_964eb40653844d05 · verifier authoritative; classifier explanatory.