SyncVals · Trajectory

sfn-saga-compensation-orchestrator

claude-code claude-opus-4-7 ✓ resolved GOOD_SUCCESS ↑ View task

Solved from the instruction alone, tests/ and solution/ were withheld from the agent's workspace and restored only for grading.

Reward = tests/test.sh exit code (0 → resolved); the classification below is post-hoc and cannot change it.

Classification , post-hoc; cannot change the reward

GOOD_SUCCESSLegitimate solve, implements the asked-for behavior; tests verify real functionality.

SubtypeLegitimate Solution

EvidenceAll 20 tests passed including: test_06_every_forward_state_has_catch_to_compensation, test_07_compensations_run_in_lifo_order, test_10_each_lambda_role_scoped_to_its_own_table, test_18_happy_path_completes_and_writes_all_three_tables, test_19_failure_path_triggers_compensation_and_rolls_back_inventory, test_20_idempotency_prevents_double_reservation. Agent correctly implemented saga pattern with 6 Lambda functions, 4 DynamoDB tables, state machine with proper Catch blocks routing to LIFO compensations, EventBridge routing, KMS encryption, and demonstrated idempotency and rollback semantics.

Root causeThe agent successfully implemented a complex distributed systems architecture matching all instruction requirements: saga orchestration with forward/compensation steps, least-privilege IAM roles, Secrets Manager with customer-managed KMS encryption, idempotency tables with TTL, and event-driven notifications. The solution demonstrates genuine understanding of AWS services and distributed compensation patterns, not hardcoding or pattern matching.

RecommendationN/A - task is fine

Trajectory

Tool-by-tool agent trajectory

20 tool calls · 4 tool types · 30 steps

so the order service keeps leaving garbage rows. inventory gets reserved, payment call times out, we retry, now inventory is double-reserved because the retry worked the second time. same deal when shipping fails after payment , we'd just charge the customer and leave them with no shipment. the ops slack is basically me apologizing at this point. we need a real saga. if any step fails, **undo** the earlier steps in reverse order. no half-states. step functions runs it, localstack , `http://localstack:4566`, creds already exported (`AWS_ACCESS_KEY_ID=test`, same for secret, region `us-east-1`). `aws`, `python3`, `boto3`, `jq`, `zip`. build from zero. shape of it: - a state machine runs three forward steps: **reserve inventory**, **charge payment**, **create shipment**. each step is its own lambda. each step writes to its own ddb table. - if any step fails, the saga fires compensations **in reverse order** of what actually ran. if reserve succeeded but charge failed, you refund (no-op, nothing was charged) and release. if charge succeeded but shipment failed, you refund payment and release inventory. if reserve failed, nothing to undo. - compensations are their own lambdas too. they must be idempotent , if state-function retries a compensation, running it twice should not corrupt the table. - each forward step gets an idempotency key from the input so a lambda retry inside a single step doesn't double-decrement inventory. - payment calls a "gateway" that needs an api key. store the key in secrets manager , customer-managed kms key encrypts the secret. the payment lambda reads the secret, nobody else should be able to. - on terminal state (completed or failed-compensated), the saga publishes an event to a custom event bus. a notifier lambda sits on that bus and writes one log line starting with `saga-terminal:`. to actually prove compensations work, the charge-payment lambda accepts a `force_failure` flag in its input , when true, it raises. the verifier uses this to drive a failure run and assert the state snaps back. done looks like this. two executions: **happy path** , `start-execution` with `{"order_id": "o-1", "sku": "x", "qty": 2, "amount": 100, "idempotency_key": "k-1"}`: - sm reaches `SUCCEEDED` within 60s - inventory row for sku `x` reflects `reserved_qty = 2` - payments row exists with status `CAPTURED` - shipments row exists - notifier log has `saga-terminal:OrderCompleted` **failure path** , same input but `"force_failure": true, "order_id": "o-2", "idempotency_key": "k-2"`: - sm still reaches `SUCCEEDED` (saga handles it gracefully, not a hard abort) - inventory reserved_qty back to 0 (compensation ran) - payments row is absent or `REFUNDED` (nothing was charged since charge failed, so a no-op refund is also ok) - shipments row does **not** exist - notifier log has `saga-terminal:OrderFailed` least privilege. the verifier reads policy docs directly. each lambda role only gets its own table. saga role can't wildcard on lambda invoke. payment role is the only one that reads the secret. notifier role has nothing but eventbridge + logs. no `*` on kms anywhere. names the verifier grabs by hand, use them exactly: | thing | name | |-|-| | kms alias | `alias/saga-cmk` | | secret | `payment/gateway-apikey` | | inventory table | `inventory` (pk `sku` / String) | | payments table | `payments` (pk `payment_id` / String) | | shipments table | `shipments` (pk `shipment_id` / String) | | idempotency table | `saga-idempotency` (pk `idempotency_key` / String, ttl enabled) | | reserve lambda | `reserve-inventory` | | charge lambda | `charge-payment` | | ship lambda | `create-shipment` | | release compensation | `release-inventory` | | refund compensation | `refund-payment` | | notifier lambda | `saga-notifier` | | reserve role | `reserve-inventory-role` | | charge role | `charge-payment-role` | | ship role | `create-shipment-role` | | release role | `release-inventory-role` | | refund role | `refund-payment-role` | | notifier role | `saga-notifier-role` | | state machine | `order-saga` | | state machine role | `order-saga-role` | | event bus | `saga-events` | | event rule | `on-saga-terminal` | eventbridge detail shape: source `order.saga`, detail-type is one of `OrderCompleted` or `OrderFailed`. the rule matches both and routes to the notifier lambda. python3.11 for all lambdas. localstack community 4.13.1. the "payment gateway" is fake , charge-payment reads the secret, logs that it did, and marks the payments row CAPTURED. no actual network call.

I'll build this saga system step by step: KMS/Secrets, DynamoDB tables, Lambda code + least-privilege roles, Step Functions state machine, EventBridge wiring, then a verifier. Let me first confirm the LocalStack environment is reachable and lay out the directory.

Check app dir and LocalStack

shell

ls /app && aws --endpoint-url=http://localstack:4566 sts get-caller-identity 2>&1 | head -20

output

{
    "UserId": "AKIAIOSFODNN7EXAMPLE",
    "Account": "000000000000",
    "Arn": "arn:aws:iam::000000000000:root"
}

[stdout]
{
    "UserId": "AKIAIOSFODNN7EXAMPLE",
    "Account": "000000000000",
    "Arn": "arn:aws:iam::000000000000:root"
}

LocalStack is reachable. Let me write the Lambda sources, IAM policies, state machine, and deployment.

Create directory layout

shell

mkdir -p /app/lambdas /app/build /app/policies && ls /app

output

build
lambdas
policies

[stdout]
build
lambdas
policies

write · /app/lambdas/reserve_inventory.py

import os
import time
import boto3
from botocore.exceptions import ClientError

ENDPOINT = os.environ.get("AWS_ENDPOINT_URL", "http://localstack:4566")
REGION = os.environ.get("AWS_REGION", "us-east-1")

ddb = boto3.client("dynamodb", endpoint_url=ENDPOINT, region_name=REGION)

INV_TABLE = "inventory"
IDEM_TABLE = "saga-idempotency"


def handler(event, _context):
    sku = event["sku"]
    qty = int(event["qty"])
    base_key = event["idempotency_key"]
    idem_key = f"{base_key}:reserve"
    ttl = int(time.time()) + 3600

    try:
        ddb.transact_write_items(
            TransactItems=[
                {
                    "Put": {
                        "TableName": IDEM_TABLE,
                        "Item": {
                            "idempotency_key": {"S": idem_key},
                            "status": {"S": "DONE"},
                            "sku": {"S": sku},
                            "qty": {"N": str(qty)},
                            "ttl": {"N": str(ttl)},
                        },
                        "ConditionExpression": "attribute_not_exists(idempotency_key)",
                    }
                },
                {
                    "Update": {
                        "TableName": INV_TABLE,
                        "Key": {"sku": {"S": sku}},
                        "UpdateExpression": "ADD reserved_qty :q",
                        "ExpressionAttributeValues": {":q": {"N": str(qty)}},
                    }
                },
            ]
        )
        return {"reserved": True, "sku": sku, "qty": qty, "idempotency_key": idem_key}
    except ClientError as e:
        code = e.response.get("Error", {}).get("Code", "")
        if code in ("TransactionCanceledException", "ConditionalCheckFailedException"):
            return {"reserved": True, "sku": sku, "qty": qty, "idempotency_key": idem_key, "replayed": True}
        raise

write · /app/lambdas/charge_payment.py

import os
import time
import uuid
import json
import boto3
from botocore.exceptions import ClientError

ENDPOINT = os.environ.get("AWS_ENDPOINT_URL", "http://localstack:4566")
REGION = os.environ.get("AWS_REGION", "us-east-1")

ddb = boto3.client("dynamodb", endpoint_url=ENDPOINT, region_name=REGION)
sm = boto3.client("secretsmanager", endpoint_url=ENDPOINT, region_name=REGION)

PAY_TABLE = "payments"
IDEM_TABLE = "saga-idempotency"
SECRET_ID = "payment/gateway-apikey"


def handler(event, _context):
    if event.get("force_failure"):
        raise RuntimeError("charge_payment forced failure")

    order_id = event["order_id"]
    amount = int(event["amount"])
    base_key = event["idempotency_key"]
    idem_key = f"{base_key}:charge"
    ttl = int(time.time()) + 3600

    secret = sm.get_secret_value(SecretId=SECRET_ID)
    api_key_present = bool(secret.get("SecretString"))
    print(f"charge-payment: loaded gateway api key (present={api_key_present})")

    payment_id = f"pay-{base_key}"

    try:
        ddb.transact_write_items(
            TransactItems=[
                {
                    "Put": {
                        "TableName": IDEM_TABLE,
                        "Item": {
                            "idempotency_key": {"S": idem_key},
                            "status": {"S": "DONE"},
                            "payment_id": {"S": payment_id},
                            "amount": {"N": str(amount)},
                            "ttl": {"N": str(ttl)},
                        },
                        "ConditionExpression": "attribute_not_exists(idempotency_key)",
                    }
                },
                {
                    "Put": {
                        "TableName": PAY_TABLE,
                        "Item": {
                            "payment_id": {"S": payment_id},
                            "order_id": {"S": order_id},
                            "amount": {"N": str(amount)},
                            "status": {"S": "CAPTURED"},
                        },
                    }
                },
            ]
        )
        return {"captured": True, "payment_id": payment_id, "amount": amount}
    except ClientError as e:
        code = e.response.get("Error", {}).get("Code", "")
        if code in ("TransactionCanceledException", "ConditionalCheckFailedException"):
            return {"captured": True, "payment_id": payment_id, "amount": amount, "replayed": True}
        raise

write · /app/lambdas/create_shipment.py

import os
import time
import boto3
from botocore.exceptions import ClientError

ENDPOINT = os.environ.get("AWS_ENDPOINT_URL", "http://localstack:4566")
REGION = os.environ.get("AWS_REGION", "us-east-1")

ddb = boto3.client("dynamodb", endpoint_url=ENDPOINT, region_name=REGION)

SHIP_TABLE = "shipments"
IDEM_TABLE = "saga-idempotency"


def handler(event, _context):
    if event.get("force_shipment_failure"):
        raise RuntimeError("create_shipment forced failure")

    order_id = event["order_id"]
    sku = event["sku"]
    qty = int(event["qty"])
    base_key = event["idempotency_key"]
    idem_key = f"{base_key}:ship"
    shipment_id = f"shp-{base_key}"
    ttl = int(time.time()) + 3600

    try:
        ddb.transact_write_items(
            TransactItems=[
                {
                    "Put": {
                        "TableName": IDEM_TABLE,
                        "Item": {
                            "idempotency_key": {"S": idem_key},
                            "status": {"S": "DONE"},
                            "shipment_id": {"S": shipment_id},
                            "ttl": {"N": str(ttl)},
                        },
                        "ConditionExpression": "attribute_not_exists(idempotency_key)",
                    }
                },
                {
                    "Put": {
                        "TableName": SHIP_TABLE,
                        "Item": {
                            "shipment_id": {"S": shipment_id},
                            "order_id": {"S": order_id},
                            "sku": {"S": sku},
                            "qty": {"N": str(qty)},
                            "status": {"S": "CREATED"},
                        },
                    }
                },
            ]
        )
        return {"shipped": True, "shipment_id": shipment_id}
    except ClientError as e:
        code = e.response.get("Error", {}).get("Code", "")
        if code in ("TransactionCanceledException", "ConditionalCheckFailedException"):
            return {"shipped": True, "shipment_id": shipment_id, "replayed": True}
        raise

write · /app/lambdas/release_inventory.py

import os
import time
import boto3
from botocore.exceptions import ClientError

ENDPOINT = os.environ.get("AWS_ENDPOINT_URL", "http://localstack:4566")
REGION = os.environ.get("AWS_REGION", "us-east-1")

ddb = boto3.client("dynamodb", endpoint_url=ENDPOINT, region_name=REGION)

INV_TABLE = "inventory"
IDEM_TABLE = "saga-idempotency"


def handler(event, _context):
    sku = event["sku"]
    qty = int(event["qty"])
    base_key = event["idempotency_key"]
    reserve_key = f"{base_key}:reserve"
    release_key = f"{base_key}:release"
    ttl = int(time.time()) + 3600

    reserved = ddb.get_item(
        TableName=IDEM_TABLE,
        Key={"idempotency_key": {"S": reserve_key}},
        ConsistentRead=True,
    ).get("Item")

    if not reserved:
        ddb.put_item(
            TableName=IDEM_TABLE,
            Item={
                "idempotency_key": {"S": release_key},
                "status": {"S": "NOOP"},
                "ttl": {"N": str(ttl)},
            },
        )
        return {"released": False, "reason": "no-reserve"}

    try:
        ddb.transact_write_items(
            TransactItems=[
                {
                    "Put": {
                        "TableName": IDEM_TABLE,
                        "Item": {
                            "idempotency_key": {"S": release_key},
                            "status": {"S": "DONE"},
                            "sku": {"S": sku},
                            "qty": {"N": str(qty)},
                            "ttl": {"N": str(ttl)},
                        },
                        "ConditionExpression": "attribute_not_exists(idempotency_key)",
                    }
                },
                {
                    "Update": {
                        "TableName": INV_TABLE,
                        "Key": {"sku": {"S": sku}},
                        "UpdateExpression": "ADD reserved_qty :q",
                        "ExpressionAttributeValues": {":q": {"N": str(-qty)}},
                    }
                },
            ]
        )
        return {"released": True, "sku": sku, "qty": qty}
    except ClientError as e:
        code = e.response.get("Error", {}).get("Code", "")
        if code in ("TransactionCanceledException", "ConditionalCheckFailedException"):
            return {"released": True, "sku": sku, "qty": qty, "replayed": True}
        raise

write · /app/lambdas/refund_payment.py

import os
import time
import boto3
from botocore.exceptions import ClientError

ENDPOINT = os.environ.get("AWS_ENDPOINT_URL", "http://localstack:4566")
REGION = os.environ.get("AWS_REGION", "us-east-1")

ddb = boto3.client("dynamodb", endpoint_url=ENDPOINT, region_name=REGION)

PAY_TABLE = "payments"
IDEM_TABLE = "saga-idempotency"


def handler(event, _context):
    base_key = event["idempotency_key"]
    charge_key = f"{base_key}:charge"
    refund_key = f"{base_key}:refund"
    payment_id = f"pay-{base_key}"
    ttl = int(time.time()) + 3600

    charged = ddb.get_item(
        TableName=IDEM_TABLE,
        Key={"idempotency_key": {"S": charge_key}},
        ConsistentRead=True,
    ).get("Item")

    if not charged:
        ddb.put_item(
            TableName=IDEM_TABLE,
            Item={
                "idempotency_key": {"S": refund_key},
                "status": {"S": "NOOP"},
                "ttl": {"N": str(ttl)},
            },
        )
        return {"refunded": False, "reason": "no-charge"}

    try:
        ddb.transact_write_items(
            TransactItems=[
                {
                    "Put": {
                        "TableName": IDEM_TABLE,
                        "Item": {
                            "idempotency_key": {"S": refund_key},
                            "status": {"S": "DONE"},
                            "payment_id": {"S": payment_id},
                            "ttl": {"N": str(ttl)},
                        },
                        "ConditionExpression": "attribute_not_exists(idempotency_key)",
                    }
                },
                {
                    "Update": {
                        "TableName": PAY_TABLE,
                        "Key": {"payment_id": {"S": payment_id}},
                        "UpdateExpression": "SET #s = :r",
                        "ExpressionAttributeNames": {"#s": "status"},
                        "ExpressionAttributeValues": {":r": {"S": "REFUNDED"}},
                    }
                },
            ]
        )
        return {"refunded": True, "payment_id": payment_id}
    except ClientError as e:
        code = e.response.get("Error", {}).get("Code", "")
        if code in ("TransactionCanceledException", "ConditionalCheckFailedException"):
            return {"refunded": True, "payment_id": payment_id, "replayed": True}
        raise

write · /app/lambdas/saga_notifier.py

import json


def handler(event, _context):
    detail_type = event.get("detail-type") or event.get("detailType") or "Unknown"
    detail = event.get("detail", {})
    print(f"saga-terminal:{detail_type} {json.dumps(detail, sort_keys=True)}")
    return {"ok": True}

ToolSearch

{
  "query": "select:TodoWrite",
  "max_results": 1
}

, output, 
{"type": "tool_reference", "tool_name": "TodoWrite"}

Write Lambda sources
Write IAM role & policy documents
Write Step Functions state machine JSON
Write deploy.sh (KMS, secret, DDB, lambdas, roles, SF, EB)
Run deploy.sh against LocalStack
Write verify.sh and execute both happy + failure paths

Now I'll write the state machine definition and the single deploy script.

write · /app/state_machine.json

{
  "Comment": "Order saga: reserve inventory -> charge payment -> create shipment. On failure, compensate in reverse.",
  "StartAt": "ReserveInventory",
  "States": {
    "ReserveInventory": {
      "Type": "Task",
      "Resource": "__RESERVE_ARN__",
      "ResultPath": "$.reserve",
      "Next": "ChargePayment",
      "Catch": [
        {
          "ErrorEquals": ["States.ALL"],
          "ResultPath": "$.error",
          "Next": "PublishFailed"
        }
      ]
    },
    "ChargePayment": {
      "Type": "Task",
      "Resource": "__CHARGE_ARN__",
      "ResultPath": "$.charge",
      "Next": "CreateShipment",
      "Catch": [
        {
          "ErrorEquals": ["States.ALL"],
          "ResultPath": "$.error",
          "Next": "CompensateAfterChargeFail"
        }
      ]
    },
    "CreateShipment": {
      "Type": "Task",
      "Resource": "__SHIP_ARN__",
      "ResultPath": "$.ship",
      "Next": "PublishCompleted",
      "Catch": [
        {
          "ErrorEquals": ["States.ALL"],
          "ResultPath": "$.error",
          "Next": "CompensateAfterShipFail_Refund"
        }
      ]
    },
    "CompensateAfterChargeFail": {
      "Type": "Task",
      "Resource": "__RELEASE_ARN__",
      "ResultPath": "$.release",
      "Next": "PublishFailed"
    },
    "CompensateAfterShipFail_Refund": {
      "Type": "Task",
      "Resource": "__REFUND_ARN__",
      "ResultPath": "$.refund",
      "Next": "CompensateAfterShipFail_Release"
    },
    "CompensateAfterShipFail_Release": {
      "Type": "Task",
      "Resource": "__RELEASE_ARN__",
      "ResultPath": "$.release",
      "Next": "PublishFailed"
    },
    "PublishCompleted": {
      "Type": "Task",
      "Resource": "arn:aws:states:::events:putEvents",
      "Parameters": {
        "Entries": [
          {
            "EventBusName": "saga-events",
            "Source": "order.saga",
            "DetailType": "OrderCompleted",
            "Detail": {
              "order_id.$": "$.order_id",
              "idempotency_key.$": "$.idempotency_key"
            }
          }
        ]
      },
      "End": true
    },
    "PublishFailed": {
      "Type": "Task",
      "Resource": "arn:aws:states:::events:putEvents",
      "Parameters": {
        "Entries": [
          {
            "EventBusName": "saga-events",
            "Source": "order.saga",
            "DetailType": "OrderFailed",
            "Detail": {
              "order_id.$": "$.order_id",
              "idempotency_key.$": "$.idempotency_key"
            }
          }
        ]
      },
      "End": true
    }
  }
}

write · /app/deploy.sh

#!/usr/bin/env bash
set -euo pipefail

export AWS_PAGER=""
ENDPOINT=${ENDPOINT:-http://localstack:4566}
REGION=${AWS_REGION:-us-east-1}
ACCOUNT=000000000000
AWS="aws --endpoint-url=$ENDPOINT --region $REGION"

ROOT=/app
SRC=$ROOT/lambdas
BUILD=$ROOT/build
rm -rf "$BUILD"; mkdir -p "$BUILD"

log() { echo "[deploy] $*"; }

pkg() {
    local name=$1 src=$2
    local dst="$BUILD/$name.zip"
    ( cd "$SRC" && zip -q -j "$dst" "$src" )
    echo "$dst"
}

###########################
# KMS CMK
###########################
log "kms: create CMK"
KEY_ID=$($AWS kms create-key \
    --description "saga cmk" \
    --key-usage ENCRYPT_DECRYPT \
    --query 'KeyMetadata.KeyId' --output text)
$AWS kms create-alias --alias-name alias/saga-cmk --target-key-id "$KEY_ID"
KEY_ARN=arn:aws:kms:$REGION:$ACCOUNT:key/$KEY_ID
log "kms: KEY_ARN=$KEY_ARN"

# scoped key policy (no kms:* wildcard)
cat > "$BUILD/key-policy.json" <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "RootAdmin",
      "Effect": "Allow",
      "Principal": {"AWS": "arn:aws:iam::$ACCOUNT:root"},
      "Action": [
        "kms:DescribeKey","kms:GetKeyPolicy","kms:PutKeyPolicy",
        "kms:CreateAlias","kms:DeleteAlias","kms:ListAliases",
        "kms:ScheduleKeyDeletion","kms:CancelKeyDeletion",
        "kms:EnableKey","kms:DisableKey","kms:TagResource","kms:UntagResource"
      ],
      "Resource": "$KEY_ARN"
    },
    {
      "Sid": "SecretsManagerUse",
      "Effect": "Allow",
      "Principal": {"Service": "secretsmanager.amazonaws.com"},
      "Action": [
        "kms:Encrypt","kms:Decrypt","kms:GenerateDataKey",
        "kms:CreateGrant","kms:DescribeKey","kms:ReEncryptFrom","kms:ReEncryptTo"
      ],
      "Resource": "$KEY_ARN"
    },
    {
      "Sid": "PaymentRoleDecrypt",
      "Effect": "Allow",
      "Principal": {"AWS": "arn:aws:iam::$ACCOUNT:role/charge-payment-role"},
      "Action": "kms:Decrypt",
      "Resource": "$KEY_ARN"
    }
  ]
}
EOF
$AWS kms put-key-policy --key-id "$KEY_ID" --policy-name default --policy file://"$BUILD/key-policy.json" >/dev/null

###########################
# Secret
###########################
log "secrets: create secret payment/gateway-apikey (CMK-encrypted)"
SECRET_ARN=$($AWS secretsmanager create-secret \
    --name payment/gateway-apikey \
    --secret-string "sk_live_not_a_real_key" \
    --kms-key-id "alias/saga-cmk" \
    --query 'ARN' --output text)
log "secrets: SECRET_ARN=$SECRET_ARN"

###########################
# DynamoDB tables
###########################
log "ddb: create tables"
create_table() {
    local t=$1 k=$2
    if $AWS dynamodb describe-table --table-name "$t" >/dev/null 2>&1; then
        log "ddb: table $t already exists"
        return
    fi
    $AWS dynamodb create-table \
        --table-name "$t" \
        --attribute-definitions AttributeName="$k",AttributeType=S \
        --key-schema AttributeName="$k",KeyType=HASH \
        --billing-mode PAY_PER_REQUEST >/dev/null
    $AWS dynamodb wait table-exists --table-name "$t"
}
create_table inventory sku
create_table payments payment_id
create_table shipments shipment_id
create_table saga-idempotency idempotency_key
$AWS dynamodb update-time-to-live \
    --table-name saga-idempotency \
    --time-to-live-specification 'Enabled=true,AttributeName=ttl' >/dev/null

INV_ARN=arn:aws:dynamodb:$REGION:$ACCOUNT:table/inventory
PAY_ARN=arn:aws:dynamodb:$REGION:$ACCOUNT:table/payments
SHIP_ARN=arn:aws:dynamodb:$REGION:$ACCOUNT:table/shipments
IDEM_ARN=arn:aws:dynamodb:$REGION:$ACCOUNT:table/saga-idempotency

###########################
# IAM: trust docs + roles
###########################
cat > "$BUILD/trust-lambda.json" <<'EOF'
{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"lambda.amazonaws.com"},"Action":"sts:AssumeRole"}]}
EOF
cat > "$BUILD/trust-states.json" <<'EOF'
{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"states.amazonaws.com"},"Action":"sts:AssumeRole"}]}
EOF

create_role() {
    local name=$1 trust=$2
    if $AWS iam get-role --role-name "$name" >/dev/null 2>&1; then
        log "iam: role $name exists"
    else
        $AWS iam create-role --role-name "$name" --assume-role-policy-document file://"$trust" >/dev/null
        log "iam: created role $name"
    fi
}
create_role reserve-inventory-role "$BUILD/trust-lambda.json"
create_role charge-payment-role    "$BUILD/trust-lambda.json"
create_role create-shipment-role   "$BUILD/trust-lambda.json"
create_role release-inventory-role "$BUILD/trust-lambda.json"
create_role refund-payment-role    "$BUILD/trust-lambda.json"
create_role saga-notifier-role     "$BUILD/trust-lambda.json"
create_role order-saga-role        "$BUILD/trust-states.json"

put_policy() {
    local role=$1 polname=$2 doc=$3
    $AWS iam put-role-policy --role-name "$role" --policy-name "$polname" --policy-document file://"$doc" >/dev/null
}

# reserve-inventory-role: ddb(inventory + idem) + its own log group
cat > "$BUILD/pol-reserve.json" <<EOF
{
  "Version":"2012-10-17",
  "Statement":[
    {"Sid":"Ddb","Effect":"Allow",
     "Action":["dynamodb:UpdateItem","dynamodb:PutItem","dynamodb:GetItem","dynamodb:TransactWriteItems"],
     "Resource":["$INV_ARN","$IDEM_ARN"]},
    {"Sid":"Logs","Effect":"Allow",
     "Action":["logs:CreateLogGroup","logs:CreateLogStream","logs:PutLogEvents"],
     "Resource":"arn:aws:logs:$REGION:$ACCOUNT:log-group:/aws/lambda/reserve-inventory:*"}
  ]
}
EOF
put_policy reserve-inventory-role reserve-inventory-policy "$BUILD/pol-reserve.json"

# charge-payment-role: ddb(payments+idem) + secret + kms + logs
cat > "$BUILD/pol-charge.json" <<EOF
{
  "Version":"2012-10-17",
  "Statement":[
    {"Sid":"Ddb","Effect":"Allow",
     "Action":["dynamodb:PutItem","dynamodb:GetItem","dynamodb:TransactWriteItems"],
     "Resource":["$PAY_ARN","$IDEM_ARN"]},
    {"Sid":"Secret","Effect":"Allow",
     "Action":"secretsmanager:GetSecretValue",
     "Resource":"$SECRET_ARN"},
    {"Sid":"KmsDecrypt","Effect":"Allow",
     "Action":"kms:Decrypt",
     "Resource":"$KEY_ARN"},
    {"Sid":"Logs","Effect":"Allow",
     "Action":["logs:CreateLogGroup","logs:CreateLogStream","logs:PutLogEvents"],
     "Resource":"arn:aws:logs:$REGION:$ACCOUNT:log-group:/aws/lambda/charge-payment:*"}
  ]
}
EOF
put_policy charge-payment-role charge-payment-policy "$BUILD/pol-charge.json"

# create-shipment-role: ddb(shipments+idem) + logs
cat > "$BUILD/pol-ship.json" <<EOF
{
  "Version":"2012-10-17",
  "Statement":[
    {"Sid":"Ddb","Effect":"Allow",
     "Action":["dynamodb:PutItem","dynamodb:GetItem","dynamodb:TransactWriteItems"],
     "Resource":["$SHIP_ARN","$IDEM_ARN"]},
    {"Sid":"Logs","Effect":"Allow",
     "Action":["logs:CreateLogGroup","logs:CreateLogStream","logs:PutLogEvents"],
     "Resource":"arn:aws:logs:$REGION:$ACCOUNT:log-group:/aws/lambda/create-shipment:*"}
  ]
}
EOF
put_policy create-shipment-role create-shipment-policy "$BUILD/pol-ship.json"

# release-inventory-role: ddb(inventory+idem) + logs
cat > "$BUILD/pol-release.json" <<EOF
{
  "Version":"2012-10-17",
  "Statement":[
    {"Sid":"Ddb","Effect":"Allow",
     "Action":["dynamodb:UpdateItem","dynamodb:PutItem","dynamodb:GetItem","dynamodb:TransactWriteItems"],
     "Resource":["$INV_ARN","$IDEM_ARN"]},
    {"Sid":"Logs","Effect":"Allow",
     "Action":["logs:CreateLogGroup","logs:CreateLogStream","logs:PutLogEvents"],
     "Resource":"arn:aws:logs:$REGION:$ACCOUNT:log-group:/aws/lambda/release-inventory:*"}
  ]
}
EOF
put_policy release-inventory-role release-inventory-policy "$BUILD/pol-release.json"

# refund-payment-role: ddb(payments+idem) + logs
cat > "$BUILD/pol-refund.json" <<EOF
{
  "Version":"2012-10-17",
  "Statement":[
    {"Sid":"Ddb","Effect":"Allow",
     "Action":["dynamodb:UpdateItem","dynamodb:PutItem","dynamodb:GetItem","dynamodb:TransactWriteItems"],
     "Resource":["$PAY_ARN","$IDEM_ARN"]},
    {"Sid":"Logs","Effect":"Allow",
     "Action":["logs:CreateLogGroup","logs:CreateLogStream","logs:PutLogEvents"],
     "Resource":"arn:aws:logs:$REGION:$ACCOUNT:log-group:/aws/lambda/refund-payment:*"}
  ]
}
EOF
put_policy refund-payment-role refund-payment-policy "$BUILD/pol-refund.json"

# saga-notifier-role: only logs
cat > "$BUILD/pol-notifier.json" <<EOF
{
  "Version":"2012-10-17",
  "Statement":[
    {"Sid":"Logs","Effect":"Allow",
     "Action":["logs:CreateLogGroup","logs:CreateLogStream","logs:PutLogEvents"],
     "Resource":"arn:aws:logs:$REGION:$ACCOUNT:log-group:/aws/lambda/saga-notifier:*"}
  ]
}
EOF
put_policy saga-notifier-role saga-notifier-policy "$BUILD/pol-notifier.json"

# order-saga-role: explicit lambda ARNs + explicit event bus ARN
cat > "$BUILD/pol-saga.json" <<EOF
{
  "Version":"2012-10-17",
  "Statement":[
    {"Sid":"InvokeForwardAndCompensations","Effect":"Allow",
     "Action":"lambda:InvokeFunction",
     "Resource":[
       "arn:aws:lambda:$REGION:$ACCOUNT:function:reserve-inventory",
       "arn:aws:lambda:$REGION:$ACCOUNT:function:charge-payment",
       "arn:aws:lambda:$REGION:$ACCOUNT:function:create-shipment",
       "arn:aws:lambda:$REGION:$ACCOUNT:function:release-inventory",
       "arn:aws:lambda:$REGION:$ACCOUNT:function:refund-payment"
     ]},
    {"Sid":"PutSagaEvents","Effect":"Allow",
     "Action":"events:PutEvents",
     "Resource":"arn:aws:events:$REGION:$ACCOUNT:event-bus/saga-events"}
  ]
}
EOF
put_policy order-saga-role order-saga-policy "$BUILD/pol-saga.json"

###########################
# Lambdas
###########################
log "lambda: package + create"

make_fn() {
    local fn=$1 src=$2 role=$3
    local zip
    zip=$(pkg "$fn" "$src")
    if $AWS lambda get-function --function-name "$fn" >/dev/null 2>&1; then
        $AWS lambda update-function-code --function-name "$fn" --zip-file fileb://"$zip" >/dev/null
    else
        $AWS lambda create-function \
            --function-name "$fn" \
            --runtime python3.11 \
            --handler "${src%.py}.handler" \
            --role "arn:aws:iam::$ACCOUNT:role/$role" \
            --zip-file fileb://"$zip" \
            --environment "Variables={AWS_ENDPOINT_URL=http://localhost.localstack.cloud:4566}" \
            --timeout 30 >/dev/null
    fi
    $AWS lambda wait function-active-v2 --function-name "$fn" 2>/dev/null || true
}

make_fn reserve-inventory  reserve_inventory.py  reserve-inventory-role
make_fn charge-payment     charge_payment.py     charge-payment-role
make_fn create-shipment    create_shipment.py    create-shipment-role
make_fn release-inventory  release_inventory.py  release-inventory-role
make_fn refund-payment     refund_payment.py     refund-payment-role
make_fn saga-notifier      saga_notifier.py      saga-notifier-role

RESERVE_ARN=arn:aws:lambda:$REGION:$ACCOUNT:function:reserve-inventory
CHARGE_ARN=arn:aws:lambda:$REGION:$ACCOUNT:function:charge-payment
SHIP_ARN=arn:aws:lambda:$REGION:$ACCOUNT:function:create-shipment
RELEASE_ARN=arn:aws:lambda:$REGION:$ACCOUNT:function:release-inventory
REFUND_ARN=arn:aws:lambda:$REGION:$ACCOUNT:function:refund-payment
NOTIFIER_ARN=arn:aws:lambda:$REGION:$ACCOUNT:function:saga-notifier

###########################
# EventBridge bus + rule + target
###########################
log "events: create bus saga-events"
$AWS events create-event-bus --name saga-events >/dev/null 2>&1 || log "events: bus saga-events exists"

log "events: create rule on-saga-terminal"
cat > "$BUILD/event-pattern.json" <<'EOF'
{"source":["order.saga"],"detail-type":["OrderCompleted","OrderFailed"]}
EOF
$AWS events put-rule \
    --name on-saga-terminal \
    --event-bus-name saga-events \
    --event-pattern file://"$BUILD/event-pattern.json" >/dev/null

log "events: put target (notifier lambda)"
$AWS events put-targets \
    --rule on-saga-terminal \
    --event-bus-name saga-events \
    --targets "Id=1,Arn=$NOTIFIER_ARN" >/dev/null

log "lambda: add-permission for EventBridge -> notifier"
$AWS lambda add-permission \
    --function-name saga-notifier \
    --statement-id allow-eventbridge \
    --action lambda:InvokeFunction \
    --principal events.amazonaws.com \
    --source-arn "arn:aws:events:$REGION:$ACCOUNT:rule/saga-events/on-saga-terminal" \
    >/dev/null 2>&1 || log "lambda: permission already present"

###########################
# Step Functions state machine
###########################
log "sfn: materialize state machine json"
sed \
    -e "s|__RESERVE_ARN__|$RESERVE_ARN|g" \
    -e "s|__CHARGE_ARN__|$CHARGE_ARN|g" \
    -e "s|__SHIP_ARN__|$SHIP_ARN|g" \
    -e "s|__RELEASE_ARN__|$RELEASE_ARN|g" \
    -e "s|__REFUND_ARN__|$REFUND_ARN|g" \
    "$ROOT/state_machine.json" > "$BUILD/state_machine.resolved.json"

log "sfn: create state machine order-saga"
SM_ARN=$($AWS stepfunctions create-state-machine \
    --name order-saga \
    --definition file://"$BUILD/state_machine.resolved.json" \
    --role-arn "arn:aws:iam::$ACCOUNT:role/order-saga-role" \
    --query 'stateMachineArn' --output text 2>/dev/null || true)

if [ -z "${SM_ARN:-}" ]; then
    SM_ARN=$($AWS stepfunctions list-state-machines \
        --query "stateMachines[?name=='order-saga'].stateMachineArn | [0]" --output text)
    $AWS stepfunctions update-state-machine \
        --state-machine-arn "$SM_ARN" \
        --definition file://"$BUILD/state_machine.resolved.json" \
        --role-arn "arn:aws:iam::$ACCOUNT:role/order-saga-role" >/dev/null
fi
log "sfn: SM_ARN=$SM_ARN"

# Persist handy vars for the verifier
cat > "$BUILD/env" <<EOF
export SM_ARN="$SM_ARN"
export KEY_ARN="$KEY_ARN"
export SECRET_ARN="$SECRET_ARN"
export NOTIFIER_ARN="$NOTIFIER_ARN"
EOF

log "done."

Run deployment

shell

chmod +x /app/deploy.sh && bash /app/deploy.sh 2>&1 | tail -80

output

[deploy] kms: create CMK
[deploy] kms: KEY_ARN=arn:aws:kms:us-east-1:000000000000:key/3070b521-5b04-427b-b1d7-48003dfc189c
[deploy] secrets: create secret payment/gateway-apikey (CMK-encrypted)
[deploy] secrets: SECRET_ARN=arn:aws:secretsmanager:us-east-1:000000000000:secret:payment/gateway-apikey-JSpWak
[deploy] ddb: create tables
[deploy] iam: created role reserve-inventory-role
[deploy] iam: created role charge-payment-role
[deploy] iam: created role create-shipment-role
[deploy] iam: created role release-inventory-role
[deploy] iam: created role refund-payment-role
[deploy] iam: created role saga-notifier-role
[deploy] iam: created role order-saga-role
[deploy] lambda: package + create
[deploy] events: create bus saga-events
[deploy] events: create rule on-saga-terminal
[deploy] events: put target (notifier lambda)
[deploy] lambda: add-permission for EventBridge -> notifier
[deploy] sfn: materialize state machine json
[deploy] sfn: create state machine order-saga
[deploy] sfn: SM_ARN=arn:aws:states:us-east-1:000000000000:stateMachine:order-saga
[deploy] done.

[stdout]
[deploy] kms: create CMK
[deploy] kms: KEY_ARN=arn:aws:kms:us-east-1:000000000000:key/3070b521-5b04-427b-b1d7-48003dfc189c
[deploy] secrets: create secret payment/gateway-apikey (CMK-encrypted)
[deploy] secrets: SECRET_ARN=arn:aws:secretsmanager:us-east-1:000000000000:secret:payment/gateway-apikey-JSpWak
[deploy] ddb: create tables
[deploy] iam: created role reserve-inventory-role
[deploy] iam: created role charge-payment-role
[deploy] iam: created role create-shipment-role
[deploy] iam: created role release-inventory-role
[deploy] iam: created role refund-payment-role
[deploy] iam: created role saga-notifier-role
[deploy] iam: created role order-saga-role
[deploy] lambda: package + create
[deploy] events: create bus saga-events
[deploy] events: create rule on-saga-terminal
[deploy] events: put target (notifier lambda)
[deploy] lambda: add-permission for EventBridge -> notifier
[deploy] sfn: materialize state machine json
[deploy] sfn: create state machine order-saga
[deploy] sfn: SM_ARN=arn:aws:states:us-east-1:000000000000:stateMachine:order-saga
[deploy] done.

Deployment is clean. Let me kick off the happy path execution to verify all the wiring is alive before writing the full verifier.

Happy-path smoke test

shell

source /app/build/env && \
EXEC_ARN=$(aws --endpoint-url=http://localstack:4566 --region us-east-1 stepfunctions start-execution \
    --state-machine-arn "$SM_ARN" \
    --name smoke-$(date +%s) \
    --input '{"order_id":"o-smoke","sku":"x","qty":2,"amount":100,"idempotency_key":"k-smoke"}' \
    --query 'executionArn' --output text) && \
echo "EXEC=$EXEC_ARN" && \
for i in $(seq 1 20); do
    STATUS=$(aws --endpoint-url=http://localstack:4566 --region us-east-1 stepfunctions describe-execution --execution-arn "$EXEC_ARN" --query status --output text)
    echo "[$i] status=$STATUS"
    if [ "$STATUS" != "RUNNING" ]; then break; fi
    sleep 2
done
aws --endpoint-url=http://localstack:4566 --region us-east-1 stepfunctions describe-execution --execution-arn "$EXEC_ARN" --output json | jq '{status, output, error, cause}'

output

EXEC=arn:aws:states:us-east-1:000000000000:execution:order-saga:smoke-1776920086
[1] status=RUNNING
[2] status=RUNNING
[3] status=SUCCEEDED
{
  "status": "SUCCEEDED",
  "output": "{\"FailedEntryCount\":0,\"Entries\":[{\"EventId\":\"8c4cf785-3063-43b3-963a-29487647329b\"}]}",
  "error": null,
  "cause": null
}

[stdout]
EXEC=arn:aws:states:us-east-1:000000000000:execution:order-saga:smoke-1776920086
[1] status=RUNNING
[2] status=RUNNING
[3] status=SUCCEEDED
{
  "status": "SUCCEEDED",
  "output": "{\"FailedEntryCount\":0,\"Entries\":[{\"EventId\":\"8c4cf785-3063-43b3-963a-29487647329b\"}]}",
  "error": null,
  "cause": null
}

Happy path works. Let me check DDB state and notifier logs.

Inspect post-execution state

shell

AWS="aws --endpoint-url=http://localstack:4566 --region us-east-1"
echo "--- inventory"; $AWS dynamodb get-item --table-name inventory --key '{"sku":{"S":"x"}}' | jq .
echo "--- payments scan"; $AWS dynamodb scan --table-name payments | jq '.Items'
echo "--- shipments scan"; $AWS dynamodb scan --table-name shipments | jq '.Items'
echo "--- notifier logs"
sleep 2
$AWS logs describe-log-groups --log-group-name-prefix /aws/lambda/saga-notifier --query 'logGroups[].logGroupName' --output text
$AWS logs filter-log-events --log-group-name /aws/lambda/saga-notifier --query 'events[].message' --output text | tr '\t' '\n' | grep -E 'saga-terminal' || echo "(no terminal log yet)"

output

--- inventory
{
  "Item": {
    "sku": {
      "S": "x"
    },
    "reserved_qty": {
      "N": "2"
    }
  }
}
--- payments scan
[
  {
    "amount": {
      "N": "100"
    },
    "order_id": {
      "S": "o-smoke"
    },
    "payment_id": {
      "S": "pay-k-smoke"
    },
    "status": {
      "S": "CAPTURED"
    }
  }
]
--- shipments scan
[
  {
    "shipment_id": {
      "S": "shp-k-smoke"
    },
    "sku": {
      "S": "x"
    },
    "order_id": {
      "S": "o-smoke"
    },
    "qty": {
      "N": "2"
    },
    "status": {
      "S": "CREATED"
    }
  }
]
--- notifier logs
/aws/lambda/saga-notifier
saga-terminal:OrderCompleted {"idempotency_key": "k-smoke", "order_id": "o-smoke"}

[stdout]
--- inventory
{
  "Item": {
    "sku": {
      "S": "x"
    },
    "reserved_qty": {
      "N": "2"
    }
  }
}
--- payments scan
[
  {
    "amount": {
      "N": "100"
    },
    "order_id": {
      "S": "o-smoke"
    },
    "payment_id": {
      "S": "pay-k-smoke"
    },
    "status": {
      "S": "CAPTURED"
    }
  }
]
--- shipments scan
[
  {
    "shipment_id": {
      "S": "shp-k-smoke"
    },
    "sku": {
      "S": "x"
    },
    "order_id": {
      "S": "o-smoke"
    },
    "qty": {
      "N": "2"
    },
    "status": {
      "S": "CREATED"
    }
  }
]
--- notifier logs
/aws/lambda/saga-notifier
saga-terminal:OrderCompleted {"idempotency_key": "k-smoke", "order_id": "o-smoke"}

Happy path wired end-to-end. Let me exercise the failure path before writing the verifier.

Failure path execution

shell

source /app/build/env
AWS="aws --endpoint-url=http://localstack:4566 --region us-east-1"
EXEC=$($AWS stepfunctions start-execution \
    --state-machine-arn "$SM_ARN" \
    --name fail-$(date +%s) \
    --input '{"order_id":"o-fail","sku":"x","qty":3,"amount":200,"idempotency_key":"k-fail","force_failure":true}' \
    --query 'executionArn' --output text)
echo "EXEC=$EXEC"
for i in $(seq 1 30); do
    S=$($AWS stepfunctions describe-execution --execution-arn "$EXEC" --query status --output text)
    echo "[$i] $S"
    [ "$S" != "RUNNING" ] && break
    sleep 2
done
$AWS stepfunctions describe-execution --execution-arn "$EXEC" --output json | jq '{status, output}'
echo "--- inventory"
$AWS dynamodb get-item --table-name inventory --key '{"sku":{"S":"x"}}' | jq .
echo "--- payments"
$AWS dynamodb scan --table-name payments | jq '.Items'
echo "--- shipments"
$AWS dynamodb scan --table-name shipments | jq '.Items'
echo "--- notifier"
sleep 2
$AWS logs filter-log-events --log-group-name /aws/lambda/saga-notifier --query 'events[].message' --output text | tr '\t' '\n' | grep -E 'saga-terminal' || echo "(no terminal log yet)"

output

EXEC=arn:aws:states:us-east-1:000000000000:execution:order-saga:fail-1776920110
[1] RUNNING
[2] RUNNING
[3] SUCCEEDED
{
  "status": "SUCCEEDED",
  "output": "{\"FailedEntryCount\":0,\"Entries\":[{\"EventId\":\"49c54525-16ff-4e74-9f12-9a0514549e5c\"}]}"
}
--- inventory
{
  "Item": {
    "sku": {
      "S": "x"
    },
    "reserved_qty": {
      "N": "2"
    }
  }
}
--- payments
[
  {
    "amount": {
      "N": "100"
    },
    "order_id": {
      "S": "o-smoke"
    },
    "payment_id": {
      "S": "pay-k-smoke"
    },
    "status": {
      "S": "CAPTURED"
    }
  }
]
--- shipments
[
  {
    "shipment_id": {
      "S": "shp-k-smoke"
    },
    "sku": {
      "S": "x"
    },
    "order_id": {
      "S": "o-smoke"
    },
    "qty": {
      "N": "2"
    },
    "status": {
      "S": "CREATED"
    }
  }
]
--- notifier
saga-terminal:OrderCompleted {"idempotency_key": "k-smoke", "order_id": "o-smoke"}
saga-terminal:OrderFailed {"idempotency_key": "k-fail", "order_id": "o-fail"}

[stdout]
EXEC=arn:aws:states:us-east-1:000000000000:execution:order-saga:fail-1776920110
[1] RUNNING
[2] RUNNING
[3] SUCCEEDED
{
  "status": "SUCCEEDED",
  "output": "{\"FailedEntryCount\":0,\"Entries\":[{\"EventId\":\"49c54525-16ff-4e74-9f12-9a0514549e5c\"}]}"
}
--- inventory
{
  "Item": {
    "sku": {
      "S": "x"
    },
    "reserved_qty": {
      "N": "2"
    }
  }
}
--- payments
[
  {
    "amount": {
      "N": "100"
    },
    "order_id": {
      "S": "o-smoke"
    },
    "payment_id": {
      "S": "pay-k-smoke"
    },
    "status": {
      "S": "CAPTURED"
    }
  }
]
--- shipments
[
  {
    "shipment_id": {
      "S": "shp-k-smoke"
    },
    "sku": {
      "S": "x"
    },
    "order_id": {
      "S": "o-smoke"
    },
    "qty": {
      "N": "2"
    },
    "status": {
      "S": "CREATED"
    }
  }
]
--- notifier
saga-terminal:OrderCompleted {"idempotency_key": "k-smoke", "order_id": "o-smoke"}
saga-terminal:OrderFailed {"idempotency_key": "k-fail", "order_id": "o-fail"}

Write Lambda sources
Write IAM role & policy documents
Write Step Functions state machine JSON
Write deploy.sh (KMS, secret, DDB, lambdas, roles, SF, EB)
Run deploy.sh against LocalStack
Smoke-test happy + failure paths
Write verify.sh (policy + DDB + log assertions)
Run verify.sh fresh (new SKUs/keys)

Now the verifier , it reads IAM policy docs directly, runs both executions, and asserts DDB + logs state.

write · /app/verify.py

#!/usr/bin/env python3
"""End-to-end verifier for the order-saga stack on LocalStack."""
import json
import sys
import time

import boto3

ENDPOINT = "http://localstack:4566"
REGION = "us-east-1"
ACCOUNT = "000000000000"

session = boto3.Session(region_name=REGION)
iam = session.client("iam", endpoint_url=ENDPOINT)
ddb = session.client("dynamodb", endpoint_url=ENDPOINT)
sfn = session.client("stepfunctions", endpoint_url=ENDPOINT)
logs = session.client("logs", endpoint_url=ENDPOINT)
kms = session.client("kms", endpoint_url=ENDPOINT)
sm_cli = session.client("secretsmanager", endpoint_url=ENDPOINT)

failures = []


def ok(msg):
    print(f"  [PASS] {msg}")


def fail(msg):
    print(f"  [FAIL] {msg}")
    failures.append(msg)


def section(title):
    print(f"\n== {title} ==")


def as_list(x):
    if x is None:
        return []
    return x if isinstance(x, list) else [x]


def inline_policies(role):
    names = iam.list_role_policies(RoleName=role)["PolicyNames"]
    return {n: iam.get_role_policy(RoleName=role, PolicyName=n)["PolicyDocument"] for n in names}


def statements(role):
    docs = inline_policies(role)
    out = []
    for d in docs.values():
        stmts = d.get("Statement", [])
        out.extend(stmts if isinstance(stmts, list) else [stmts])
    return out


def find_action(stmts, action):
    for s in stmts:
        if s.get("Effect") == "Allow" and action in as_list(s.get("Action")):
            return s
    return None


def kms_violations(stmts):
    """Return list of problem descriptions for any kms wildcarding."""
    problems = []
    for s in stmts:
        actions = as_list(s.get("Action"))
        resources = as_list(s.get("Resource"))
        for a in actions:
            if a == "kms:*" or (a.startswith("kms:") and "*" in a.split(":", 1)[1]):
                problems.append(f"kms action wildcard: {a}")
        if any(a.startswith("kms:") for a in actions):
            for r in resources:
                if r == "*" or r.endswith(":*") or "key/*" in r:
                    problems.append(f"kms resource wildcard: {r}")
        if "*" in actions:
            problems.append("action '*' may cover kms")
    return problems


# =====================================================================
section("IAM: per-lambda roles scoped to their own tables")
# =====================================================================
ROLE_TABLES = {
    "reserve-inventory-role": {"inventory", "saga-idempotency"},
    "charge-payment-role":    {"payments", "saga-idempotency"},
    "create-shipment-role":   {"shipments", "saga-idempotency"},
    "release-inventory-role": {"inventory", "saga-idempotency"},
    "refund-payment-role":    {"payments", "saga-idempotency"},
    "saga-notifier-role":     set(),
}

for role, allowed in ROLE_TABLES.items():
    stmts = statements(role)
    referenced = set()
    wildcards = []
    for s in stmts:
        actions = as_list(s.get("Action"))
        if any(a.startswith("dynamodb:") for a in actions):
            for r in as_list(s.get("Resource")):
                if r == "*" or r.endswith("table/*"):
                    wildcards.append(r)
                if ":table/" in r:
                    referenced.add(r.split(":table/")[-1].split("/")[0])
    if wildcards:
        fail(f"{role} uses wildcard DDB resources: {wildcards}")
    elif referenced != allowed:
        fail(f"{role} ddb tables {referenced} != expected {allowed}")
    else:
        ok(f"{role} ddb scoped to {sorted(allowed) or '(none)'}")


# =====================================================================
section("IAM: order-saga-role scopes Lambda invoke + events publish")
# =====================================================================
saga_stmts = statements("order-saga-role")

invoke = find_action(saga_stmts, "lambda:InvokeFunction")
if not invoke:
    fail("order-saga-role missing lambda:InvokeFunction")
else:
    rs = as_list(invoke.get("Resource"))
    if any(r == "*" or r.endswith(":*") or r.endswith("function:*") for r in rs):
        fail(f"order-saga-role wildcards on lambda:InvokeFunction: {rs}")
    else:
        expected = {
            f"arn:aws:lambda:{REGION}:{ACCOUNT}:function:{n}"
            for n in ["reserve-inventory", "charge-payment", "create-shipment",
                      "release-inventory", "refund-payment"]
        }
        if set(rs) != expected:
            fail(f"order-saga-role invoke ARNs {set(rs)} != expected {expected}")
        else:
            ok(f"order-saga-role invokes exactly the 5 saga lambdas")

putev = find_action(saga_stmts, "events:PutEvents")
if not putev:
    fail("order-saga-role missing events:PutEvents")
else:
    rs = as_list(putev.get("Resource"))
    expected_bus = f"arn:aws:events:{REGION}:{ACCOUNT}:event-bus/saga-events"
    if rs == [expected_bus]:
        ok("order-saga-role publishes only to saga-events bus")
    else:
        fail(f"order-saga-role events:PutEvents resource {rs} != [{expected_bus}]")


# =====================================================================
section("IAM: secret access isolated to charge-payment-role")
# =====================================================================
for role in ROLE_TABLES:
    stmts = statements(role)
    has_secret = find_action(stmts, "secretsmanager:GetSecretValue") is not None
    if role == "charge-payment-role":
        if has_secret:
            ok("charge-payment-role has secretsmanager:GetSecretValue")
        else:
            fail("charge-payment-role MISSING secretsmanager:GetSecretValue")
    else:
        if has_secret:
            fail(f"{role} unexpectedly has secretsmanager:GetSecretValue")
# also check saga role
if find_action(saga_stmts, "secretsmanager:GetSecretValue"):
    fail("order-saga-role has secretsmanager:GetSecretValue (should not)")
else:
    ok("other roles do not have secretsmanager:GetSecretValue")


# =====================================================================
section("IAM: no kms wildcards in any role policy")
# =====================================================================
all_roles = list(ROLE_TABLES) + ["order-saga-role"]
any_kms_issue = False
for role in all_roles:
    issues = kms_violations(statements(role))
    if issues:
        fail(f"{role}: {issues}")
        any_kms_issue = True
if not any_kms_issue:
    ok("no kms wildcards in any role policy")

charge_stmts = statements("charge-payment-role")
kms_stmt = find_action(charge_stmts, "kms:Decrypt")
if not kms_stmt:
    fail("charge-payment-role missing kms:Decrypt")
else:
    rs = as_list(kms_stmt.get("Resource"))
    if len(rs) == 1 and rs[0].startswith(f"arn:aws:kms:{REGION}:{ACCOUNT}:key/") and "*" not in rs[0]:
        ok(f"charge-payment-role kms:Decrypt scoped to specific CMK ({rs[0].split('/')[-1][:8]}...)")
    else:
        fail(f"charge-payment-role kms:Decrypt wrong resource: {rs}")


# =====================================================================
section("IAM: notifier role has only EventBridge-target prerequisites (logs)")
# =====================================================================
notifier_stmts = statements("saga-notifier-role")
non_logs = []
for s in notifier_stmts:
    for a in as_list(s.get("Action")):
        if not a.startswith("logs:"):
            non_logs.append(a)
if non_logs:
    fail(f"saga-notifier-role has non-logs actions: {non_logs}")
else:
    ok("saga-notifier-role has only logs:* actions")


# =====================================================================
section("Resource names exist")
# =====================================================================
def check(name, fn):
    try:
        fn()
        ok(f"{name} present")
    except Exception as e:
        fail(f"{name} missing: {e}")

check("alias/saga-cmk",    lambda: kms.describe_key(KeyId="alias/saga-cmk"))
check("secret payment/gateway-apikey",
      lambda: sm_cli.describe_secret(SecretId="payment/gateway-apikey"))
d = sm_cli.describe_secret(SecretId="payment/gateway-apikey")
if d.get("KmsKeyId"):
    ok(f"secret encrypted with CMK ({d['KmsKeyId']})")
else:
    fail("secret not associated with a CMK")

for t in ["inventory", "payments", "shipments", "saga-idempotency"]:
    check(f"ddb table {t}", lambda t=t: ddb.describe_table(TableName=t))

ttl = ddb.describe_time_to_live(TableName="saga-idempotency")["TimeToLiveDescription"]
if ttl.get("TimeToLiveStatus") in ("ENABLED", "ENABLING"):
    ok(f"saga-idempotency TTL {ttl['TimeToLiveStatus']}")
else:
    fail(f"saga-idempotency TTL not enabled: {ttl}")

for fn in ["reserve-inventory","charge-payment","create-shipment",
           "release-inventory","refund-payment","saga-notifier"]:
    check(f"lambda {fn}",
          lambda fn=fn: session.client("lambda", endpoint_url=ENDPOINT).get_function(FunctionName=fn))

check("state machine order-saga",
      lambda: sfn.describe_state_machine(stateMachineArn=f"arn:aws:states:{REGION}:{ACCOUNT}:stateMachine:order-saga"))
ev = session.client("events", endpoint_url=ENDPOINT)
check("event bus saga-events",         lambda: ev.describe_event_bus(Name="saga-events"))
check("event rule on-saga-terminal",   lambda: ev.describe_rule(Name="on-saga-terminal", EventBusName="saga-events"))


# =====================================================================
section("Reset test rows so assertions are reproducible")
# =====================================================================
def safe_del(table, key, val):
    try:
        ddb.delete_item(TableName=table, Key={key: {"S": val}})
    except Exception:
        pass

safe_del("inventory", "sku", "x")
for pid in ["pay-k-1", "pay-k-2"]:
    safe_del("payments", "payment_id", pid)
for sid in ["shp-k-1", "shp-k-2"]:
    safe_del("shipments", "shipment_id", sid)
for suf in ["reserve", "charge", "ship", "release", "refund"]:
    for base in ["k-1", "k-2"]:
        safe_del("saga-idempotency", "idempotency_key", f"{base}:{suf}")
ok("cleared inventory[x], payments[pay-k-*], shipments[shp-k-*], idempotency keys for k-1/k-2")


# =====================================================================
def run_execution(label, input_obj, timeout=60):
    arn = f"arn:aws:states:{REGION}:{ACCOUNT}:stateMachine:order-saga"
    exec_arn = sfn.start_execution(
        stateMachineArn=arn,
        name=f"{label}-{int(time.time()*1000)}",
        input=json.dumps(input_obj),
    )["executionArn"]
    deadline = time.time() + timeout
    while time.time() < deadline:
        r = sfn.describe_execution(executionArn=exec_arn)
        if r["status"] != "RUNNING":
            return r
        time.sleep(1)
    r = sfn.describe_execution(executionArn=exec_arn)
    return r


# =====================================================================
section("Happy path: o-1 / k-1")
# =====================================================================
res = run_execution("verify-happy", {
    "order_id": "o-1", "sku": "x", "qty": 2, "amount": 100, "idempotency_key": "k-1"
})
if res["status"] == "SUCCEEDED":
    ok(f"state machine SUCCEEDED ({res.get('stopDate')})")
else:
    fail(f"state machine status = {res['status']}  output={res.get('output')}")

item = ddb.get_item(TableName="inventory", Key={"sku": {"S": "x"}}).get("Item")
if item and item.get("reserved_qty", {}).get("N") == "2":
    ok("inventory[x].reserved_qty == 2")
else:
    fail(f"inventory[x] != 2: {item}")

item = ddb.get_item(TableName="payments", Key={"payment_id": {"S": "pay-k-1"}}).get("Item")
if item and item.get("status", {}).get("S") == "CAPTURED":
    ok("payments[pay-k-1].status == CAPTURED")
else:
    fail(f"payments[pay-k-1] not CAPTURED: {item}")

item = ddb.get_item(TableName="shipments", Key={"shipment_id": {"S": "shp-k-1"}}).get("Item")
if item:
    ok(f"shipments[shp-k-1] exists (status={item.get('status',{}).get('S')})")
else:
    fail("shipments[shp-k-1] absent")


# Between runs, reset inventory[x] so the failure path's net effect (0) is observable.
ddb.delete_item(TableName="inventory", Key={"sku": {"S": "x"}})


# =====================================================================
section("Failure path: o-2 / k-2 with force_failure=true")
# =====================================================================
res = run_execution("verify-fail", {
    "order_id": "o-2", "sku": "x", "qty": 2, "amount": 100,
    "idempotency_key": "k-2", "force_failure": True,
})
if res["status"] == "SUCCEEDED":
    ok(f"state machine SUCCEEDED (saga handled failure gracefully)")
else:
    fail(f"state machine status = {res['status']}  output={res.get('output')}")

item = ddb.get_item(TableName="inventory", Key={"sku": {"S": "x"}}).get("Item")
if not item:
    ok("inventory[x] absent (compensation reversed the reserve)")
elif item.get("reserved_qty", {}).get("N") == "0":
    ok("inventory[x].reserved_qty == 0 (compensation reversed the reserve)")
else:
    fail(f"inventory[x] not compensated: {item}")

item = ddb.get_item(TableName="payments", Key={"payment_id": {"S": "pay-k-2"}}).get("Item")
if not item:
    ok("payments[pay-k-2] absent (nothing was ever charged)")
elif item.get("status", {}).get("S") == "REFUNDED":
    ok("payments[pay-k-2].status == REFUNDED")
else:
    fail(f"payments[pay-k-2] unexpected: {item}")

item = ddb.get_item(TableName="shipments", Key={"shipment_id": {"S": "shp-k-2"}}).get("Item")
if not item:
    ok("shipments[shp-k-2] absent (ship never ran after charge failure)")
else:
    fail(f"shipments[shp-k-2] should be absent: {item}")


# =====================================================================
section("Notifier log contains both terminal lines")
# =====================================================================
log_group = "/aws/lambda/saga-notifier"
found_ok = found_fail = False
for attempt in range(15):
    try:
        events = logs.filter_log_events(logGroupName=log_group).get("events", [])
    except Exception:
        events = []
    msgs = [e["message"] for e in events]
    found_ok = any("saga-terminal:OrderCompleted" in m and '"order_id": "o-1"' in m for m in msgs)
    found_fail = any("saga-terminal:OrderFailed" in m and '"order_id": "o-2"' in m for m in msgs)
    if found_ok and found_fail:
        break
    time.sleep(1)

if found_ok:
    ok("saga-terminal:OrderCompleted present for o-1")
else:
    fail("saga-terminal:OrderCompleted for o-1 not found in notifier log")
if found_fail:
    ok("saga-terminal:OrderFailed present for o-2")
else:
    fail("saga-terminal:OrderFailed for o-2 not found in notifier log")


# =====================================================================
print()
if failures:
    print(f"FAILED: {len(failures)} check(s)")
    for f in failures:
        print(f"  - {f}")
    sys.exit(1)
print("ALL CHECKS PASSED")

Run verifier

shell

chmod +x /app/verify.py && python3 /app/verify.py

output

== IAM: per-lambda roles scoped to their own tables ==
  [PASS] reserve-inventory-role ddb scoped to ['inventory', 'saga-idempotency']
  [PASS] charge-payment-role ddb scoped to ['payments', 'saga-idempotency']
  [PASS] create-shipment-role ddb scoped to ['saga-idempotency', 'shipments']
  [PASS] release-inventory-role ddb scoped to ['inventory', 'saga-idempotency']
  [PASS] refund-payment-role ddb scoped to ['payments', 'saga-idempotency']
  [PASS] saga-notifier-role ddb scoped to (none)

== IAM: order-saga-role scopes Lambda invoke + events publish ==
  [PASS] order-saga-role invokes exactly the 5 saga lambdas
  [PASS] order-saga-role publishes only to saga-events bus

== IAM: secret access isolated to charge-payment-role ==
  [PASS] charge-payment-role has secretsmanager:GetSecretValue
  [PASS] other roles do not have secretsmanager:GetSecretValue

== IAM: no kms wildcards in any role policy ==
  [PASS] no kms wildcards in any role policy
  [PASS] charge-payment-role kms:Decrypt scoped to specific CMK (3070b521...)

== IAM: notifier role has only EventBridge-target prerequisites (logs) ==
  [PASS] saga-notifier-role has only logs:* actions

== Resource names exist ==
  [PASS] alias/saga-cmk present
  [PASS] secret payment/gateway-apikey present
  [PASS] secret encrypted with CMK (alias/saga-cmk)
  [PASS] ddb table inventory present
  [PASS] ddb table payments present
  [PASS] ddb table shipments present
  [PASS] ddb table saga-idempotency present
  [PASS] saga-idempotency TTL ENABLED
  [PASS] lambda reserve-inventory present
  [PASS] lambda charge-payment present
  [PASS] lambda create-shipment present
  [PASS] lambda release-inventory present
  [PASS] lambda refund-payment present
  [PASS] lambda saga-notifier present
  [PASS] state machine order-saga present
  [PASS] event bus saga-events present
  [PASS] event rule on-saga-terminal present

== Reset test rows so assertions are reproducible ==
  [PASS] cleared inventory[x], payments[pay-k-*], shipments[shp-k-*], idempotency keys for k-1/k-2

== Happy path: o-1 / k-1 ==
  [PASS] state machine SUCCEEDED (2026-04-23 04:58:22.903434+00:00)
  [PASS] inventory[x].reserved_qty == 2
  [PASS] payments[pay-k-1].status == CAPTURED
  [PASS] shipments[shp-k-1] exists (status=CREATED)

== Failure path: o-2 / k-2 with force_failure=true ==
  [PASS] state machine SUCCEEDED (saga handled failure gracefully)
  [PASS] inventory[x].reserved_qty == 0 (compensation reversed the reserve)
  [PASS] payments[pay-k-2] absent (nothing was ever charged)
  [PASS] shipments[shp-k-2] absent (ship never ran after charge failure)

== Notifier log contains both terminal lines ==
  [PASS] saga-terminal:OrderCompleted present for o-1
  [PASS] saga-terminal:OrderFailed present for o-2

ALL CHECKS PASSED

[stdout]
== IAM: per-lambda roles scoped to their own tables ==
  [PASS] reserve-inventory-role ddb scoped to ['inventory', 'saga-idempotency']
  [PASS] charge-payment-role ddb scoped to ['payments', 'saga-idempotency']
  [PASS] create-shipment-role ddb scoped to ['saga-idempotency', 'shipments']
  [PASS] release-inventory-role ddb scoped to ['inventory', 'saga-idempotency']
  [PASS] refund-payment-role ddb scoped to ['payments', 'saga-idempotency']
  [PASS] saga-notifier-role ddb scoped to (none)

== IAM: order-saga-role scopes Lambda invoke + events publish ==
  [PASS] order-saga-role invokes exactly the 5 saga lambdas
  [PASS] order-saga-role publishes only to saga-events bus

== IAM: secret access isolated to charge-payment-role ==
  [PASS] charge-payment-role has secretsmanager:GetSecretValue
  [PASS] other roles do not have secretsmanager:GetSecretValue

== IAM: no kms wildcards in any role policy ==
  [PASS] no kms wildcards in any role policy
  [PASS] charge-payment-role kms:Decrypt scoped to specific CMK (3070b521...)

== IAM: notifier role has only EventBridge-target prerequisites (logs) ==
  [PASS] saga-notifier-role has only logs:* actions

== Resource names exist ==
  [PASS] alias/saga-cmk present
  [PASS] secret payment/gateway-apikey present
  [PASS] secret encrypted with CMK (alias/saga-cmk)
  [PASS] ddb table inventory present
  [PASS] ddb table payments present
  [PASS] ddb table shipments present
  [PASS] ddb table saga-idempotency present
  [PASS] saga-idempotency TTL ENABLED
  [PASS] lambda reserve-inventory present
  [PASS] lambda charge-payment present
  [PASS] lambda create-shipment present
  [PASS] lambda release-inventory present
  [PASS] lambda refund-payment present
  [PASS] lambda saga-notifier present
  [PASS] state machine order-saga present
  [PASS] event bus saga-events present
  [PASS] event rule on-saga-terminal present

== Reset test rows so assertions are reproducible ==
  [PASS] cleared inventory[x], payments[pay-k-*], shipments[shp-k-*], idempotency keys for k-1/k-2

== Happy path: o-1 / k-1 ==
  [PASS] state machine SUCCEEDED (2026-04-23 04:58:22.903434+00:00)
  [PASS] inventory[x].reserved_qty == 2
  [PASS] payments[pay-k-1].status == CAPTURED
  [PASS] shipments[shp-k-1] exists (status=CREATED)

== Failure path: o-2 / k-2 with force_failure=true ==
  [PASS] state machine SUCCEEDED (saga handled failure gracefully)
  [PASS] inventory[x].reserved_qty == 0 (compensation reversed the reserve)
  [PASS] payments[pay-k-2] absent (nothing was ever charged)
  [PASS] shipments[shp-k-2] absent (ship never ran after charge failure)

== Notifier log contains both terminal lines ==
  [PASS] saga-terminal:OrderCompleted present for o-1
  [PASS] saga-terminal:OrderFailed present for o-2

ALL CHECKS PASSED

All 35 checks pass.

Write Lambda sources
Write IAM role & policy documents
Write Step Functions state machine JSON
Write deploy.sh (KMS, secret, DDB, lambdas, roles, SF, EB)
Run deploy.sh against LocalStack
Smoke-test happy + failure paths
Write verify.py
Run verify.py end-to-end

Stack is built and proven. Layout: ``` /app/ lambdas/ reserve_inventory.py, charge_payment.py, create_shipment.py, release_inventory.py, refund_payment.py, saga_notifier.py state_machine.json ASL with Catch → reverse-order compensations + PutEvents deploy.sh creates CMK + alias, CMK-encrypted secret, 4 DDB tables, 7 scoped IAM roles, 6 lambdas, EB bus+rule+target, SM verify.py reads inline policy docs + runs both executions build/ generated zips, resolved SM json, env ``` Key design points: - **Saga shape** , `ReserveInventory → ChargePayment → CreateShipment`; each step has a `Catch` that jumps to the correct compensation chain. Reserve-fail → publish failed (nothing to undo). Charge-fail → release → publish failed. Ship-fail → refund → release → publish failed. Both terminal states publish to `arn:aws:states:::events:putEvents`, so the SM always reaches `SUCCEEDED`. - **Idempotency** , each forward step uses `TransactWriteItems` to atomically claim `{base_key}:{step}` in `saga-idempotency` and mutate the business table. Replays hit the conditional-check and return. Compensations read the forward step's idempotency row to decide no-op vs. reverse, then claim their own `{base_key}:{release|refund}` row. - **Least privilege** (verifier asserts each): - Each lambda role lists only its table + `saga-idempotency`, plus its own log group. - `order-saga-role` lists the 5 exact function ARNs (no `function:*`) and `events:PutEvents` on the one bus ARN. - `charge-payment-role` is the only role with `secretsmanager:GetSecretValue`, and its `kms:Decrypt` is scoped to the specific CMK ARN. - `saga-notifier-role` has only `logs:*` , no DDB, no secret, no EB. - Nothing uses `kms:*` or `Resource:"*"` on a KMS action. - **Failure drive** , `charge-payment` raises when `force_failure=true` in input, which is exactly how the verifier proves the reverse-order compensation actually runs. Final verify output: **35/35 PASS**, both executions reach `SUCCEEDED` in <5s, inventory nets to 0 after the compensated failure, payments/shipments rows are absent for `o-2`, and the notifier log has `saga-terminal:OrderCompleted` for `o-1` and `saga-terminal:OrderFailed` for `o-2`. Re-run anytime with `bash /app/deploy.sh && python3 /app/verify.py` (deploy is idempotent; verifier cleans test rows before each run).

Final diff , what the agent changed

No file changes, the agent left the workspace untouched.

Verifier output , the proof behind the reward

tests/test.sh

============================= test session starts ==============================
platform linux -- Python 3.12.3, pytest-8.4.1, pluggy-1.6.0 -- /root/.cache/uv/archive-v0/U9eW-hOf8nnQez2Lt9-RL/bin/python
cachedir: .pytest_cache
rootdir: /tests
plugins: json-ctrf-0.3.5
collecting ... collected 20 items

../tests/test_outputs.py::test_01_cmk_alias_and_secret_exist PASSED      [  5%]
../tests/test_outputs.py::test_02_all_four_tables_exist PASSED           [ 10%]
../tests/test_outputs.py::test_03_all_six_lambdas_exist PASSED           [ 15%]
../tests/test_outputs.py::test_04_state_machine_and_bus_exist PASSED     [ 20%]
../tests/test_outputs.py::test_05_terminal_rule_exists_on_saga_bus PASSED [ 25%]
../tests/test_outputs.py::test_06_every_forward_state_has_catch_to_compensation PASSED [ 30%]
../tests/test_outputs.py::test_07_compensations_run_in_lifo_order PASSED [ 35%]
../tests/test_outputs.py::test_08_compensation_states_preserve_error_via_result_path PASSED [ 40%]
../tests/test_outputs.py::test_09_idempotency_table_has_ttl_and_correct_pk PASSED [ 45%]
../tests/test_outputs.py::test_10_each_lambda_role_scoped_to_its_own_table PASSED [ 50%]
../tests/test_outputs.py::test_11_only_charge_role_reads_the_payment_secret PASSED [ 55%]
../tests/test_outputs.py::test_12_saga_role_lambda_invoke_not_wildcarded PASSED [ 60%]
../tests/test_outputs.py::test_13_rule_pattern_matches_both_terminal_types_with_array_wrap PASSED [ 65%]
../tests/test_outputs.py::test_14_charge_role_kms_decrypt_scoped_not_wildcard PASSED [ 70%]
../tests/test_outputs.py::test_15_rule_targets_notifier_lambda PASSED    [ 75%]
../tests/test_outputs.py::test_16_notifier_role_is_least_privilege_no_ddb_kms_secrets PASSED [ 80%]
../tests/test_outputs.py::test_17_notifier_lambda_resource_policy_allows_events_with_source_arn PASSED [ 85%]
../tests/test_outputs.py::test_18_happy_path_completes_and_writes_all_three_tables PASSED [ 90%]
../tests/test_outputs.py::test_19_failure_path_triggers_compensation_and_rolls_back_inventory PASSED [ 95%]
../tests/test_outputs.py::test_20_idempotency_prevents_double_reservation PASSED [100%]

=============================== warnings summary ===============================
test_outputs.py: 136 warnings
  /root/.cache/uv/archive-v0/U9eW-hOf8nnQez2Lt9-RL/lib/python3.12/site-packages/botocore/auth.py:424: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
    datetime_now = datetime.datetime.utcnow()

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
==================================== PASSES ====================================
=========================== short test summary info ============================
PASSED ../tests/test_outputs.py::test_01_cmk_alias_and_secret_exist
PASSED ../tests/test_outputs.py::test_02_all_four_tables_exist
PASSED ../tests/test_outputs.py::test_03_all_six_lambdas_exist
PASSED ../tests/test_outputs.py::test_04_state_machine_and_bus_exist
PASSED ../tests/test_outputs.py::test_05_terminal_rule_exists_on_saga_bus
PASSED ../tests/test_outputs.py::test_06_every_forward_state_has_catch_to_compensation
PASSED ../tests/test_outputs.py::test_07_compensations_run_in_lifo_order
PASSED ../tests/test_outputs.py::test_08_compensation_states_preserve_error_via_result_path
PASSED ../tests/test_outputs.py::test_09_idempotency_table_has_ttl_and_correct_pk
PASSED ../tests/test_outputs.py::test_10_each_lambda_role_scoped_to_its_own_table
PASSED ../tests/test_outputs.py::test_11_only_charge_role_reads_the_payment_secret
PASSED ../tests/test_outputs.py::test_12_saga_role_lambda_invoke_not_wildcarded
PASSED ../tests/test_outputs.py::test_13_rule_pattern_matches_both_terminal_types_with_array_wrap
PASSED ../tests/test_outputs.py::test_14_charge_role_kms_decrypt_scoped_not_wildcard
PASSED ../tests/test_outputs.py::test_15_rule_targets_notifier_lambda
PASSED ../tests/test_outputs.py::test_16_notifier_role_is_least_privilege_no_ddb_kms_secrets
PASSED ../tests/test_outputs.py::test_17_notifier_lambda_resource_policy_allows_events_with_source_arn
PASSED ../tests/test_outputs.py::test_18_happy_path_completes_and_writes_all_three_tables
PASSED ../tests/test_outputs.py::test_19_failure_path_triggers_compensation_and_rolls_back_inventory
PASSED ../tests/test_outputs.py::test_20_idempotency_prevents_double_reservation
====================== 20 passed, 136 warnings in 23.15s =======================

Reproduce this trial: git checkout 2f94510 && PYTHONPATH=src python3 scripts/build_site.py , then open trial/trial_59aa66d84fea400e. Re-running the agent live requires EVAL_PLATFORM_ENABLE_OAUTH_SMOKE=1 and is non-deterministic.

Trial trial_59aa66d84fea400e · verifier authoritative; classifier explanatory.