SyncVals · Trajectory

sfn-saga-compensation-orchestrator

claude-code claude-opus-4-7 ✗ failed BAD_FAILURE ↑ View task

Solved from the instruction alone, tests/ and solution/ were withheld from the agent's workspace and restored only for grading.

Reward = tests/test.sh exit code (0 → resolved); the classification below is post-hoc and cannot change it.

Classification , post-hoc; cannot change the reward

BAD_FAILUREThe task is at fault, underspecified/contradictory instruction, brittle/flaky tests, or tests demanding undiscoverable behavior.

SubtypeUnderspecified Instruction

EvidenceTest `test_08_compensation_states_preserve_error_via_result_path` fails because compensation state 'RefundAfterShipFail' is missing the `ResultPath` field. The test explicitly states: 'Compensation Task states must use ResultPath that doesn't clobber the error payload - either ResultPath:null or a side path like $.compensationResult. LLMs commonly omit ResultPath and let the compensation's return value overwrite the error context.' The instruction.md discusses compensations extensively (idempotency, reverse order, when they fire) but never mentions ResultPath. task.toml metadata documents that tests check 'ResultPath preservation of the original error', but agents only see instruction.md, not task.toml metadata. 19/20 tests passed, confirming the agent understood the overall saga architecture."

Root causeThe instruction.md describes the saga compensation logic and requirements comprehensively but omits the Step Functions-specific implementation detail of ResultPath preservation. This detail is documented in task.toml metadata (which agents cannot see) but not in the instruction that agents actually receive, creating a specification gap.

RecommendationUpdate instruction.md to explicitly document that compensation Task states in the state machine must use `ResultPath` set to either `null` or a side path like `$.compensationResult` to preserve the original error context for downstream states. Add a note that omitting ResultPath will cause the compensation's return value to clobber the error payload used by downstream logging/routing. Alternatively, if this is truly an advanced Step Functions best practice beyond the scope of the task, adjust the test to not require it or clearly signal in the instruction that Step Functions state machine best practices for error handling will be tested."

Trajectory

Tool-by-tool agent trajectory

38 tool calls · 4 tool types · 49 steps

so the order service keeps leaving garbage rows. inventory gets reserved, payment call times out, we retry, now inventory is double-reserved because the retry worked the second time. same deal when shipping fails after payment , we'd just charge the customer and leave them with no shipment. the ops slack is basically me apologizing at this point. we need a real saga. if any step fails, **undo** the earlier steps in reverse order. no half-states. step functions runs it, localstack , `http://localstack:4566`, creds already exported (`AWS_ACCESS_KEY_ID=test`, same for secret, region `us-east-1`). `aws`, `python3`, `boto3`, `jq`, `zip`. build from zero. shape of it: - a state machine runs three forward steps: **reserve inventory**, **charge payment**, **create shipment**. each step is its own lambda. each step writes to its own ddb table. - if any step fails, the saga fires compensations **in reverse order** of what actually ran. if reserve succeeded but charge failed, you refund (no-op, nothing was charged) and release. if charge succeeded but shipment failed, you refund payment and release inventory. if reserve failed, nothing to undo. - compensations are their own lambdas too. they must be idempotent , if state-function retries a compensation, running it twice should not corrupt the table. - each forward step gets an idempotency key from the input so a lambda retry inside a single step doesn't double-decrement inventory. - payment calls a "gateway" that needs an api key. store the key in secrets manager , customer-managed kms key encrypts the secret. the payment lambda reads the secret, nobody else should be able to. - on terminal state (completed or failed-compensated), the saga publishes an event to a custom event bus. a notifier lambda sits on that bus and writes one log line starting with `saga-terminal:`. to actually prove compensations work, the charge-payment lambda accepts a `force_failure` flag in its input , when true, it raises. the verifier uses this to drive a failure run and assert the state snaps back. done looks like this. two executions: **happy path** , `start-execution` with `{"order_id": "o-1", "sku": "x", "qty": 2, "amount": 100, "idempotency_key": "k-1"}`: - sm reaches `SUCCEEDED` within 60s - inventory row for sku `x` reflects `reserved_qty = 2` - payments row exists with status `CAPTURED` - shipments row exists - notifier log has `saga-terminal:OrderCompleted` **failure path** , same input but `"force_failure": true, "order_id": "o-2", "idempotency_key": "k-2"`: - sm still reaches `SUCCEEDED` (saga handles it gracefully, not a hard abort) - inventory reserved_qty back to 0 (compensation ran) - payments row is absent or `REFUNDED` (nothing was charged since charge failed, so a no-op refund is also ok) - shipments row does **not** exist - notifier log has `saga-terminal:OrderFailed` least privilege. the verifier reads policy docs directly. each lambda role only gets its own table. saga role can't wildcard on lambda invoke. payment role is the only one that reads the secret. notifier role has nothing but eventbridge + logs. no `*` on kms anywhere. names the verifier grabs by hand, use them exactly: | thing | name | |-|-| | kms alias | `alias/saga-cmk` | | secret | `payment/gateway-apikey` | | inventory table | `inventory` (pk `sku` / String) | | payments table | `payments` (pk `payment_id` / String) | | shipments table | `shipments` (pk `shipment_id` / String) | | idempotency table | `saga-idempotency` (pk `idempotency_key` / String, ttl enabled) | | reserve lambda | `reserve-inventory` | | charge lambda | `charge-payment` | | ship lambda | `create-shipment` | | release compensation | `release-inventory` | | refund compensation | `refund-payment` | | notifier lambda | `saga-notifier` | | reserve role | `reserve-inventory-role` | | charge role | `charge-payment-role` | | ship role | `create-shipment-role` | | release role | `release-inventory-role` | | refund role | `refund-payment-role` | | notifier role | `saga-notifier-role` | | state machine | `order-saga` | | state machine role | `order-saga-role` | | event bus | `saga-events` | | event rule | `on-saga-terminal` | eventbridge detail shape: source `order.saga`, detail-type is one of `OrderCompleted` or `OrderFailed`. the rule matches both and routes to the notifier lambda. python3.11 for all lambdas. localstack community 4.13.1. the "payment gateway" is fake , charge-payment reads the secret, logs that it did, and marks the payments row CAPTURED. no actual network call.

ToolSearch

{
  "query": "select:TodoWrite",
  "max_results": 1
}

, output, 
{"type": "tool_reference", "tool_name": "TodoWrite"}

Explore /app working directory and check LocalStack connectivity
Create KMS CMK, alias, and encrypted secret
Create DynamoDB tables (inventory, payments, shipments, saga-idempotency)
Create EventBridge bus and rule target
Write and package all Lambda code (reserve/charge/ship/release/refund/notifier)
Create least-privilege IAM roles and policies for each Lambda
Deploy Lambda functions with their roles
Create state machine role and state machine
Run happy-path and failure-path verifier scenarios

List /app and verify LocalStack connectivity

shell

ls -la /app && echo "---" && aws --endpoint-url=http://localstack:4566 sts get-caller-identity

output

total 8
drwxr-xr-x 2 root root 4096 Apr 21 20:05 .
drwxr-xr-x 1 root root 4096 Apr 23 03:40 ..
---
{
    "UserId": "AKIAIOSFODNN7EXAMPLE",
    "Account": "000000000000",
    "Arn": "arn:aws:iam::000000000000:root"
}

[stdout]
total 8
drwxr-xr-x 2 root root 4096 Apr 21 20:05 .
drwxr-xr-x 1 root root 4096 Apr 23 03:40 ..
---
{
    "UserId": "AKIAIOSFODNN7EXAMPLE",
    "Account": "000000000000",
    "Arn": "arn:aws:iam::000000000000:root"
}

Create workspace dirs and verify tooling

shell

mkdir -p /app/lambdas /app/policies /app/build && ls && which zip python3 jq

output

build
lambdas
policies
/usr/bin/zip
/opt/venv/bin/python3
/usr/bin/jq

[stdout]
build
lambdas
policies
/usr/bin/zip
/opt/venv/bin/python3
/usr/bin/jq

Create KMS CMK and alias saga-cmk

shell

export AWS_ENDPOINT_URL=http://localstack:4566
echo "Creating KMS key..."
KEY_META=$(aws --endpoint-url=$AWS_ENDPOINT_URL kms create-key --description "saga cmk" --key-usage ENCRYPT_DECRYPT --origin AWS_KMS)
KEY_ID=$(echo "$KEY_META" | jq -r '.KeyMetadata.KeyId')
echo "KeyId=$KEY_ID"
aws --endpoint-url=$AWS_ENDPOINT_URL kms create-alias --alias-name alias/saga-cmk --target-key-id $KEY_ID
aws --endpoint-url=$AWS_ENDPOINT_URL kms list-aliases | jq '.Aliases[] | select(.AliasName=="alias/saga-cmk")'
echo "$KEY_ID" > /app/build/kms_key_id

output

Creating KMS key...
KeyId=30f1d7f1-8060-45df-b508-8248de376c35
{
  "AliasName": "alias/saga-cmk",
  "AliasArn": "arn:aws:kms:us-east-1:000000000000:alias/saga-cmk",
  "TargetKeyId": "30f1d7f1-8060-45df-b508-8248de376c35",
  "CreationDate": "2026-04-23T03:42:41.318726+00:00"
}

[stdout]
Creating KMS key...
KeyId=30f1d7f1-8060-45df-b508-8248de376c35
{
  "AliasName": "alias/saga-cmk",
  "AliasArn": "arn:aws:kms:us-east-1:000000000000:alias/saga-cmk",
  "TargetKeyId": "30f1d7f1-8060-45df-b508-8248de376c35",
  "CreationDate": "2026-04-23T03:42:41.318726+00:00"
}

Create Secrets Manager secret with CMK encryption

shell

export AWS_ENDPOINT_URL=http://localstack:4566
aws --endpoint-url=$AWS_ENDPOINT_URL secretsmanager create-secret \
  --name "payment/gateway-apikey" \
  --description "payment gateway api key" \
  --kms-key-id "alias/saga-cmk" \
  --secret-string "sk_test_fakegatewaykey_xyz" | jq '.ARN,.Name'

output

"arn:aws:secretsmanager:us-east-1:000000000000:secret:payment/gateway-apikey-hTcpFU"
"payment/gateway-apikey"

[stdout]
"arn:aws:secretsmanager:us-east-1:000000000000:secret:payment/gateway-apikey-hTcpFU"
"payment/gateway-apikey"

Explore /app working directory and check LocalStack connectivity
Create KMS CMK, alias, and encrypted secret
Create DynamoDB tables (inventory, payments, shipments, saga-idempotency)
Create EventBridge bus and rule target
Write and package all Lambda code (reserve/charge/ship/release/refund/notifier)
Create least-privilege IAM roles and policies for each Lambda
Deploy Lambda functions with their roles
Create state machine role and state machine
Run happy-path and failure-path verifier scenarios

Create DynamoDB tables and enable TTL on idempotency

shell

export AWS_ENDPOINT_URL=http://localstack:4566
for t in \
  "inventory sku" \
  "payments payment_id" \
  "shipments shipment_id" \
  "saga-idempotency idempotency_key" ; do
  name=$(echo $t | cut -d' ' -f1)
  key=$(echo $t | cut -d' ' -f2)
  echo "Creating $name (pk=$key)"
  aws --endpoint-url=$AWS_ENDPOINT_URL dynamodb create-table \
    --table-name "$name" \
    --attribute-definitions AttributeName=$key,AttributeType=S \
    --key-schema AttributeName=$key,KeyType=HASH \
    --billing-mode PAY_PER_REQUEST > /dev/null
done

# enable TTL on saga-idempotency
aws --endpoint-url=$AWS_ENDPOINT_URL dynamodb update-time-to-live \
  --table-name saga-idempotency \
  --time-to-live-specification "Enabled=true, AttributeName=expires_at" | jq '.'

aws --endpoint-url=$AWS_ENDPOINT_URL dynamodb list-tables

output

Creating inventory (pk=sku)
Creating payments (pk=payment_id)
Creating shipments (pk=shipment_id)
Creating saga-idempotency (pk=idempotency_key)
{
  "TimeToLiveSpecification": {
    "Enabled": true,
    "AttributeName": "expires_at"
  }
}
{
    "TableNames": [
        "inventory",
        "payments",
        "saga-idempotency",
        "shipments"
    ]
}

[stdout]
Creating inventory (pk=sku)
Creating payments (pk=payment_id)
Creating shipments (pk=shipment_id)
Creating saga-idempotency (pk=idempotency_key)
{
  "TimeToLiveSpecification": {
    "Enabled": true,
    "AttributeName": "expires_at"
  }
}
{
    "TableNames": [
        "inventory",
        "payments",
        "saga-idempotency",
        "shipments"
    ]
}

Create EventBridge custom bus saga-events

shell

export AWS_ENDPOINT_URL=http://localstack:4566
aws --endpoint-url=$AWS_ENDPOINT_URL events create-event-bus --name saga-events | jq '.EventBusArn'

output

"arn:aws:events:us-east-1:000000000000:event-bus/saga-events"

[stdout]
"arn:aws:events:us-east-1:000000000000:event-bus/saga-events"

Now I'll write all the Lambda code.

write · /app/lambdas/reserve_inventory.py

import os
import time
import boto3
from botocore.config import Config

ENDPOINT = os.environ.get("AWS_ENDPOINT_URL", "http://localstack:4566")
TABLE = os.environ["INVENTORY_TABLE"]
IDEMPOTENCY_TABLE = os.environ["IDEMPOTENCY_TABLE"]
STEP = "reserve"
TTL_SECONDS = 24 * 60 * 60

cfg = Config(retries={"max_attempts": 3})
ddb = boto3.client("dynamodb", endpoint_url=ENDPOINT, config=cfg)


def _idem_key(key):
    return f"{STEP}:{key}"


def lambda_handler(event, context):
    idem = event["idempotency_key"]
    sku = event["sku"]
    qty = int(event["qty"])
    stored_key = _idem_key(idem)

    # If this step already ran for this idempotency key, return the prior result.
    existing = ddb.get_item(
        TableName=IDEMPOTENCY_TABLE,
        Key={"idempotency_key": {"S": stored_key}},
        ConsistentRead=True,
    )
    if "Item" in existing:
        return {**event, "reserve_status": "ALREADY_RESERVED"}

    # Atomic increment of reserved_qty on the inventory row.
    ddb.update_item(
        TableName=TABLE,
        Key={"sku": {"S": sku}},
        UpdateExpression="ADD reserved_qty :q",
        ExpressionAttributeValues={":q": {"N": str(qty)}},
    )

    ddb.put_item(
        TableName=IDEMPOTENCY_TABLE,
        Item={
            "idempotency_key": {"S": stored_key},
            "step": {"S": STEP},
            "sku": {"S": sku},
            "qty": {"N": str(qty)},
            "expires_at": {"N": str(int(time.time()) + TTL_SECONDS)},
        },
        ConditionExpression="attribute_not_exists(idempotency_key)",
    )

    return {**event, "reserve_status": "RESERVED"}

write · /app/lambdas/charge_payment.py

import os
import time
import boto3
from botocore.config import Config

ENDPOINT = os.environ.get("AWS_ENDPOINT_URL", "http://localstack:4566")
PAYMENTS_TABLE = os.environ["PAYMENTS_TABLE"]
IDEMPOTENCY_TABLE = os.environ["IDEMPOTENCY_TABLE"]
SECRET_ID = os.environ["SECRET_ID"]
STEP = "charge"
TTL_SECONDS = 24 * 60 * 60

cfg = Config(retries={"max_attempts": 3})
ddb = boto3.client("dynamodb", endpoint_url=ENDPOINT, config=cfg)
sm = boto3.client("secretsmanager", endpoint_url=ENDPOINT, config=cfg)


def _idem_key(key):
    return f"{STEP}:{key}"


def lambda_handler(event, context):
    if event.get("force_failure"):
        raise RuntimeError("forced failure: charge-payment")

    idem = event["idempotency_key"]
    order_id = event["order_id"]
    amount = int(event["amount"])
    payment_id = f"pay-{order_id}"
    stored_key = _idem_key(idem)

    existing = ddb.get_item(
        TableName=IDEMPOTENCY_TABLE,
        Key={"idempotency_key": {"S": stored_key}},
        ConsistentRead=True,
    )
    if "Item" in existing:
        return {**event, "payment_id": payment_id, "charge_status": "ALREADY_CAPTURED"}

    # Read (but do not log/return) the gateway api key before charging.
    sm.get_secret_value(SecretId=SECRET_ID)
    print(f"charge-payment: fetched gateway credential, simulating charge for {order_id}")

    ddb.put_item(
        TableName=PAYMENTS_TABLE,
        Item={
            "payment_id": {"S": payment_id},
            "order_id": {"S": order_id},
            "amount": {"N": str(amount)},
            "status": {"S": "CAPTURED"},
        },
    )

    ddb.put_item(
        TableName=IDEMPOTENCY_TABLE,
        Item={
            "idempotency_key": {"S": stored_key},
            "step": {"S": STEP},
            "payment_id": {"S": payment_id},
            "expires_at": {"N": str(int(time.time()) + TTL_SECONDS)},
        },
        ConditionExpression="attribute_not_exists(idempotency_key)",
    )

    return {**event, "payment_id": payment_id, "charge_status": "CAPTURED"}

write · /app/lambdas/create_shipment.py

import os
import time
import boto3
from botocore.config import Config

ENDPOINT = os.environ.get("AWS_ENDPOINT_URL", "http://localstack:4566")
SHIPMENTS_TABLE = os.environ["SHIPMENTS_TABLE"]
IDEMPOTENCY_TABLE = os.environ["IDEMPOTENCY_TABLE"]
STEP = "ship"
TTL_SECONDS = 24 * 60 * 60

cfg = Config(retries={"max_attempts": 3})
ddb = boto3.client("dynamodb", endpoint_url=ENDPOINT, config=cfg)


def _idem_key(key):
    return f"{STEP}:{key}"


def lambda_handler(event, context):
    if event.get("force_shipment_failure"):
        raise RuntimeError("forced failure: create-shipment")

    idem = event["idempotency_key"]
    order_id = event["order_id"]
    shipment_id = f"ship-{order_id}"
    stored_key = _idem_key(idem)

    existing = ddb.get_item(
        TableName=IDEMPOTENCY_TABLE,
        Key={"idempotency_key": {"S": stored_key}},
        ConsistentRead=True,
    )
    if "Item" in existing:
        return {**event, "shipment_id": shipment_id, "ship_status": "ALREADY_SHIPPED"}

    ddb.put_item(
        TableName=SHIPMENTS_TABLE,
        Item={
            "shipment_id": {"S": shipment_id},
            "order_id": {"S": order_id},
            "sku": {"S": event["sku"]},
            "qty": {"N": str(int(event["qty"]))},
            "status": {"S": "CREATED"},
        },
    )

    ddb.put_item(
        TableName=IDEMPOTENCY_TABLE,
        Item={
            "idempotency_key": {"S": stored_key},
            "step": {"S": STEP},
            "shipment_id": {"S": shipment_id},
            "expires_at": {"N": str(int(time.time()) + TTL_SECONDS)},
        },
        ConditionExpression="attribute_not_exists(idempotency_key)",
    )

    return {**event, "shipment_id": shipment_id, "ship_status": "CREATED"}

write · /app/lambdas/release_inventory.py

import os
import time
import boto3
from botocore.config import Config

ENDPOINT = os.environ.get("AWS_ENDPOINT_URL", "http://localstack:4566")
TABLE = os.environ["INVENTORY_TABLE"]
IDEMPOTENCY_TABLE = os.environ["IDEMPOTENCY_TABLE"]
STEP = "release"
TTL_SECONDS = 24 * 60 * 60

cfg = Config(retries={"max_attempts": 3})
ddb = boto3.client("dynamodb", endpoint_url=ENDPOINT, config=cfg)


def lambda_handler(event, context):
    idem = event["idempotency_key"]
    sku = event["sku"]
    qty = int(event["qty"])
    compensation_key = f"{STEP}:{idem}"

    # If compensation already ran, do nothing.
    existing = ddb.get_item(
        TableName=IDEMPOTENCY_TABLE,
        Key={"idempotency_key": {"S": compensation_key}},
        ConsistentRead=True,
    )
    if "Item" in existing:
        return {**event, "release_status": "ALREADY_RELEASED"}

    # Only decrement if the forward step actually wrote a reservation.
    forward = ddb.get_item(
        TableName=IDEMPOTENCY_TABLE,
        Key={"idempotency_key": {"S": f"reserve:{idem}"}},
        ConsistentRead=True,
    )
    if "Item" in forward:
        ddb.update_item(
            TableName=TABLE,
            Key={"sku": {"S": sku}},
            UpdateExpression="ADD reserved_qty :q",
            ExpressionAttributeValues={":q": {"N": str(-qty)}},
        )

    ddb.put_item(
        TableName=IDEMPOTENCY_TABLE,
        Item={
            "idempotency_key": {"S": compensation_key},
            "step": {"S": STEP},
            "sku": {"S": sku},
            "qty": {"N": str(qty)},
            "expires_at": {"N": str(int(time.time()) + TTL_SECONDS)},
        },
        ConditionExpression="attribute_not_exists(idempotency_key)",
    )

    return {**event, "release_status": "RELEASED"}

write · /app/lambdas/refund_payment.py

import os
import time
import boto3
from botocore.config import Config

ENDPOINT = os.environ.get("AWS_ENDPOINT_URL", "http://localstack:4566")
PAYMENTS_TABLE = os.environ["PAYMENTS_TABLE"]
IDEMPOTENCY_TABLE = os.environ["IDEMPOTENCY_TABLE"]
STEP = "refund"
TTL_SECONDS = 24 * 60 * 60

cfg = Config(retries={"max_attempts": 3})
ddb = boto3.client("dynamodb", endpoint_url=ENDPOINT, config=cfg)


def lambda_handler(event, context):
    idem = event["idempotency_key"]
    order_id = event["order_id"]
    payment_id = f"pay-{order_id}"
    compensation_key = f"{STEP}:{idem}"

    existing = ddb.get_item(
        TableName=IDEMPOTENCY_TABLE,
        Key={"idempotency_key": {"S": compensation_key}},
        ConsistentRead=True,
    )
    if "Item" in existing:
        return {**event, "refund_status": "ALREADY_REFUNDED"}

    # Charge might not have written a row (failed before capture). No-op refund if absent.
    got = ddb.get_item(
        TableName=PAYMENTS_TABLE,
        Key={"payment_id": {"S": payment_id}},
        ConsistentRead=True,
    )
    status = "NOOP"
    if "Item" in got:
        ddb.update_item(
            TableName=PAYMENTS_TABLE,
            Key={"payment_id": {"S": payment_id}},
            UpdateExpression="SET #s = :s",
            ExpressionAttributeNames={"#s": "status"},
            ExpressionAttributeValues={":s": {"S": "REFUNDED"}},
        )
        status = "REFUNDED"

    ddb.put_item(
        TableName=IDEMPOTENCY_TABLE,
        Item={
            "idempotency_key": {"S": compensation_key},
            "step": {"S": STEP},
            "payment_id": {"S": payment_id},
            "expires_at": {"N": str(int(time.time()) + TTL_SECONDS)},
        },
        ConditionExpression="attribute_not_exists(idempotency_key)",
    )

    return {**event, "refund_status": status}

write · /app/lambdas/saga_notifier.py

import json


def lambda_handler(event, context):
    # EventBridge delivers one event per invocation.
    detail_type = event.get("detail-type", "UnknownDetailType")
    detail = event.get("detail", {})
    print(f"saga-terminal:{detail_type} {json.dumps(detail, sort_keys=True)}")
    return {"ok": True}

Now packaging the lambdas.

Package each Lambda into its own zip

shell

cd /app/lambdas
for f in reserve_inventory charge_payment create_shipment release_inventory refund_payment saga_notifier; do
  zip -qj "/app/build/${f}.zip" "${f}.py"
done
ls -la /app/build

output

total 36
drwxr-xr-x 2 root root 4096 Apr 23 03:44 .
drwxr-xr-x 1 root root 4096 Apr 23 03:42 ..
-rw-r--r-- 1 root root 1021 Apr 23 03:44 charge_payment.zip
-rw-r--r-- 1 root root  896 Apr 23 03:44 create_shipment.zip
-rw-r--r-- 1 root root   37 Apr 23 03:42 kms_key_id
-rw-r--r-- 1 root root  931 Apr 23 03:44 refund_payment.zip
-rw-r--r-- 1 root root  925 Apr 23 03:44 release_inventory.zip
-rw-r--r-- 1 root root  912 Apr 23 03:44 reserve_inventory.zip
-rw-r--r-- 1 root root  382 Apr 23 03:44 saga_notifier.zip

[stdout]
total 36
drwxr-xr-x 2 root root 4096 Apr 23 03:44 .
drwxr-xr-x 1 root root 4096 Apr 23 03:42 ..
-rw-r--r-- 1 root root 1021 Apr 23 03:44 charge_payment.zip
-rw-r--r-- 1 root root  896 Apr 23 03:44 create_shipment.zip
-rw-r--r-- 1 root root   37 Apr 23 03:42 kms_key_id
-rw-r--r-- 1 root root  931 Apr 23 03:44 refund_payment.zip
-rw-r--r-- 1 root root  925 Apr 23 03:44 release_inventory.zip
-rw-r--r-- 1 root root  912 Apr 23 03:44 reserve_inventory.zip
-rw-r--r-- 1 root root  382 Apr 23 03:44 saga_notifier.zip

Explore /app working directory and check LocalStack connectivity
Create KMS CMK, alias, and encrypted secret
Create DynamoDB tables (inventory, payments, shipments, saga-idempotency)
Create EventBridge bus and rule target
Write and package all Lambda code (reserve/charge/ship/release/refund/notifier)
Create least-privilege IAM roles and policies for each Lambda
Deploy Lambda functions with their roles
Create state machine role and state machine
Run happy-path and failure-path verifier scenarios

Capture KMS and Secret ARNs for IAM scoping

shell

export AWS_ENDPOINT_URL=http://localstack:4566
aws --endpoint-url=$AWS_ENDPOINT_URL kms describe-key --key-id alias/saga-cmk | jq -r '.KeyMetadata.Arn' > /app/build/kms_key_arn
aws --endpoint-url=$AWS_ENDPOINT_URL secretsmanager describe-secret --secret-id payment/gateway-apikey | jq -r '.ARN' > /app/build/secret_arn
cat /app/build/kms_key_arn /app/build/secret_arn

output

arn:aws:kms:us-east-1:000000000000:key/30f1d7f1-8060-45df-b508-8248de376c35
arn:aws:secretsmanager:us-east-1:000000000000:secret:payment/gateway-apikey-hTcpFU

[stdout]
arn:aws:kms:us-east-1:000000000000:key/30f1d7f1-8060-45df-b508-8248de376c35
arn:aws:secretsmanager:us-east-1:000000000000:secret:payment/gateway-apikey-hTcpFU

write · /app/policies/trust-lambda.json

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "Service": "lambda.amazonaws.com" },
      "Action": "sts:AssumeRole"
    }
  ]
}

write · /app/policies/trust-states.json

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "Service": "states.amazonaws.com" },
      "Action": "sts:AssumeRole"
    }
  ]
}

Emit least-privilege policy JSON for all roles

shell

cat > /app/build/write-policies.sh <<'EOF'
#!/bin/bash
set -euo pipefail
ACC=000000000000
REGION=us-east-1
KMS_ARN="$(cat /app/build/kms_key_arn)"
SECRET_ARN="$(cat /app/build/secret_arn)"
INV_ARN="arn:aws:dynamodb:${REGION}:${ACC}:table/inventory"
PAY_ARN="arn:aws:dynamodb:${REGION}:${ACC}:table/payments"
SHIP_ARN="arn:aws:dynamodb:${REGION}:${ACC}:table/shipments"
IDEM_ARN="arn:aws:dynamodb:${REGION}:${ACC}:table/saga-idempotency"
BUS_ARN="arn:aws:events:${REGION}:${ACC}:event-bus/saga-events"

LOGS='{"Effect":"Allow","Action":["logs:CreateLogGroup","logs:CreateLogStream","logs:PutLogEvents"],"Resource":"*"}'

write() {
  local file="$1"; shift
  local statements="$1"
  cat > "$file" <<JSON
{"Version":"2012-10-17","Statement":${statements}}
JSON
}

# reserve-inventory: UpdateItem on inventory + Get/Put on idempotency + logs
write /app/policies/reserve-inventory-policy.json "[
  ${LOGS},
  {\"Effect\":\"Allow\",\"Action\":[\"dynamodb:UpdateItem\"],\"Resource\":\"${INV_ARN}\"},
  {\"Effect\":\"Allow\",\"Action\":[\"dynamodb:GetItem\",\"dynamodb:PutItem\"],\"Resource\":\"${IDEM_ARN}\"}
]"

# charge-payment: PutItem on payments + Get/Put on idempotency + secret read + kms decrypt scoped + logs
write /app/policies/charge-payment-policy.json "[
  ${LOGS},
  {\"Effect\":\"Allow\",\"Action\":[\"dynamodb:PutItem\"],\"Resource\":\"${PAY_ARN}\"},
  {\"Effect\":\"Allow\",\"Action\":[\"dynamodb:GetItem\",\"dynamodb:PutItem\"],\"Resource\":\"${IDEM_ARN}\"},
  {\"Effect\":\"Allow\",\"Action\":[\"secretsmanager:GetSecretValue\"],\"Resource\":\"${SECRET_ARN}\"},
  {\"Effect\":\"Allow\",\"Action\":[\"kms:Decrypt\"],\"Resource\":\"${KMS_ARN}\"}
]"

# create-shipment: PutItem on shipments + Get/Put on idempotency + logs
write /app/policies/create-shipment-policy.json "[
  ${LOGS},
  {\"Effect\":\"Allow\",\"Action\":[\"dynamodb:PutItem\"],\"Resource\":\"${SHIP_ARN}\"},
  {\"Effect\":\"Allow\",\"Action\":[\"dynamodb:GetItem\",\"dynamodb:PutItem\"],\"Resource\":\"${IDEM_ARN}\"}
]"

# release-inventory: UpdateItem on inventory + Get/Put on idempotency + logs
write /app/policies/release-inventory-policy.json "[
  ${LOGS},
  {\"Effect\":\"Allow\",\"Action\":[\"dynamodb:UpdateItem\"],\"Resource\":\"${INV_ARN}\"},
  {\"Effect\":\"Allow\",\"Action\":[\"dynamodb:GetItem\",\"dynamodb:PutItem\"],\"Resource\":\"${IDEM_ARN}\"}
]"

# refund-payment: Get/UpdateItem on payments + Get/Put on idempotency + logs
write /app/policies/refund-payment-policy.json "[
  ${LOGS},
  {\"Effect\":\"Allow\",\"Action\":[\"dynamodb:GetItem\",\"dynamodb:UpdateItem\"],\"Resource\":\"${PAY_ARN}\"},
  {\"Effect\":\"Allow\",\"Action\":[\"dynamodb:GetItem\",\"dynamodb:PutItem\"],\"Resource\":\"${IDEM_ARN}\"}
]"

# notifier: only logs
write /app/policies/saga-notifier-policy.json "[
  ${LOGS}
]"

# state machine role: invoke exactly the 5 step lambdas + PutEvents to the specific bus
LAMBDAS=(
  "arn:aws:lambda:${REGION}:${ACC}:function:reserve-inventory"
  "arn:aws:lambda:${REGION}:${ACC}:function:charge-payment"
  "arn:aws:lambda:${REGION}:${ACC}:function:create-shipment"
  "arn:aws:lambda:${REGION}:${ACC}:function:release-inventory"
  "arn:aws:lambda:${REGION}:${ACC}:function:refund-payment"
)
LAMBDA_JSON=$(printf ',\"%s\"' "${LAMBDAS[@]}")
LAMBDA_JSON="[${LAMBDA_JSON:1}]"
write /app/policies/order-saga-policy.json "[
  ${LOGS},
  {\"Effect\":\"Allow\",\"Action\":[\"lambda:InvokeFunction\"],\"Resource\":${LAMBDA_JSON}},
  {\"Effect\":\"Allow\",\"Action\":[\"events:PutEvents\"],\"Resource\":\"${BUS_ARN}\"}
]"

echo "policies written:"
ls /app/policies
EOF
bash /app/build/write-policies.sh

output

policies written:
charge-payment-policy.json
create-shipment-policy.json
order-saga-policy.json
refund-payment-policy.json
release-inventory-policy.json
reserve-inventory-policy.json
saga-notifier-policy.json
trust-lambda.json
trust-states.json

[stdout]
policies written:
charge-payment-policy.json
create-shipment-policy.json
order-saga-policy.json
refund-payment-policy.json
release-inventory-policy.json
reserve-inventory-policy.json
saga-notifier-policy.json
trust-lambda.json
trust-states.json

Sanity check policy JSON

shell

for f in /app/policies/*-policy.json; do
  echo "=== $f ==="
  jq . "$f"
done

output

=== /app/policies/charge-payment-policy.json ===
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:PutItem"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:000000000000:table/payments"
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:PutItem"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:000000000000:table/saga-idempotency"
    },
    {
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetSecretValue"
      ],
      "Resource": "arn:aws:secretsmanager:us-east-1:000000000000:secret:payment/gateway-apikey-hTcpFU"
    },
    {
      "Effect": "Allow",
      "Action": [
        "kms:Decrypt"
      ],
      "Resource": "arn:aws:kms:us-east-1:000000000000:key/30f1d7f1-8060-45df-b508-8248de376c35"
    }
  ]
}
=== /app/policies/create-shipment-policy.json ===
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:PutItem"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:000000000000:table/shipments"
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:PutItem"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:000000000000:table/saga-idempotency"
    }
  ]
}
=== /app/policies/order-saga-policy.json ===
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "lambda:InvokeFunction"
      ],
      "Resource": [
        "arn:aws:lambda:us-east-1:000000000000:function:reserve-inventory",
        "arn:aws:lambda:us-east-1:000000000000:function:charge-payment",
        "arn:aws:lambda:us-east-1:000000000000:function:create-shipment",
        "arn:aws:lambda:us-east-1:000000000000:function:release-inventory",
        "arn:aws:lambda:us-east-1:000000000000:function:refund-payment"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "events:PutEvents"
      ],
      "Resource": "arn:aws:events:us-east-1:000000000000:event-bus/saga-events"
    }
  ]
}
=== /app/policies/refund-payment-policy.json ===
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:UpdateItem"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:000000000000:table/payments"
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:PutItem"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:000000000000:table/saga-idempotency"
    }
  ]
}
=== /app/policies/release-inventory-policy.json ===
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:UpdateItem"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:000000000000:table/inventory"
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:PutItem"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:000000000000:table/saga-idempotency"
    }
  ]
}
=== /app/policies/reserve-inventory-policy.json ===
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:UpdateItem"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:000000000000:table/inventory"
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:PutItem"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:000000000000:table/saga-idempotency"
    }
  ]
}
=== /app/policies/saga-notifier-policy.json ===
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "*"
    }
  ]
}

[stdout]
=== /app/policies/charge-payment-policy.json ===
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:PutItem"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:000000000000:table/payments"
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:PutItem"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:000000000000:table/saga-idempotency"
    },
    {
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetSecretValue"
      ],
      "Resource": "arn:aws:secretsmanager:us-east-1:000000000000:secret:payment/gateway-apikey-hTcpFU"
    },
    {
      "Effect": "Allow",
      "Action": [
        "kms:Decrypt"
      ],
      "Resource": "arn:aws:kms:us-east-1:000000000000:key/30f1d7f1-8060-45df-b508-8248de376c35"
    }
  ]
}
=== /app/policies/create-shipment-policy.json ===
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:PutItem"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:000000000000:table/shipments"
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:PutItem"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:000000000000:table/saga-idempotency"
    }
  ]
}
=== /app/policies/order-saga-policy.json ===
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "lambda:InvokeFunction"
      ],
      "Resource": [
        "arn:aws:lambda:us-east-1:000000000000:function:reserve-inventory",
        "arn:aws:lambda:us-east-1:000000000000:function:charge-payment",
        "arn:aws:lambda:us-east-1:000000000000:function:create-shipment",
        "arn:aws:lambda:us-east-1:000000000000:function:release-inventory",
        "arn:aws:lambda:us-east-1:000000000000:function:refund-payment"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "events:PutEvents"
      ],
      "Resource": "arn:aws:events:us-east-1:000000000000:event-bus/saga-events"
    }
  ]
}
=== /app/policies/refund-payment-policy.json ===
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:UpdateItem"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:000000000000:table/payments"
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:PutItem"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:000000000000:table/saga-idempotency"
    }
  ]
}
=== /app/policies/release-inventory-policy.json ===
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:UpdateItem"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:000000000000:table/inventory"
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:PutItem"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:000000000000:table/saga-idempotency"
    }
  ]
}
=== /app/policies/reserve-inventory-policy.json ===
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:UpdateItem"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:000000000000:table/inventory"
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:PutItem"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:000000000000:table/saga-idempotency"
    }
  ]
}
=== /app/policies/saga-notifier-policy.json ===
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "*"
    }
  ]
}

Now creating the IAM roles and attaching policies.

Create all IAM roles and attach inline policies

shell

export AWS_ENDPOINT_URL=http://localstack:4566

# Lambda roles
for role in reserve-inventory charge-payment create-shipment release-inventory refund-payment saga-notifier; do
  aws --endpoint-url=$AWS_ENDPOINT_URL iam create-role \
    --role-name "${role}-role" \
    --assume-role-policy-document file:///app/policies/trust-lambda.json > /dev/null
  aws --endpoint-url=$AWS_ENDPOINT_URL iam put-role-policy \
    --role-name "${role}-role" \
    --policy-name "${role}-policy" \
    --policy-document file:///app/policies/${role}-policy.json
  echo "role ${role}-role ready"
done

# State machine role
aws --endpoint-url=$AWS_ENDPOINT_URL iam create-role \
  --role-name order-saga-role \
  --assume-role-policy-document file:///app/policies/trust-states.json > /dev/null
aws --endpoint-url=$AWS_ENDPOINT_URL iam put-role-policy \
  --role-name order-saga-role \
  --policy-name order-saga-policy \
  --policy-document file:///app/policies/order-saga-policy.json
echo "role order-saga-role ready"

aws --endpoint-url=$AWS_ENDPOINT_URL iam list-roles | jq -r '.Roles[].RoleName' | sort

output

role reserve-inventory-role ready
role charge-payment-role ready
role create-shipment-role ready
role release-inventory-role ready
role refund-payment-role ready
role saga-notifier-role ready
role order-saga-role ready
charge-payment-role
create-shipment-role
order-saga-role
refund-payment-role
release-inventory-role
reserve-inventory-role
saga-notifier-role

[stdout]
role reserve-inventory-role ready
role charge-payment-role ready
role create-shipment-role ready
role release-inventory-role ready
role refund-payment-role ready
role saga-notifier-role ready
role order-saga-role ready
charge-payment-role
create-shipment-role
order-saga-role
refund-payment-role
release-inventory-role
reserve-inventory-role
saga-notifier-role

Explore /app working directory and check LocalStack connectivity
Create KMS CMK, alias, and encrypted secret
Create DynamoDB tables (inventory, payments, shipments, saga-idempotency)
Create EventBridge bus and rule target
Write and package all Lambda code (reserve/charge/ship/release/refund/notifier)
Create least-privilege IAM roles and policies for each Lambda
Deploy Lambda functions with their roles
Create state machine role and state machine
Run happy-path and failure-path verifier scenarios

Create all six Lambda functions

shell

export AWS_ENDPOINT_URL=http://localstack:4566
ACC=000000000000
REGION=us-east-1

create_fn() {
  local name="$1"
  local handler="$2"
  local zipfile="$3"
  local role="$4"
  local envjson="$5"
  aws --endpoint-url=$AWS_ENDPOINT_URL lambda create-function \
    --function-name "$name" \
    --runtime python3.11 \
    --handler "$handler" \
    --role "arn:aws:iam::${ACC}:role/${role}" \
    --zip-file "fileb:///app/build/${zipfile}" \
    --environment "$envjson" \
    --timeout 30 > /dev/null
  echo "created $name"
}

COMMON_DDB_ENV='{"Variables":{"AWS_ENDPOINT_URL":"http://localstack:4566"}}'

create_fn reserve-inventory reserve_inventory.lambda_handler reserve_inventory.zip reserve-inventory-role \
  '{"Variables":{"AWS_ENDPOINT_URL":"http://localstack:4566","INVENTORY_TABLE":"inventory","IDEMPOTENCY_TABLE":"saga-idempotency"}}'

create_fn charge-payment charge_payment.lambda_handler charge_payment.zip charge-payment-role \
  '{"Variables":{"AWS_ENDPOINT_URL":"http://localstack:4566","PAYMENTS_TABLE":"payments","IDEMPOTENCY_TABLE":"saga-idempotency","SECRET_ID":"payment/gateway-apikey"}}'

create_fn create-shipment create_shipment.lambda_handler create_shipment.zip create-shipment-role \
  '{"Variables":{"AWS_ENDPOINT_URL":"http://localstack:4566","SHIPMENTS_TABLE":"shipments","IDEMPOTENCY_TABLE":"saga-idempotency"}}'

create_fn release-inventory release_inventory.lambda_handler release_inventory.zip release-inventory-role \
  '{"Variables":{"AWS_ENDPOINT_URL":"http://localstack:4566","INVENTORY_TABLE":"inventory","IDEMPOTENCY_TABLE":"saga-idempotency"}}'

create_fn refund-payment refund_payment.lambda_handler refund_payment.zip refund-payment-role \
  '{"Variables":{"AWS_ENDPOINT_URL":"http://localstack:4566","PAYMENTS_TABLE":"payments","IDEMPOTENCY_TABLE":"saga-idempotency"}}'

create_fn saga-notifier saga_notifier.lambda_handler saga_notifier.zip saga-notifier-role \
  '{"Variables":{}}'

aws --endpoint-url=$AWS_ENDPOINT_URL lambda list-functions | jq -r '.Functions[].FunctionName' | sort

output

created reserve-inventory
created charge-payment
created create-shipment
created release-inventory
created refund-payment
created saga-notifier
charge-payment
create-shipment
refund-payment
release-inventory
reserve-inventory
saga-notifier

[stdout]
created reserve-inventory
created charge-payment
created create-shipment
created release-inventory
created refund-payment
created saga-notifier
charge-payment
create-shipment
refund-payment
release-inventory
reserve-inventory
saga-notifier

Confirm all Lambdas are Active

shell

export AWS_ENDPOINT_URL=http://localstack:4566
for fn in reserve-inventory charge-payment create-shipment release-inventory refund-payment saga-notifier; do
  st=$(aws --endpoint-url=$AWS_ENDPOINT_URL lambda get-function --function-name "$fn" | jq -r '.Configuration.State')
  echo "$fn -> $st"
done

output

reserve-inventory -> Active
charge-payment -> Active
create-shipment -> Active
release-inventory -> Active
refund-payment -> Active
saga-notifier -> Active

[stdout]
reserve-inventory -> Active
charge-payment -> Active
create-shipment -> Active
release-inventory -> Active
refund-payment -> Active
saga-notifier -> Active

Now wire EventBridge rule to the notifier.

Create EventBridge rule and attach notifier Lambda target

shell

export AWS_ENDPOINT_URL=http://localstack:4566
ACC=000000000000
REGION=us-east-1

aws --endpoint-url=$AWS_ENDPOINT_URL events put-rule \
  --name on-saga-terminal \
  --event-bus-name saga-events \
  --event-pattern '{"source":["order.saga"],"detail-type":["OrderCompleted","OrderFailed"]}' | jq '.RuleArn'

aws --endpoint-url=$AWS_ENDPOINT_URL lambda add-permission \
  --function-name saga-notifier \
  --statement-id allow-eventbridge \
  --action lambda:InvokeFunction \
  --principal events.amazonaws.com \
  --source-arn "arn:aws:events:${REGION}:${ACC}:rule/saga-events/on-saga-terminal" | jq '.Statement' -r | head -c 200
echo

aws --endpoint-url=$AWS_ENDPOINT_URL events put-targets \
  --rule on-saga-terminal \
  --event-bus-name saga-events \
  --targets "Id=1,Arn=arn:aws:lambda:${REGION}:${ACC}:function:saga-notifier" | jq '.FailedEntryCount'

output

"arn:aws:events:us-east-1:000000000000:rule/saga-events/on-saga-terminal"
{"Sid": "allow-eventbridge", "Effect": "Allow", "Action": "lambda:InvokeFunction", "Resource": "arn:aws:lambda:us-east-1:000000000000:function:saga-notifier", "Principal": {"Service": "events.amazonaw
0

[stdout]
"arn:aws:events:us-east-1:000000000000:rule/saga-events/on-saga-terminal"
{"Sid": "allow-eventbridge", "Effect": "Allow", "Action": "lambda:InvokeFunction", "Resource": "arn:aws:lambda:us-east-1:000000000000:function:saga-notifier", "Principal": {"Service": "events.amazonaw
0

Now the state machine definition.

write · /app/build/state-machine.json

{
  "Comment": "Order saga: reserve inventory -> charge payment -> create shipment, with reverse-order compensations",
  "StartAt": "ReserveInventory",
  "States": {
    "ReserveInventory": {
      "Type": "Task",
      "Resource": "arn:aws:states:::lambda:invoke",
      "Parameters": {
        "FunctionName": "arn:aws:lambda:us-east-1:000000000000:function:reserve-inventory",
        "Payload.$": "$"
      },
      "OutputPath": "$.Payload",
      "Retry": [
        {
          "ErrorEquals": ["Lambda.ServiceException", "Lambda.AWSLambdaException", "Lambda.SdkClientException", "Lambda.TooManyRequestsException"],
          "IntervalSeconds": 1,
          "MaxAttempts": 2,
          "BackoffRate": 2.0
        }
      ],
      "Catch": [
        {
          "ErrorEquals": ["States.ALL"],
          "ResultPath": "$.error",
          "Next": "PublishFailed"
        }
      ],
      "Next": "ChargePayment"
    },
    "ChargePayment": {
      "Type": "Task",
      "Resource": "arn:aws:states:::lambda:invoke",
      "Parameters": {
        "FunctionName": "arn:aws:lambda:us-east-1:000000000000:function:charge-payment",
        "Payload.$": "$"
      },
      "OutputPath": "$.Payload",
      "Catch": [
        {
          "ErrorEquals": ["States.ALL"],
          "ResultPath": "$.error",
          "Next": "ReleaseAfterChargeFail"
        }
      ],
      "Next": "CreateShipment"
    },
    "CreateShipment": {
      "Type": "Task",
      "Resource": "arn:aws:states:::lambda:invoke",
      "Parameters": {
        "FunctionName": "arn:aws:lambda:us-east-1:000000000000:function:create-shipment",
        "Payload.$": "$"
      },
      "OutputPath": "$.Payload",
      "Catch": [
        {
          "ErrorEquals": ["States.ALL"],
          "ResultPath": "$.error",
          "Next": "RefundAfterShipFail"
        }
      ],
      "Next": "PublishCompleted"
    },
    "RefundAfterShipFail": {
      "Type": "Task",
      "Resource": "arn:aws:states:::lambda:invoke",
      "Parameters": {
        "FunctionName": "arn:aws:lambda:us-east-1:000000000000:function:refund-payment",
        "Payload.$": "$"
      },
      "OutputPath": "$.Payload",
      "Next": "ReleaseAfterShipFail"
    },
    "ReleaseAfterShipFail": {
      "Type": "Task",
      "Resource": "arn:aws:states:::lambda:invoke",
      "Parameters": {
        "FunctionName": "arn:aws:lambda:us-east-1:000000000000:function:release-inventory",
        "Payload.$": "$"
      },
      "OutputPath": "$.Payload",
      "Next": "PublishFailed"
    },
    "ReleaseAfterChargeFail": {
      "Type": "Task",
      "Resource": "arn:aws:states:::lambda:invoke",
      "Parameters": {
        "FunctionName": "arn:aws:lambda:us-east-1:000000000000:function:release-inventory",
        "Payload.$": "$"
      },
      "OutputPath": "$.Payload",
      "Next": "PublishFailed"
    },
    "PublishCompleted": {
      "Type": "Task",
      "Resource": "arn:aws:states:::events:putEvents",
      "Parameters": {
        "Entries": [
          {
            "EventBusName": "saga-events",
            "Source": "order.saga",
            "DetailType": "OrderCompleted",
            "Detail": {
              "order_id.$": "$.order_id",
              "sku.$": "$.sku",
              "qty.$": "$.qty",
              "amount.$": "$.amount",
              "payment_id.$": "$.payment_id",
              "shipment_id.$": "$.shipment_id",
              "idempotency_key.$": "$.idempotency_key",
              "status": "completed"
            }
          }
        ]
      },
      "End": true
    },
    "PublishFailed": {
      "Type": "Task",
      "Resource": "arn:aws:states:::events:putEvents",
      "Parameters": {
        "Entries": [
          {
            "EventBusName": "saga-events",
            "Source": "order.saga",
            "DetailType": "OrderFailed",
            "Detail": {
              "order_id.$": "$.order_id",
              "sku.$": "$.sku",
              "qty.$": "$.qty",
              "idempotency_key.$": "$.idempotency_key",
              "status": "failed"
            }
          }
        ]
      },
      "End": true
    }
  }
}

Create the order-saga state machine

shell

export AWS_ENDPOINT_URL=http://localstack:4566
ACC=000000000000
aws --endpoint-url=$AWS_ENDPOINT_URL stepfunctions create-state-machine \
  --name order-saga \
  --definition file:///app/build/state-machine.json \
  --role-arn "arn:aws:iam::${ACC}:role/order-saga-role" \
  --type STANDARD | jq '.stateMachineArn'

output

"arn:aws:states:us-east-1:000000000000:stateMachine:order-saga"

[stdout]
"arn:aws:states:us-east-1:000000000000:stateMachine:order-saga"

Explore /app working directory and check LocalStack connectivity
Create KMS CMK, alias, and encrypted secret
Create DynamoDB tables (inventory, payments, shipments, saga-idempotency)
Create EventBridge bus and rule target
Write and package all Lambda code (reserve/charge/ship/release/refund/notifier)
Create least-privilege IAM roles and policies for each Lambda
Deploy Lambda functions with their roles
Create state machine role and state machine
Run happy-path and failure-path verifier scenarios

Now let's seed inventory and run the happy path.

Seed inventory and run happy path execution

shell

export AWS_ENDPOINT_URL=http://localstack:4566

# Seed inventory row for sku=x with reserved_qty=0
aws --endpoint-url=$AWS_ENDPOINT_URL dynamodb put-item \
  --table-name inventory \
  --item '{"sku":{"S":"x"},"on_hand":{"N":"100"},"reserved_qty":{"N":"0"}}'

SM_ARN="arn:aws:states:us-east-1:000000000000:stateMachine:order-saga"
EXEC=$(aws --endpoint-url=$AWS_ENDPOINT_URL stepfunctions start-execution \
  --state-machine-arn "$SM_ARN" \
  --name "happy-$(date +%s)" \
  --input '{"order_id":"o-1","sku":"x","qty":2,"amount":100,"idempotency_key":"k-1"}' | jq -r '.executionArn')
echo "exec=$EXEC"
echo "$EXEC" > /app/build/happy_exec

# Wait for terminal status
for i in $(seq 1 30); do
  ST=$(aws --endpoint-url=$AWS_ENDPOINT_URL stepfunctions describe-execution --execution-arn "$EXEC" | jq -r '.status')
  echo "attempt $i: $ST"
  if [ "$ST" != "RUNNING" ]; then break; fi
  sleep 1
done
aws --endpoint-url=$AWS_ENDPOINT_URL stepfunctions describe-execution --execution-arn "$EXEC" | jq '{status,input,output}'

output

exec=arn:aws:states:us-east-1:000000000000:execution:order-saga:happy-1776916025
attempt 1: RUNNING
attempt 2: RUNNING
attempt 3: RUNNING
attempt 4: SUCCEEDED
{
  "status": "SUCCEEDED",
  "input": "{\"order_id\":\"o-1\",\"sku\":\"x\",\"qty\":2,\"amount\":100,\"idempotency_key\":\"k-1\"}",
  "output": "{\"FailedEntryCount\":0,\"Entries\":[{\"EventId\":\"97fff07b-ea96-4a8d-b37f-f31ce308e23f\"}]}"
}

[stdout]
exec=arn:aws:states:us-east-1:000000000000:execution:order-saga:happy-1776916025
attempt 1: RUNNING
attempt 2: RUNNING
attempt 3: RUNNING
attempt 4: SUCCEEDED
{
  "status": "SUCCEEDED",
  "input": "{\"order_id\":\"o-1\",\"sku\":\"x\",\"qty\":2,\"amount\":100,\"idempotency_key\":\"k-1\"}",
  "output": "{\"FailedEntryCount\":0,\"Entries\":[{\"EventId\":\"97fff07b-ea96-4a8d-b37f-f31ce308e23f\"}]}"
}

Happy path SUCCEEDED. Now verify the data.

Verify inventory/payments/shipments state after happy path

shell

export AWS_ENDPOINT_URL=http://localstack:4566
echo "--- inventory sku=x ---"
aws --endpoint-url=$AWS_ENDPOINT_URL dynamodb get-item --table-name inventory --key '{"sku":{"S":"x"}}' --consistent-read | jq
echo "--- payments pay-o-1 ---"
aws --endpoint-url=$AWS_ENDPOINT_URL dynamodb get-item --table-name payments --key '{"payment_id":{"S":"pay-o-1"}}' --consistent-read | jq
echo "--- shipments ship-o-1 ---"
aws --endpoint-url=$AWS_ENDPOINT_URL dynamodb get-item --table-name shipments --key '{"shipment_id":{"S":"ship-o-1"}}' --consistent-read | jq

output

--- inventory sku=x ---
{
  "Item": {
    "sku": {
      "S": "x"
    },
    "on_hand": {
      "N": "100"
    },
    "reserved_qty": {
      "N": "2"
    }
  }
}
--- payments pay-o-1 ---
{
  "Item": {
    "amount": {
      "N": "100"
    },
    "order_id": {
      "S": "o-1"
    },
    "payment_id": {
      "S": "pay-o-1"
    },
    "status": {
      "S": "CAPTURED"
    }
  }
}
--- shipments ship-o-1 ---
{
  "Item": {
    "shipment_id": {
      "S": "ship-o-1"
    },
    "sku": {
      "S": "x"
    },
    "order_id": {
      "S": "o-1"
    },
    "qty": {
      "N": "2"
    },
    "status": {
      "S": "CREATED"
    }
  }
}

[stdout]
--- inventory sku=x ---
{
  "Item": {
    "sku": {
      "S": "x"
    },
    "on_hand": {
      "N": "100"
    },
    "reserved_qty": {
      "N": "2"
    }
  }
}
--- payments pay-o-1 ---
{
  "Item": {
    "amount": {
      "N": "100"
    },
    "order_id": {
      "S": "o-1"
    },
    "payment_id": {
      "S": "pay-o-1"
    },
    "status": {
      "S": "CAPTURED"
    }
  }
}
--- shipments ship-o-1 ---
{
  "Item": {
    "shipment_id": {
      "S": "ship-o-1"
    },
    "sku": {
      "S": "x"
    },
    "order_id": {
      "S": "o-1"
    },
    "qty": {
      "N": "2"
    },
    "status": {
      "S": "CREATED"
    }
  }
}

Inspect saga-notifier CloudWatch logs for happy path

shell

export AWS_ENDPOINT_URL=http://localstack:4566
LG="/aws/lambda/saga-notifier"
STREAMS=$(aws --endpoint-url=$AWS_ENDPOINT_URL logs describe-log-streams --log-group-name "$LG" --order-by LastEventTime --descending --limit 5 2>/dev/null | jq -r '.logStreams[].logStreamName')
echo "streams: $STREAMS"
for s in $STREAMS; do
  aws --endpoint-url=$AWS_ENDPOINT_URL logs get-log-events --log-group-name "$LG" --log-stream-name "$s" 2>/dev/null | jq -r '.events[].message'
done

output

streams: 2026/04/23/[$LATEST]fa2ee38107d830c6277f27698282c74e
START RequestId: dd5c3063-c0eb-4b1c-8064-6f68db54da81 Version: $LATEST

saga-terminal:OrderCompleted {"amount": 100, "idempotency_key": "k-1", "order_id": "o-1", "payment_id": "pay-o-1", "qty": 2, "shipment_id": "ship-o-1", "sku": "x", "status": "completed"}

END RequestId: dd5c3063-c0eb-4b1c-8064-6f68db54da81

REPORT RequestId: dd5c3063-c0eb-4b1c-8064-6f68db54da81	Duration: 9.65 ms	Billed Duration: 10 ms	Memory Size: 128 MB	Max Memory Used: 128 MB

[stdout]
streams: 2026/04/23/[$LATEST]fa2ee38107d830c6277f27698282c74e
START RequestId: dd5c3063-c0eb-4b1c-8064-6f68db54da81 Version: $LATEST

saga-terminal:OrderCompleted {"amount": 100, "idempotency_key": "k-1", "order_id": "o-1", "payment_id": "pay-o-1", "qty": 2, "shipment_id": "ship-o-1", "sku": "x", "status": "completed"}

END RequestId: dd5c3063-c0eb-4b1c-8064-6f68db54da81

REPORT RequestId: dd5c3063-c0eb-4b1c-8064-6f68db54da81	Duration: 9.65 ms	Billed Duration: 10 ms	Memory Size: 128 MB	Max Memory Used: 128 MB

Happy path verified end-to-end. Now the failure path.

Run failure path execution with force_failure=true

shell

export AWS_ENDPOINT_URL=http://localstack:4566
SM_ARN="arn:aws:states:us-east-1:000000000000:stateMachine:order-saga"
EXEC=$(aws --endpoint-url=$AWS_ENDPOINT_URL stepfunctions start-execution \
  --state-machine-arn "$SM_ARN" \
  --name "fail-$(date +%s)" \
  --input '{"order_id":"o-2","sku":"x","qty":3,"amount":250,"idempotency_key":"k-2","force_failure":true}' | jq -r '.executionArn')
echo "exec=$EXEC"

for i in $(seq 1 30); do
  ST=$(aws --endpoint-url=$AWS_ENDPOINT_URL stepfunctions describe-execution --execution-arn "$EXEC" | jq -r '.status')
  echo "attempt $i: $ST"
  if [ "$ST" != "RUNNING" ]; then break; fi
  sleep 1
done
aws --endpoint-url=$AWS_ENDPOINT_URL stepfunctions describe-execution --execution-arn "$EXEC" | jq '{status,output}'

output

exec=arn:aws:states:us-east-1:000000000000:execution:order-saga:fail-1776916048
attempt 1: RUNNING
attempt 2: RUNNING
attempt 3: RUNNING
attempt 4: SUCCEEDED
{
  "status": "SUCCEEDED",
  "output": "{\"FailedEntryCount\":0,\"Entries\":[{\"EventId\":\"21b36e98-e0ff-4647-a7c3-b99c9cf8683f\"}]}"
}

[stdout]
exec=arn:aws:states:us-east-1:000000000000:execution:order-saga:fail-1776916048
attempt 1: RUNNING
attempt 2: RUNNING
attempt 3: RUNNING
attempt 4: SUCCEEDED
{
  "status": "SUCCEEDED",
  "output": "{\"FailedEntryCount\":0,\"Entries\":[{\"EventId\":\"21b36e98-e0ff-4647-a7c3-b99c9cf8683f\"}]}"
}

Verify data rolled back after failure path

shell

export AWS_ENDPOINT_URL=http://localstack:4566
echo "--- inventory sku=x (should be reserved_qty=2: o-1 reserved 2, o-2 rolled back) ---"
aws --endpoint-url=$AWS_ENDPOINT_URL dynamodb get-item --table-name inventory --key '{"sku":{"S":"x"}}' --consistent-read | jq '.Item.reserved_qty'
echo "--- payments pay-o-2 (should be absent since charge failed before capture) ---"
aws --endpoint-url=$AWS_ENDPOINT_URL dynamodb get-item --table-name payments --key '{"payment_id":{"S":"pay-o-2"}}' --consistent-read | jq '.Item // "ABSENT"'
echo "--- shipments ship-o-2 (should be absent) ---"
aws --endpoint-url=$AWS_ENDPOINT_URL dynamodb get-item --table-name shipments --key '{"shipment_id":{"S":"ship-o-2"}}' --consistent-read | jq '.Item // "ABSENT"'

output

--- inventory sku=x (should be reserved_qty=2: o-1 reserved 2, o-2 rolled back) ---
{
  "N": "2"
}
--- payments pay-o-2 (should be absent since charge failed before capture) ---
--- shipments ship-o-2 (should be absent) ---

[stdout]
--- inventory sku=x (should be reserved_qty=2: o-1 reserved 2, o-2 rolled back) ---
{
  "N": "2"
}
--- payments pay-o-2 (should be absent since charge failed before capture) ---
--- shipments ship-o-2 (should be absent) ---

Double-check absence of payments/shipments rows for o-2

shell

export AWS_ENDPOINT_URL=http://localstack:4566
P=$(aws --endpoint-url=$AWS_ENDPOINT_URL dynamodb get-item --table-name payments --key '{"payment_id":{"S":"pay-o-2"}}' --consistent-read)
S=$(aws --endpoint-url=$AWS_ENDPOINT_URL dynamodb get-item --table-name shipments --key '{"shipment_id":{"S":"ship-o-2"}}' --consistent-read)
echo "payments raw='$P'"
echo "shipments raw='$S'"

output

payments raw=''
shipments raw=''

[stdout]
payments raw=''
shipments raw=''

Both absent as expected. Now the notifier log.

Grep notifier logs for saga-terminal lines

shell

export AWS_ENDPOINT_URL=http://localstack:4566
LG="/aws/lambda/saga-notifier"
STREAMS=$(aws --endpoint-url=$AWS_ENDPOINT_URL logs describe-log-streams --log-group-name "$LG" --order-by LastEventTime --descending --limit 5 | jq -r '.logStreams[].logStreamName')
for s in $STREAMS; do
  aws --endpoint-url=$AWS_ENDPOINT_URL logs get-log-events --log-group-name "$LG" --log-stream-name "$s" | jq -r '.events[].message' | grep -E '^saga-terminal:' || true
done

output

saga-terminal:OrderFailed {"idempotency_key": "k-2", "order_id": "o-2", "qty": 3, "sku": "x", "status": "failed"}
saga-terminal:OrderCompleted {"amount": 100, "idempotency_key": "k-1", "order_id": "o-1", "payment_id": "pay-o-1", "qty": 2, "shipment_id": "ship-o-1", "sku": "x", "status": "completed"}

[stdout]
saga-terminal:OrderFailed {"idempotency_key": "k-2", "order_id": "o-2", "qty": 3, "sku": "x", "status": "failed"}
saga-terminal:OrderCompleted {"amount": 100, "idempotency_key": "k-1", "order_id": "o-1", "payment_id": "pay-o-1", "qty": 2, "shipment_id": "ship-o-1", "sku": "x", "status": "completed"}

Explore /app working directory and check LocalStack connectivity
Create KMS CMK, alias, and encrypted secret
Create DynamoDB tables (inventory, payments, shipments, saga-idempotency)
Create EventBridge bus and rule target
Write and package all Lambda code (reserve/charge/ship/release/refund/notifier)
Create least-privilege IAM roles and policies for each Lambda
Deploy Lambda functions with their roles
Create state machine role and state machine
Run happy-path and failure-path verifier scenarios

Both scenarios pass. Here is what's deployed and how it was proven: **Infrastructure** (names per spec, LocalStack at `http://localstack:4566`): - KMS CMK aliased `alias/saga-cmk`; Secrets Manager secret `payment/gateway-apikey` encrypted with that CMK - DynamoDB tables `inventory` (pk `sku`), `payments` (pk `payment_id`), `shipments` (pk `shipment_id`), `saga-idempotency` (pk `idempotency_key`, TTL on `expires_at` enabled) - EventBridge bus `saga-events` with rule `on-saga-terminal` matching `source=order.saga` and `detail-type ∈ {OrderCompleted, OrderFailed}`, routed to `saga-notifier` - Step Functions STANDARD state machine `order-saga` **Lambdas** (python3.11, code under `/app/lambdas`): - `reserve-inventory`, `charge-payment`, `create-shipment` , forward steps; each writes to its own table and records its action in `saga-idempotency` keyed `{step}:{idempotency_key}` to short-circuit retries - `release-inventory`, `refund-payment` , compensations; idempotent via `saga-idempotency` (`release:` / `refund:` keys) and no-op if the forward step never actually wrote (refund-payment checks for the row first) - `saga-notifier` , prints `saga-terminal:<DetailType> <json>` **Saga shape** (`/app/build/state-machine.json`) , reverse-order compensations: - Reserve fails → `PublishFailed` (nothing to undo) - Charge fails → `ReleaseAfterChargeFail` → `PublishFailed` - Shipment fails → `RefundAfterShipFail` → `ReleaseAfterShipFail` → `PublishFailed` - All-good → `PublishCompleted` Both terminal paths emit to `saga-events` and end in a success state, so the execution itself reaches `SUCCEEDED` even when the business flow rolled back. **Least privilege** (`/app/policies/*.json`): - Each Lambda role has only its own table ARNs + `saga-idempotency` + `logs:*` (no wildcards on DDB) - `charge-payment-role` is the only principal with `secretsmanager:GetSecretValue` on the specific secret ARN and `kms:Decrypt` on the specific CMK ARN - `order-saga-role` lists the five step-lambda ARNs explicitly for `lambda:InvokeFunction` (no `*`) and `events:PutEvents` only on the `saga-events` bus ARN - `saga-notifier-role` has only CloudWatch logs **Verification results** Happy path , input `{"order_id":"o-1","sku":"x","qty":2,"amount":100,"idempotency_key":"k-1"}`: - execution `SUCCEEDED` in ~4s - `inventory[sku=x].reserved_qty = 2` - `payments[pay-o-1].status = CAPTURED` - `shipments[ship-o-1]` present (`status=CREATED`) - notifier logged `saga-terminal:OrderCompleted {...}` Failure path , same input with `force_failure:true, order_id:"o-2", idempotency_key:"k-2", qty:3`: - execution `SUCCEEDED` in ~4s (saga handled gracefully) - `inventory[sku=x].reserved_qty` still `2` , the 3 reserved for o-2 was released - `payments[pay-o-2]` absent (charge raised before capture; refund was a no-op) - `shipments[ship-o-2]` absent - notifier logged `saga-terminal:OrderFailed {...}`

Final diff , what the agent changed

No file changes, the agent left the workspace untouched.

Verifier output , the proof behind the reward

tests/test.sh

============================= test session starts ==============================
platform linux -- Python 3.12.3, pytest-8.4.1, pluggy-1.6.0 -- /root/.cache/uv/archive-v0/34iwuekVN0XuKYERm_tEm/bin/python
cachedir: .pytest_cache
rootdir: /tests
plugins: json-ctrf-0.3.5
collecting ... collected 20 items

../tests/test_outputs.py::test_01_cmk_alias_and_secret_exist PASSED      [  5%]
../tests/test_outputs.py::test_02_all_four_tables_exist PASSED           [ 10%]
../tests/test_outputs.py::test_03_all_six_lambdas_exist PASSED           [ 15%]
../tests/test_outputs.py::test_04_state_machine_and_bus_exist PASSED     [ 20%]
../tests/test_outputs.py::test_05_terminal_rule_exists_on_saga_bus PASSED [ 25%]
../tests/test_outputs.py::test_06_every_forward_state_has_catch_to_compensation PASSED [ 30%]
../tests/test_outputs.py::test_07_compensations_run_in_lifo_order PASSED [ 35%]
../tests/test_outputs.py::test_08_compensation_states_preserve_error_via_result_path FAILED [ 40%]
../tests/test_outputs.py::test_09_idempotency_table_has_ttl_and_correct_pk PASSED [ 45%]
../tests/test_outputs.py::test_10_each_lambda_role_scoped_to_its_own_table PASSED [ 50%]
../tests/test_outputs.py::test_11_only_charge_role_reads_the_payment_secret PASSED [ 55%]
../tests/test_outputs.py::test_12_saga_role_lambda_invoke_not_wildcarded PASSED [ 60%]
../tests/test_outputs.py::test_13_rule_pattern_matches_both_terminal_types_with_array_wrap PASSED [ 65%]
../tests/test_outputs.py::test_14_charge_role_kms_decrypt_scoped_not_wildcard PASSED [ 70%]
../tests/test_outputs.py::test_15_rule_targets_notifier_lambda PASSED    [ 75%]
../tests/test_outputs.py::test_16_notifier_role_is_least_privilege_no_ddb_kms_secrets PASSED [ 80%]
../tests/test_outputs.py::test_17_notifier_lambda_resource_policy_allows_events_with_source_arn PASSED [ 85%]
../tests/test_outputs.py::test_18_happy_path_completes_and_writes_all_three_tables PASSED [ 90%]
../tests/test_outputs.py::test_19_failure_path_triggers_compensation_and_rolls_back_inventory PASSED [ 95%]
../tests/test_outputs.py::test_20_idempotency_prevents_double_reservation PASSED [100%]

=================================== FAILURES ===================================
__________ test_08_compensation_states_preserve_error_via_result_path __________

    def test_08_compensation_states_preserve_error_via_result_path():
        """Compensation Task states must use ResultPath that doesn't clobber
        the error payload - either ResultPath:null or a side path like
        $.compensationResult. LLMs commonly omit ResultPath and let the
        compensation's return value overwrite the error context."""
        definition = _sm_definition()
        states = _collect_states(definition)
        comp_states = []
        for name, state in states.items():
            if state.get("Type") != "Task":
                continue
            resource = state.get("Resource", "") or ""
            params = state.get("Parameters") or {}
            ref = params.get("FunctionName") or resource
            ref_str = json.dumps(ref) if not isinstance(ref, str) else ref
            if FN_REFUND in ref_str or FN_RELEASE in ref_str:
                comp_states.append((name, state))
        assert len(comp_states) >= 2, (
            f"expected at least 2 compensation task states; found {len(comp_states)}"
        )
        for name, s in comp_states:
            rp = s.get("ResultPath", "__MISSING__")
>           assert rp != "__MISSING__", (
                f"compensation state '{name}' missing ResultPath - the "
                "compensation's return will clobber the error context used "
                "by downstream logging"
            )
E           AssertionError: compensation state 'RefundAfterShipFail' missing ResultPath - the compensation's return will clobber the error context used by downstream logging
E           assert '__MISSING__' != '__MISSING__'

/tests/test_outputs.py:409: AssertionError
=============================== warnings summary ===============================
test_outputs.py: 131 warnings
  /root/.cache/uv/archive-v0/34iwuekVN0XuKYERm_tEm/lib/python3.12/site-packages/botocore/auth.py:424: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
    datetime_now = datetime.datetime.utcnow()

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
==================================== PASSES ====================================
=========================== short test summary info ============================
PASSED ../tests/test_outputs.py::test_01_cmk_alias_and_secret_exist
PASSED ../tests/test_outputs.py::test_02_all_four_tables_exist
PASSED ../tests/test_outputs.py::test_03_all_six_lambdas_exist
PASSED ../tests/test_outputs.py::test_04_state_machine_and_bus_exist
PASSED ../tests/test_outputs.py::test_05_terminal_rule_exists_on_saga_bus
PASSED ../tests/test_outputs.py::test_06_every_forward_state_has_catch_to_compensation
PASSED ../tests/test_outputs.py::test_07_compensations_run_in_lifo_order
PASSED ../tests/test_outputs.py::test_09_idempotency_table_has_ttl_and_correct_pk
PASSED ../tests/test_outputs.py::test_10_each_lambda_role_scoped_to_its_own_table
PASSED ../tests/test_outputs.py::test_11_only_charge_role_reads_the_payment_secret
PASSED ../tests/test_outputs.py::test_12_saga_role_lambda_invoke_not_wildcarded
PASSED ../tests/test_outputs.py::test_13_rule_pattern_matches_both_terminal_types_with_array_wrap
PASSED ../tests/test_outputs.py::test_14_charge_role_kms_decrypt_scoped_not_wildcard
PASSED ../tests/test_outputs.py::test_15_rule_targets_notifier_lambda
PASSED ../tests/test_outputs.py::test_16_notifier_role_is_least_privilege_no_ddb_kms_secrets
PASSED ../tests/test_outputs.py::test_17_notifier_lambda_resource_policy_allows_events_with_source_arn
PASSED ../tests/test_outputs.py::test_18_happy_path_completes_and_writes_all_three_tables
PASSED ../tests/test_outputs.py::test_19_failure_path_triggers_compensation_and_rolls_back_inventory
PASSED ../tests/test_outputs.py::test_20_idempotency_prevents_double_reservation
FAILED ../tests/test_outputs.py::test_08_compensation_states_preserve_error_via_result_path
================= 1 failed, 19 passed, 131 warnings in 20.91s ==================

Reproduce this trial: git checkout 2f94510 && PYTHONPATH=src python3 scripts/build_site.py , then open trial/trial_c34b8cf9daed42fa. Re-running the agent live requires EVAL_PLATFORM_ENABLE_OAUTH_SMOKE=1 and is non-deterministic.

Trial trial_c34b8cf9daed42fa · verifier authoritative; classifier explanatory.