tests/test.sh exit code (0 → resolved); the classification below is post-hoc and cannot change it.Verify environment
aws --version && python3 --version && which jq zip curl && aws --endpoint-url=http://localstack:4566 sts get-caller-identity
aws-cli/2.34.33 Python/3.14.4 Linux/6.10.14-linuxkit exe/aarch64.ubuntu.24
Python 3.12.3
/usr/bin/jq
/usr/bin/zip
/usr/bin/curl
{
"UserId": "AKIAIOSFODNN7EXAMPLE",
"Account": "000000000000",
"Arn": "arn:aws:iam::000000000000:root"
}
[stdout]
aws-cli/2.34.33 Python/3.14.4 Linux/6.10.14-linuxkit exe/aarch64.ubuntu.24
Python 3.12.3
/usr/bin/jq
/usr/bin/zip
/usr/bin/curl
{
"UserId": "AKIAIOSFODNN7EXAMPLE",
"Account": "000000000000",
"Arn": "arn:aws:iam::000000000000:root"
}Check current LocalStack state
aws --endpoint-url=http://localstack:4566 --region us-east-1 secretsmanager list-secrets --query 'SecretList[].Name' && echo "---" && aws --endpoint-url=http://localstack:4566 --region us-east-1 lambda list-functions --query 'Functions[].FunctionName' && echo "---" && aws --endpoint-url=http://localstack:4566 --region us-east-1 stepfunctions list-state-machines
[]
---
[]
---
{
"stateMachines": []
}
[stdout]
[]
---
[]
---
{
"stateMachines": []
}Create working directory
mkdir -p /tmp/rotation && cd /tmp/rotation && ls
Shell cwd was reset to /app [stderr] Shell cwd was reset to /app
Create subdirs
mkdir -p /tmp/rotation/rotator /tmp/rotation/notifier /tmp/rotation/policies /tmp/rotation/sm && ls /tmp/rotation
notifier policies rotator sm [stdout] notifier policies rotator sm
Create KMS key and alias
set -e export AWS_PAGER="" ENDPOINT="http://localstack:4566" REGION="us-east-1" # Create KMS key + alias KEY_ID=$(aws --endpoint-url=$ENDPOINT --region $REGION kms create-key --description "app rotation key" --query 'KeyMetadata.KeyId' --output text) echo "KEY_ID=$KEY_ID" aws --endpoint-url=$ENDPOINT --region $REGION kms create-alias --alias-name alias/app-rotation-key --target-key-id $KEY_ID KEY_ARN=$(aws --endpoint-url=$ENDPOINT --region $REGION kms describe-key --key-id alias/app-rotation-key --query 'KeyMetadata.Arn' --output text) echo "KEY_ARN=$KEY_ARN" echo $KEY_ARN > /tmp/rotation/key_arn echo $KEY_ID > /tmp/rotation/key_id
KEY_ID=57fb9d95-f011-4af5-85a5-7ec3a0d5775b KEY_ARN=arn:aws:kms:us-east-1:000000000000:key/57fb9d95-f011-4af5-85a5-7ec3a0d5775b [stdout] KEY_ID=57fb9d95-f011-4af5-85a5-7ec3a0d5775b KEY_ARN=arn:aws:kms:us-east-1:000000000000:key/57fb9d95-f011-4af5-85a5-7ec3a0d5775b
Create secret with CMK
set -e
export AWS_PAGER=""
ENDPOINT="http://localstack:4566"
REGION="us-east-1"
KEY_ARN=$(cat /tmp/rotation/key_arn)
# Create secret encrypted with CMK
aws --endpoint-url=$ENDPOINT --region $REGION secretsmanager create-secret \
--name "app/db/password" \
--description "app db password" \
--kms-key-id $KEY_ARN \
--secret-string '{"username":"appuser","password":"initial-placeholder"}'
aws --endpoint-url=$ENDPOINT --region $REGION secretsmanager describe-secret --secret-id app/db/password --query '{Name:Name,KmsKeyId:KmsKeyId}'{
"ARN": "arn:aws:secretsmanager:us-east-1:000000000000:secret:app/db/password-jbuLIu",
"Name": "app/db/password",
"VersionId": "68dd6dd2-226b-4b06-b655-fbdcd60b0b4f"
}
{
"Name": "app/db/password",
"KmsKeyId": "arn:aws:kms:us-east-1:000000000000:key/57fb9d95-f011-4af5-85a5-7ec3a0d5775b"
}
[stdout]
{
"ARN": "arn:aws:secretsmanager:us-east-1:000000000000:secret:app/db/password-jbuLIu",
"Name": "app/db/password",
"VersionId": "68dd6dd2-226b-4b06-b655-fbdcd60b0b4f"
}
{
"Name": "app/db/password",
"KmsKeyId": "arn:aws:kms:us-east-1:000000000000:key/57fb9d95-f011-4af5-85a5-7ec3a0d5775b"
}import json
import os
import secrets
import string
import boto3
ENDPOINT = os.environ.get("AWS_ENDPOINT_URL", "http://localstack:4566")
SECRET_ID = os.environ["SECRET_ID"]
_sm = boto3.client("secretsmanager", endpoint_url=ENDPOINT)
def _generate_password(length: int = 32) -> str:
alphabet = string.ascii_letters + string.digits + "-_"
return "".join(secrets.choice(alphabet) for _ in range(length))
def handler(event, context):
current = _sm.get_secret_value(SecretId=SECRET_ID, VersionStage="AWSCURRENT")
payload = json.loads(current["SecretString"])
payload["password"] = _generate_password()
put = _sm.put_secret_value(
SecretId=SECRET_ID,
SecretString=json.dumps(payload),
VersionStages=["AWSCURRENT"],
)
return {
"secretId": SECRET_ID,
"newVersionId": put["VersionId"],
"previousVersionId": current["VersionId"],
}
import json
def handler(event, context):
detail = event.get("detail", {})
print(f"notified: RotationComplete secretId={detail.get('secretId')} newVersionId={detail.get('newVersionId')} event={json.dumps(event)}")
return {"ok": True}
{
"query": "select:TodoWrite",
"max_results": 1
}
, output,
{"type": "tool_reference", "tool_name": "TodoWrite"}
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {"Service": "lambda.amazonaws.com"},
"Action": "sts:AssumeRole"
}
]
}
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {"Service": "states.amazonaws.com"},
"Action": "sts:AssumeRole"
}
]
}
Capture secret ARN
set -e export AWS_PAGER="" ENDPOINT="http://localstack:4566" REGION="us-east-1" SECRET_ARN=$(aws --endpoint-url=$ENDPOINT --region $REGION secretsmanager describe-secret --secret-id app/db/password --query 'ARN' --output text) echo $SECRET_ARN > /tmp/rotation/secret_arn echo "SECRET_ARN=$SECRET_ARN"
SECRET_ARN=arn:aws:secretsmanager:us-east-1:000000000000:secret:app/db/password-jbuLIu [stdout] SECRET_ARN=arn:aws:secretsmanager:us-east-1:000000000000:secret:app/db/password-jbuLIu
Write rotator least-priv policy
set -e
KEY_ARN=$(cat /tmp/rotation/key_arn)
SECRET_ARN=$(cat /tmp/rotation/secret_arn)
cat > /tmp/rotation/policies/rotator-policy.json <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Logs",
"Effect": "Allow",
"Action": ["logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents"],
"Resource": "arn:aws:logs:us-east-1:000000000000:log-group:/aws/lambda/secrets-rotator*"
},
{
"Sid": "SecretRW",
"Effect": "Allow",
"Action": [
"secretsmanager:GetSecretValue",
"secretsmanager:PutSecretValue",
"secretsmanager:DescribeSecret",
"secretsmanager:UpdateSecretVersionStage"
],
"Resource": "${SECRET_ARN}"
},
{
"Sid": "KmsForSecret",
"Effect": "Allow",
"Action": ["kms:Decrypt", "kms:GenerateDataKey"],
"Resource": "${KEY_ARN}",
"Condition": {
"StringEquals": {
"kms:ViaService": "secretsmanager.us-east-1.amazonaws.com"
}
}
}
]
}
EOF
cat /tmp/rotation/policies/rotator-policy.json | jq .{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Logs",
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "arn:aws:logs:us-east-1:000000000000:log-group:/aws/lambda/secrets-rotator*"
},
{
"Sid": "SecretRW",
"Effect": "Allow",
"Action": [
"secretsmanager:GetSecretValue",
"secretsmanager:PutSecretValue",
"secretsmanager:DescribeSecret",
"secretsmanager:UpdateSecretVersionStage"
],
"Resource": "arn:aws:secretsmanager:us-east-1:000000000000:secret:app/db/password-jbuLIu"
},
{
"Sid": "KmsForSecret",
"Effect": "Allow",
"Action": [
"kms:Decrypt",
"kms:GenerateDataKey"
],
"Resource": "arn:aws:kms:us-east-1:000000000000:key/57fb9d95-f011-4af5-85a5-7ec3a0d5775b",
"Condition": {
"StringEquals": {
"kms:ViaService": "secretsmanager.us-east-1.amazonaws.com"
}
}
}
]
}
[stdout]
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Logs",
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "arn:aws:logs:us-east-1:000000000000:log-group:/aws/lambda/secrets-rotator*"
},
{
"Sid": "SecretRW",
"Effect": "Allow",
"Action": [
"secretsmanager:GetSecretValue",
"secretsmanager:PutSecretValue",
"secretsmanager:DescribeSecret",
"secretsmanager:UpdateSecretVersionStage"
],
"Resource": "arn:aws:secretsmanager:us-east-1:000000000000:secret:app/db/password-jbuLIu"
},
{
"Sid": "KmsForSecret",
"Effect": "Allow",
"Action": [
"kms:Decrypt",
"kms:GenerateDataKey"
],
"Resource": "arn:aws:kms:us-east-1:000000000000:key/57fb9d95-f011-4af5-85a5-7ec3a0d5775b",
"Condition": {
"StringEquals": {
"kms:ViaService": "secretsmanager.us-east-1.amazonaws.com"
}
}
}
]
}Create rotator role + inline policy
set -e export AWS_PAGER="" ENDPOINT="http://localstack:4566" REGION="us-east-1" # Rotator role aws --endpoint-url=$ENDPOINT --region $REGION iam create-role \ --role-name secrets-rotator-role \ --assume-role-policy-document file:///tmp/rotation/policies/lambda-trust.json \ --query 'Role.Arn' --output text > /tmp/rotation/rotator_role_arn cat /tmp/rotation/rotator_role_arn aws --endpoint-url=$ENDPOINT --region $REGION iam put-role-policy \ --role-name secrets-rotator-role \ --policy-name rotator-inline \ --policy-document file:///tmp/rotation/policies/rotator-policy.json
arn:aws:iam::000000000000:role/secrets-rotator-role [stdout] arn:aws:iam::000000000000:role/secrets-rotator-role
Package and deploy rotator lambda
set -e
export AWS_PAGER=""
ENDPOINT="http://localstack:4566"
REGION="us-east-1"
# Package rotator lambda
cd /tmp/rotation/rotator && zip -q rotator.zip handler.py && ls -la rotator.zip
ROLE_ARN=$(cat /tmp/rotation/rotator_role_arn)
aws --endpoint-url=$ENDPOINT --region $REGION lambda create-function \
--function-name secrets-rotator \
--runtime python3.11 \
--role $ROLE_ARN \
--handler handler.handler \
--zip-file fileb:///tmp/rotation/rotator/rotator.zip \
--environment "Variables={SECRET_ID=app/db/password,AWS_ENDPOINT_URL=http://localstack:4566}" \
--timeout 30 \
--query 'FunctionArn' --output text > /tmp/rotation/rotator_fn_arn
cat /tmp/rotation/rotator_fn_arn
# Wait for active
for i in 1 2 3 4 5; do
STATE=$(aws --endpoint-url=$ENDPOINT --region $REGION lambda get-function --function-name secrets-rotator --query 'Configuration.State' --output text)
echo "state=$STATE"
[ "$STATE" = "Active" ] && break
sleep 2
done-rw-r--r-- 1 root root 629 Apr 22 06:38 rotator.zip arn:aws:lambda:us-east-1:000000000000:function:secrets-rotator state=Pending state=Active Shell cwd was reset to /app [stdout] -rw-r--r-- 1 root root 629 Apr 22 06:38 rotator.zip arn:aws:lambda:us-east-1:000000000000:function:secrets-rotator state=Pending state=Active [stderr] Shell cwd was reset to /app
Smoke test rotator lambda
set -e
export AWS_PAGER=""
ENDPOINT="http://localstack:4566"
REGION="us-east-1"
# Quick smoke test to make sure rotator works end-to-end
aws --endpoint-url=$ENDPOINT --region $REGION lambda invoke \
--function-name secrets-rotator \
--payload '{}' \
--cli-binary-format raw-in-base64-out \
/tmp/rotation/rotator_test.out
cat /tmp/rotation/rotator_test.out; echo
aws --endpoint-url=$ENDPOINT --region $REGION secretsmanager get-secret-value --secret-id app/db/password --version-stage AWSCURRENT --query 'SecretString' --output text
aws --endpoint-url=$ENDPOINT --region $REGION secretsmanager get-secret-value --secret-id app/db/password --version-stage AWSPREVIOUS --query 'SecretString' --output text{
"StatusCode": 200,
"ExecutedVersion": "$LATEST"
}
{"secretId": "app/db/password", "newVersionId": "5f47d313-5a43-40de-aa00-52e0dc509a12", "previousVersionId": "68dd6dd2-226b-4b06-b655-fbdcd60b0b4f"}
{"username": "appuser", "password": "juOqj_kYYia-uR1vrq2yvgNFgpBfetNe"}
{"username":"appuser","password":"initial-placeholder"}
[stdout]
{
"StatusCode": 200,
"ExecutedVersion": "$LATEST"
}
{"secretId": "app/db/password", "newVersionId": "5f47d313-5a43-40de-aa00-52e0dc509a12", "previousVersionId": "68dd6dd2-226b-4b06-b655-fbdcd60b0b4f"}
{"username": "appuser", "password": "juOqj_kYYia-uR1vrq2yvgNFgpBfetNe"}
{"username":"appuser","password":"initial-placeholder"}Create notifier role and lambda
set -e
export AWS_PAGER=""
ENDPOINT="http://localstack:4566"
REGION="us-east-1"
cat > /tmp/rotation/policies/notifier-policy.json <<'EOF'
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Logs",
"Effect": "Allow",
"Action": ["logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents"],
"Resource": "arn:aws:logs:us-east-1:000000000000:log-group:/aws/lambda/notification-lambda*"
}
]
}
EOF
# Role for notifier - only logs, no other perms since it's invoked by EventBridge
aws --endpoint-url=$ENDPOINT --region $REGION iam create-role \
--role-name notification-lambda-role \
--assume-role-policy-document file:///tmp/rotation/policies/lambda-trust.json \
--query 'Role.Arn' --output text > /tmp/rotation/notifier_role_arn
cat /tmp/rotation/notifier_role_arn
aws --endpoint-url=$ENDPOINT --region $REGION iam put-role-policy \
--role-name notification-lambda-role \
--policy-name notifier-inline \
--policy-document file:///tmp/rotation/policies/notifier-policy.json
cd /tmp/rotation/notifier && zip -q notifier.zip handler.py
ROLE_ARN=$(cat /tmp/rotation/notifier_role_arn)
aws --endpoint-url=$ENDPOINT --region $REGION lambda create-function \
--function-name notification-lambda \
--runtime python3.11 \
--role $ROLE_ARN \
--handler handler.handler \
--zip-file fileb:///tmp/rotation/notifier/notifier.zip \
--timeout 15 \
--query 'FunctionArn' --output text > /tmp/rotation/notifier_fn_arn
cat /tmp/rotation/notifier_fn_arn
for i in 1 2 3 4 5; do
STATE=$(aws --endpoint-url=$ENDPOINT --region $REGION lambda get-function --function-name notification-lambda --query 'Configuration.State' --output text)
echo "state=$STATE"
[ "$STATE" = "Active" ] && break
sleep 2
donearn:aws:iam::000000000000:role/notification-lambda-role arn:aws:lambda:us-east-1:000000000000:function:notification-lambda state=Active Shell cwd was reset to /app [stdout] arn:aws:iam::000000000000:role/notification-lambda-role arn:aws:lambda:us-east-1:000000000000:function:notification-lambda state=Active [stderr] Shell cwd was reset to /app
Create bus, rule, target, and lambda invoke permission
set -e
export AWS_PAGER=""
ENDPOINT="http://localstack:4566"
REGION="us-east-1"
# Custom EventBridge bus
aws --endpoint-url=$ENDPOINT --region $REGION events create-event-bus --name rotation-events --query 'EventBusArn' --output text > /tmp/rotation/bus_arn
cat /tmp/rotation/bus_arn
# Rule scoped to custom bus
cat > /tmp/rotation/rule-pattern.json <<'EOF'
{"source": ["rotation.pipeline"], "detail-type": ["RotationComplete"]}
EOF
aws --endpoint-url=$ENDPOINT --region $REGION events put-rule \
--name on-rotation-success \
--event-bus-name rotation-events \
--event-pattern file:///tmp/rotation/rule-pattern.json \
--state ENABLED \
--query 'RuleArn' --output text > /tmp/rotation/rule_arn
cat /tmp/rotation/rule_arn
# Target: notification lambda
NOTIFIER_ARN=$(cat /tmp/rotation/notifier_fn_arn)
aws --endpoint-url=$ENDPOINT --region $REGION events put-targets \
--rule on-rotation-success \
--event-bus-name rotation-events \
--targets "Id=notifier,Arn=${NOTIFIER_ARN}"
# Lambda permission for EventBridge to invoke notifier (scoped to this rule)
RULE_ARN=$(cat /tmp/rotation/rule_arn)
aws --endpoint-url=$ENDPOINT --region $REGION lambda add-permission \
--function-name notification-lambda \
--statement-id allow-eventbridge-rotation-rule \
--action lambda:InvokeFunction \
--principal events.amazonaws.com \
--source-arn $RULE_ARNarn:aws:events:us-east-1:000000000000:event-bus/rotation-events
arn:aws:events:us-east-1:000000000000:rule/rotation-events/on-rotation-success
{
"FailedEntryCount": 0,
"FailedEntries": []
}
{
"Statement": "{\"Sid\": \"allow-eventbridge-rotation-rule\", \"Effect\": \"Allow\", \"Action\": \"lambda:InvokeFunction\", \"Resource\": \"arn:aws:lambda:us-east-1:000000000000:function:notification-lambda\", \"Principal\": {\"Service\": \"events.amazonaws.com\"}, \"Condition\": {\"ArnLike\": {\"AWS:SourceArn\": \"arn:aws:events:us-east-1:000000000000:rule/rotation-events/on-rotation-success\"}}}"
}
[stdout]
arn:aws:events:us-east-1:000000000000:event-bus/rotation-events
arn:aws:events:us-east-1:000000000000:rule/rotation-events/on-rotation-success
{
"FailedEntryCount": 0,
"FailedEntries": []
}
{
"Statement": "{\"Sid\": \"allow-eventbridge-rotation-rule\", \"Effect\": \"Allow\", \"Action\": \"lambda:InvokeFunction\", \"Resource\": \"arn:aws:lambda:us-east-1:000000000000:function:notification-lambda\", \"Principal\": {\"Service\": \"events.amazonaws.com\"}, \"Condition\": {\"ArnLike\": {\"AWS:SourceArn\": \"arn:aws:events:us-east-1:000000000000:rule/rotation-events/on-rotation-success\"}}}"
}Test event bus wiring
set -e
export AWS_PAGER=""
ENDPOINT="http://localstack:4566"
REGION="us-east-1"
# Quick test: put an event and see if notifier fires
aws --endpoint-url=$ENDPOINT --region $REGION events put-events \
--entries 'Source=rotation.pipeline,DetailType=RotationComplete,EventBusName=rotation-events,Detail={"secretId":"app/db/password","newVersionId":"test"}'
sleep 3
# Check logs
aws --endpoint-url=$ENDPOINT --region $REGION logs describe-log-groups --query 'logGroups[].logGroupName'aws: [ERROR]: An error occurred (ParamValidation): Error parsing parameter '--entries': Expected: '=', received: '"' for input:
Source=rotation.pipeline,DetailType=RotationComplete,EventBusName=rotation-events,Detail={"secretId":"app/db/password","newVersionId":"test"}
^
[
"/aws/lambda/secrets-rotator"
]
[stdout]
aws: [ERROR]: An error occurred (ParamValidation): Error parsing parameter '--entries': Expected: '=', received: '"' for input:
Source=rotation.pipeline,DetailType=RotationComplete,EventBusName=rotation-events,Detail={"secretId":"app/db/password","newVersionId":"test"}
^
[
"/aws/lambda/secrets-rotator"
]Test event bus delivers to lambda
set -e
export AWS_PAGER=""
ENDPOINT="http://localstack:4566"
REGION="us-east-1"
cat > /tmp/rotation/test-event.json <<'EOF'
[{
"Source": "rotation.pipeline",
"DetailType": "RotationComplete",
"EventBusName": "rotation-events",
"Detail": "{\"secretId\":\"app/db/password\",\"newVersionId\":\"test\"}"
}]
EOF
aws --endpoint-url=$ENDPOINT --region $REGION events put-events --entries file:///tmp/rotation/test-event.json
sleep 4
aws --endpoint-url=$ENDPOINT --region $REGION logs describe-log-groups --query 'logGroups[].logGroupName'
# tail
LG=/aws/lambda/notification-lambda
STREAMS=$(aws --endpoint-url=$ENDPOINT --region $REGION logs describe-log-streams --log-group-name $LG --query 'logStreams[].logStreamName' --output text 2>/dev/null || echo "")
echo "streams=$STREAMS"
for s in $STREAMS; do
aws --endpoint-url=$ENDPOINT --region $REGION logs get-log-events --log-group-name $LG --log-stream-name $s --query 'events[].message' --output text
done{
"FailedEntryCount": 0,
"Entries": [
{
"EventId": "9b4a6699-358d-40b2-a5ee-e550b731373c"
}
]
}
[
"/aws/lambda/notification-lambda",
"/aws/lambda/secrets-rotator"
]
streams=2026/04/22/[$LATEST]277eb76dbfd530d0294975397e07cb1d
START RequestId: 008c276c-a3aa-4729-8764-c70ffb9a84a6 Version: $LATEST
notified: RotationComplete secretId=app/db/password newVersionId=test event={"version": "0", "id": "9b4a6699-358d-40b2-a5ee-e550b731373c", "detail-type": "RotationComplete", "source": "rotation.pipeline", "account": "000000000000", "time": "2026-04-22T06:39:49Z", "region": "us-east-1", "resources": [], "detail": {"secretId": "app/db/password", "newVersionId": "test"}}
END RequestId: 008c276c-a3aa-4729-8764-c70ffb9a84a6
REPORT RequestId: 008c276c-a3aa-4729-8764-c70ffb9a84a6 Duration: 9.38 ms Billed Duration: 10 ms Memory Size: 128 MB Max Memory Used: 128 MB
[stdout]
{
"FailedEntryCount": 0,
"Entries": [
{
"EventId": "9b4a6699-358d-40b2-a5ee-e550b731373c"
}
]
}
[
"/aws/lambda/notification-lambda",
"/aws/lambda/secrets-rotator"
]
streams=2026/04/22/[$LATEST]277eb76dbfd530d0294975397e07cb1d
START RequestId: 008c276c-a3aa-4729-8764-c70ffb9a84a6 Version: $LATEST
notified: RotationComplete secretId=app/db/password newVersionId=test event={"version": "0", "id": "9b4a6699-358d-40b2-a5ee-e550b731373c", "detail-type": "RotationComplete", "source": "rotation.pipeline", "account": "000000000000", "time": "2026-04-22T06:39:49Z", "region": "us-east-1", "resources": [], "detail": {"secretId": "app/db/password", "newVersionId": "test"}}
END RequestId: 008c276c-a3aa-4729-8764-c70ffb9a84a6
REPORT RequestId: 008c276c-a3aa-4729-8764-c70ffb9a84a6 Duration: 9.38 ms Billed Duration: 10 ms Memory Size: 128 MB Max Memory Used: 128 MB{
"Comment": "Rotate secret then emit RotationComplete",
"StartAt": "Rotate",
"States": {
"Rotate": {
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"Parameters": {
"FunctionName": "arn:aws:lambda:us-east-1:000000000000:function:secrets-rotator",
"Payload": {}
},
"ResultSelector": {
"secretId.$": "$.Payload.secretId",
"newVersionId.$": "$.Payload.newVersionId",
"previousVersionId.$": "$.Payload.previousVersionId"
},
"ResultPath": "$.rotation",
"Next": "EmitEvent"
},
"EmitEvent": {
"Type": "Task",
"Resource": "arn:aws:states:::events:putEvents",
"Parameters": {
"Entries": [
{
"EventBusName": "rotation-events",
"Source": "rotation.pipeline",
"DetailType": "RotationComplete",
"Detail": {
"secretId.$": "$.rotation.secretId",
"newVersionId.$": "$.rotation.newVersionId",
"previousVersionId.$": "$.rotation.previousVersionId"
}
}
]
},
"End": true
}
}
}
Create child SM role, policy, and state machine
set -e
export AWS_PAGER=""
ENDPOINT="http://localstack:4566"
REGION="us-east-1"
ROTATOR_ARN=$(cat /tmp/rotation/rotator_fn_arn)
BUS_ARN=$(cat /tmp/rotation/bus_arn)
cat > /tmp/rotation/policies/child-sm-policy.json <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "InvokeRotator",
"Effect": "Allow",
"Action": "lambda:InvokeFunction",
"Resource": "${ROTATOR_ARN}"
},
{
"Sid": "PutEventsOnBus",
"Effect": "Allow",
"Action": "events:PutEvents",
"Resource": "${BUS_ARN}"
}
]
}
EOF
jq . /tmp/rotation/policies/child-sm-policy.json
aws --endpoint-url=$ENDPOINT --region $REGION iam create-role \
--role-name rotation-child-role \
--assume-role-policy-document file:///tmp/rotation/policies/states-trust.json \
--query 'Role.Arn' --output text > /tmp/rotation/child_role_arn
cat /tmp/rotation/child_role_arn
aws --endpoint-url=$ENDPOINT --region $REGION iam put-role-policy \
--role-name rotation-child-role \
--policy-name child-sm-inline \
--policy-document file:///tmp/rotation/policies/child-sm-policy.json
CHILD_ROLE=$(cat /tmp/rotation/child_role_arn)
aws --endpoint-url=$ENDPOINT --region $REGION stepfunctions create-state-machine \
--name rotation-child \
--type STANDARD \
--role-arn $CHILD_ROLE \
--definition file:///tmp/rotation/sm/child.asl.json \
--query 'stateMachineArn' --output text > /tmp/rotation/child_sm_arn
cat /tmp/rotation/child_sm_arn{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "InvokeRotator",
"Effect": "Allow",
"Action": "lambda:InvokeFunction",
"Resource": "arn:aws:lambda:us-east-1:000000000000:function:secrets-rotator"
},
{
"Sid": "PutEventsOnBus",
"Effect": "Allow",
"Action": "events:PutEvents",
"Resource": "arn:aws:events:us-east-1:000000000000:event-bus/rotation-events"
}
]
}
arn:aws:iam::000000000000:role/rotation-child-role
arn:aws:states:us-east-1:000000000000:stateMachine:rotation-child
[stdout]
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "InvokeRotator",
"Effect": "Allow",
"Action": "lambda:InvokeFunction",
"Resource": "arn:aws:lambda:us-east-1:000000000000:function:secrets-rotator"
},
{
"Sid": "PutEventsOnBus",
"Effect": "Allow",
"Action": "events:PutEvents",
"Resource": "arn:aws:events:us-east-1:000000000000:event-bus/rotation-events"
}
]
}
arn:aws:iam::000000000000:role/rotation-child-role
arn:aws:states:us-east-1:000000000000:stateMachine:rotation-childSmoke test child SM
set -e
export AWS_PAGER=""
ENDPOINT="http://localstack:4566"
REGION="us-east-1"
# Smoke test child SM standalone
CHILD_SM=$(cat /tmp/rotation/child_sm_arn)
EXEC=$(aws --endpoint-url=$ENDPOINT --region $REGION stepfunctions start-execution --state-machine-arn $CHILD_SM --input '{}' --query 'executionArn' --output text)
echo "exec=$EXEC"
for i in 1 2 3 4 5 6 7 8 9 10; do
STATUS=$(aws --endpoint-url=$ENDPOINT --region $REGION stepfunctions describe-execution --execution-arn $EXEC --query 'status' --output text)
echo "i=$i status=$STATUS"
[ "$STATUS" = "SUCCEEDED" ] && break
[ "$STATUS" = "FAILED" ] && break
sleep 2
done
aws --endpoint-url=$ENDPOINT --region $REGION stepfunctions describe-execution --execution-arn $EXEC --query '{status:status,output:output,error:error,cause:cause}'exec=arn:aws:states:us-east-1:000000000000:execution:rotation-child:fc3d60b2-a481-4143-9328-4730ae52f810
i=1 status=RUNNING
i=2 status=SUCCEEDED
{
"status": "SUCCEEDED",
"output": "{\"FailedEntryCount\":0,\"Entries\":[{\"EventId\":\"b0e3a178-7121-47f8-8e09-9005e438c56f\"}]}",
"error": null,
"cause": null
}
[stdout]
exec=arn:aws:states:us-east-1:000000000000:execution:rotation-child:fc3d60b2-a481-4143-9328-4730ae52f810
i=1 status=RUNNING
i=2 status=SUCCEEDED
{
"status": "SUCCEEDED",
"output": "{\"FailedEntryCount\":0,\"Entries\":[{\"EventId\":\"b0e3a178-7121-47f8-8e09-9005e438c56f\"}]}",
"error": null,
"cause": null
}{
"Comment": "Parent orchestrator - delegates to child SM sync:2",
"StartAt": "InvokeChild",
"States": {
"InvokeChild": {
"Type": "Task",
"Resource": "arn:aws:states:::states:startExecution.sync:2",
"Parameters": {
"StateMachineArn": "arn:aws:states:us-east-1:000000000000:stateMachine:rotation-child",
"Input": {}
},
"End": true
}
}
}
Create parent role, policy, and SM
set -e
export AWS_PAGER=""
ENDPOINT="http://localstack:4566"
REGION="us-east-1"
CHILD_SM=$(cat /tmp/rotation/child_sm_arn)
# Parent role: startExecution.sync:2 needs StartExecution on child + managed rule perms + Describe/StopExecution
cat > /tmp/rotation/policies/parent-sm-policy.json <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "StartChild",
"Effect": "Allow",
"Action": "states:StartExecution",
"Resource": "${CHILD_SM}"
},
{
"Sid": "MonitorChild",
"Effect": "Allow",
"Action": ["states:DescribeExecution", "states:StopExecution"],
"Resource": "arn:aws:states:us-east-1:000000000000:execution:rotation-child:*"
},
{
"Sid": "ManagedRuleForSync2",
"Effect": "Allow",
"Action": ["events:PutRule", "events:PutTargets", "events:DescribeRule"],
"Resource": "arn:aws:events:us-east-1:000000000000:rule/StepFunctionsGetEventsForStepFunctionsExecutionRule"
}
]
}
EOF
jq . /tmp/rotation/policies/parent-sm-policy.json
aws --endpoint-url=$ENDPOINT --region $REGION iam create-role \
--role-name rotation-orchestrator-role \
--assume-role-policy-document file:///tmp/rotation/policies/states-trust.json \
--query 'Role.Arn' --output text > /tmp/rotation/parent_role_arn
cat /tmp/rotation/parent_role_arn
aws --endpoint-url=$ENDPOINT --region $REGION iam put-role-policy \
--role-name rotation-orchestrator-role \
--policy-name parent-sm-inline \
--policy-document file:///tmp/rotation/policies/parent-sm-policy.json
PARENT_ROLE=$(cat /tmp/rotation/parent_role_arn)
aws --endpoint-url=$ENDPOINT --region $REGION stepfunctions create-state-machine \
--name rotation-orchestrator \
--type STANDARD \
--role-arn $PARENT_ROLE \
--definition file:///tmp/rotation/sm/parent.asl.json \
--query 'stateMachineArn' --output text > /tmp/rotation/parent_sm_arn
cat /tmp/rotation/parent_sm_arn{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "StartChild",
"Effect": "Allow",
"Action": "states:StartExecution",
"Resource": "arn:aws:states:us-east-1:000000000000:stateMachine:rotation-child"
},
{
"Sid": "MonitorChild",
"Effect": "Allow",
"Action": [
"states:DescribeExecution",
"states:StopExecution"
],
"Resource": "arn:aws:states:us-east-1:000000000000:execution:rotation-child:*"
},
{
"Sid": "ManagedRuleForSync2",
"Effect": "Allow",
"Action": [
"events:PutRule",
"events:PutTargets",
"events:DescribeRule"
],
"Resource": "arn:aws:events:us-east-1:000000000000:rule/StepFunctionsGetEventsForStepFunctionsExecutionRule"
}
]
}
arn:aws:iam::000000000000:role/rotation-orchestrator-role
arn:aws:states:us-east-1:000000000000:stateMachine:rotation-orchestrator
[stdout]
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "StartChild",
"Effect": "Allow",
"Action": "states:StartExecution",
"Resource": "arn:aws:states:us-east-1:000000000000:stateMachine:rotation-child"
},
{
"Sid": "MonitorChild",
"Effect": "Allow",
"Action": [
"states:DescribeExecution",
"states:StopExecution"
],
"Resource": "arn:aws:states:us-east-1:000000000000:execution:rotation-child:*"
},
{
"Sid": "ManagedRuleForSync2",
"Effect": "Allow",
"Action": [
"events:PutRule",
"events:PutTargets",
"events:DescribeRule"
],
"Resource": "arn:aws:events:us-east-1:000000000000:rule/StepFunctionsGetEventsForStepFunctionsExecutionRule"
}
]
}
arn:aws:iam::000000000000:role/rotation-orchestrator-role
arn:aws:states:us-east-1:000000000000:stateMachine:rotation-orchestratorRun parent SM and poll to completion
set -e
export AWS_PAGER=""
ENDPOINT="http://localstack:4566"
REGION="us-east-1"
# Capture pre-state
BEFORE_CURRENT_VERSION=$(aws --endpoint-url=$ENDPOINT --region $REGION secretsmanager get-secret-value --secret-id app/db/password --version-stage AWSCURRENT --query 'VersionId' --output text)
BEFORE_CURRENT_STRING=$(aws --endpoint-url=$ENDPOINT --region $REGION secretsmanager get-secret-value --secret-id app/db/password --version-stage AWSCURRENT --query 'SecretString' --output text)
echo "BEFORE_CURRENT_VERSION=$BEFORE_CURRENT_VERSION"
echo "BEFORE_CURRENT_STRING=$BEFORE_CURRENT_STRING"
PARENT_SM=$(cat /tmp/rotation/parent_sm_arn)
START_TS=$(date +%s)
EXEC=$(aws --endpoint-url=$ENDPOINT --region $REGION stepfunctions start-execution --state-machine-arn $PARENT_SM --query 'executionArn' --output text)
echo "PARENT_EXEC=$EXEC START_TS=$START_TS"
# Poll until success or 90s
for i in $(seq 1 45); do
STATUS=$(aws --endpoint-url=$ENDPOINT --region $REGION stepfunctions describe-execution --execution-arn $EXEC --query 'status' --output text)
ELAPSED=$(( $(date +%s) - START_TS ))
echo "i=$i t=${ELAPSED}s status=$STATUS"
[ "$STATUS" = "SUCCEEDED" ] && break
[ "$STATUS" = "FAILED" ] && break
[ "$STATUS" = "TIMED_OUT" ] && break
sleep 2
done
aws --endpoint-url=$ENDPOINT --region $REGION stepfunctions describe-execution --execution-arn $EXEC --query '{status:status,error:error,cause:cause,output:output}'BEFORE_CURRENT_VERSION=d1803e8a-9899-4a9e-994d-bd594cd4ba67
BEFORE_CURRENT_STRING={"username": "appuser", "password": "3efZa_Iai2Je2hUnVNO_rY51pTzd66_k"}
PARENT_EXEC=arn:aws:states:us-east-1:000000000000:execution:rotation-orchestrator:f7ee46b5-2603-4b69-b615-8246717a06cb START_TS=1776840089
i=1 t=0s status=RUNNING
i=2 t=3s status=SUCCEEDED
{
"status": "SUCCEEDED",
"error": null,
"cause": null,
"output": "{\"ExecutionArn\":\"arn:aws:states:us-east-1:000000000000:execution:rotation-child:b5d3ddab-73eb-4c3b-9b31-a3b60b8238a9\",\"StateMachineArn\":\"arn:aws:states:us-east-1:000000000000:stateMachine:rotation-child\",\"Name\":\"b5d3ddab-73eb-4c3b-9b31-a3b60b8238a9\",\"Status\":\"SUCCEEDED\",\"StartDate\":\"2026-04-22T06:41:29.333290+00:00\",\"StopDate\":\"2026-04-22T06:41:30.695444+00:00\",\"Input\":{},\"InputDetails\":{\"Included\":true},\"Output\":{\"FailedEntryCount\":0,\"Entries\":[{\"EventId\":\"9d4395d3-d27b-4d9c-b751-e0f8e3a852fc\"}]},\"OutputDetails\":{\"Included\":true}}"
}
[stdout]
BEFORE_CURRENT_VERSION=d1803e8a-9899-4a9e-994d-bd594cd4ba67
BEFORE_CURRENT_STRING={"username": "appuser", "password": "3efZa_Iai2Je2hUnVNO_rY51pTzd66_k"}
PARENT_EXEC=arn:aws:states:us-east-1:000000000000:execution:rotation-orchestrator:f7ee46b5-2603-4b69-b615-8246717a06cb START_TS=1776840089
i=1 t=0s status=RUNNING
i=2 t=3s status=SUCCEEDED
{
"status": "SUCCEEDED",
"error": null,
"cause": null,
"output": "{\"ExecutionArn\":\"arn:aws:states:us-east-1:000000000000:execution:rotation-child:b5d3ddab-73eb-4c3b-9b31-a3b60b8238a9\",\"StateMachineArn\":\"arn:aws:states:us-east-1:000000000000:stateMachine:rotation-child\",\"Name\":\"b5d3ddab-73eb-4c3b-9b31-a3b60b8238a9\",\"Status\":\"SUCCEEDED\",\"StartDate\":\"2026-04-22T06:41:29.333290+00:00\",\"StopDate\":\"2026-04-22T06:41:30.695444+00:00\",\"Input\":{},\"InputDetails\":{\"Included\":true},\"Output\":{\"FailedEntryCount\":0,\"Entries\":[{\"EventId\":\"9d4395d3-d27b-4d9c-b751-e0f8e3a852fc\"}]},\"OutputDetails\":{\"Included\":true}}"
}Verify end-to-end success criteria
set -e
export AWS_PAGER=""
ENDPOINT="http://localstack:4566"
REGION="us-east-1"
echo "=== AWSCURRENT ==="
aws --endpoint-url=$ENDPOINT --region $REGION secretsmanager get-secret-value --secret-id app/db/password --version-stage AWSCURRENT --query '{VersionId:VersionId,SecretString:SecretString,VersionStages:VersionStages}'
echo "=== AWSPREVIOUS ==="
aws --endpoint-url=$ENDPOINT --region $REGION secretsmanager get-secret-value --secret-id app/db/password --version-stage AWSPREVIOUS --query '{VersionId:VersionId,SecretString:SecretString,VersionStages:VersionStages}'
echo "=== list-secret-version-ids ==="
aws --endpoint-url=$ENDPOINT --region $REGION secretsmanager list-secret-version-ids --secret-id app/db/password
echo "=== notification lambda logs ==="
sleep 3
LG=/aws/lambda/notification-lambda
for s in $(aws --endpoint-url=$ENDPOINT --region $REGION logs describe-log-streams --log-group-name $LG --query 'logStreams[].logStreamName' --output text); do
aws --endpoint-url=$ENDPOINT --region $REGION logs get-log-events --log-group-name $LG --log-stream-name $s --query 'events[].message' --output text
done=== AWSCURRENT ===
{
"VersionId": "d896b705-d9b2-494b-9dee-81430b41449a",
"SecretString": "{\"username\": \"appuser\", \"password\": \"F6tIzTN4fJWn6xHarXwoi8WkKVNoZYWM\"}",
"VersionStages": [
"AWSCURRENT"
]
}
=== AWSPREVIOUS ===
{
"VersionId": "d1803e8a-9899-4a9e-994d-bd594cd4ba67",
"SecretString": "{\"username\": \"appuser\", \"password\": \"3efZa_Iai2Je2hUnVNO_rY51pTzd66_k\"}",
"VersionStages": [
"AWSPREVIOUS"
]
}
=== list-secret-version-ids ===
{
"Versions": [
{
"VersionId": "d896b705-d9b2-494b-9dee-81430b41449a",
"VersionStages": [
"AWSCURRENT"
],
"LastAccessedDate": "2026-04-22T00:00:00+00:00",
"CreatedDate": "2026-04-22T06:41:30+00:00"
},
{
"VersionId": "d1803e8a-9899-4a9e-994d-bd594cd4ba67",
"VersionStages": [
"AWSPREVIOUS"
],
"LastAccessedDate": "2026-04-22T00:00:00+00:00",
"CreatedDate": "2026-04-22T06:40:26+00:00"
}
],
"ARN": "arn:aws:secretsmanager:us-east-1:000000000000:secret:app/db/password-jbuLIu",
"Name": "app/db/password"
}
=== notification lambda logs ===
START RequestId: 008c276c-a3aa-4729-8764-c70ffb9a84a6 Version: $LATEST
notified: RotationComplete secretId=app/db/password newVersionId=test event={"version": "0", "id": "9b4a6699-358d-40b2-a5ee-e550b731373c", "detail-type": "RotationComplete", "source": "rotation.pipeline", "account": "000000000000", "time": "2026-04-22T06:39:49Z", "region": "us-east-1", "resources": [], "detail": {"secretId": "app/db/password", "newVersionId": "test"}}
END RequestId: 008c276c-a3aa-4729-8764-c70ffb9a84a6
REPORT RequestId: 008c276c-a3aa-4729-8764-c70ffb9a84a6 Duration: 9.38 ms Billed Duration: 10 ms Memory Size: 128 MB Max Memory Used: 128 MB
START RequestId: b3a4dd9b-ffbd-457a-8b60-86e5054742a5 Version: $LATEST
notified: RotationComplete secretId=app/db/password newVersionId=d896b705-d9b2-494b-9dee-81430b41449a event={"version": "0", "id": "9d4395d3-d27b-4d9c-b751-e0f8e3a852fc", "detail-type": "RotationComplete", "source": "rotation.pipeline", "account": "000000000000", "time": "2026-04-22T06:41:30Z", "region": "us-east-1", "resources": ["arn:aws:states:us-east-1:000000000000:stateMachine:rotation-child", "arn:aws:states:us-east-1:000000000000:execution:rotation-child:b5d3ddab-73eb-4c3b-9b31-a3b60b8238a9"], "detail": {"secretId": "app/db/password", "newVersionId": "d896b705-d9b2-494b-9dee-81430b41449a", "previousVersionId": "d1803e8a-9899-4a9e-994d-bd594cd4ba67"}}
END RequestId: b3a4dd9b-ffbd-457a-8b60-86e5054742a5
REPORT RequestId: b3a4dd9b-ffbd-457a-8b60-86e5054742a5 Duration: 9.07 ms Billed Duration: 10 ms Memory Size: 128 MB Max Memory Used: 128 MB
START RequestId: 8d5ebe7e-4d7b-4fa8-867f-75a6a3706ba4 Version: $LATEST
notified: RotationComplete secretId=app/db/password newVersionId=d1803e8a-9899-4a9e-994d-bd594cd4ba67 event={"version": "0", "id": "b0e3a178-7121-47f8-8e09-9005e438c56f", "detail-type": "RotationComplete", "source": "rotation.pipeline", "account": "000000000000", "time": "2026-04-22T06:40:27Z", "region": "us-east-1", "resources": ["arn:aws:states:us-east-1:000000000000:stateMachine:rotation-child", "arn:aws:states:us-east-1:000000000000:execution:rotation-child:fc3d60b2-a481-4143-9328-4730ae52f810"], "detail": {"secretId": "app/db/password", "newVersionId": "d1803e8a-9899-4a9e-994d-bd594cd4ba67", "previousVersionId": "5f47d313-5a43-40de-aa00-52e0dc509a12"}}
END RequestId: 8d5ebe7e-4d7b-4fa8-867f-75a6a3706ba4
REPORT RequestId: 8d5ebe7e-4d7b-4fa8-867f-75a6a3706ba4 Duration: 9.32 ms Billed Duration: 10 ms Memory Size: 128 MB Max Memory Used: 128 MB
[stdout]
=== AWSCURRENT ===
{
"VersionId": "d896b705-d9b2-494b-9dee-81430b41449a",
"SecretString": "{\"username\": \"appuser\", \"password\": \"F6tIzTN4fJWn6xHarXwoi8WkKVNoZYWM\"}",
"VersionStages": [
"AWSCURRENT"
]
}
=== AWSPREVIOUS ===
{
"VersionId": "d1803e8a-9899-4a9e-994d-bd594cd4ba67",
"SecretString": "{\"username\": \"appuser\", \"password\": \"3efZa_Iai2Je2hUnVNO_rY51pTzd66_k\"}",
"VersionStages": [
"AWSPREVIOUS"
]
}
=== list-secret-version-ids ===
{
"Versions": [
{
"VersionId": "d896b705-d9b2-494b-9dee-81430b41449a",
"VersionStages": [
"AWSCURRENT"
],
"LastAccessedDate": "2026-04-22T00:00:00+00:00",
"CreatedDate": "2026-04-22T06:41:30+00:00"
},
{
"VersionId": "d1803e8a-9899-4a9e-994d-bd594cd4ba67",
"VersionStages": [
"AWSPREVIOUS"
],
"LastAccessedDate": "2026-04-22T00:00:00+00:00",
"CreatedDate": "2026-04-22T06:40:26+00:00"
}
],
"ARN": "arn:aws:secretsmanager:us-east-1:000000000000:secret:app/db/password-jbuLIu",
"Name": "app/db/password"
}
=== notification lambda logs ===
START RequestId: 008c276c-a3aa-4729-8764-c70ffb9a84a6 Version: $LATEST
notified: RotationComplete secretId=app/db/password newVersionId=test event={"version": "0", "id": "9b4a6699-358d-40b2-a5ee-e550b731373c", "detail-type": "RotationComplete", "source": "rotation.pipeline", "account": "000000000000", "time": "2026-04-22T06:39:49Z", "region": "us-east-1", "resources": [], "detail": {"secretId": "app/db/password", "newVersionId": "test"}}
END RequestId: 008c276c-a3aa-4729-8764-c70ffb9a84a6
REPORT RequestId: 008c276c-a3aa-4729-8764-c70ffb9a84a6 Duration: 9.38 ms Billed Duration: 10 ms Memory Size: 128 MB Max Memory Used: 128 MB
START RequestId: b3a4dd9b-ffbd-457a-8b60-86e5054742a5 Version: $LATEST
notified: RotationComplete secretId=app/db/password newVersionId=d896b705-d9b2-494b-9dee-81430b41449a event={"version": "0", "id": "9d4395d3-d27b-4d9c-b751-e0f8e3a852fc", "detail-type": "RotationComplete", "source": "rotation.pipeline", "account": "000000000000", "time": "2026-04-22T06:41:30Z", "region": "us-east-1", "resources": ["arn:aws:states:us-east-1:000000000000:stateMachine:rotation-child", "arn:aws:states:us-east-1:000000000000:execution:rotation-child:b5d3ddab-73eb-4c3b-9b31-a3b60b8238a9"], "detail": {"secretId": "app/db/password", "newVersionId": "d896b705-d9b2-494b-9dee-81430b41449a", "previousVersionId": "d1803e8a-9899-4a9e-994d-bd594cd4ba67"}}
END RequestId: b3a4dd9b-ffbd-457a-8b60-86e5054742a5
REPORT RequestId: b3a4dd9b-ffbd-457a-8b60-86e5054742a5 Duration: 9.07 ms Billed Duration: 10 ms Memory Size: 128 MB Max Memory Used: 128 MB
START RequestId: 8d5ebe7e-4d7b-4fa8-867f-75a6a3706ba4 Version: $LATEST
notified: RotationComplete secretId=app/db/password newVersionId=d1803e8a-9899-4a9e-994d-bd594cd4ba67 event={"version": "0", "id": "b0e3a178-7121-47f8-8e09-9005e438c56f", "detail-type": "RotationComplete", "source": "rotation.pipeline", "account": "000000000000", "time": "2026-04-22T06:40:27Z", "region": "us-east-1", "resources": ["arn:aws:states:us-east-1:000000000000:stateMachine:rotation-child", "arn:aws:states:us-east-1:000000000000:execution:rotation-child:fc3d60b2-a481-4143-9328-4730ae52f810"], "detail": {"secretId": "app/db/password", "newVersionId": "d1803e8a-9899-4a9e-994d-bd594cd4ba67", "previousVersionId": "5f47d313-5a43-40de-aa00-52e0dc509a12"}}
END RequestId: 8d5ebe7e-4d7b-4fa8-867f-75a6a3706ba4
REPORT RequestId: 8d5ebe7e-4d7b-4fa8-867f-75a6a3706ba4 Duration: 9.32 ms Billed Duration: 10 ms Memory Size: 128 MB Max Memory Used: 128 MBNo file changes, the agent left the workspace untouched.
============================= test session starts ==============================
platform linux -- Python 3.12.3, pytest-8.4.1, pluggy-1.6.0 -- /root/.cache/uv/archive-v0/YVYuQMk4lFdbtxBOM5w8K/bin/python
cachedir: .pytest_cache
rootdir: /tests
plugins: json-ctrf-0.3.5
collecting ... collected 15 items
../tests/test_state.py::test_aws_cli_binary_is_real PASSED [ 6%]
../tests/test_state.py::test_localstack_reachable PASSED [ 13%]
../tests/test_state.py::test_secret_exists_and_uses_customer_cmk PASSED [ 20%]
../tests/test_state.py::test_rotation_lambda_exists PASSED [ 26%]
../tests/test_state.py::test_notification_lambda_exists PASSED [ 33%]
../tests/test_state.py::test_parent_and_child_state_machines_exist PASSED [ 40%]
../tests/test_state.py::test_event_bus_and_rule_exist PASSED [ 46%]
../tests/test_state.py::test_rule_target_points_at_notification_lambda PASSED [ 53%]
../tests/test_state.py::test_rotator_role_grants_kms_actions PASSED [ 60%]
../tests/test_state.py::test_kms_key_policy_grants_rotator_role FAILED [ 66%]
../tests/test_state.py::test_rotator_lambda_allows_secretsmanager_invoke FAILED [ 73%]
../tests/test_state.py::test_parent_role_grants_sync2_managed_rule_perms FAILED [ 80%]
../tests/test_state.py::test_child_role_can_rotate_and_publish FAILED [ 86%]
../tests/test_state.py::test_notification_lambda_allows_eventbridge_invoke PASSED [ 93%]
../tests/test_state.py::test_end_to_end_parent_execution_rotates_and_notifies PASSED [100%]
=================================== FAILURES ===================================
___________________ test_kms_key_policy_grants_rotator_role ____________________
iam = <botocore.client.IAM object at 0xffffb24cda90>
kms = <botocore.client.KMS object at 0xffffb2a182c0>
def test_kms_key_policy_grants_rotator_role(iam, kms):
role_arn = iam.get_role(RoleName=ROTATOR_ROLE)["Role"]["Arn"]
policy_str = kms.get_key_policy(KeyId=KEY_ALIAS, PolicyName="default")[
"Policy"
]
policy = json.loads(policy_str)
match = False
for st in policy.get("Statement", []):
if st.get("Effect") != "Allow":
continue
principal = st.get("Principal") or {}
if not isinstance(principal, dict):
continue
arns = set(_normalise_list(principal.get("AWS")))
if role_arn not in arns:
continue
actions = set(_normalise_list(st.get("Action")))
if _actions_cover(actions, REQUIRED_KMS_ACTIONS):
match = True
break
> assert match, (
f"KMS key policy on {KEY_ALIAS} has no Allow statement whose "
f"Principal.AWS includes {role_arn} and whose Action covers "
f"{sorted(REQUIRED_KMS_ACTIONS)}. Key policy: {policy_str}"
)
E AssertionError: KMS key policy on alias/app-rotation-key has no Allow statement whose Principal.AWS includes arn:aws:iam::000000000000:role/secrets-rotator-role and whose Action covers ['kms:Decrypt', 'kms:GenerateDataKey']. Key policy: {"Version": "2012-10-17", "Id": "key-default-1", "Statement": [{"Sid": "Enable IAM User Permissions", "Effect": "Allow", "Principal": {"AWS": "arn:aws:iam::000000000000:root"}, "Action": "kms:*", "Resource": "*"}]}
E assert False
/tests/test_state.py:322: AssertionError
_______________ test_rotator_lambda_allows_secretsmanager_invoke _______________
lmb = <botocore.client.Lambda object at 0xffffb29930b0>
def test_rotator_lambda_allows_secretsmanager_invoke(lmb):
try:
policy_doc = json.loads(
> lmb.get_policy(FunctionName=ROTATOR_FUNCTION)["Policy"]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
/tests/test_state.py:332:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/root/.cache/uv/archive-v0/YVYuQMk4lFdbtxBOM5w8K/lib/python3.12/site-packages/botocore/client.py:569: in _api_call
return self._make_api_call(operation_name, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <botocore.client.Lambda object at 0xffffb29930b0>
operation_name = 'GetPolicy', api_params = {'FunctionName': 'secrets-rotator'}
def _make_api_call(self, operation_name, api_params):
operation_model = self._service_model.operation_model(operation_name)
service_name = self._service_model.service_name
history_recorder.record(
'API_CALL',
{
'service': service_name,
'operation': operation_name,
'params': api_params,
},
)
if operation_model.deprecated:
logger.debug(
'Warning: %s.%s() is deprecated', service_name, operation_name
)
request_context = {
'client_region': self.meta.region_name,
'client_config': self.meta.config,
'has_streaming_input': operation_model.has_streaming_input,
'auth_type': operation_model.resolved_auth_type,
'unsigned_payload': operation_model.unsigned_payload,
}
api_params = self._emit_api_params(
api_params=api_params,
operation_model=operation_model,
context=request_context,
)
(
endpoint_url,
additional_headers,
properties,
) = self._resolve_endpoint_ruleset(
operation_model, api_params, request_context
)
if properties:
# Pass arbitrary endpoint info with the Request
# for use during construction.
request_context['endpoint_properties'] = properties
request_dict = self._convert_to_request_dict(
api_params=api_params,
operation_model=operation_model,
endpoint_url=endpoint_url,
context=request_context,
headers=additional_headers,
)
resolve_checksum_context(request_dict, operation_model, api_params)
service_id = self._service_model.service_id.hyphenize()
handler, event_response = self.meta.events.emit_until_response(
f'before-call.{service_id}.{operation_name}',
model=operation_model,
params=request_dict,
request_signer=self._request_signer,
context=request_context,
)
if event_response is not None:
http, parsed_response = event_response
else:
maybe_compress_request(
self.meta.config, request_dict, operation_model
)
apply_request_checksum(request_dict)
http, parsed_response = self._make_request(
operation_model, request_dict, request_context
)
self.meta.events.emit(
f'after-call.{service_id}.{operation_name}',
http_response=http,
parsed=parsed_response,
model=operation_model,
context=request_context,
)
if http.status_code >= 300:
error_info = parsed_response.get("Error", {})
error_code = error_info.get("QueryErrorCode") or error_info.get(
"Code"
)
error_class = self.exceptions.from_code(error_code)
> raise error_class(parsed_response, operation_name)
E botocore.errorfactory.ResourceNotFoundException: An error occurred (ResourceNotFoundException) when calling the GetPolicy operation: The resource you requested does not exist.
/root/.cache/uv/archive-v0/YVYuQMk4lFdbtxBOM5w8K/lib/python3.12/site-packages/botocore/client.py:1023: ResourceNotFoundException
During handling of the above exception, another exception occurred:
lmb = <botocore.client.Lambda object at 0xffffb29930b0>
def test_rotator_lambda_allows_secretsmanager_invoke(lmb):
try:
policy_doc = json.loads(
lmb.get_policy(FunctionName=ROTATOR_FUNCTION)["Policy"]
)
except ClientError as e:
> pytest.fail(
f"Lambda {ROTATOR_FUNCTION} has no resource-based policy , "
f"secretsmanager.amazonaws.com cannot invoke it: {e}"
)
E Failed: Lambda secrets-rotator has no resource-based policy , secretsmanager.amazonaws.com cannot invoke it: An error occurred (ResourceNotFoundException) when calling the GetPolicy operation: The resource you requested does not exist.
/tests/test_state.py:335: Failed
_______________ test_parent_role_grants_sync2_managed_rule_perms _______________
iam = <botocore.client.IAM object at 0xffffb24cda90>
def test_parent_role_grants_sync2_managed_rule_perms(iam):
"""The .sync:2 integration registers an EventBridge-managed callback
rule. Parent role needs events:PutRule/PutTargets/DescribeRule plus
full states:* on the child."""
found_events = False
found_states = False
for _pname, st in _inline_statements(iam, PARENT_ROLE):
actions = set(_normalise_list(st.get("Action")))
if _actions_cover(actions, REQUIRED_PARENT_EVENTS_ACTIONS):
found_events = True
if _actions_cover(actions, REQUIRED_PARENT_STATES_ACTIONS):
found_states = True
assert found_events, (
f"{PARENT_ROLE} missing events:PutRule/PutTargets/DescribeRule. "
f".sync:2 uses an EventBridge-managed callback rule; without "
f"these the parent cannot set up the callback."
)
> assert found_states, (
f"{PARENT_ROLE} missing states:StartExecution/DescribeExecution/"
f"StopExecution. .sync:2 requires all three on the child SM."
)
E AssertionError: rotation-orchestrator-role missing states:StartExecution/DescribeExecution/StopExecution. .sync:2 requires all three on the child SM.
E assert False
/tests/test_state.py:377: AssertionError
____________________ test_child_role_can_rotate_and_publish ____________________
iam = <botocore.client.IAM object at 0xffffb24cda90>
def test_child_role_can_rotate_and_publish(iam):
found_rotate = False
found_putevents = False
for _pname, st in _inline_statements(iam, CHILD_ROLE):
actions = set(_normalise_list(st.get("Action")))
if _actions_cover(actions, {"secretsmanager:RotateSecret"}):
found_rotate = True
if _actions_cover(actions, {"events:PutEvents"}):
found_putevents = True
> assert found_rotate, (
f"{CHILD_ROLE} lacks secretsmanager:RotateSecret , the child "
f"state machine cannot trigger rotation."
)
E AssertionError: rotation-child-role lacks secretsmanager:RotateSecret , the child state machine cannot trigger rotation.
E assert False
/tests/test_state.py:392: AssertionError
=============================== warnings summary ===============================
test_state.py: 33 warnings
/root/.cache/uv/archive-v0/YVYuQMk4lFdbtxBOM5w8K/lib/python3.12/site-packages/botocore/auth.py:424: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
datetime_now = datetime.datetime.utcnow()
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
==================================== PASSES ====================================
=========================== short test summary info ============================
PASSED ../tests/test_state.py::test_aws_cli_binary_is_real
PASSED ../tests/test_state.py::test_localstack_reachable
PASSED ../tests/test_state.py::test_secret_exists_and_uses_customer_cmk
PASSED ../tests/test_
… (truncated at 12,000 chars, full verifier log is in the trial artifacts)Reproduce this trial: git checkout 2f94510 && PYTHONPATH=src python3 scripts/build_site.py , then open trial/trial_6968318bdf68402d. Re-running the agent live requires EVAL_PLATFORM_ENABLE_OAUTH_SMOKE=1 and is non-deterministic.
Trial trial_6968318bdf68402d · verifier authoritative; classifier explanatory.