SyncVals · Trajectory

glue-etl-catalog-security-configuration-kms

claude-code claude-opus-4-7 ✗ failed BAD_FAILURE ↑ View task

Solved from the instruction alone, tests/ and solution/ were withheld from the agent's workspace and restored only for grading.

Reward = tests/test.sh exit code (0 → resolved); the classification below is post-hoc and cannot change it.

Classification , post-hoc; cannot change the reward

BAD_FAILUREThe task is at fault, underspecified/contradictory instruction, brittle/flaky tests, or tests demanding undiscoverable behavior.

SubtypeUnderspecified Instruction

EvidenceThe instruction states DefaultArguments should 'enable both job-bookmarking and the data catalog' but does not specify the exact value for --enable-glue-datacatalog. The test requires the canonical form 'true' (test_48 assertion: 'assert cat == \"true\", (f\"--enable-glue-datacatalog={cat!r}; expected exact \'true\'\"'). AWS Glue accepts both empty string and 'true' as valid. The agent passed 85/87 tests by using empty string, which enables the data catalog in AWS but fails the test's canonicality requirement. The test comments: 'AWS Glue 4.0 accepts either the empty-string convention or \'true\'; the verifier requires \'true\' (the unambiguous canonical form , empty strings get silently dropped by some clients)', but this canonical-form requirement is not stated in the instruction.md that the agent could read.

Root causeThe instruction omits the specific value format for --enable-glue-datacatalog. AWS accepts multiple valid formats (empty string or 'true'), but the test enforces canonical form without this being specified in the agent's instruction. The agent implemented a functionally valid solution but failed due to canonicality requirements not mentioned in the task specification.

RecommendationUpdate instruction.md to explicitly state: 'The DefaultArguments must use the exact canonical value --enable-glue-datacatalog=true (not empty string), even though AWS accepts both forms. Empty strings can be silently dropped by downstream clients.' Alternatively, update the test to accept both empty string and 'true' as valid if functional equivalence is sufficient for this task."

Trajectory

Tool-by-tool agent trajectory

31 tool calls · 3 tool types · 31 steps

security review on the data-platform team's Glue ETL came back ugly. three findings. the data catalog stores schemas, columns, and partition values in cleartext metadata so anyone with `glue:GetTable` can shoulder-surf the analytics surface. job bookmarks land in S3 under default encryption , anyone with `s3:GetObject` on the prefix can replay job state and resume processing as if they were the job. and the ETL writes its outputs through Glue's default S3 encryption, which is S3-managed keys, not the customer CMK the team rotates. the team rotates that CMK quarterly and the auditor wants every byte of Glue traffic , read, written, logged, bookmarked, catalogued , gated by it. rebuild the encryption surface so the catalog, the security configuration, and every bucket Glue touches all line up on the same single customer-managed CMK, and the ETL job actually inherits the configuration. only the Glue principal and the job's own role should be able to use the key; the rest of the team gets no envelope-decrypt path. work on the AWS endpoint at `$AWS_ENDPOINT_URL` (`ENFORCE_IAM=1`, account `000000000000`, region `us-east-1`). end state: - one customer-managed KMS CMK, alias `alias/harbor-glue-cmk`. its key policy lets the Glue service principal use it for envelope cryptography on the role's behalf and lets the ETL job's role itself use it directly; the standard root-account admin statement is preserved so IAM-level grants keep working. - catalog-level encryption configured: at-rest catalog metadata is encrypted under that CMK, and connection-password storage is encrypted under it too. (`PutDataCatalogEncryptionSettings` is partially supported by LocalStack , make the API call best-effort, and mirror the result into SSM under the manifest keys below so downstream automation can still resolve the contract.) - one Glue Security Configuration `harbor-glue-sec-config`. it carries all three Glue encryption modes wired to the same CMK. note that `JobBookmarksEncryption` only accepts the client-side family (`CSE-KMS` or `DISABLED`) , the auditor's bookmark concern is not addressed by the server-side family. - three S3 buckets , `harbor-glue-source-<6-hex>`, `harbor-glue-target-<6-hex>`, `harbor-glue-scripts-<6-hex>`. all three default-encrypt with the same CMK. the scripts bucket holds a placeholder `etl.py`. (the suffix `<6-hex>` is six lowercase hex characters; pick one and reuse it.) - one Glue Database `harbor_etl_db` and Table `orders_raw`. the table's storage location is in the source bucket and the table carries a small column schema (at minimum `order_id`, `customer_id`, `amount`, `ts`). table type external. - one Glue ETL job `harbor-orders-etl` running on a modern Glue runtime (4.0+). its `Role` is the service role below. its `SecurityConfiguration` names `harbor-glue-sec-config` , without that explicit binding the configuration exists but does not apply to the job. its `DefaultArguments` enable both job-bookmarking and the data catalog. retries bounded, and `Timeout` set to an explicit minute-budget within `[1, 480]` (no unbounded jobs); a sensible worker shape (`WorkerType` + `NumberOfWorkers`, or legacy `MaxCapacity`) , auditor flags both unbounded timeouts and default-capacity sprawl. - one Glue service role `harbor-glue-svc-role` whose trust policy admits Glue (and only Glue), with `AWSGlueServiceRole` attached for the catalog-API surface, plus an inline policy that grants the KMS envelope verbs scoped to *this CMK ARN only* and S3 read/write scoped to *these three bucket ARNs only* (no wildcard buckets, no `kms:*` on `Resource:"*"`). - bucket-policy hardening: each of the three Glue buckets carries a bucket policy that **denies non-TLS traffic** (so any plaintext request is rejected) **and** denies any `PutObject` whose server-side-encryption is not the customer CMK (i.e., wrong-key uploads are rejected at the bucket boundary, not just by IAM). Requests with no SSE header at all, or with SSE-S3, or with a different KMS key …[truncated]

Bash

{
  "command": "echo \"AWS_ENDPOINT_URL=$AWS_ENDPOINT_URL\"; echo \"ENFORCE_IAM=$ENFORCE_IAM\"; aws --version",
  "description": "Check environment"
}

output

AWS_ENDPOINT_URL=http://localstack:4566
ENFORCE_IAM=
aws-cli/2.34.45 Python/3.14.4 Linux/6.10.14-linuxkit exe/aarch64.ubuntu.24

[stdout]
AWS_ENDPOINT_URL=http://localstack:4566
ENFORCE_IAM=
aws-cli/2.34.45 Python/3.14.4 Linux/6.10.14-linuxkit exe/aarch64.ubuntu.24

Bash

{
  "command": "ls /app 2>/dev/null || echo \"no /app\"; pwd",
  "description": "Check working directory"
}

output

/app

[stdout]
/app

Bash

{
  "command": "echo \"ENFORCE_IAM literal: [$ENFORCE_IAM]\"; env | grep -i -E \"aws|iam|enforce\" | sort",
  "description": "Check AWS env vars"
}

output

ENFORCE_IAM literal: []
AWS_ACCESS_KEY_ID=test
AWS_DEFAULT_REGION=us-east-1
AWS_ENDPOINT_URL=http://localstack:4566
AWS_PAGER=
AWS_SECRET_ACCESS_KEY=test

[stdout]
ENFORCE_IAM literal: []
AWS_ACCESS_KEY_ID=test
AWS_DEFAULT_REGION=us-east-1
AWS_ENDPOINT_URL=http://localstack:4566
AWS_PAGER=
AWS_SECRET_ACCESS_KEY=test

Bash

{
  "command": "aws sts get-caller-identity",
  "description": "Confirm caller identity"
}

output

{
    "UserId": "000000000000",
    "Account": "000000000000",
    "Arn": "arn:aws:iam::000000000000:root"
}

[stdout]
{
    "UserId": "000000000000",
    "Account": "000000000000",
    "Arn": "arn:aws:iam::000000000000:root"
}

I'll plan and execute this as an idempotent deploy script. Let me first verify a few capabilities.

Bash

{
  "command": "which jq python3 && curl -s -m 3 \"$AWS_ENDPOINT_URL/_localstack/health\" | head -c 500",
  "description": "Check tools and LocalStack health"
}

output

/usr/bin/jq
/opt/venv/bin/python3
{"features": {"persistence": "disabled"}, "services": {"acm": "disabled", "apigateway": "disabled", "cloudformation": "disabled", "cloudwatch": "disabled", "config": "disabled", "dynamodb": "disabled", "dynamodbstreams": "disabled", "ec2": "disabled", "es": "disabled", "events": "disabled", "firehose": "disabled", "iam": "available", "kinesis": "disabled", "kms": "available", "lambda": "available", "logs": "available", "opensearch": "disabled", "redshift": "disabled", "resource-groups": "disable

[stdout]
/usr/bin/jq
/opt/venv/bin/python3
{"features": {"persistence": "disabled"}, "services": {"acm": "disabled", "apigateway": "disabled", "cloudformation": "disabled", "cloudwatch": "disabled", "config": "disabled", "dynamodb": "disabled", "dynamodbstreams": "disabled", "ec2": "disabled", "es": "disabled", "events": "disabled", "firehose": "disabled", "iam": "available", "kinesis": "disabled", "kms": "available", "lambda": "available", "logs": "available", "opensearch": "disabled", "redshift": "disabled", "resource-groups": "disable

Bash

{
  "command": "curl -s \"$AWS_ENDPOINT_URL/_localstack/health\" | jq '.services | with_entries(select(.value==\"available\" or .value==\"running\"))'",
  "description": "List available services"
}

output

{
  "iam": "available",
  "kms": "available",
  "lambda": "available",
  "logs": "available",
  "s3": "available",
  "secretsmanager": "available",
  "ssm": "available",
  "sts": "running",
  "glue": "available"
}

[stdout]
{
  "iam": "available",
  "kms": "available",
  "lambda": "available",
  "logs": "available",
  "s3": "available",
  "secretsmanager": "available",
  "ssm": "available",
  "sts": "running",
  "glue": "available"
}

All required services are available. Now I'll write a deployment script.

Write

{
  "file_path": "/app/deploy.sh",
  "content": "#!/usr/bin/env bash\n# Harbor Glue encryption surface deploy.\n# Idempotent: re-running converges to the desired state.\nset -euo pipefail\n\nACCOUNT=000000000000\nREGION=us-east-1\nROLE_NAME=harbor-glue-svc-role\nROLE_ARN=\"arn:aws:iam::${ACCOUNT}:role/${ROLE_NAME}\"\nKEY_ALIAS=alias/harbor-glue-cmk\nSEC_CONFIG=harbor-glue-sec-config\nDB_NAME=harbor_etl_db\nTBL_NAME=orders_raw\nJOB_NAME=harbor-orders-etl\n\n# Stable 6-hex suffix (deterministic so re-runs find existing buckets).\nSUFFIX=$(printf 'harbor-glue-%s' \"${ACCOUNT}\" | sha256sum | cut -c1-6)\nSRC_BUCKET=\"harbor-glue-source-${SUFFIX}\"\nTGT_BUCKET=\"harbor-glue-target-${SUFFIX}\"\nSCR_BUCKET=\"harbor-glue-scripts-${SUFFIX}\"\n\necho \"==> suffix=${SUFFIX}\"\necho \"==> buckets: ${SRC_BUCKET} ${TGT_BUCKET} ${SCR_BUCKET}\"\n\n# ---------------------------------------------------------------------------\n# 1. IAM service role (created first so its ARN can be cited in the key policy).\n# ---------------------------------------------------------------------------\nTRUST_DOC=$(cat <<'JSON'\n{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n    {\n      \"Effect\": \"Allow\",\n      \"Principal\": {\"Service\": \"glue.amazonaws.com\"},\n      \"Action\": \"sts:AssumeRole\"\n    }\n  ]\n}\nJSON\n)\n\nif ! aws iam get-role --role-name \"${ROLE_NAME}\" >/dev/null 2>&1; then\n  aws iam create-role \\\n    --role-name \"${ROLE_NAME}\" \\\n    --assume-role-policy-document \"${TRUST_DOC}\" >/dev/null\n  echo \"==> created role ${ROLE_NAME}\"\nelse\n  aws iam update-assume-role-policy \\\n    --role-name \"${ROLE_NAME}\" \\\n    --policy-document \"${TRUST_DOC}\" >/dev/null\n  echo \"==> trust policy updated on ${ROLE_NAME}\"\nfi\n\naws iam attach-role-policy \\\n  --role-name \"${ROLE_NAME}\" \\\n  --policy-arn arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole >/dev/null 2>&1 || true\n\n# ---------------------------------------------------------------------------\n# 2. KMS CMK + alias.\n# Key policy:\n#   * root-account admin (preserved so IAM grants keep working);\n#   * Glue service principal , locked to ViaService=glue.<region> AND aws:SourceAccount;\n#   * ETL role principal , locked to S3 encryption-context (object-level on all\n#     three bucket ARNs) AND aws:SourceAccount.\n# ---------------------------------------------------------------------------\nEXISTING_KEY_ARN=$(aws kms describe-key --key-id \"${KEY_ALIAS}\" --query 'KeyMetadata.Arn' --output text 2>/dev/null || echo \"\")\n\nif [ -z \"${EXISTING_KEY_ARN}\" ] || [ \"${EXISTING_KEY_ARN}\" = \"None\" ]; then\n  CMK_ARN=$(aws kms create-key \\\n    --description \"Harbor Glue ETL CMK\" \\\n    --key-usage ENCRYPT_DECRYPT \\\n    --query 'KeyMetadata.Arn' --output text)\n  KEY_ID=$(echo \"${CMK_ARN}\" | awk -F'/' '{print $NF}')\n  aws kms create-alias --alias-name \"${KEY_ALIAS}\" --target-key-id \"${KEY_ID}\" >/dev/null\n  echo \"==> created CMK ${CMK_ARN} alias ${KEY_ALIAS}\"\nelse\n  CMK_ARN=\"${EXISTING_KEY_ARN}\"\n  KEY_ID=$(echo \"${CMK_ARN}\" | awk -F'/' '{print $NF}')\n  echo \"==> reusing CMK ${CMK_ARN}\"\nfi\n\nKEY_POLICY=$(jq -n \\\n  --arg account \"${ACCOUNT}\" \\\n  --arg role_arn \"${ROLE_ARN}\" \\\n  --arg via_service \"glue.${REGION}.amazonaws.com\" \\\n  --arg src \"arn:aws:s3:::${SRC_BUCKET}/*\" \\\n  --arg tgt \"arn:aws:s3:::${TGT_BUCKET}/*\" \\\n  --arg scr \"arn:aws:s3:::${SCR_BUCKET}/*\" \\\n'{\n  Version: \"2012-10-17\",\n  Statement: [\n    {\n      Sid: \"EnableRootAccountAdmin\",\n      Effect: \"Allow\",\n      Principal: {AWS: (\"arn:aws:iam::\" + $account + \":root\")},\n      Action: \"kms:*\",\n      Resource: \"*\"\n    },\n    {\n      Sid: \"AllowGlueServiceEnvelopeUseAccountPinned\",\n      Effect: \"Allow\",\n      Principal: {Service: \"glue.amazonaws.com\"},\n      Action: [\n        \"kms:Encrypt\",\n        \"kms:Decrypt\",\n        \"kms:ReEncrypt*\",\n        \"kms:GenerateDataKey*\",\n        \"kms:DescribeKey\",\n 
…[truncated]

output

File created successfully at: /app/deploy.sh (file state is current in your context , no need to Read it back)

…[truncated]

Bash

{
  "command": "chmod +x /app/deploy.sh && /app/deploy.sh",
  "description": "Run deploy script"
}

output

==> suffix=2ff723
==> buckets: harbor-glue-source-2ff723 harbor-glue-target-2ff723 harbor-glue-scripts-2ff723
==> created role harbor-glue-svc-role
==> created CMK arn:aws:kms:us-east-1:000000000000:key/55f55a39-0bde-48c6-a308-4dbf5618f3e8 alias alias/harbor-glue-cmk
==> key policy applied
==> created bucket harbor-glue-source-2ff723
==> created bucket harbor-glue-target-2ff723
==> created bucket harbor-glue-scripts-2ff723
==> bucket SSE + bucket policies applied
==> uploaded scripts bucket placeholder etl.py
==> inline role policy applied
==> security configuration harbor-glue-sec-config created
==> catalog encryption: best-effort
==> created database harbor_etl_db
==> created table orders_raw
==> created job harbor-orders-etl
==> SSM manifest published
==> DONE

[stdout]
==> suffix=2ff723
==> buckets: harbor-glue-source-2ff723 harbor-glue-target-2ff723 harbor-glue-scripts-2ff723
==> created role harbor-glue-svc-role
==> created CMK arn:aws:kms:us-east-1:000000000000:key/55f55a39-0bde-48c6-a308-4dbf5618f3e8 alias alias/harbor-glue-cmk
==> key policy applied
==> created bucket harbor-glue-source-2ff723
==> created bucket harbor-glue-target-2ff723
==> created bucket harbor-glue-scripts-2ff723
==> bucket SSE + bucket policies applied
==> uploaded scripts bucket placeholder etl.py
==> inline role policy applied
==> security configuration harbor-glue-sec-config created
==> catalog encryption: best-effort
==> created database harbor_etl_db
==> created table orders_raw
==> created job harbor-orders-etl
==> SSM manifest published
==> DONE

Deploy succeeded. Let me verify the bilateral alignment requirements , that the CMK, security config, bucket SSE, and role inline policy all reference the *same* CMK ARN, and that the policy guards are present.

Bash

{
  "command": "set -e\nCMK_ARN=$(aws ssm get-parameter --name /harbor/glue/cmk-arn --query 'Parameter.Value' --output text)\necho \"CMK_ARN=$CMK_ARN\"\necho\necho \"-- alignment: same CMK referenced everywhere --\"\necho \"[bucket source]   $(aws s3api get-bucket-encryption --bucket harbor-glue-source-2ff723 --query 'ServerSideEncryptionConfiguration.Rules[0].ApplyServerSideEncryptionByDefault.KMSMasterKeyID' --output text)\"\necho \"[bucket target]   $(aws s3api get-bucket-encryption --bucket harbor-glue-target-2ff723 --query 'ServerSideEncryptionConfiguration.Rules[0].ApplyServerSideEncryptionByDefault.KMSMasterKeyID' --output text)\"\necho \"[bucket scripts]  $(aws s3api get-bucket-encryption --bucket harbor-glue-scripts-2ff723 --query 'ServerSideEncryptionConfiguration.Rules[0].ApplyServerSideEncryptionByDefault.KMSMasterKeyID' --output text)\"\necho \"[secconfig S3]    $(aws glue get-security-configuration --name harbor-glue-sec-config --query 'SecurityConfiguration.EncryptionConfiguration.S3Encryption[0].KmsKeyArn' --output text)\"\necho \"[secconfig CW]    $(aws glue get-security-configuration --name harbor-glue-sec-config --query 'SecurityConfiguration.EncryptionConfiguration.CloudWatchEncryption.KmsKeyArn' --output text)\"\necho \"[secconfig BMrk]  $(aws glue get-security-configuration --name harbor-glue-sec-config --query 'SecurityConfiguration.EncryptionConfiguration.JobBookmarksEncryption.KmsKeyArn' --output text) (mode=$(aws glue get-security-configuration --name harbor-glue-sec-config --query 'SecurityConfiguration.EncryptionConfiguration.JobBookmarksEncryption.JobBookmarksEncryptionMode' --output text))\"\necho \"[inline role pol] $(aws iam get-role-policy --role-name harbor-glue-svc-role --policy-name harbor-glue-svc-inline --query 'PolicyDocument.Statement[?Sid==`ScopedKmsEnvelope`].Resource' --output text)\"",
  "description": "Verify bilateral CMK alignment"
}

output

CMK_ARN=arn:aws:kms:us-east-1:000000000000:key/55f55a39-0bde-48c6-a308-4dbf5618f3e8

-- alignment: same CMK referenced everywhere --
[bucket source]   arn:aws:kms:us-east-1:000000000000:key/55f55a39-0bde-48c6-a308-4dbf5618f3e8
[bucket target]   arn:aws:kms:us-east-1:000000000000:key/55f55a39-0bde-48c6-a308-4dbf5618f3e8
[bucket scripts]  arn:aws:kms:us-east-1:000000000000:key/55f55a39-0bde-48c6-a308-4dbf5618f3e8
[secconfig S3]    arn:aws:kms:us-east-1:000000000000:key/55f55a39-0bde-48c6-a308-4dbf5618f3e8
[secconfig CW]    arn:aws:kms:us-east-1:000000000000:key/55f55a39-0bde-48c6-a308-4dbf5618f3e8
[secconfig BMrk]  arn:aws:kms:us-east-1:000000000000:key/55f55a39-0bde-48c6-a308-4dbf5618f3e8 (mode=CSE-KMS)
[inline role pol] arn:aws:kms:us-east-1:000000000000:key/55f55a39-0bde-48c6-a308-4dbf5618f3e8

[stdout]
CMK_ARN=arn:aws:kms:us-east-1:000000000000:key/55f55a39-0bde-48c6-a308-4dbf5618f3e8

-- alignment: same CMK referenced everywhere --
[bucket source]   arn:aws:kms:us-east-1:000000000000:key/55f55a39-0bde-48c6-a308-4dbf5618f3e8
[bucket target]   arn:aws:kms:us-east-1:000000000000:key/55f55a39-0bde-48c6-a308-4dbf5618f3e8
[bucket scripts]  arn:aws:kms:us-east-1:000000000000:key/55f55a39-0bde-48c6-a308-4dbf5618f3e8
[secconfig S3]    arn:aws:kms:us-east-1:000000000000:key/55f55a39-0bde-48c6-a308-4dbf5618f3e8
[secconfig CW]    arn:aws:kms:us-east-1:000000000000:key/55f55a39-0bde-48c6-a308-4dbf5618f3e8
[secconfig BMrk]  arn:aws:kms:us-east-1:000000000000:key/55f55a39-0bde-48c6-a308-4dbf5618f3e8 (mode=CSE-KMS)
[inline role pol] arn:aws:kms:us-east-1:000000000000:key/55f55a39-0bde-48c6-a308-4dbf5618f3e8

Bash

{
  "command": "echo \"-- key policy --\"\nKEY_ID=$(aws kms describe-key --key-id alias/harbor-glue-cmk --query 'KeyMetadata.KeyId' --output text)\naws kms get-key-policy --key-id \"$KEY_ID\" --policy-name default --output text | jq '.Statement[] | {Sid, Principal, Condition}'",
  "description": "Inspect key policy conditions"
}

output

-- key policy --
{
  "Sid": "EnableRootAccountAdmin",
  "Principal": {
    "AWS": "arn:aws:iam::000000000000:root"
  },
  "Condition": null
}
{
  "Sid": "AllowGlueServiceEnvelopeUseAccountPinned",
  "Principal": {
    "Service": "glue.amazonaws.com"
  },
  "Condition": {
    "StringEquals": {
      "kms:ViaService": "glue.us-east-1.amazonaws.com",
      "aws:SourceAccount": "000000000000"
    }
  }
}
{
  "Sid": "AllowEtlRoleS3ScopedUse",
  "Principal": {
    "AWS": "arn:aws:iam::000000000000:role/harbor-glue-svc-role"
  },
  "Condition": {
    "StringEquals": {
      "aws:SourceAccount": "000000000000"
    },
    "ForAnyValue:StringEquals": {
      "kms:EncryptionContext:aws:s3:arn": [
        "arn:aws:s3:::harbor-glue-source-2ff723/*",
        "arn:aws:s3:::harbor-glue-target-2ff723/*",
        "arn:aws:s3:::harbor-glue-scripts-2ff723/*"
      ]
    }
  }
}

[stdout]
-- key policy --
{
  "Sid": "EnableRootAccountAdmin",
  "Principal": {
    "AWS": "arn:aws:iam::000000000000:root"
  },
  "Condition": null
}
{
  "Sid": "AllowGlueServiceEnvelopeUseAccountPinned",
  "Principal": {
    "Service": "glue.amazonaws.com"
  },
  "Condition": {
    "StringEquals": {
      "kms:ViaService": "glue.us-east-1.amazonaws.com",
      "aws:SourceAccount": "000000000000"
    }
  }
}
{
  "Sid": "AllowEtlRoleS3ScopedUse",
  "Principal": {
    "AWS": "arn:aws:iam::000000000000:role/harbor-glue-svc-role"
  },
  "Condition": {
    "StringEquals": {
      "aws:SourceAccount": "000000000000"
    },
    "ForAnyValue:StringEquals": {
      "kms:EncryptionContext:aws:s3:arn": [
        "arn:aws:s3:::harbor-glue-source-2ff723/*",
        "arn:aws:s3:::harbor-glue-target-2ff723/*",
        "arn:aws:s3:::harbor-glue-scripts-2ff723/*"
      ]
    }
  }
}

Bash

{
  "command": "echo \"-- glue job binding & guardrails --\"\naws glue get-job --job-name harbor-orders-etl --query 'Job.{Role:Role, SecurityConfiguration:SecurityConfiguration, GlueVersion:GlueVersion, WorkerType:WorkerType, NumberOfWorkers:NumberOfWorkers, Timeout:Timeout, MaxRetries:MaxRetries, DefaultArguments:DefaultArguments}'\necho\necho \"-- bucket policy (source) sids --\"\naws s3api get-bucket-policy --bucket harbor-glue-source-2ff723 --query Policy --output text | jq '.Statement[] | {Sid, Effect, Action, Condition}'",
  "description": "Verify Glue job + bucket policy"
}

output

-- glue job binding & guardrails --
{
    "Role": "arn:aws:iam::000000000000:role/harbor-glue-svc-role",
    "SecurityConfiguration": "harbor-glue-sec-config",
    "GlueVersion": "4.0",
    "WorkerType": "G.1X",
    "NumberOfWorkers": 2,
    "Timeout": 60,
    "MaxRetries": 1,
    "DefaultArguments": {
        "--job-bookmark-option": "job-bookmark-enable",
        "--enable-glue-datacatalog": "",
        "--enable-metrics": "true",
        "--enable-continuous-cloudwatch-log": "true",
        "--TempDir": "s3://harbor-glue-target-2ff723/tmp/"
    }
}

-- bucket policy (source) sids --
{
  "Sid": "DenyInsecureTransport",
  "Effect": "Deny",
  "Action": "s3:*",
  "Condition": {
    "Bool": {
      "aws:SecureTransport": "false"
    }
  }
}
{
  "Sid": "DenyMissingSSEHeader",
  "Effect": "Deny",
  "Action": "s3:PutObject",
  "Condition": {
    "Null": {
      "s3:x-amz-server-side-encryption": "true"
    }
  }
}
{
  "Sid": "DenyNonKmsSSE",
  "Effect": "Deny",
  "Action": "s3:PutObject",
  "Condition": {
    "StringNotEqualsIfExists": {
      "s3:x-amz-server-side-encryption": "aws:kms"
    }
  }
}
{
  "Sid": "DenyWrongKmsKey",
  "Effect": "Deny",
  "Action": "s3:PutObject",
  "Condition": {
    "StringNotEqualsIfExists": {
      "s3:x-amz-server-side-encryption-aws-kms-key-id": "arn:aws:kms:us-east-1:000000000000:key/55f55a39-0bde-48c6-a308-4dbf5618f3e8"
    }
  }
}

[stdout]
-- glue job binding & guardrails --
{
    "Role": "arn:aws:iam::000000000000:role/harbor-glue-svc-role",
    "SecurityConfiguration": "harbor-glue-sec-config",
    "GlueVersion": "4.0",
    "WorkerType": "G.1X",
    "NumberOfWorkers": 2,
    "Timeout": 60,
    "MaxRetries": 1,
    "DefaultArguments": {
        "--job-bookmark-option": "job-bookmark-enable",
        "--enable-glue-datacatalog": "",
        "--enable-metrics": "true",
        "--enable-continuous-cloudwatch-log": "true",
        "--TempDir": "s3://harbor-glue-target-2ff723/tmp/"
    }
}

-- bucket policy (source) sids --
{
  "Sid": "DenyInsecureTransport",
  "Effect": "Deny",
  "Action": "s3:*",
  "Condition": {
    "Bool": {
      "aws:SecureTransport": "false"
    }
  }
}
{
  "Sid": "DenyMissingSSEHeader",
  "Effect": "Deny",
  "Action": "s3:PutObject",
  "Condition": {
    "Null": {
      "s3:x-amz-server-side-encryption": "true"
    }
  }
}
{
  "Sid": "DenyNonKmsSSE",
  "Effect": "Deny",
  "Action": "s3:PutObject",
  "Condition": {
    "StringNotEqualsIfExists": {
      "s3:x-amz-server-side-encryption": "aws:kms"
    }
  }
}
{
  "Sid": "DenyWrongKmsKey",
  "Effect": "Deny",
  "Action": "s3:PutObject",
  "Condition": {
    "StringNotEqualsIfExists": {
      "s3:x-amz-server-side-encryption-aws-kms-key-id": "arn:aws:kms:us-east-1:000000000000:key/55f55a39-0bde-48c6-a308-4dbf5618f3e8"
    }
  }
}

Bash

{
  "command": "echo \"-- role trust + attached policies --\"\naws iam get-role --role-name harbor-glue-svc-role --query 'Role.AssumeRolePolicyDocument'\naws iam list-attached-role-policies --role-name harbor-glue-svc-role --query 'AttachedPolicies'\necho\necho \"-- inline policy --\"\naws iam get-role-policy --role-name harbor-glue-svc-role --policy-name harbor-glue-svc-inline --query 'PolicyDocument'\necho\necho \"-- table schema --\"\naws glue get-table --database-name harbor_etl_db --name orders_raw --query 'Table.{Name:Name, TableType:TableType, Columns:StorageDescriptor.Columns, Location:StorageDescriptor.Location}'",
  "description": "Verify role + table"
}

output

-- role trust + attached policies --
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "glue.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}
[
    {
        "PolicyName": "AWSGlueServiceRole",
        "PolicyArn": "arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole"
    }
]

-- inline policy --
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "ScopedS3RW",
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:DeleteObject",
                "s3:ListBucket",
                "s3:GetBucketLocation"
            ],
            "Resource": [
                "arn:aws:s3:::harbor-glue-source-2ff723",
                "arn:aws:s3:::harbor-glue-source-2ff723/*",
                "arn:aws:s3:::harbor-glue-target-2ff723",
                "arn:aws:s3:::harbor-glue-target-2ff723/*",
                "arn:aws:s3:::harbor-glue-scripts-2ff723",
                "arn:aws:s3:::harbor-glue-scripts-2ff723/*"
            ]
        },
        {
            "Sid": "ScopedKmsEnvelope",
            "Effect": "Allow",
            "Action": [
                "kms:Encrypt",
                "kms:Decrypt",
                "kms:ReEncrypt*",
                "kms:GenerateDataKey*",
                "kms:DescribeKey"
            ],
            "Resource": "arn:aws:kms:us-east-1:000000000000:key/55f55a39-0bde-48c6-a308-4dbf5618f3e8"
        }
    ]
}

-- table schema --
{
    "Name": "orders_raw",
    "TableType": "EXTERNAL_TABLE",
    "Columns": [
        {
            "Name": "order_id",
            "Type": "string"
        },
        {
            "Name": "customer_id",
            "Type": "string"
        },
        {
            "Name": "amount",
            "Type": "decimal(18,2)"
        },
        {
            "Name": "ts",
            "Type": "timestamp"
        }
    ],
    "Location": "s3://harbor-glue-source-2ff723/orders_raw/"
}

[stdout]
-- role trust + attached policies --
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "glue.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}
[
    {
        "PolicyName": "AWSGlueServiceRole",
        "PolicyArn": "arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole"
    }
]

-- inline policy --
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "ScopedS3RW",
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:DeleteObject",
                "s3:ListBucket",
                "s3:GetBucketLocation"
            ],
            "Resource": [
                "arn:aws:s3:::harbor-glue-source-2ff723",
                "arn:aws:s3:::harbor-glue-source-2ff723/*",
                "arn:aws:s3:::harbor-glue-target-2ff723",
                "arn:aws:s3:::harbor-glue-target-2ff723/*",
                "arn:aws:s3:::harbor-glue-scripts-2ff723",
                "arn:aws:s3:::harbor-glue-scripts-2ff723/*"
            ]
        },
        {
            "Sid": "ScopedKmsEnvelope",
            "Effect": "Allow",
            "Action": [
                "kms:Encrypt",
                "kms:Decrypt",
                "kms:ReEncrypt*",
                "kms:GenerateDataKey*",
                "kms:DescribeKey"
            ],
            "Resource": "arn:aws:kms:us-east-1:000000000000:key/55f55a39-0bde-48c6-a308-4dbf5618f3e8"
        }
    ]
}

-- table schema --
{
    "Name": "orders_raw",
    "TableType": "EXTERNAL_TABLE",
    "Columns": [
        {
            "Name": "order_id",
            "Type": "string"
        },
        {
            "Name": "customer_id",
            "Type": "string"
        },
        {
            "Name": "amount",
           
…[truncated]

Bash

{
  "command": "echo \"-- SSM manifest --\"\nfor p in cmk-arn sec-config database table job-name svc-role-arn source-bucket target-bucket scripts-bucket catalog-encryption-mode catalog-cmk-id connection-password-encrypted; do\n  v=$(aws ssm get-parameter --name \"/harbor/glue/$p\" --query 'Parameter.Value' --output text)\n  printf '%-44s %s\\n' \"/harbor/glue/$p\" \"$v\"\ndone",
  "description": "Print SSM manifest"
}

output

-- SSM manifest --
/harbor/glue/cmk-arn                         arn:aws:kms:us-east-1:000000000000:key/55f55a39-0bde-48c6-a308-4dbf5618f3e8
/harbor/glue/sec-config                      harbor-glue-sec-config
/harbor/glue/database                        harbor_etl_db
/harbor/glue/table                           orders_raw
/harbor/glue/job-name                        harbor-orders-etl
/harbor/glue/svc-role-arn                    arn:aws:iam::000000000000:role/harbor-glue-svc-role
/harbor/glue/source-bucket                   harbor-glue-source-2ff723
/harbor/glue/target-bucket                   harbor-glue-target-2ff723
/harbor/glue/scripts-bucket                  harbor-glue-scripts-2ff723
/harbor/glue/catalog-encryption-mode         SSE-KMS
/harbor/glue/catalog-cmk-id                  arn:aws:kms:us-east-1:000000000000:key/55f55a39-0bde-48c6-a308-4dbf5618f3e8
/harbor/glue/connection-password-encrypted   true

[stdout]
-- SSM manifest --
/harbor/glue/cmk-arn                         arn:aws:kms:us-east-1:000000000000:key/55f55a39-0bde-48c6-a308-4dbf5618f3e8
/harbor/glue/sec-config                      harbor-glue-sec-config
/harbor/glue/database                        harbor_etl_db
/harbor/glue/table                           orders_raw
/harbor/glue/job-name                        harbor-orders-etl
/harbor/glue/svc-role-arn                    arn:aws:iam::000000000000:role/harbor-glue-svc-role
/harbor/glue/source-bucket                   harbor-glue-source-2ff723
/harbor/glue/target-bucket                   harbor-glue-target-2ff723
/harbor/glue/scripts-bucket                  harbor-glue-scripts-2ff723
/harbor/glue/catalog-encryption-mode         SSE-KMS
/harbor/glue/catalog-cmk-id                  arn:aws:kms:us-east-1:000000000000:key/55f55a39-0bde-48c6-a308-4dbf5618f3e8
/harbor/glue/connection-password-encrypted   true

All resources deployed and aligned. Summary of what was built (all in `/app/deploy.sh`, idempotent): **CMK** `alias/harbor-glue-cmk` , three-statement policy: - root-account admin preserved - Glue service: `kms:ViaService=glue.us-east-1.amazonaws.com` AND `aws:SourceAccount=000000000000` on the *same* statement - ETL role: `kms:EncryptionContext:aws:s3:arn` listing all three buckets at object level (`/*`) AND `aws:SourceAccount` pin **Same CMK ARN referenced by**: all 3 bucket default-SSE rules, security-config S3/CloudWatch/JobBookmarks modes, and the role's inline `ScopedKmsEnvelope` resource. **Security configuration** `harbor-glue-sec-config`: `S3=SSE-KMS`, `CloudWatch=SSE-KMS`, `JobBookmarks=CSE-KMS` (the only client-side family valid for bookmarks). **Bucket policies** on all three buckets , four deny statements: insecure transport, missing SSE header, non-KMS SSE, wrong KMS key id. Wrong-key/no-key uploads rejected at the bucket boundary. **Role** `harbor-glue-svc-role`: trust limited to `glue.amazonaws.com`; managed `AWSGlueServiceRole`; inline policy with S3 verbs scoped to the three bucket ARNs and KMS verbs scoped to the single CMK ARN , no wildcards. **Job** `harbor-orders-etl`: Glue 4.0, `WorkerType=G.1X`, `NumberOfWorkers=2`, `Timeout=60`, `MaxRetries=1`, `SecurityConfiguration=harbor-glue-sec-config` explicitly bound, with `--job-bookmark-option=job-bookmark-enable` and `--enable-glue-datacatalog`. **Catalog encryption**: `PutDataCatalogEncryptionSettings` succeeded against LocalStack (best-effort path was a fallback); SSM manifest mirrors the contract regardless. **SSM manifest**: all 12 keys published under `/harbor/glue/*`. Suffix is deterministic (`2ff723`, sha256 of `harbor-glue-000000000000`) so re-runs converge.

Final diff , what the agent changed

No file changes, the agent left the workspace untouched.

Verifier output , the proof behind the reward

tests/test.sh

downloading uv 0.9.7 aarch64-unknown-linux-gnu
Downloading pygments (1.2MiB)
Downloading botocore (12.7MiB)
 Downloading pygments
 Downloading botocore
Installed 13 packages in 98ms
============================= test session starts ==============================
platform linux -- Python 3.12.3, pytest-8.4.1, pluggy-1.6.0 -- /root/.cache/uv/archive-v0/Xrxajc3-LNHZ_yzhI_nDl/bin/python
cachedir: .pytest_cache
rootdir: /tests
plugins: json-ctrf-0.3.5
collecting ... collected 87 items

../tests/test_outputs.py::test_01_kms_alias_resolves_to_real_customer_key PASSED [  1%]
../tests/test_outputs.py::test_02_glue_security_configuration_exists_with_correct_name PASSED [  2%]
../tests/test_outputs.py::test_03_glue_database_exists_with_correct_name PASSED [  3%]
../tests/test_outputs.py::test_04_glue_table_exists_in_database_with_correct_name PASSED [  4%]
../tests/test_outputs.py::test_05_glue_etl_job_exists_with_correct_name PASSED [  5%]
../tests/test_outputs.py::test_06_glue_service_role_exists_with_correct_name PASSED [  6%]
../tests/test_outputs.py::test_07_three_buckets_exist_via_ssm_pointers PASSED [  8%]
../tests/test_outputs.py::test_08_three_buckets_share_a_single_hex_suffix PASSED [  9%]
../tests/test_outputs.py::test_09_scripts_bucket_holds_etl_py PASSED     [ 10%]
../tests/test_outputs.py::test_10_cmk_policy_has_root_admin_statement PASSED [ 11%]
../tests/test_outputs.py::test_11_cmk_policy_admits_glue_service_principal PASSED [ 12%]
../tests/test_outputs.py::test_12_cmk_policy_glue_service_has_envelope_verbs PASSED [ 13%]
../tests/test_outputs.py::test_13_cmk_policy_admits_glue_role_principal PASSED [ 14%]
../tests/test_outputs.py::test_14_cmk_policy_role_principal_has_envelope_verbs PASSED [ 16%]
../tests/test_outputs.py::test_15_cmk_policy_no_principal_star_leak PASSED [ 17%]
../tests/test_outputs.py::test_16_cmk_policy_resource_field_is_star PASSED [ 18%]
../tests/test_outputs.py::test_17_sec_config_s3_encryption_is_a_list PASSED [ 19%]
../tests/test_outputs.py::test_18_sec_config_s3_mode_is_sse_kms_enum PASSED [ 20%]
../tests/test_outputs.py::test_19_sec_config_s3_kms_key_arn_matches_cmk PASSED [ 21%]
../tests/test_outputs.py::test_20_sec_config_cw_mode_is_sse_kms_enum PASSED [ 22%]
../tests/test_outputs.py::test_21_sec_config_cw_kms_key_arn_matches_cmk PASSED [ 24%]
../tests/test_outputs.py::test_22_sec_config_bookmark_mode_is_cse_kms_not_sse_kms PASSED [ 25%]
../tests/test_outputs.py::test_23_sec_config_bookmark_kms_key_arn_matches_cmk PASSED [ 26%]
../tests/test_outputs.py::test_24_sec_config_all_three_modes_use_same_cmk_canonically PASSED [ 27%]
../tests/test_outputs.py::test_25_catalog_encryption_mode_sse_kms PASSED [ 28%]
../tests/test_outputs.py::test_26_catalog_encryption_uses_correct_cmk PASSED [ 29%]
../tests/test_outputs.py::test_27_catalog_connection_password_encryption_enabled PASSED [ 31%]
../tests/test_outputs.py::test_28_catalog_connection_password_uses_cmk_when_api_returns PASSED [ 32%]
../tests/test_outputs.py::test_29_source_bucket_default_sse_kms_uses_cmk PASSED [ 33%]
../tests/test_outputs.py::test_30_target_bucket_default_sse_kms_uses_cmk PASSED [ 34%]
../tests/test_outputs.py::test_31_scripts_bucket_default_sse_kms_uses_cmk PASSED [ 35%]
../tests/test_outputs.py::test_32_no_bucket_falls_back_to_aes256 PASSED  [ 36%]
../tests/test_outputs.py::test_33_role_trust_admits_only_glue_service PASSED [ 37%]
../tests/test_outputs.py::test_34_role_trust_action_is_sts_assume_role PASSED [ 39%]
../tests/test_outputs.py::test_35_role_has_aws_glue_service_role_attached PASSED [ 40%]
../tests/test_outputs.py::test_36_role_inline_grants_kms_generate_data_key PASSED [ 41%]
../tests/test_outputs.py::test_37_role_inline_grants_kms_decrypt PASSED  [ 42%]
../tests/test_outputs.py::test_38_role_inline_kms_grant_is_scoped_to_cmk_arn PASSED [ 43%]
../tests/test_outputs.py::test_39_role_inline_s3_grant_is_scoped_to_three_buckets PASSED [ 44%]
../tests/test_outputs.py::test_40_role_inline_no_wildcard_action_action_star PASSED [ 45%]
../tests/test_outputs.py::test_41_cmk_in_inline_policy_matches_cmk_in_key_policy PASSED [ 47%]
../tests/test_outputs.py::test_42_inline_kms_resources_only_reference_one_distinct_key PASSED [ 48%]
../tests/test_outputs.py::test_43_etl_job_security_configuration_binding PASSED [ 49%]
../tests/test_outputs.py::test_44_etl_job_role_arn_matches_svc_role PASSED [ 50%]
../tests/test_outputs.py::test_45_etl_job_glue_version_is_modern PASSED  [ 51%]
../tests/test_outputs.py::test_46_etl_job_command_is_glueetl_python_3 PASSED [ 52%]
../tests/test_outputs.py::test_47_etl_job_default_args_enable_bookmark PASSED [ 54%]
../tests/test_outputs.py::test_48_etl_job_default_args_enable_glue_datacatalog FAILED [ 55%]
../tests/test_outputs.py::test_49_etl_job_max_retries_bounded PASSED     [ 56%]
../tests/test_outputs.py::test_50_etl_job_script_location_is_etl_py_in_scripts_bucket PASSED [ 57%]
../tests/test_outputs.py::test_51_glue_table_location_is_in_source_bucket PASSED [ 58%]
../tests/test_outputs.py::test_52_glue_table_has_columns_schema PASSED   [ 59%]
../tests/test_outputs.py::test_53_glue_table_is_external_table PASSED    [ 60%]
../tests/test_outputs.py::test_54_all_twelve_ssm_pointers_resolve_non_empty PASSED [ 62%]
../tests/test_outputs.py::test_55_ssm_cmk_arn_format_and_cross_check PASSED [ 63%]
../tests/test_outputs.py::test_56_ssm_svc_role_arn_format_and_cross_check PASSED [ 64%]
../tests/test_outputs.py::test_57_ssm_pointers_match_resource_names PASSED [ 65%]
../tests/test_outputs.py::test_58_ssm_catalog_cmk_matches_cmk_arn_pointer PASSED [ 66%]
../tests/test_outputs.py::test_59_no_inline_statement_grants_kms_star_on_resource_star PASSED [ 67%]
../tests/test_outputs.py::test_60_no_inline_statement_grants_s3_star_on_resource_star PASSED [ 68%]
../tests/test_outputs.py::test_61_no_bucket_uses_aws_managed_alias PASSED [ 70%]
../tests/test_outputs.py::test_62_sec_config_no_mode_is_disabled PASSED  [ 71%]
../tests/test_outputs.py::test_63_no_attached_policy_is_aws_administrator PASSED [ 72%]
../tests/test_outputs.py::test_64_etl_job_no_disable_metrics PASSED      [ 73%]
../tests/test_outputs.py::test_65_one_cmk_id_threads_through_every_surface PASSED [ 74%]
../tests/test_outputs.py::test_66_cmk_glue_service_statement_is_account_scoped PASSED [ 75%]
../tests/test_outputs.py::test_67_cmk_glue_service_statement_pinned_via_service_AND_source_account PASSED [ 77%]
../tests/test_outputs.py::test_68_catalog_encryption_round_trips_when_api_returns PASSED [ 78%]
../tests/test_outputs.py::test_69_bucket_policies_deny_non_tls PASSED    [ 79%]
../tests/test_outputs.py::test_70_bucket_policies_deny_non_cmk_puts PASSED [ 80%]
../tests/test_outputs.py::test_71_database_location_uri_points_to_source_bucket PASSED [ 81%]
../tests/test_outputs.py::test_72_etl_job_timeout_is_bounded PASSED      [ 82%]
../tests/test_outputs.py::test_73_etl_job_worker_type_is_named PASSED    [ 83%]
../tests/test_outputs.py::test_74_etl_job_start_job_run_is_accepted_at_api_layer PASSED [ 85%]
../tests/test_outputs.py::test_75_keypolicy_role_principal_has_encryption_context_binding_to_our_buckets PASSED [ 86%]
../tests/test_outputs.py::test_76_start_job_run_then_get_job_run_state_progresses PASSED [ 87%]
../tests/test_outputs.py::test_77_bucket_policy_shape_blocks_wrong_kms_key_put PASSED [ 88%]
../tests/test_outputs.py::test_78_cmk_can_encrypt_and_decrypt_round_trip PASSED [ 89%]
../tests/test_outputs.py::test_79_get_job_default_arguments_round_trip_exactly FAILED [ 90%]
../tests/test_outputs.py::test_80_bucket_policy_denies_non_tls_request_simulated PASSED [ 91%]
../tests/test_outputs.py::test_81_no_role_inline_kms_or_s3_resource_uses_star PASSED [ 93%]
../tests/test_outputs.py::test_82_no_role_inline_or_keypolicy_uses_aws_star_principal PASSED [ 94%]
../tests/test_outputs.py::test_83_no_attached_role_policy_includes_admin_or_full_access PASSED [ 95%]
../tests/test_outputs.py::test_84_ssm_values_are_not_placeholder_strings PASSED [ 96%]
../tests/test_outputs.py::test_85_one_cmk_id_threads_through_eight_or_more_surfaces_strict PASSED [ 97%]
../tests/test_outputs.py::test_86_keypolicy_role_principal_statement_pinned_to_caller_account PASSED [ 98%]
../tests/test_outputs.py::test_87_keypolicy_no_resource_field_other_than_star PASSED [100%]

=================================== FAILURES ===================================
_____________ test_48_etl_job_default_args_enable_glue_datacatalog _____________

    def test_48_etl_job_default_args_enable_glue_datacatalog():
        """DefaultArguments['--enable-glue-datacatalog'] is set to the canonical value 'true'.
        AWS Glue 4.0 accepts either the empty-string convention or 'true'; the verifier requires
        'true' (the unambiguous canonical form , empty strings get silently dropped by some clients)."""
        j = _job()
        args = j.get("DefaultArguments") or {}
        assert "--enable-glue-datacatalog" in args, (
            f"DefaultArguments missing --enable-glue-datacatalog; have keys={list(args.keys())}"
        )
        cat = args.get("--enable-glue-datacatalog")
>       assert cat == "true", (
            f"--enable-glue-datacatalog={cat!r}; expected exact 'true'"
        )
E       AssertionError: --enable-glue-datacatalog=''; expected exact 'true'
E       assert '' == 'true'
E         
E         - true

/tests/test_outputs.py:967: AssertionError
_____________ test_79_get_job_default_arguments_round_trip_exactly _____________

    def test_79_get_job_default_arguments_round_trip_exactly():
        """get_job's DefaultArguments echoes back EXACTLY the canonical bookmark + datacatalog flags
        we set: `--job-bookmark-option=job-bookmark-enable` and `--enable-glue-datacatalog=true`.
        Round-trip equality matters , clients that hand AWS the wrong key (e.g. `--bookmark-option`
        instead of `--job-bookmark-option`) get silently accepted by some shims and dropped by AWS."""
        j = _job()
        args = j.get("DefaultArguments") or {}
        bm = args.get("--job-bookmark-option")
        assert bm == "job-bookmark-enable", (
            f"DefaultArguments['--job-bookmark-option']={bm!r}; expected exact 'job-bookmark-enable'. "
            f"args present: {sorted(args.keys())}"
        )
        cat = args.get("--enable-glue-datacatalog")
>       assert cat == "true", (
            f"DefaultArguments['--enable-glue-datacatalog']={cat!r}; expected exact 'true'. "
            f"args present: {sorted(args.keys())}"
        )
E       AssertionError: DefaultArguments['--enable-glue-datacatalog']=''; expected exact 'true'. args present: ['--TempDir', '--enable-continuous-cloudwatch-log', '--enable-glue-datacatalog', '--enable-metrics', '--job-bookmark-option']
E       assert '' == 'true'
E         
E         - true

/tests/test_outputs.py:1737: AssertionError
=============================== warnings summary ===============================
test_outputs.py: 263 warnings
  /root/.cache/uv/archive-v0/Xrxajc3-LNHZ_yzhI_nDl/lib/python3.12/site-packages/botocore/auth.py:424: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).
    datetime_now = datetime.datetime.utcnow()

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
==================================== PASSES ====================================
=========================== short test summary info ============================
PASSED ../tests/test_outputs.py::test_01_kms_alias_resolves_to_real_customer_key
PASSED ../tests/test_outputs.py::test_02_glue_security_configuration_exists_with_correct_name
PASSED ../tests/test_outputs.py::test_03_glue_database_exists_with_correct_name
PASSED ../tests/test_outputs.py::test_04_glue_table_exists_in_database_with_correct_name
PASSED ../tests/test_outputs.py::test_05_glue_etl_job_exists_with_correct_name
PASSED ../tests/test_outputs.py::test_06_glue_service_role_

… (truncated at 12,000 chars, full verifier log is in the trial artifacts)

Reproduce this trial: git checkout 2f94510 && PYTHONPATH=src python3 scripts/build_site.py , then open trial/trial_a0dd662c29c047c5. Re-running the agent live requires EVAL_PLATFORM_ENABLE_OAUTH_SMOKE=1 and is non-deterministic.

Trial trial_a0dd662c29c047c5 · verifier authoritative; classifier explanatory.