fix(s3-deployment): handle properly quoted strings in JSON files

[scorbiere]

The S3 deployment custom resource now properly handles strings containing quotes when used in JSON files, while maintaining the existing behavior for other file types including YAML files.

Issue #22661

Closes #22661 .

Reason for this change

Investigation Findings

1. At the Source.jsonData level, we only have markers, not actual values:

// User code
Source.jsonData('config.json', {
  secret_value: param.stringValue // Token containing "test"with"quotes"
})

// What actually gets written:
{
  "secret_value": "<<marker:0xbaba:0>>" // Marker, not the actual value
}

2. The actual value substitution happens in the S3 deployment Lambda (custom resource), where the markers are replaced with actual values.

BEFORE Fix (Current Broken State)

When the Lambda replaces markers:

// Lambda receives:
{"secret_value":"<<marker:0xbaba:0>>"}
// And the mapping:
markers = {
  "<<marker:0xbaba:0>>": "test"with"quotes"
}

// Simple string replacement results in:
{"secret_value":"test"with"quotes"}  // Invalid JSON

AFTER (With Fix)

The Lambda now detects JSON files and handles them specially:

1. Detects if the file is JSON
2. If JSON:
   • Parses the JSON structure
   • Properly escapes strings during marker replacement
   • Re-serializes to valid JSON
3. If not JSON (including YAML):
   • Uses simple string replacement (which works fine for YAML)

// Lambda receives same input:
{"secret_value":"<<marker:0xbaba:0>>"}
// But now properly escapes quotes in JSON context:
{"secret_value":"test\"with\"quotes"}  // Valid JSON

Key Insights

1. The issue wasn't in the CDK token system but in the marker replacement in the Lambda
2. YAML files work without special handling because YAML is more permissive with quotes
3. The fix:
   • Maintains existing behavior for non-JSON files (YAML, text, etc.)
   • Properly handles JSON string escaping
   • Potential Breaking Change: This fix might cause double-escaping for customers who implemented workarounds (e.g., manually adding escape characters in marker values to handle JSON)

Description of changes

Changes:

• Added JSON detection and special handling in the Lambda custom resource
• Added integration tests for both JSON and YAML files with quoted values
• Added bucket cleanup configuration to the test stack

Describe any new or updated permissions being added

N/A

Description of how you validated changes

Test cases show that:

• JSON files properly escape quotes: {"secret_value": "test\"with\"quotes"}
• YAML files work as-is: secret_value: test"with"quotes

Checklist

• My code adheres to the CONTRIBUTING GUIDE and DESIGN GUIDELINES

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.37%. Comparing base (d12854a) to head (a6658d3).
Report is 7 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #33698      +/-   ##
==========================================
+ Coverage   82.24%   82.37%   +0.13%     
==========================================
  Files         119      120       +1     
  Lines        6875     6933      +58     
  Branches     1161     1169       +8     
==========================================
+ Hits         5654     5711      +57     
- Misses       1118     1119       +1     
  Partials      103      103              

Flag	Coverage Δ
suite.unit	82.37% <ø> (+0.13%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
packages/aws-cdk	∅ <ø> (∅)
packages/aws-cdk-lib/core	82.37% <ø> (+0.13%)	⬆️

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[aws-cdk-automation]

AWS CodeBuild CI Report

• CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
• Commit ID: a6658d3
• Result: SUCCEEDED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository