Setting CloudWatch Log Retention Across Your AWS Organization 🚀

TL;DR

This Terraform based solution automatically sets retention periods for CloudWatch Log Groups as they’re created across your entire AWS organization. No more manual cleanup or forgetting to set retention periods! Just deploy it once in your management account and forget about it. Made this solution in about 8 hours thanks to Cursor – The AI Code Editor !

Check it out on Github here: main-salman/aws-org-wide-log-retention-policy

Quick Deploy 🛠️

Prerequisites

• An AWS Organization (with all features enabled)
• Access to the management account
• A coffee ☕ (this will only take 5 minutes)

Deployment Steps

1. Clone this repo:

git clone https://github.com/yourusername/aws-org-wide-log-retention-policy.git
cd aws-org-wide-log-retention-policy

2. Set up your AWS credentials for the management account:

export AWS_PROFILE=your-management-account-profile

3. Register your management account as the delegated admin for EventBridge:

aws organizations register-delegated-administrator \
    --account-id YOUR_MANAGEMENT_ACCOUNT_ID \
    --service-principal events.amazonaws.com

4. Deploy with Terraform:

terraform init
terraform apply

That’s it! Now any new log groups created in your organization will automatically get a retention period set.

How It Works 🔧

The Big Picture

When someone creates a new CloudWatch Log Group anywhere in your AWS organization, this solution jumps into action:

1. CloudTrail captures the CreateLogGroup API call
2. EventBridge notices this event and triggers a Lambda function
3. The Lambda function sets the retention period for the new log group

The Technical Details

Event Flow

• CloudTrail is set up as an organization trail
• EventBridge rules in member accounts forward CreateLogGroup events to the management account
• The management account’s EventBridge rule triggers the Lambda function

Cross-Account Access

• CloudFormation StackSets deploy IAM roles to all member accounts
• The Lambda function assumes these roles to set retention periods
• Everything is properly locked down with least-privilege permissions

Components

1. Lambda Function: The brain of the operation
   • Written in Python
   • Handles both management and member accounts
   • Uses STS to assume roles in member accounts
2. IAM Roles & Policies: The security layer
   • Member account roles trust only the Lambda function
   • Minimal permissions for setting retention periods
3. EventBridge Rules: The event router
   • Captures log group creation events
   • Routes them to the Lambda function

Testing It Out

Create a test log group in any account:

aws logs create-log-group --log-group-name test-group

Check its retention period:

aws logs describe-log-groups --log-group-name-prefix test-group

You should see the retention period set to your configured value (default: 30 days)!

Troubleshooting 🔍

If things aren’t working:

1. Check the Lambda function logs
2. Verify the CloudTrail trail is active
3. Make sure the IAM roles are deployed to member accounts

Need more help? Open an issue!

Contributing 🤝

Found a bug? Want to add a feature? PRs are welcome!

License

MIT – Go wild! 🎉