Skip to main content

CloudWatch Log Group Retention Policy

Overview​

This check verifies that Amazon CloudWatch Log Groups have a retention policy that meets or exceeds a configured minimum number of days (typically 365 days) or are set to never expire. Log groups with shorter retention periods or no retention policy fail this check.

Risk​

Short or missing log retention policies create significant security and compliance risks:

  • Lost audit evidence: Log events deleted before investigation can destroy critical forensic data
  • Compliance violations: Many regulations (PCI-DSS, HIPAA, SOC 2) require specific log retention periods
  • Detection gaps: Attackers can simply wait out short retention windows before acting, knowing their activities will be erased
  • Incident response failures: Without historical logs, security teams cannot trace the full scope of a breach or establish timelines
  • Accountability issues: Inability to demonstrate what happened and when during security audits

Remediation Steps​

Prerequisites​

You need:

  • AWS Console access with permissions to modify CloudWatch Log Groups
  • Knowledge of your organization's log retention requirements
Required IAM permissions (for administrators)

Your IAM user or role needs these permissions:

  • logs:DescribeLogGroups
  • logs:PutRetentionPolicy
  • logs:DeleteRetentionPolicy (only if setting to "Never expire")

AWS Console Method​

  1. Open CloudWatch in the AWS Console

  2. Navigate to Log Groups

    • In the left sidebar, expand Logs
    • Click Log groups

  3. Find the log group to update

    • Use the search box to find your log group by name
    • Or scroll through the list to locate it

  4. Update the retention setting

    • In the Expire events after column, click on the current retention value (e.g., "Never expire" or "7 days")
    • A dropdown menu appears with available retention options

  5. Select the appropriate retention period

    • Choose a value that meets your requirements (e.g., 1 year (365 days) or longer)
    • Or select Never expire to retain logs indefinitely
    • Valid options: 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1096, 1827, 2192, 2557, 2922, 3288, or 3653 days

  6. Confirm the change

    • The retention policy updates immediately after selection
AWS CLI (optional)

Set a retention policy on a log group​

aws logs put-retention-policy \
    --log-group-name <your-log-group-name> \
    --retention-in-days 365 \
    --region us-east-1

Replace <your-log-group-name> with the actual log group name.

Valid retention values: 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1096, 1827, 2192, 2557, 2922, 3288, 3653

Set to never expire​

To configure a log group to retain logs indefinitely, delete the retention policy:

aws logs delete-retention-policy \
    --log-group-name <your-log-group-name> \
    --region us-east-1

Update multiple log groups​

To update all log groups without a retention policy:

# List log groups with no retention policy
aws logs describe-log-groups \
    --query 'logGroups[?retentionInDays==`null`].logGroupName' \
    --output text \
    --region us-east-1

# Set retention on each (one at a time)
for lg in $(aws logs describe-log-groups \
    --query 'logGroups[?retentionInDays==`null`].logGroupName' \
    --output text \
    --region us-east-1); do
    aws logs put-retention-policy \
        --log-group-name "$lg" \
        --retention-in-days 365 \
        --region us-east-1
    echo "Set 365-day retention for: $lg"
done
CloudFormation (optional)

Create a new log group with retention policy​

AWSTemplateFormatVersion: '2010-09-09'
Description: CloudWatch Log Group with retention policy

Parameters:
  LogGroupName:
    Type: String
    Description: Name of the CloudWatch Log Group
  RetentionDays:
    Type: Number
    Description: Number of days to retain log events
    Default: 365
    AllowedValues:
      - 1
      - 3
      - 5
      - 7
      - 14
      - 30
      - 60
      - 90
      - 120
      - 150
      - 180
      - 365
      - 400
      - 545
      - 731
      - 1096
      - 1827
      - 2192
      - 2557
      - 2922
      - 3288
      - 3653

Resources:
  LogGroup:
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupName: !Ref LogGroupName
      RetentionInDays: !Ref RetentionDays
      Tags:
        - Key: ManagedBy
          Value: CloudFormation

Outputs:
  LogGroupArn:
    Description: ARN of the log group
    Value: !GetAtt LogGroup.Arn

Deploy the stack​

aws cloudformation create-stack \
    --stack-name my-log-group-stack \
    --template-body file://template.yaml \
    --parameters ParameterKey=LogGroupName,ParameterValue=/my-app/logs \
                 ParameterKey=RetentionDays,ParameterValue=365 \
    --region us-east-1

Note on existing log groups​

CloudFormation cannot modify the retention policy of an existing log group that was created outside of CloudFormation. For existing log groups, use the AWS Console or CLI method.

Terraform (optional)

Create a new log group with retention policy​

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

provider "aws" {
  region = "us-east-1"
}

variable "log_group_name" {
  description = "Name of the CloudWatch Log Group"
  type        = string
}

variable "retention_in_days" {
  description = "Number of days to retain log events"
  type        = number
  default     = 365

  validation {
    condition = contains([
      1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365,
      400, 545, 731, 1096, 1827, 2192, 2557, 2922, 3288, 3653
    ], var.retention_in_days)
    error_message = "retention_in_days must be a valid CloudWatch Logs retention value."
  }
}

resource "aws_cloudwatch_log_group" "this" {
  name              = var.log_group_name
  retention_in_days = var.retention_in_days

  tags = {
    ManagedBy = "Terraform"
  }
}

output "log_group_arn" {
  description = "ARN of the log group"
  value       = aws_cloudwatch_log_group.this.arn
}

Update an existing log group​

If the log group already exists but was created outside Terraform, import it first:

terraform import aws_cloudwatch_log_group.this /my-app/logs

Then update the retention_in_days value and apply:

terraform apply

Set to never expire​

To configure logs to never expire, set retention_in_days to 0 or omit it:

resource "aws_cloudwatch_log_group" "this" {
  name              = var.log_group_name
  retention_in_days = 0  # Never expire

  tags = {
    ManagedBy = "Terraform"
  }
}

Verification​

After updating the retention policy, verify the change was applied:

  1. In the AWS Console:

    • Go to CloudWatch > Logs > Log groups
    • Find your log group in the list
    • Check the Expire events after column shows the expected retention value

  2. Re-run the Prowler check:

    • Execute the Prowler scan again to confirm the log group now passes
CLI verification commands

Check a specific log group​

aws logs describe-log-groups \
    --log-group-name-prefix <your-log-group-name> \
    --query 'logGroups[*].[logGroupName,retentionInDays]' \
    --output table \
    --region us-east-1

List all log groups with insufficient retention​

# Find log groups with retention less than 365 days
aws logs describe-log-groups \
    --query 'logGroups[?retentionInDays!=`null` && retentionInDays<`365`].[logGroupName,retentionInDays]' \
    --output table \
    --region us-east-1

Find log groups with no retention policy (set to never expire)​

aws logs describe-log-groups \
    --query 'logGroups[?retentionInDays==`null`].[logGroupName]' \
    --output table \
    --region us-east-1

Note: Log groups with no retention policy (null) retain logs indefinitely, which passes the check if "never expire" is acceptable.

Additional Resources​

Notes​

  • Deletion timing: CloudWatch Logs takes up to 72 hours to delete log events after the retention period expires. Events are marked for deletion immediately but may remain visible temporarily.
  • Cost considerations: Longer retention periods increase storage costs. Balance security/compliance requirements with budget constraints.
  • No partial retention: Retention applies to the entire log group. You cannot set different retention policies for individual log streams within a group.
  • Compliance requirements: Check your industry-specific compliance requirements (PCI-DSS, HIPAA, SOX, etc.) for minimum retention periods before selecting a value.
  • Never expire option: Setting logs to "Never expire" (no retention policy) means logs are retained indefinitely. This passes the check but will incur ongoing storage costs.
  • Existing log data: Changing the retention policy does not immediately delete old logs. The new policy applies prospectively, and logs older than the new retention period will be deleted within 72 hours.