Elastic Beanstalk Environment Should Stream Logs to CloudWatch

Overview

This check verifies that your Elastic Beanstalk environments are configured to send logs to Amazon CloudWatch Logs. CloudWatch Logs provides centralized, durable storage for your application and system logs, making it easier to monitor, search, and troubleshoot issues.

Risk

Without CloudWatch Logs streaming enabled:

• Logs can be lost when instances are replaced, scaled down, or terminated
• Incident detection is delayed because you cannot set up alerts on log patterns
• Forensic analysis becomes difficult since attackers could delete local logs to cover their tracks
• Troubleshooting is harder without centralized access to logs across all instances

This is rated high severity because logging is fundamental to security monitoring and incident response.

Remediation Steps

Prerequisites

• Access to the AWS Console with permissions to modify Elastic Beanstalk environments
• Your environment's IAM instance profile must have permissions to write to CloudWatch Logs

IAM permissions required

The instance profile attached to your Elastic Beanstalk environment needs the following permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents",
        "logs:DescribeLogStreams"
      ],
      "Resource": "arn:aws:logs:*:*:log-group:/aws/elasticbeanstalk/*"
    }
  ]
}

The default AWSElasticBeanstalkWebTier and AWSElasticBeanstalkWorkerTier managed policies already include these permissions.

AWS Console Method

1. Open the AWS Elastic Beanstalk console at https://console.aws.amazon.com/elasticbeanstalk/
2. Select your Region (us-east-1 recommended)
3. Click on your application name
4. Click on the environment you want to configure
5. In the left navigation, click Configuration
6. Find the Updates, monitoring, and logging card and click Edit
7. Scroll down to Instance log streaming to CloudWatch Logs
8. Toggle Log streaming to Enabled
9. Configure the following settings:
   • Retention: Choose how long to keep logs (7 days is a good default)
   • Lifecycle: Select whether to delete or retain logs when the environment terminates (recommend Retain logs)
10. Click Apply at the bottom of the page
11. Wait for the environment to update (this may take a few minutes)

AWS CLI (optional)

Enable CloudWatch Logs streaming for an existing environment:

aws elasticbeanstalk update-environment \
  --region us-east-1 \
  --environment-name <your-environment-name> \
  --option-settings \
    Namespace=aws:elasticbeanstalk:cloudwatch:logs,OptionName=StreamLogs,Value=true \
    Namespace=aws:elasticbeanstalk:cloudwatch:logs,OptionName=DeleteOnTerminate,Value=false \
    Namespace=aws:elasticbeanstalk:cloudwatch:logs,OptionName=RetentionInDays,Value=7

Replace <your-environment-name> with your actual environment name.

Configuration options explained:

Option	Description
StreamLogs	Set to true to enable log streaming
DeleteOnTerminate	Set to false to retain logs after environment termination
RetentionInDays	Number of days to keep logs (1, 3, 5, 7, 14, 30, 60, 90, etc.)

To verify the current configuration:

aws elasticbeanstalk describe-configuration-settings \
  --region us-east-1 \
  --application-name <your-application-name> \
  --environment-name <your-environment-name> \
  --query "ConfigurationSettings[0].OptionSettings[?Namespace=='aws:elasticbeanstalk:cloudwatch:logs']"

CloudFormation (optional)

Use the following CloudFormation template to create or update an Elastic Beanstalk environment with CloudWatch Logs enabled:

AWSTemplateFormatVersion: '2010-09-09'
Description: Enable CloudWatch Logs streaming for Elastic Beanstalk environment

Parameters:
  ApplicationName:
    Type: String
    Description: Name of the Elastic Beanstalk application
  EnvironmentName:
    Type: String
    Description: Name of the Elastic Beanstalk environment
  SolutionStackName:
    Type: String
    Description: The Elastic Beanstalk solution stack name
    Default: 64bit Amazon Linux 2023 v4.0.0 running Python 3.11
  LogRetentionDays:
    Type: Number
    Description: Number of days to retain logs in CloudWatch
    Default: 7
    AllowedValues:
      - 1
      - 3
      - 5
      - 7
      - 14
      - 30
      - 60
      - 90
      - 120
      - 150
      - 180
      - 365
      - 400
      - 545
      - 731
      - 1827
      - 3653

Resources:
  ElasticBeanstalkEnvironment:
    Type: AWS::ElasticBeanstalk::Environment
    Properties:
      ApplicationName: !Ref ApplicationName
      EnvironmentName: !Ref EnvironmentName
      SolutionStackName: !Ref SolutionStackName
      OptionSettings:
        - Namespace: aws:elasticbeanstalk:cloudwatch:logs
          OptionName: StreamLogs
          Value: 'true'
        - Namespace: aws:elasticbeanstalk:cloudwatch:logs
          OptionName: DeleteOnTerminate
          Value: 'false'
        - Namespace: aws:elasticbeanstalk:cloudwatch:logs
          OptionName: RetentionInDays
          Value: !Ref LogRetentionDays

Outputs:
  EnvironmentName:
    Description: Name of the Elastic Beanstalk environment
    Value: !Ref ElasticBeanstalkEnvironment
  EnvironmentURL:
    Description: URL of the Elastic Beanstalk environment
    Value: !GetAtt ElasticBeanstalkEnvironment.EndpointURL

Deploy with:

aws cloudformation deploy \
  --region us-east-1 \
  --template-file template.yaml \
  --stack-name my-eb-environment \
  --parameter-overrides \
    ApplicationName=<your-application-name> \
    EnvironmentName=<your-environment-name>

Terraform (optional)

Use the following Terraform configuration:

terraform {
  required_version = ">= 1.0"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 4.0"
    }
  }
}

provider "aws" {
  region = "us-east-1"
}

variable "application_name" {
  description = "Name of the Elastic Beanstalk application"
  type        = string
}

variable "environment_name" {
  description = "Name of the Elastic Beanstalk environment"
  type        = string
}

variable "solution_stack_name" {
  description = "The Elastic Beanstalk solution stack name"
  type        = string
  default     = "64bit Amazon Linux 2023 v4.0.0 running Python 3.11"
}

variable "log_retention_days" {
  description = "Number of days to retain logs in CloudWatch"
  type        = number
  default     = 7
}

resource "aws_elastic_beanstalk_environment" "main" {
  name                = var.environment_name
  application         = var.application_name
  solution_stack_name = var.solution_stack_name

  # Enable CloudWatch Logs streaming
  setting {
    namespace = "aws:elasticbeanstalk:cloudwatch:logs"
    name      = "StreamLogs"
    value     = "true"
  }

  # Retain logs after environment termination
  setting {
    namespace = "aws:elasticbeanstalk:cloudwatch:logs"
    name      = "DeleteOnTerminate"
    value     = "false"
  }

  # Set log retention period
  setting {
    namespace = "aws:elasticbeanstalk:cloudwatch:logs"
    name      = "RetentionInDays"
    value     = tostring(var.log_retention_days)
  }
}

output "environment_name" {
  description = "Name of the Elastic Beanstalk environment"
  value       = aws_elastic_beanstalk_environment.main.name
}

output "environment_endpoint" {
  description = "URL of the Elastic Beanstalk environment"
  value       = aws_elastic_beanstalk_environment.main.endpoint_url
}

Apply with:

terraform init
terraform apply -var="application_name=<your-app>" -var="environment_name=<your-env>"

Verification

After enabling log streaming, verify it is working:

1. Go to the CloudWatch console at https://console.aws.amazon.com/cloudwatch/
2. Click Log groups in the left navigation
3. Look for log groups starting with /aws/elasticbeanstalk/<your-environment-name>/
4. You should see log groups for various log types (e.g., var/log/web.stdout.log)
5. Click on a log group and verify log events are appearing

CLI verification

List CloudWatch log groups for your environment:

aws logs describe-log-groups \
  --region us-east-1 \
  --log-group-name-prefix "/aws/elasticbeanstalk/<your-environment-name>/"

View recent log events:

aws logs filter-log-events \
  --region us-east-1 \
  --log-group-name "/aws/elasticbeanstalk/<your-environment-name>/var/log/web.stdout.log" \
  --limit 10

Additional Resources

• AWS Documentation: Streaming Elastic Beanstalk environment logs to CloudWatch Logs
• AWS Documentation: CloudWatch Logs configuration namespaces
• AWS Documentation: Viewing logs from Elastic Beanstalk environments

Notes

• Environment update required: Enabling log streaming triggers an environment update, which may cause a brief interruption depending on your deployment settings.
• Cost consideration: CloudWatch Logs incurs costs for ingestion and storage. Set appropriate retention periods to manage costs.
• Log types: By default, Elastic Beanstalk streams instance logs. You can also enable health streaming for enhanced health reporting data.
• Compliance frameworks: This control helps meet requirements for C5, ISO27001, KISA-ISMS-P, NIS2, PCI, and SOC2.