ECS Task Definitions Containers Should Have Logging Configured with Non-Blocking Mode

Overview

This check verifies that your Amazon ECS task definitions have logging configured in non-blocking mode. When containers write logs, they send output to stdout and stderr. If the logging system cannot keep up (for example, due to network issues or CloudWatch throttling), blocking mode causes your application to pause and wait, which can lead to failures.

Non-blocking mode lets your containers continue running even if there are temporary issues sending logs.

Risk

When logging is set to blocking mode (the default), your containers face these risks:

• Application hangs: If logs cannot be sent immediately, any code writing to stdout or stderr will pause
• Health check failures: Blocked containers may fail health checks and get terminated
• Startup failures: If CloudWatch Log Groups or streams cannot be created, containers will not start
• Cascading failures: One logging issue can cause multiple containers or services to become unresponsive

Severity: Low

Remediation Steps

Prerequisites

You need:

• Access to the AWS Console with permissions to modify ECS task definitions, OR
• AWS CLI configured with appropriate permissions

Required IAM permissions

Your IAM user or role needs these permissions:

• ecs:RegisterTaskDefinition
• ecs:DescribeTaskDefinition
• logs:CreateLogGroup (if creating new log groups)

AWS Console Method

1. Open the Amazon ECS console at https://console.aws.amazon.com/ecs/
2. In the left navigation, click Task definitions
3. Select the task definition that failed the check
4. Click Create new revision
5. Scroll to the Container definitions section and click on your container name
6. Expand the Logging section (under Environment)
7. Set Log driver to awslogs if not already set
8. In Log options, add these key-value pairs:
   • awslogs-group: Your CloudWatch log group (e.g., /ecs/my-task)
   • awslogs-region: us-east-1
   • awslogs-stream-prefix: ecs
   • mode: non-blocking
   • max-buffer-size: 25m (optional but recommended)
9. Click Update to save the container changes
10. Click Create to register the new task definition revision
11. Update your ECS service to use the new revision

AWS CLI

First, get your current task definition:

aws ecs describe-task-definition \
    --task-definition my-task-family \
    --region us-east-1 \
    --query 'taskDefinition' > task-def.json

Edit task-def.json to update the logConfiguration for each container:

"logConfiguration": {
    "logDriver": "awslogs",
    "options": {
        "awslogs-group": "/ecs/my-task",
        "awslogs-region": "us-east-1",
        "awslogs-stream-prefix": "ecs",
        "mode": "non-blocking",
        "max-buffer-size": "25m"
    }
}

Remove these fields from the JSON before registering (they cannot be included):

• taskDefinitionArn
• revision
• status
• requiresAttributes
• compatibilities
• registeredAt
• registeredBy

Register the updated task definition:

aws ecs register-task-definition \
    --cli-input-json file://task-def.json \
    --region us-east-1

Update your service to use the new revision:

aws ecs update-service \
    --cluster my-cluster \
    --service my-service \
    --task-definition my-task-family \
    --region us-east-1

CloudFormation

AWSTemplateFormatVersion: '2010-09-09'
Description: ECS Task Definition with non-blocking logging mode

Parameters:
  TaskFamily:
    Type: String
    Default: my-task
    Description: The family name for the task definition
  ContainerImage:
    Type: String
    Default: nginx:latest
    Description: The container image to use
  LogGroupName:
    Type: String
    Default: /ecs/my-task
    Description: CloudWatch Log Group name

Resources:
  ECSLogGroup:
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupName: !Ref LogGroupName
      RetentionInDays: 30

  ECSTaskDefinition:
    Type: AWS::ECS::TaskDefinition
    Properties:
      Family: !Ref TaskFamily
      ContainerDefinitions:
        - Name: app
          Image: !Ref ContainerImage
          Memory: 512
          Cpu: 256
          Essential: true
          LogConfiguration:
            LogDriver: awslogs
            Options:
              awslogs-group: !Ref LogGroupName
              awslogs-region: !Ref AWS::Region
              awslogs-stream-prefix: ecs
              mode: non-blocking
              max-buffer-size: 25m

Outputs:
  TaskDefinitionArn:
    Description: ARN of the created task definition
    Value: !Ref ECSTaskDefinition

Deploy the stack:

aws cloudformation deploy \
    --template-file template.yaml \
    --stack-name ecs-task-nonblocking-logging \
    --parameter-overrides TaskFamily=my-task ContainerImage=nginx:latest \
    --region us-east-1

Terraform

resource "aws_cloudwatch_log_group" "ecs" {
  name              = "/ecs/my-task"
  retention_in_days = 30
}

resource "aws_ecs_task_definition" "app" {
  family = "my-task"

  container_definitions = jsonencode([{
    name      = "app"
    image     = "nginx:latest"
    memory    = 512
    cpu       = 256
    essential = true

    logConfiguration = {
      logDriver = "awslogs"
      options = {
        awslogs-group         = aws_cloudwatch_log_group.ecs.name
        awslogs-region        = "us-east-1"
        awslogs-stream-prefix = "ecs"
        mode                  = "non-blocking"
        max-buffer-size       = "25m"
      }
    }
  }])
}

Apply the configuration:

terraform init
terraform plan
terraform apply

Verification

After updating your task definition:

1. Go to ECS > Task definitions in the AWS Console
2. Select your task definition and click on the latest revision
3. Click on your container name
4. Scroll to the Logging section
5. Confirm that mode is set to non-blocking

CLI verification

aws ecs describe-task-definition \
    --task-definition my-task-family \
    --region us-east-1 \
    --query 'taskDefinition.containerDefinitions[*].logConfiguration'

The output should show "mode": "non-blocking" for each container:

[
    {
        "logDriver": "awslogs",
        "options": {
            "awslogs-group": "/ecs/my-task",
            "awslogs-region": "us-east-1",
            "awslogs-stream-prefix": "ecs",
            "mode": "non-blocking",
            "max-buffer-size": "25m"
        }
    }
]

Additional Resources

• Amazon ECS awslogs Log Driver
• Preventing Log Loss with Non-Blocking Mode
• ECS Task Definition Parameters

Notes

• Existing services must be updated: After creating a new task definition revision, you must update your ECS services to use it. Running tasks will continue using the old definition until replaced.

• max-buffer-size is recommended: When using non-blocking mode, set max-buffer-size (e.g., 25m) to control how much log data can be buffered. If the buffer fills up, the oldest logs are dropped rather than blocking the container.

• Log loss is possible: Non-blocking mode prioritizes application availability over log completeness. If CloudWatch cannot keep up, some logs may be dropped. For critical audit logs, consider additional logging mechanisms.

• All containers need configuration: Each container in your task definition has its own logging configuration. Make sure to update all containers, not just one.

• Compliance frameworks: This check is relevant for C5, ISO27001, KISA-ISMS-P, and NIS2 compliance frameworks.