Glue Development Endpoint S3 Encryption Enabled

Overview

This check verifies that your AWS Glue development endpoints have an attached security configuration with S3 encryption enabled. Development endpoints are used for interactive data exploration and script development. Without encryption, any data written to S3 during development (temporary files, test outputs, scripts) is stored unencrypted.

Risk

When S3 encryption is not enabled on Glue development endpoints, sensitive data processed during ETL development may be exposed:

• Unprotected data at rest: Temporary data, ETL outputs, and scripts written to S3 remain readable if bucket permissions are misconfigured
• Credential exposure: If AWS credentials are compromised, attackers can read all unencrypted development data
• Compliance violations: Many security frameworks (PCI-DSS, HIPAA, SOC 2) require encryption of data at rest
• Data leakage: Development often uses production-like data, which may contain sensitive information

Remediation Steps

Prerequisites

You need:

• AWS Console access with permissions to manage AWS Glue resources
• Permission to create Glue security configurations

Required IAM permissions (for administrators)

Your IAM user or role needs these permissions:

• glue:CreateSecurityConfiguration
• glue:GetSecurityConfiguration
• glue:CreateDevEndpoint
• glue:UpdateDevEndpoint
• glue:GetDevEndpoint
• glue:DeleteDevEndpoint
• kms:CreateKey (if using SSE-KMS)
• kms:DescribeKey (if using SSE-KMS)

AWS Console Method

Step 1: Create a Security Configuration with S3 Encryption

1. Open AWS Glue in the Console

   • Go to AWS Glue Console in us-east-1

2. Navigate to Security configurations

   • In the left sidebar, expand Data Catalog or scroll down to find Security configurations
   • Click Security configurations

3. Create a new security configuration

   • Click Add security configuration
   • Enter a name like glue-s3-encrypted

4. Enable S3 encryption

   • Under S3 encryption, select Enable
   • Choose an encryption mode:
     • SSE-S3 (Amazon S3-managed keys) - simplest option
     • SSE-KMS (AWS KMS-managed keys) - recommended for more control
   • If using SSE-KMS, select or create a KMS key
   • Click Create

Step 2: Create a New Development Endpoint with the Security Configuration

Because existing development endpoints cannot be updated to add a security configuration, you must create a new endpoint.

1. Navigate to Dev endpoints

   • In the left sidebar under ETL, click Dev endpoints

2. Create a new development endpoint

   • Click Add endpoint
   • Enter an endpoint name
   • Select an IAM role with appropriate permissions

3. Attach the security configuration

   • In the Security configuration dropdown, select the configuration you created (e.g., glue-s3-encrypted)
   • Complete the remaining configuration options as needed
   • Click Create

4. Delete the old unencrypted endpoint

   • Once your new endpoint is ready, delete the old endpoint without encryption
   • Select the old endpoint and click Delete

AWS CLI (optional)

Step 1: Create a security configuration with S3 encryption

Using SSE-S3 (simpler):

aws glue create-security-configuration \
    --name glue-s3-encrypted \
    --encryption-configuration '{
        "S3Encryption": [
            {
                "S3EncryptionMode": "SSE-S3"
            }
        ]
    }' \
    --region us-east-1

Using SSE-KMS (recommended for more control):

aws glue create-security-configuration \
    --name glue-s3-encrypted-kms \
    --encryption-configuration '{
        "S3Encryption": [
            {
                "S3EncryptionMode": "SSE-KMS",
                "KmsKeyArn": "arn:aws:kms:us-east-1:<your-account-id>:key/<your-key-id>"
            }
        ]
    }' \
    --region us-east-1

Replace <your-account-id> and <your-key-id> with your actual values.

Step 2: Create a new development endpoint with the security configuration

aws glue create-dev-endpoint \
    --endpoint-name my-secure-dev-endpoint \
    --role-arn arn:aws:iam::<your-account-id>:role/GlueServiceRole \
    --security-configuration glue-s3-encrypted \
    --worker-type G.1X \
    --number-of-workers 2 \
    --glue-version 3.0 \
    --region us-east-1

Step 3: Delete the old unencrypted endpoint

aws glue delete-dev-endpoint \
    --endpoint-name <old-endpoint-name> \
    --region us-east-1

CloudFormation (optional)

This template creates a security configuration with S3 encryption and a development endpoint that uses it:

AWSTemplateFormatVersion: '2010-09-09'
Description: Glue development endpoint with S3 encryption

Parameters:
  EndpointName:
    Type: String
    Description: Name of the Glue development endpoint
    Default: my-secure-dev-endpoint

  GlueServiceRoleArn:
    Type: String
    Description: ARN of the IAM role for the Glue endpoint

  SecurityConfigName:
    Type: String
    Description: Name of the security configuration
    Default: glue-s3-encrypted

Resources:
  GlueSecurityConfiguration:
    Type: AWS::Glue::SecurityConfiguration
    Properties:
      Name: !Ref SecurityConfigName
      EncryptionConfiguration:
        S3Encryptions:
          - S3EncryptionMode: SSE-S3

  GlueDevEndpoint:
    Type: AWS::Glue::DevEndpoint
    DependsOn: GlueSecurityConfiguration
    Properties:
      EndpointName: !Ref EndpointName
      RoleArn: !Ref GlueServiceRoleArn
      SecurityConfiguration: !Ref SecurityConfigName
      WorkerType: G.1X
      NumberOfWorkers: 2
      GlueVersion: '3.0'

Outputs:
  SecurityConfigurationName:
    Description: Name of the security configuration
    Value: !Ref GlueSecurityConfiguration

  DevEndpointName:
    Description: Name of the development endpoint
    Value: !Ref GlueDevEndpoint

Deploy with:

aws cloudformation deploy \
    --template-file glue-secure-endpoint.yaml \
    --stack-name glue-secure-dev-endpoint \
    --parameter-overrides \
        EndpointName=my-secure-dev-endpoint \
        GlueServiceRoleArn=arn:aws:iam::123456789012:role/GlueServiceRole \
    --region us-east-1

Terraform (optional)

# Variables
variable "endpoint_name" {
  description = "Name of the Glue development endpoint"
  type        = string
  default     = "my-secure-dev-endpoint"
}

variable "glue_service_role_arn" {
  description = "ARN of the IAM role for the Glue endpoint"
  type        = string
}

variable "security_config_name" {
  description = "Name of the security configuration"
  type        = string
  default     = "glue-s3-encrypted"
}

# Security configuration with S3 encryption
resource "aws_glue_security_configuration" "encrypted" {
  name = var.security_config_name

  encryption_configuration {
    s3_encryption {
      s3_encryption_mode = "SSE-S3"
    }
  }
}

# Development endpoint with security configuration
resource "aws_glue_dev_endpoint" "secure" {
  name                   = var.endpoint_name
  role_arn               = var.glue_service_role_arn
  security_configuration = aws_glue_security_configuration.encrypted.name
  worker_type            = "G.1X"
  number_of_workers      = 2
  glue_version           = "3.0"
}

# Outputs
output "security_configuration_name" {
  description = "Name of the security configuration"
  value       = aws_glue_security_configuration.encrypted.name
}

output "dev_endpoint_name" {
  description = "Name of the development endpoint"
  value       = aws_glue_dev_endpoint.secure.name
}

For SSE-KMS encryption instead:

resource "aws_glue_security_configuration" "encrypted_kms" {
  name = "glue-s3-encrypted-kms"

  encryption_configuration {
    s3_encryption {
      s3_encryption_mode = "SSE-KMS"
      kms_key_arn        = aws_kms_key.glue.arn
    }
  }
}

resource "aws_kms_key" "glue" {
  description             = "KMS key for Glue S3 encryption"
  deletion_window_in_days = 30
  enable_key_rotation     = true
}

Deploy with:

terraform init
terraform plan -var="glue_service_role_arn=arn:aws:iam::123456789012:role/GlueServiceRole"
terraform apply -var="glue_service_role_arn=arn:aws:iam::123456789012:role/GlueServiceRole"

Verification

After creating the new endpoint, verify encryption is enabled:

1. In the AWS Console:

   • Go to AWS Glue > Dev endpoints
   • Click on your development endpoint name
   • In the endpoint details, verify the Security configuration field shows your encrypted configuration

2. Check the security configuration:

   • Go to AWS Glue > Security configurations
   • Click on your security configuration name
   • Verify S3 encryption mode shows SSE-S3 or SSE-KMS

CLI verification commands

Check the development endpoint configuration:

aws glue get-dev-endpoint \
    --endpoint-name <endpoint-name> \
    --region us-east-1 \
    --query '{EndpointName:DevEndpoint.EndpointName,SecurityConfiguration:DevEndpoint.SecurityConfiguration}'

Expected output:

{
    "EndpointName": "my-secure-dev-endpoint",
    "SecurityConfiguration": "glue-s3-encrypted"
}

Verify the security configuration has S3 encryption enabled:

aws glue get-security-configuration \
    --name glue-s3-encrypted \
    --region us-east-1 \
    --query 'SecurityConfiguration.EncryptionConfiguration.S3Encryption'

Expected output:

[
    {
        "S3EncryptionMode": "SSE-S3"
    }
]

If SecurityConfiguration is null or empty in the endpoint output, the endpoint is not using encryption.

Additional Resources

• AWS Documentation: Encrypting Data Written by AWS Glue
• AWS Documentation: Working with Security Configurations
• AWS Documentation: Managing Development Endpoints
• AWS Documentation: AWS KMS Keys

Notes

• Existing endpoints cannot be updated: You cannot add a security configuration to an existing development endpoint. You must create a new endpoint with the security configuration attached.
• Deprecation notice: AWS has deprecated Glue development endpoints in favor of AWS Glue Studio notebooks and Glue interactive sessions. Consider migrating to these newer options, which also support security configurations.
• All-or-nothing encryption: When you attach a security configuration, the S3 encryption setting applies to all S3 writes from that endpoint.
• KMS vs S3-managed keys: SSE-KMS provides more control (key policies, audit trails via CloudTrail, key rotation control) but has a small cost. SSE-S3 is simpler and free.
• Job inheritance: If you use the development endpoint to develop ETL jobs, remember to also attach a security configuration to the jobs when you deploy them to production.
• Costs: Development endpoints incur charges while running. Delete unused endpoints to avoid unnecessary costs.