Skip to main content

Glue ETL Job S3 Encryption Enabled

Overview​

This check verifies that AWS Glue ETL jobs are configured to encrypt data at rest when writing outputs to Amazon S3. Glue jobs can be secured by attaching a security configuration that specifies S3 encryption settings (SSE-S3 or SSE-KMS). Without this, job outputs are stored as plaintext in S3.

Risk​

When Glue ETL jobs write unencrypted data to S3:

  • Data exposure: Plaintext data can be accessed by anyone with S3 bucket permissions, even if access was unintended
  • Credential compromise impact: If AWS credentials are stolen, attackers can read sensitive ETL outputs directly
  • Compliance violations: Many regulations (HIPAA, PCI-DSS, GDPR) require encryption of data at rest
  • No audit trail for access: Without KMS encryption, you lose the ability to track who decrypted your data via CloudTrail
  • Bucket misconfiguration risk: If an S3 bucket is accidentally made public, unencrypted data is immediately exposed

Remediation Steps​

Prerequisites​

  • AWS account access with permissions to manage Glue jobs and security configurations
  • For SSE-KMS: An existing KMS key, or permission to create one
Required IAM permissions

To create a Glue security configuration and update jobs, you need:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "glue:CreateSecurityConfiguration",
        "glue:GetSecurityConfiguration",
        "glue:GetJob",
        "glue:UpdateJob"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "kms:DescribeKey",
        "kms:ListAliases"
      ],
      "Resource": "*"
    }
  ]
}

AWS Console Method​

Step 1: Create a Glue security configuration​

  1. Open the AWS Glue Console at https://console.aws.amazon.com/glue
  2. Ensure you are in the us-east-1 region (top-right corner)
  3. In the left sidebar, click Security configurations (under "Security")
  4. Click Add security configuration
  5. Enter a Name (e.g., glue-s3-encryption-config)
  6. Under S3 encryption, check Enable
  7. Choose an encryption mode:
    • SSE-S3: Amazon S3 manages the encryption keys (simpler setup)
    • SSE-KMS: You control the encryption key (recommended for sensitive data)
  8. If using SSE-KMS, select or enter your KMS key ARN
  9. Click Create

Step 2: Attach the security configuration to your Glue job​

  1. In the Glue Console, click Jobs in the left sidebar
  2. Select the job you want to secure
  3. Click Actions > Edit job
  4. Scroll down to Security configuration, script libraries, and job parameters
  5. Click to expand this section
  6. Under Security configuration, select the configuration you created (e.g., glue-s3-encryption-config)
  7. Click Save

Repeat Step 2 for each Glue ETL job that needs encryption.

AWS CLI (optional)

Create a security configuration with SSE-S3​

aws glue create-security-configuration \
  --name glue-s3-encryption-config \
  --encryption-configuration '{
    "S3Encryption": [
      {
        "S3EncryptionMode": "SSE-S3"
      }
    ]
  }' \
  --region us-east-1

Create a security configuration with SSE-KMS​

aws glue create-security-configuration \
  --name glue-s3-kms-encryption-config \
  --encryption-configuration '{
    "S3Encryption": [
      {
        "S3EncryptionMode": "SSE-KMS",
        "KmsKeyArn": "arn:aws:kms:us-east-1:<YOUR_ACCOUNT_ID>:key/<YOUR_KEY_ID>"
      }
    ]
  }' \
  --region us-east-1

Replace:

  • <YOUR_ACCOUNT_ID> with your 12-digit AWS account ID
  • <YOUR_KEY_ID> with your KMS key ID

Update an existing Glue job to use the security configuration​

First, get the current job definition:

aws glue get-job \
  --job-name <YOUR_JOB_NAME> \
  --region us-east-1

Then update the job with the security configuration. Note that update-job requires you to provide the Role and Command parameters even if you are not changing them:

aws glue update-job \
  --job-name <YOUR_JOB_NAME> \
  --job-update '{
    "Role": "arn:aws:iam::<YOUR_ACCOUNT_ID>:role/<YOUR_GLUE_ROLE>",
    "Command": {
      "Name": "glueetl",
      "ScriptLocation": "s3://<YOUR_BUCKET>/<YOUR_SCRIPT_PATH>"
    },
    "SecurityConfiguration": "glue-s3-encryption-config"
  }' \
  --region us-east-1

List existing security configurations​

aws glue get-security-configurations \
  --region us-east-1
CloudFormation (optional)

This template creates a Glue security configuration with S3 encryption and a sample Glue job that uses it.

AWSTemplateFormatVersion: '2010-09-09'
Description: Glue ETL job with S3 encryption enabled

Parameters:
  SecurityConfigName:
    Type: String
    Description: Name for the Glue security configuration
    Default: glue-s3-encryption-config
  EncryptionMode:
    Type: String
    Description: S3 encryption mode
    Default: SSE-S3
    AllowedValues:
      - SSE-S3
      - SSE-KMS
  KmsKeyArn:
    Type: String
    Description: KMS key ARN (required only for SSE-KMS)
    Default: ''
  JobName:
    Type: String
    Description: Name for the Glue job
    Default: my-encrypted-glue-job
  ScriptLocation:
    Type: String
    Description: S3 path to the Glue job script
  GlueRoleArn:
    Type: String
    Description: ARN of the IAM role for the Glue job

Conditions:
  UseKms: !Equals [!Ref EncryptionMode, 'SSE-KMS']

Resources:
  GlueSecurityConfiguration:
    Type: AWS::Glue::SecurityConfiguration
    Properties:
      Name: !Ref SecurityConfigName
      EncryptionConfiguration:
        S3Encryptions:
          - S3EncryptionMode: !Ref EncryptionMode
            KmsKeyArn: !If [UseKms, !Ref KmsKeyArn, !Ref 'AWS::NoValue']

  GlueJob:
    Type: AWS::Glue::Job
    Properties:
      Name: !Ref JobName
      Role: !Ref GlueRoleArn
      Command:
        Name: glueetl
        ScriptLocation: !Ref ScriptLocation
        PythonVersion: '3'
      GlueVersion: '4.0'
      SecurityConfiguration: !Ref GlueSecurityConfiguration
      DefaultArguments:
        '--enable-metrics': 'true'
        '--enable-continuous-cloudwatch-log': 'true'

Outputs:
  SecurityConfigurationName:
    Description: Name of the Glue security configuration
    Value: !Ref GlueSecurityConfiguration
  JobName:
    Description: Name of the Glue job
    Value: !Ref GlueJob

Deploy with:

aws cloudformation deploy \
  --template-file template.yaml \
  --stack-name glue-encrypted-job \
  --parameter-overrides \
    ScriptLocation=s3://my-bucket/scripts/my-job.py \
    GlueRoleArn=arn:aws:iam::123456789012:role/GlueServiceRole \
  --region us-east-1
Terraform (optional)
variable "security_config_name" {
  description = "Name for the Glue security configuration"
  type        = string
  default     = "glue-s3-encryption-config"
}

variable "encryption_mode" {
  description = "S3 encryption mode (SSE-S3 or SSE-KMS)"
  type        = string
  default     = "SSE-S3"
}

variable "kms_key_arn" {
  description = "KMS key ARN for SSE-KMS encryption (optional)"
  type        = string
  default     = null
}

variable "job_name" {
  description = "Name for the Glue job"
  type        = string
  default     = "my-encrypted-glue-job"
}

variable "script_location" {
  description = "S3 path to the Glue job script"
  type        = string
}

variable "glue_role_arn" {
  description = "ARN of the IAM role for the Glue job"
  type        = string
}

resource "aws_glue_security_configuration" "s3_encryption" {
  name = var.security_config_name

  encryption_configuration {
    s3_encryption {
      s3_encryption_mode = var.encryption_mode
      kms_key_arn        = var.encryption_mode == "SSE-KMS" ? var.kms_key_arn : null
    }
  }
}

resource "aws_glue_job" "encrypted_job" {
  name              = var.job_name
  role_arn          = var.glue_role_arn
  glue_version      = "4.0"
  security_configuration = aws_glue_security_configuration.s3_encryption.name

  command {
    name            = "glueetl"
    script_location = var.script_location
    python_version  = "3"
  }

  default_arguments = {
    "--enable-metrics"                      = "true"
    "--enable-continuous-cloudwatch-log"    = "true"
  }
}

output "security_configuration_name" {
  description = "Name of the Glue security configuration"
  value       = aws_glue_security_configuration.s3_encryption.name
}

output "job_name" {
  description = "Name of the Glue job"
  value       = aws_glue_job.encrypted_job.name
}

Deploy with:

terraform init
terraform plan -var="script_location=s3://my-bucket/scripts/my-job.py" \
               -var="glue_role_arn=arn:aws:iam::123456789012:role/GlueServiceRole"
terraform apply

Verification​

After making changes, verify S3 encryption is enabled:

  1. Open the AWS Glue Console at https://console.aws.amazon.com/glue
  2. Click Jobs in the left sidebar and select your job
  3. In the Job details tab, look for the Security configuration field
  4. Confirm it shows the name of your security configuration
  5. Click on the security configuration name to verify S3 encryption is enabled
Verify with AWS CLI

Check the job's security configuration:

aws glue get-job \
  --job-name <YOUR_JOB_NAME> \
  --region us-east-1 \
  --query 'Job.SecurityConfiguration'

Then verify the security configuration has S3 encryption enabled:

aws glue get-security-configuration \
  --name <SECURITY_CONFIG_NAME> \
  --region us-east-1 \
  --query 'SecurityConfiguration.EncryptionConfiguration.S3Encryption'

A properly configured security configuration will show output like:

[
  {
    "S3EncryptionMode": "SSE-KMS",
    "KmsKeyArn": "arn:aws:kms:us-east-1:123456789012:key/12345678-1234-1234-1234-123456789012"
  }
]

Or for SSE-S3:

[
  {
    "S3EncryptionMode": "SSE-S3"
  }
]

Additional Resources​

Notes​

  • SSE-S3 vs SSE-KMS: SSE-S3 is simpler to set up with no key management required. SSE-KMS provides additional benefits: custom key policies, CloudTrail logging of key usage, and the ability to immediately revoke access by disabling the key.
  • Existing jobs: Adding a security configuration to an existing job does not retroactively encrypt previously written data. Only new job outputs will be encrypted.
  • Job parameters alternative: Instead of using a security configuration, you can enable encryption via job parameters (--encryption-type, --s3-encryption-type). However, security configurations are recommended as they provide centralized management.
  • Multiple encryption settings: Security configurations can also enable CloudWatch Logs encryption and job bookmarks encryption in addition to S3 encryption.
  • Streaming jobs: This check applies to both Glue ETL batch jobs and Glue Streaming jobs.
  • Cost considerations: SSE-S3 has no additional cost. SSE-KMS incurs charges for KMS key storage and API calls.
  • Key permissions: If using SSE-KMS, ensure the Glue job's IAM role has kms:GenerateDataKey and kms:Decrypt permissions on the KMS key.