Glue ETL Job Bookmark Encryption Enabled

Overview

This check verifies that your AWS Glue ETL jobs have job bookmark encryption enabled through a security configuration. Job bookmarks track the state of your ETL jobs between runs, helping Glue process only new or changed data. When encryption is enabled, these bookmarks are protected using AWS KMS client-side encryption (CSE-KMS).

Risk

Without job bookmark encryption, your Glue job state data is stored unencrypted:

• Data exposure: Job bookmarks contain execution state and data pointers that could reveal information about your data processing pipelines
• Tampering risk: Unencrypted bookmarks could be modified, causing jobs to skip data, reprocess data, or behave unexpectedly
• Compliance gaps: Many security frameworks require encryption of data at rest, including job metadata
• Audit trail gaps: Without encryption, you lose KMS-level access logging for bookmark operations

Remediation Steps

Prerequisites

You need:

• AWS Console access with permissions to create Glue security configurations and modify Glue jobs
• An existing KMS key or permission to create one

Required IAM permissions (for administrators)

Your IAM user or role needs these permissions:

• glue:CreateSecurityConfiguration
• glue:GetSecurityConfiguration
• glue:UpdateJob
• glue:GetJob
• kms:CreateKey (if creating a new key)
• kms:DescribeKey
• kms:ListKeys

AWS Console Method

Step 1: Create a Security Configuration

1. Open AWS Glue in the Console

   • Go to AWS Glue Console in us-east-1

2. Navigate to Security configurations

   • In the left sidebar, click Security configurations under Data Catalog
   • Click Add security configuration

3. Configure the security settings

   • Enter a name like glue-job-bookmark-encryption
   • Under Encryption settings, find Job bookmark encryption
   • Check the box to enable it
   • Select CSE-KMS as the encryption mode
   • Choose an existing KMS key or enter a KMS key ARN
   • Click Create

Step 2: Attach the Security Configuration to Your Glue Job

1. Navigate to ETL jobs

   • In the left sidebar, click ETL jobs
   • Select the job you want to configure

2. Edit the job

   • Click Actions and select Edit job
   • Go to the Job details tab

3. Assign the security configuration

   • Scroll to Advanced properties
   • Under Security configuration, select the configuration you created (e.g., glue-job-bookmark-encryption)
   • Click Save

4. Repeat for other jobs

   • Repeat step 2-3 for each Glue ETL job that needs protection

AWS CLI (optional)

Step 1: Create a Security Configuration

Create a security configuration with job bookmark encryption enabled:

aws glue create-security-configuration \
    --name glue-job-bookmark-encryption \
    --encryption-configuration '{
        "JobBookmarksEncryption": {
            "JobBookmarksEncryptionMode": "CSE-KMS",
            "KmsKeyArn": "arn:aws:kms:us-east-1:<your-account-id>:key/<your-key-id>"
        }
    }' \
    --region us-east-1

Replace:

• <your-account-id> with your 12-digit AWS account ID
• <your-key-id> with your KMS key ID

Step 2: Update Your Glue Job

First, get your current job configuration:

aws glue get-job \
    --job-name <your-job-name> \
    --region us-east-1 > /tmp/current-job.json

Then update the job to use the security configuration. Note that update-job overwrites the entire job configuration, so you must include all existing settings:

aws glue update-job \
    --job-name <your-job-name> \
    --job-update '{
        "Role": "<your-existing-role-arn>",
        "Command": {
            "Name": "glueetl",
            "ScriptLocation": "<your-existing-script-location>",
            "PythonVersion": "3"
        },
        "SecurityConfiguration": "glue-job-bookmark-encryption"
    }' \
    --region us-east-1

Replace the placeholders with values from your current job configuration.

CloudFormation (optional)

This template creates a security configuration with job bookmark encryption and a sample Glue job that uses it:

AWSTemplateFormatVersion: '2010-09-09'
Description: AWS Glue security configuration with job bookmark encryption

Parameters:
  SecurityConfigName:
    Type: String
    Description: Name of the security configuration
    Default: glue-job-bookmark-encryption

  KmsKeyArn:
    Type: String
    Description: ARN of the KMS key for encryption

  JobName:
    Type: String
    Description: Name of the Glue job
    Default: my-glue-etl-job

  GlueRoleArn:
    Type: String
    Description: ARN of the IAM role for the Glue job

  ScriptLocation:
    Type: String
    Description: S3 path to the Glue job script

Resources:
  GlueSecurityConfiguration:
    Type: AWS::Glue::SecurityConfiguration
    Properties:
      Name: !Ref SecurityConfigName
      EncryptionConfiguration:
        JobBookmarksEncryption:
          JobBookmarksEncryptionMode: CSE-KMS
          KmsKeyArn: !Ref KmsKeyArn

  GlueJob:
    Type: AWS::Glue::Job
    Properties:
      Name: !Ref JobName
      Role: !Ref GlueRoleArn
      Command:
        Name: glueetl
        ScriptLocation: !Ref ScriptLocation
        PythonVersion: '3'
      SecurityConfiguration: !Ref SecurityConfigName
      GlueVersion: '4.0'
      WorkerType: G.1X
      NumberOfWorkers: 2

Outputs:
  SecurityConfigurationName:
    Description: Name of the security configuration
    Value: !Ref GlueSecurityConfiguration

  JobName:
    Description: Name of the Glue job
    Value: !Ref GlueJob

Deploy with:

aws cloudformation deploy \
    --template-file glue-security-config.yaml \
    --stack-name glue-bookmark-encryption \
    --parameter-overrides \
        KmsKeyArn=arn:aws:kms:us-east-1:123456789012:key/your-key-id \
        GlueRoleArn=arn:aws:iam::123456789012:role/GlueServiceRole \
        ScriptLocation=s3://your-bucket/scripts/etl-script.py \
    --region us-east-1

Terraform (optional)

# Variables
variable "security_config_name" {
  description = "Name of the Glue security configuration"
  type        = string
  default     = "glue-job-bookmark-encryption"
}

variable "kms_key_arn" {
  description = "ARN of the KMS key for encryption"
  type        = string
}

variable "job_name" {
  description = "Name of the Glue job"
  type        = string
  default     = "my-glue-etl-job"
}

variable "glue_role_arn" {
  description = "ARN of the IAM role for the Glue job"
  type        = string
}

variable "script_location" {
  description = "S3 path to the Glue job script"
  type        = string
}

# Security configuration with job bookmark encryption
resource "aws_glue_security_configuration" "main" {
  name = var.security_config_name

  encryption_configuration {
    job_bookmarks_encryption {
      job_bookmarks_encryption_mode = "CSE-KMS"
      kms_key_arn                   = var.kms_key_arn
    }
  }
}

# Glue job with security configuration attached
resource "aws_glue_job" "main" {
  name              = var.job_name
  role_arn          = var.glue_role_arn
  glue_version      = "4.0"
  worker_type       = "G.1X"
  number_of_workers = 2

  command {
    name            = "glueetl"
    script_location = var.script_location
    python_version  = "3"
  }

  security_configuration = aws_glue_security_configuration.main.name
}

# Outputs
output "security_configuration_name" {
  description = "Name of the security configuration"
  value       = aws_glue_security_configuration.main.name
}

output "job_name" {
  description = "Name of the Glue job"
  value       = aws_glue_job.main.name
}

Deploy with:

terraform init
terraform plan \
    -var="kms_key_arn=arn:aws:kms:us-east-1:123456789012:key/your-key-id" \
    -var="glue_role_arn=arn:aws:iam::123456789012:role/GlueServiceRole" \
    -var="script_location=s3://your-bucket/scripts/etl-script.py"
terraform apply \
    -var="kms_key_arn=arn:aws:kms:us-east-1:123456789012:key/your-key-id" \
    -var="glue_role_arn=arn:aws:iam::123456789012:role/GlueServiceRole" \
    -var="script_location=s3://your-bucket/scripts/etl-script.py"

Verification

After making changes, verify job bookmark encryption is enabled:

1. In the AWS Console:

   • Go to Glue > Security configurations
   • Click on your security configuration name
   • Verify that Job bookmark encryption shows CSE-KMS with your KMS key

2. Check your job:

   • Go to Glue > ETL jobs
   • Select your job and view its details
   • Under Advanced properties, confirm the security configuration is attached

CLI verification commands

Check the security configuration:

aws glue get-security-configuration \
    --name glue-job-bookmark-encryption \
    --region us-east-1 \
    --query 'SecurityConfiguration.EncryptionConfiguration.JobBookmarksEncryption'

Expected output:

{
    "JobBookmarksEncryptionMode": "CSE-KMS",
    "KmsKeyArn": "arn:aws:kms:us-east-1:123456789012:key/your-key-id"
}

Check that your job is using the security configuration:

aws glue get-job \
    --job-name <your-job-name> \
    --region us-east-1 \
    --query 'Job.SecurityConfiguration'

This should return the name of your security configuration.

Additional Resources

• AWS Documentation: Working with Security Configurations
• AWS Documentation: Encrypting Data Written by Crawlers, Jobs, and Development Endpoints
• AWS Documentation: Job Bookmarks

Notes

• Existing jobs: You must manually update each existing Glue job to use the security configuration. Creating a security configuration alone does not automatically apply it.
• Job must have bookmarks enabled: Job bookmark encryption only matters if your job is configured to use bookmarks. Jobs with bookmarks disabled do not store bookmark data.
• One security configuration per job: Each Glue job can only have one security configuration attached. If you need different encryption settings for different data types, create separate security configurations.
• KMS key permissions: The IAM role used by your Glue job must have kms:Encrypt and kms:Decrypt permissions on the KMS key specified in the security configuration.
• Cannot modify security configuration: Once created, you cannot modify a security configuration. To change encryption settings, create a new security configuration and update your jobs to use it.
• S3 and CloudWatch encryption: Consider also enabling S3 encryption and CloudWatch Logs encryption in the same security configuration for comprehensive protection.