No Potential LLM Jacking Activity Detected in CloudTrail

Overview

This check analyzes CloudTrail activity related to Amazon Bedrock to detect potential "LLM jacking" attacks. Prowler monitors for identities that make an unusually high volume or diversity of LLM-related API calls (such as InvokeModel, InvokeModelWithResponseStream, and GetFoundationModelAvailability). When an identity's share of these actions exceeds configured thresholds, the system flags it as potential credential abuse.

What is LLM jacking? Attackers who steal AWS credentials may use them to access generative AI services like Amazon Bedrock. They often resell this access through reverse proxies, running up large bills while obscuring the true source of the activity.

Risk

If this check fails, one or more identities in your account are exhibiting behavior patterns consistent with stolen credential abuse:

• Cost exhaustion: Attackers can rack up massive Bedrock usage charges in hours or days
• Service disruption: Quota exhaustion may prevent legitimate use of Bedrock
• Data leakage: Prompts and model outputs may contain sensitive information
• Model exposure: Attackers gain insight into your model configurations and capabilities
• Lateral movement: Compromised credentials may have broader permissions beyond Bedrock

Attackers frequently use reverse proxy services to resell access to stolen AI credentials, making it difficult to trace activity back to its source.

Remediation Steps

This check detects a potential security incident. Remediation involves both immediate response (if you have a finding) and preventive controls (to reduce future risk).

Prerequisites

You need:

• AWS Console access with permissions to view CloudTrail, IAM, and Bedrock settings
• For preventive controls: permissions to modify IAM policies and create SCPs (if using AWS Organizations)

Immediate Response (If Alert Triggered)

If Prowler flagged suspicious activity, take these steps:

1. Identify the affected identity

   • Review the Prowler finding to see which IAM user, role, or access key is involved
   • Note the time window and specific API calls flagged

2. Disable or rotate the credential

   • For IAM users: deactivate the access key in IAM > Users > Security credentials
   • For roles: review and revoke any active sessions if possible
   • Generate new credentials only after investigating the compromise source

3. Review CloudTrail logs

   • Go to CloudTrail > Event history
   • Filter by the affected identity to see all recent activity
   • Look for unusual patterns: unfamiliar IP addresses, odd hours, high-volume requests

4. Check for broader compromise

   • Review what other permissions the affected identity has
   • Check for any IAM changes, new resources, or data access during the suspicious period

CLI commands for incident investigation

List recent Bedrock activity for a specific identity:

aws cloudtrail lookup-events \
    --lookup-attributes AttributeKey=Username,AttributeValue=<username-or-role> \
    --start-time $(date -u -d '7 days ago' +%Y-%m-%dT%H:%M:%SZ) \
    --region us-east-1 \
    --query 'Events[?contains(EventName, `Invoke`) || contains(EventName, `Model`)].{Time:EventTime,Event:EventName,Source:SourceIPAddress}' \
    --output table

Check for active access keys on a user:

aws iam list-access-keys \
    --user-name <username> \
    --region us-east-1

Deactivate a compromised access key:

aws iam update-access-key \
    --user-name <username> \
    --access-key-id <access-key-id> \
    --status Inactive \
    --region us-east-1

Preventive Controls

Implement these controls to reduce the risk of LLM jacking:

1. Apply Least Privilege to Bedrock

Restrict Bedrock Invoke* actions to only the roles that genuinely need them.

In the AWS Console:

1. Go to IAM > Policies
2. Review any policies that grant bedrock:Invoke* or bedrock:*
3. Narrow these permissions to specific models and conditions where possible

Example restrictive IAM policy

This policy allows invoking only specific models:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowSpecificBedrockModels",
            "Effect": "Allow",
            "Action": [
                "bedrock:InvokeModel",
                "bedrock:InvokeModelWithResponseStream"
            ],
            "Resource": [
                "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-sonnet-*",
                "arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-text-*"
            ]
        }
    ]
}

2. Deny Bedrock Where Unused (via SCP)

If you use AWS Organizations, create a Service Control Policy to deny Bedrock access in accounts or OUs that do not need it.

In the AWS Console:

1. Go to AWS Organizations > Policies > Service control policies
2. Click Create policy
3. Name it something like DenyBedrockAccess
4. Add the policy content (see below)
5. Attach it to the appropriate OU or accounts

Example SCP to deny all Bedrock actions

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "DenyAllBedrockActions",
            "Effect": "Deny",
            "Action": "bedrock:*",
            "Resource": "*"
        }
    ]
}

To allow specific roles to bypass this deny (for legitimate use), add a condition:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "DenyBedrockExceptApprovedRoles",
            "Effect": "Deny",
            "Action": "bedrock:*",
            "Resource": "*",
            "Condition": {
                "StringNotLike": {
                    "aws:PrincipalArn": [
                        "arn:aws:iam::*:role/ApprovedBedrockRole",
                        "arn:aws:iam::*:role/MLPlatformRole"
                    ]
                }
            }
        }
    ]
}

AWS CLI to create and attach an SCP

Create the SCP:

aws organizations create-policy \
    --name "DenyBedrockAccess" \
    --description "Deny all Bedrock actions for accounts that do not need AI services" \
    --type SERVICE_CONTROL_POLICY \
    --content '{
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "DenyAllBedrockActions",
                "Effect": "Deny",
                "Action": "bedrock:*",
                "Resource": "*"
            }
        ]
    }' \
    --region us-east-1

Note the policy ID from the output, then attach it to an OU:

aws organizations attach-policy \
    --policy-id <policy-id> \
    --target-id <ou-id-or-account-id> \
    --region us-east-1

3. Enforce MFA and Short-Lived Credentials

• Require MFA for all IAM users, especially those with Bedrock access
• Use IAM roles with temporary credentials instead of long-lived access keys
• If access keys are necessary, rotate them regularly

4. Enable Model Invocation Logging

Turn on Bedrock model invocation logging to capture detailed request/response data for forensics.

In the AWS Console:

1. Go to Amazon Bedrock > Settings
2. Under Model invocation logging, click Edit
3. Enable logging and choose your destination (S3 bucket or CloudWatch Logs)
4. Click Save

AWS CLI to enable model invocation logging

First, create an IAM role for Bedrock logging (if you do not have one):

# Create trust policy
cat > /tmp/bedrock-logging-trust.json << 'EOF'
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "bedrock.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}
EOF

# Create the role
aws iam create-role \
    --role-name BedrockLoggingRole \
    --assume-role-policy-document file:///tmp/bedrock-logging-trust.json \
    --region us-east-1

Attach a policy allowing CloudWatch Logs writes:

aws iam attach-role-policy \
    --role-name BedrockLoggingRole \
    --policy-arn arn:aws:iam::aws:policy/CloudWatchLogsFullAccess \
    --region us-east-1

Enable logging:

aws bedrock put-model-invocation-logging-configuration \
    --logging-config '{
        "cloudWatchConfig": {
            "logGroupName": "/aws/bedrock/model-invocations",
            "roleArn": "arn:aws:iam::<your-account-id>:role/BedrockLoggingRole"
        },
        "textDataDeliveryEnabled": true,
        "imageDataDeliveryEnabled": false,
        "embeddingDataDeliveryEnabled": false
    }' \
    --region us-east-1

5. Set Budgets and Quotas

Create AWS Budgets alerts to catch unexpected Bedrock spend early.

In the AWS Console:

1. Go to AWS Budgets (search in the console)
2. Click Create budget
3. Choose Cost budget with a customized setup
4. Set a monthly limit appropriate for your expected Bedrock usage
5. Add alert thresholds (e.g., 50%, 80%, 100% of budget)
6. Configure email notifications

AWS CLI to create a Bedrock budget alert

aws budgets create-budget \
    --account-id <your-account-id> \
    --budget '{
        "BudgetName": "Bedrock-Monthly-Limit",
        "BudgetLimit": {
            "Amount": "500",
            "Unit": "USD"
        },
        "BudgetType": "COST",
        "TimeUnit": "MONTHLY",
        "CostFilters": {
            "Service": ["Amazon Bedrock"]
        }
    }' \
    --notifications-with-subscribers '[
        {
            "Notification": {
                "NotificationType": "ACTUAL",
                "ComparisonOperator": "GREATER_THAN",
                "Threshold": 80,
                "ThresholdType": "PERCENTAGE"
            },
            "Subscribers": [
                {
                    "SubscriptionType": "EMAIL",
                    "Address": "security-alerts@example.com"
                }
            ]
        }
    ]' \
    --region us-east-1

CloudFormation (optional)

This template creates a budget for Bedrock with email alerts:

AWSTemplateFormatVersion: '2010-09-09'
Description: Budget alert for Amazon Bedrock usage

Parameters:
  BudgetAmount:
    Type: Number
    Description: Monthly budget limit in USD
    Default: 500

  AlertEmail:
    Type: String
    Description: Email address for budget alerts

Resources:
  BedrockBudget:
    Type: AWS::Budgets::Budget
    Properties:
      Budget:
        BudgetName: Bedrock-Monthly-Limit
        BudgetLimit:
          Amount: !Ref BudgetAmount
          Unit: USD
        BudgetType: COST
        TimeUnit: MONTHLY
        CostFilters:
          Service:
            - Amazon Bedrock
      NotificationsWithSubscribers:
        - Notification:
            NotificationType: ACTUAL
            ComparisonOperator: GREATER_THAN
            Threshold: 50
            ThresholdType: PERCENTAGE
          Subscribers:
            - SubscriptionType: EMAIL
              Address: !Ref AlertEmail
        - Notification:
            NotificationType: ACTUAL
            ComparisonOperator: GREATER_THAN
            Threshold: 80
            ThresholdType: PERCENTAGE
          Subscribers:
            - SubscriptionType: EMAIL
              Address: !Ref AlertEmail
        - Notification:
            NotificationType: ACTUAL
            ComparisonOperator: GREATER_THAN
            Threshold: 100
            ThresholdType: PERCENTAGE
          Subscribers:
            - SubscriptionType: EMAIL
              Address: !Ref AlertEmail

Deploy with:

aws cloudformation deploy \
    --template-file bedrock-budget.yaml \
    --stack-name bedrock-budget-alert \
    --parameter-overrides BudgetAmount=500 AlertEmail=security-alerts@example.com \
    --region us-east-1

Terraform (optional)

variable "budget_amount" {
  description = "Monthly budget limit in USD"
  type        = number
  default     = 500
}

variable "alert_email" {
  description = "Email address for budget alerts"
  type        = string
}

data "aws_caller_identity" "current" {}

resource "aws_budgets_budget" "bedrock" {
  name         = "Bedrock-Monthly-Limit"
  budget_type  = "COST"
  limit_amount = var.budget_amount
  limit_unit   = "USD"
  time_unit    = "MONTHLY"

  cost_filter {
    name   = "Service"
    values = ["Amazon Bedrock"]
  }

  notification {
    comparison_operator        = "GREATER_THAN"
    threshold                  = 50
    threshold_type             = "PERCENTAGE"
    notification_type          = "ACTUAL"
    subscriber_email_addresses = [var.alert_email]
  }

  notification {
    comparison_operator        = "GREATER_THAN"
    threshold                  = 80
    threshold_type             = "PERCENTAGE"
    notification_type          = "ACTUAL"
    subscriber_email_addresses = [var.alert_email]
  }

  notification {
    comparison_operator        = "GREATER_THAN"
    threshold                  = 100
    threshold_type             = "PERCENTAGE"
    notification_type          = "ACTUAL"
    subscriber_email_addresses = [var.alert_email]
  }
}

Deploy with:

terraform init
terraform plan -var="budget_amount=500" -var="alert_email=security-alerts@example.com"
terraform apply -var="budget_amount=500" -var="alert_email=security-alerts@example.com"

Verification

After implementing preventive controls:

1. Check IAM policies:

   • Review that Bedrock permissions follow least privilege
   • Verify no overly broad bedrock:* policies exist on general-use roles

2. Verify SCPs (if applicable):

   • Confirm the deny policy is attached to the correct OUs/accounts
   • Test that legitimate Bedrock workloads still function

3. Confirm logging is enabled:

   • Go to Amazon Bedrock > Settings and verify model invocation logging shows as enabled

4. Test budget alerts:

   • Create a test budget with a low threshold to verify email delivery

CLI verification commands

Check Bedrock logging configuration:

aws bedrock get-model-invocation-logging-configuration \
    --region us-east-1

List SCPs in your organization:

aws organizations list-policies \
    --filter SERVICE_CONTROL_POLICY \
    --region us-east-1

View policies attached to an OU:

aws organizations list-policies-for-target \
    --target-id <ou-id> \
    --filter SERVICE_CONTROL_POLICY \
    --region us-east-1

Additional Resources

• AWS Documentation: Amazon Bedrock Security Best Practices
• AWS Documentation: Model Invocation Logging
• AWS Documentation: Service Control Policies
• AWS Documentation: AWS Budgets
• Prowler Documentation: LLM Jacking Detection

Notes

• Detection window: This check analyzes activity over a configurable time window. After you remediate, the alert may persist until the detection window passes without suspicious activity.
• False positives: Legitimate high-volume Bedrock usage (e.g., batch processing, load testing) may trigger this check. Review the specific activity before taking action.
• Credential scope: If a compromised credential has broad permissions, the attacker may have done more than just LLM jacking. Always investigate for lateral movement.
• Regional considerations: Bedrock is available in specific regions. Apply SCPs and IAM policies consistently across all regions where Bedrock could be accessed.
• Cost recovery: AWS may provide credits for fraudulent usage in some cases. Contact AWS Support promptly if you experience credential compromise.