Envelope Encryption Pattern

Target Audience: Security Architects, Developers, Operations Engineers

Reading Time: 12 minutes

Prerequisites: Understanding of encryption basics, HSM concepts

Executive Summary

Envelope encryption is a cryptographic pattern where data is encrypted with a Data Encryption Key (DEK), and the DEK itself is encrypted with a Key Encryption Key (KEK) stored in a Hardware Security Module (HSM).

Benefits:

• HSM protection: KEK never leaves HSM (FIPS 140-2/3 certified hardware)
• Performance: Bulk data encrypted with symmetric key (fast)
• Scalability: Single KEK protects unlimited DEKs
• Key rotation: Rotate KEK without re-encrypting all data

ANKASecure uses envelope encryption when integrated with HSMs (SoftHSM, Thales Luna, AWS CloudHSM, nShield).

Table of Contents

1. The Problem: Direct HSM Encryption
2. The Solution: Envelope Encryption
3. How Envelope Encryption Works
4. KEK (Key Encryption Key)
5. DEK (Data Encryption Key)
6. Performance vs Security Tradeoff
7. Key Rotation Strategies
8. ANKASecure Implementation

The Problem: Direct HSM Encryption

Traditional Approach

Encrypt data directly with HSM-stored key:

Plaintext
   ↓
HSM (RSA/ECDH key)
   ↓
Ciphertext

Problems:

1. Performance bottleneck:
2. HSM operations ~100-1000x slower than software crypto
3. Each encryption requires HSM call

4. Limited concurrent HSM sessions (typically 10-50)

5. Scalability limits:

6. HSM throughput: ~500-5,000 ops/second
7. Modern applications need millions of ops/second

8. HSM becomes system bottleneck

9. Size limitations:

10. RSA-2048: Max 190 bytes per operation
11. Large files require chunking and multiple HSM calls
12. Impractical for bulk encryption

Example: Encrypting 1 GB File Directly with HSM

Assumptions:

• HSM: 1,000 RSA operations/second
• Chunk size: 190 bytes (RSA-2048 limit)
• File size: 1 GB = 1,073,741,824 bytes

Calculations:

Chunks needed: 1,073,741,824 / 190 = 5,651,273 chunks
HSM throughput: 1,000 ops/second
Time required: 5,651,273 / 1,000 = 5,651 seconds = 94 minutes

Result: ❌ Unacceptably slow (1.5 hours for 1 GB).

Cost: ❌ Expensive (HSM wear, licensing based on operations).

The Solution: Envelope Encryption

Architecture

Hybrid approach: symmetric encryption + asymmetric key protection

Layer 1: Data Encryption (Fast)
  Plaintext → DEK (AES-256) → Ciphertext
  (Software crypto: ~1 GB/second)

Layer 2: Key Protection (Secure)
  DEK → KEK (HSM RSA/ECDH) → Encrypted DEK
  (Single HSM operation per file)

Key insight: Use HSM for key protection, not data encryption.

Benefits

Visual Comparison

Direct HSM (slow):

File (1 GB) → [Split into 5.6M chunks]
              ↓
           [5.6M HSM calls] → 94 minutes
              ↓
           Ciphertext (1 GB)

Envelope Encryption (fast):

File (1 GB) → [Generate DEK (AES-256)]
              ↓
           [Software AES-GCM] → 1 second
              ↓
           Ciphertext (1 GB)

DEK (32 bytes) → [1 HSM call (KEK)] → Encrypted DEK (256 bytes)

Total time: ~1 second (vs 94 minutes).

How Envelope Encryption Works

Encryption Flow

Step 1: Generate DEK (Data Encryption Key)

DEK ← Random(32 bytes)  // AES-256 key

Step 2: Encrypt plaintext with DEK (software)

Ciphertext ← AES-GCM-256(Plaintext, DEK)

Step 3: Encrypt DEK with KEK (HSM)

EncryptedDEK ← HSM.Encrypt(KEK, DEK)

Step 4: Store both

Storage:
  - Ciphertext (encrypted data)
  - EncryptedDEK (encrypted key, small)
  - IV, Tag (AES-GCM metadata)

Decryption Flow

Step 1: Decrypt DEK with KEK (HSM)

DEK ← HSM.Decrypt(KEK, EncryptedDEK)

Step 2: Decrypt ciphertext with DEK (software)

Plaintext ← AES-GCM-256-Decrypt(Ciphertext, DEK)

Step 3: Zero memory

Zero(DEK)  // Sensitive key material

Security Properties

KEK (Key Encryption Key)

Characteristics

KEK (Key Encryption Key):

• Long-lived: Single KEK protects many DEKs (months/years)
• HSM-bound: Never exported from HSM in plaintext
• Asymmetric: RSA, ECDH, or ML-KEM (post-quantum)
• Backed up: Securely replicated across HSM cluster
• Rotatable: Can be rotated without re-encrypting all data

KEK Storage

HSM storage:

HSM (PKCS#11)
  └── Slot 0
       ├── KEK-prod-2025 (label, never exported)
       │   ├── Public key (exportable)
       │   └── Private key (non-exportable, CKA_EXTRACTABLE=false)
       └── KEK-prod-2024 (old KEK, retained for legacy DEKs)

Attributes:

CKA_LABEL: "KEK-prod-2025"
CKA_TOKEN: TRUE         (persistent, survives HSM restart)
CKA_PRIVATE: TRUE       (authentication required)
CKA_SENSITIVE: TRUE     (cannot be revealed)
CKA_EXTRACTABLE: FALSE  (cannot be exported)
CKA_WRAP_WITH_TRUSTED: TRUE (can only be wrapped by trusted key)

KEK Provisioning

Admin API provisions KEK:

# No endpoint provisions a KEK. HsmKekProvisioningService materializes it eagerly at tenant
# provisioning, or lazily on the tenant's first key-create.

Request:
{
  "tenantId": "tenant-a",
  "algorithm": "RSA-2048",
  "label": "KEK-tenant-a-2025"
}

Response:
{
  "label": "KEK-tenant-a-2025",
  "publicKey": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBg...",
  "status": "provisioned"
}

KEK label published to Redis (by Admin API, as a signed envelope over the full key, an issue timestamp and the label):

ankasecure:kek:tenant:tenant-a-uuid  →  {"payload":"KEK-tenant-a-2025","issuedAt":"…","kid":"…","sig":"…"}

Core API retrieves label:

// During encryption/decryption: the envelope is opened (the HMAC is verified) before the label is
// used; a value that does not verify, or an absent one, is refused — Core API never guesses a label
String kekLabel = projectionVerifier.open(key, redis.get(key)).orElseThrow(...);
// Use kekLabel to reference the HSM key

See: KEK Provisioning Architecture^*

DEK (Data Encryption Key)

Characteristics

DEK (Data Encryption Key):

• Ephemeral: Generated per-file, used once
• Symmetric: AES-256-GCM (fast, authenticated encryption)
• Short-lived: Exists in memory only during encryption/decryption
• Encrypted at rest: Stored encrypted by KEK
• Secure deletion: Zeroed from memory after use

DEK Lifecycle

1. Generation:

// Generate random DEK from the platform DRBG
SecureRandom random = CryptoRandomProvider.get();
byte[] dek = new byte[32];  // 256 bits for AES-256
random.nextBytes(dek);

2. Usage:

// Encrypt data with DEK (software AES-GCM)
Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
GCMParameterSpec gcmSpec = new GCMParameterSpec(128, iv);
cipher.init(Cipher.ENCRYPT_MODE, new SecretKeySpec(dek, "AES"), gcmSpec);
byte[] ciphertext = cipher.doFinal(plaintext);

3. Protection:

// Encrypt DEK with KEK (HSM operation)
byte[] encryptedDEK = hsm.encryptWithKEK(kekLabel, dek);

4. Secure deletion:

// Zero DEK from memory
Arrays.fill(dek, (byte) 0);

5. Storage:

Database or JWE header:
{
  "encrypted_key": "base64url(encryptedDEK)",
  "iv": "base64url(iv)",
  "tag": "base64url(authTag)"
}

DEK Size

Encrypted DEK size (RSA-OAEP):

DEK plaintext: 32 bytes (AES-256)
RSA-2048 encrypted: 256 bytes
RSA-3072 encrypted: 384 bytes
RSA-4096 encrypted: 512 bytes

Overhead: Negligible (~0.025% for 1 MB file).

Performance vs Security Tradeoff

Performance Analysis

Benchmark: Encrypt 1 GB file

Speedup: Envelope encryption is 5,640x faster than direct HSM.

Security Analysis

Attack scenarios:

Conclusion: Envelope encryption provides near-HSM security with near-software performance.

Key Rotation Strategies

KEK Rotation (Rare, Complex)

When to rotate KEK:

• Compliance requirement (e.g., annually)
• KEK compromise suspected
• Algorithm migration (RSA → ML-KEM)

Challenge: All DEKs encrypted with old KEK must be re-encrypted.

Strategy 1: Lazy Re-encryption

1. Generate new KEK in HSM
2. Keep old KEK active (decrypt-only mode)
3. New files encrypted with new KEK
4. On access: If file uses old KEK, re-encrypt DEK with new KEK
5. After transition period: Retire old KEK

Strategy 2: Batch Re-encryption

1. Generate new KEK in HSM
2. Query all files encrypted with old KEK
3. For each file:
    - Decrypt DEK with old KEK (HSM)
    - Re-encrypt DEK with new KEK (HSM)
    - Update database with new encrypted DEK
4. Retire old KEK

Benefit: No plaintext data re-encryption needed (only DEK re-encrypted).

DEK Rotation (Per-File)

DEK is inherently rotated per file.

No explicit rotation needed:

• New file → New DEK generated
• Each file has unique DEK
• Compromise of one DEK does not affect others

Re-encryption (algorithm migration):

1. Decrypt file with old DEK (old algorithm)
2. Generate new DEK (new algorithm)
3. Encrypt file with new DEK
4. Encrypt new DEK with KEK
5. Store new encrypted DEK

ANKASecure re-encryption API:

POST /api/v3/crypto/reencrypt

Request (the source key is read from the JWE header):
{
  "jweToken": { "protected": "...", "recipients": [ ... ], "iv": "...", "ciphertext": "...", "tag": "..." },
  "newKid": "new-key"
}

# Internally:
# 1. Decrypt DEK1 with KEK (HSM)
# 2. Decrypt ciphertext with DEK1 (software, in memory)
# 3. Generate DEK2
# 4. Encrypt plaintext with DEK2 (software, in memory)
# 5. Encrypt DEK2 with KEK (HSM)
# 6. Return new JWE

ANKASecure Implementation

HSM Integration

Supported HSMs:

• SoftHSM: Software-based PKCS#11 (development/testing)
• Thales Luna: Hardware HSM (production)
• AWS CloudHSM: Cloud HSM service
• Entrust nShield: Hardware HSM (production)

Integration method: PKCS#11 via Java Panama FFM (Foreign Function Memory API).

See: HSM Integration Overview^*

JWE Format with Envelope Encryption

JWE header (composite key + HSM-backed):

{
  "enc": "A256GCM",            // DEK algorithm (software)
  "kid": "my-encryption-key",  // the Stable KID of the composite key (HSM-backed components)
  "anka_kdf": "HKDF-SHA256"    // the composite key's KDF token, in the protected header
}

The protected header carries no alg: each recipient header names its component (P-384, ML-KEM-768), because RFC 7516 §7.2.1 keeps the protected and per-recipient header names disjoint.

encrypted_key fields (one recipient per component, each carrying that component's KEM ciphertext):

recipients[0].encrypted_key = c_classical      # the ephemeral P-384 public key
recipients[1].encrypted_key = c_pqc            # the ML-KEM-768 ciphertext
DEK = KeyCombine(K_classical, K_pqc, c_classical, c_pqc, ek_classical, ek_pqc, domain)   # SP 800-227 §4.6.1

Where KEK is:

• Stored in HSM (label: KEK-tenant-a-2025)
• Private key never exported
• Public key used for encryption (software)
• Private key used for decryption (HSM operation)

Caching Strategy

KEK label caching:

L1 Cache (In-Memory): 5 minutes TTL
L2 Cache (Redis): 1 hour TTL
Persistent: Database

Benefit: Avoid Redis lookup on every encryption/decryption.

See: KEK Caching Architecture^*

Monitoring

Metrics tracked:

• KEK encrypt operations/second
• KEK decrypt operations/second
• HSM session utilization
• KEK cache hit rate
• DEK generation rate

Alerts:

• KEK decrypt failures (possible HSM issue)
• High HSM latency (>100ms)
• Cache miss rate >10% (investigate)

See: HSM Monitoring^*

Quick Reference

Envelope Encryption vs Direct HSM

When to Use Envelope Encryption

Related Documentation

HSM Integration

• HSM Integration Overview^*
• KEK Provisioning Architecture^*
• KEK Caching Architecture^*
• Envelope Encryption Architecture^*
• HSM Deployment Planning^*
• HSM Monitoring^*

Key Management

• Key Management Overview^*
• Key Rotation Strategies^*
• Keystore Providers^*

Foundational Concepts

• JOSE Technical Reference
• Policy-Driven Crypto-Agility

Document Status: ✅ ACTIVE

Last Updated: January 2025

Document Location: 02-concepts/envelope-encryption-pattern.md

Target Audience: Security Architects, Developers, Operations Engineers

Feedback: For corrections or additions, contact the documentation team.