PQC Transport Handshake in the AnkaSecure SDK

Feature: PQC Transport Layer Algorithms: ML-KEM-512/768/1024, HQC-128/192/256, FRODO-976/1344 Default: ML-KEM-768 Status: Optional (configurable)

This document describes the SDK-specific implementation of the Post-Quantum Cryptography (PQC) transport handshake. For the complete architectural overview, security model, and protocol specification, see the PQC Transport Layer Architecture^*.

Overview

The AnkaSecure SDK implements a transparent PQC transport layer that automatically encrypts API payloads using quantum-resistant cryptography. This provides defense-in-depth on top of TLS 1.3.

Key Benefits:

• Quantum Resistance: Uses post-quantum KEMs (ML-KEM, HQC or FrodoKEM) designed to resist quantum attacks
• Defense-in-Depth: Adds encryption layer beyond TLS 1.3
• Transparent to Developers: No code changes required — enable via configuration
• Configurable: Algorithm, TTL, and endpoints adjustable via properties
• Session Reuse: Handshake overhead amortized across multiple requests
• Thread-Safe: Concurrent requests handled safely with session locking

How to Enable:

Add to your SDK configuration file (cli.properties or sdk-config.properties):

# Enable PQC transport layer (the SDK default is false; set it explicitly to turn it on)
pqc.transport.enabled=true

# Algorithm selection (default: ML-KEM-768)
pqc.kem.algorithm=ML-KEM-768

# Session TTL in seconds (the SDK default is 600 = 10 minutes; 14400 is a common override)
pqc.session.ttl.seconds=14400

No code changes required — the SDK automatically encrypts all eligible API requests.

For detailed information:

• Why PQC transport exists → PQC Transport Architecture: Why It Exists^*
• Security model and threat analysis → PQC Transport Architecture: Security Model^*
• Server-side session management → PQC Transport Architecture: Session Management^*

SDK Architecture

flowchart TB
    subgraph App["Application Code<br/>(uses AnkaSecureSdk)"]
    end
    subgraph Client["AnkaSecureOpenApiClient<br/>– applies auth token<br/>– applies PQC interceptor"]
    end
    subgraph Interceptor["PqcTransportInterceptor<br/>– performs handshake<br/>– encrypts request bodies<br/>– decrypts responses"]
    end
    subgraph Handshake["Server PQC Handshake<br/>– encapsulates AES key<br/>– returns {sessionId,…}"]
    end
    subgraph Protected["Protected REST APIs<br/>– decrypt request payload<br/>– process business logic<br/>– encrypt response payload"]
    end

    App -->|invoke SDK methods| Client
    Client -->|HTTP calls via OkHttp| Interceptor
    Interceptor -->|POST /api/pqc-handshake/init unmodified| Handshake
    Handshake -->|subsequent requests carry<br/>X-PQC-Session| Protected

SDK Components

1. PqcTransportConfig

Configuration reader for PQC transport settings.

Purpose: Loads and validates PQC transport configuration from properties file

Properties:

Validation:

• If pqc.transport.enabled=false, all PQC operations are disabled
• With the transport enabled, a pqc.kem.algorithm outside the supported set is refused when the client is built (InvalidInputException naming the supported set); it never falls back to the default
• TTL must be positive integer (minimum 60 seconds recommended)

2. PqcSessionManager

Orchestrates the PQC handshake and manages session lifecycle.

Purpose: Establishes and caches PQC transport sessions for encrypting API payloads

Handshake Process:

1. Generate Key Pair: Creates a key pair for the configured KEM client-side
2. Send Public Key: Calls handshake endpoint with Base64-encoded public key
3. Receive Encapsulated Key: Server returns KEM ciphertext containing AES session key
4. Decapsulate: Client uses private key to extract AES key
5. Cache Session: Stores session ID, AES key, and creation timestamp

Session Management:

• Active Session: Cached session used for all subsequent requests
• Session Expiry: Based on TTL configuration (SDK default 600 s = 10 minutes)
• Invalidation: Explicitly invalidated during authentication to force fresh handshake
• Reuse: Same session used for multiple requests to avoid handshake overhead

Thread Safety:

Uses ReentrantLock to prevent concurrent handshakes when multiple threads request sessions simultaneously. Only one handshake occurs at a time; other threads wait and reuse the newly established session.

3. PqcTransportInterceptor

OkHttp interceptor that transparently encrypts/decrypts HTTP request/response bodies.

Purpose: Automatically applies PQC transport encryption to eligible API calls

Interception Flow:

Phase 1 - Request Processing:

1. Check eligibility: Determine if request should be encrypted
2. Obtain session: Get active PQC session or trigger handshake if needed
3. Encrypt payload: Apply AES-GCM encryption to JSON request body
4. Modify headers:
   • Change Content-Type to application/octet-stream
   • Add X-PQC-Transport: true header
   • Add X-PQC-Session: <sessionId> header
5. Send request: Forward encrypted binary payload to server

Phase 2 - Response Processing: the response is classified by its Content-Type and, for a problem document, by whether its status is an error.

1. application/octet-stream: the body is decrypted with the session AES key whatever the status, so an encrypted 4xx or 5xx reaches the caller as the service's own problem. A body that does not open, and an empty encrypted body (the services never encrypt an empty body, and it carries no authentication tag to check), are refused with PqcTransportException (UNDECRYPTABLE_RESPONSE). A network failure while the body is read (a read timeout, a reset connection) is not a decryption failure: it reaches the caller as the I/O error it is, so a read timeout stays SdkErrorCode.TIMEOUT.
2. application/problem+json on a 4xx or 5xx status: a refusal written in plain text before the request was decrypted (the session is unknown, the body did not open, the session store could not be read), or by a filter ahead of the transport layer (401, 403, 413, 421). It is never decrypted and reaches the caller as AnkaSecureSdkException with the problem's status; the caller reads type and extensions.code from the problem document in getResponseBody(), since the exception has no accessor for them.
3. application/problem+json on any other status: refused with PqcTransportException (UNENCRYPTED_RESPONSE) and never read. A refusal is never a success, so a plain-text problem with a 2xx status is what a forging intermediary would send.
4. Any other content type, or none: refused with PqcTransportException (UNENCRYPTED_RESPONSE), including an empty 2xx or 204 without a media type. An unencrypted body is never handed to the caller as if it had been decrypted.

A lost session is re-established once. When a plain-text 400 problem carries extensions.code pqc.session.notfound (core-api, auth-api and admin-api all carry it), the interceptor replaces that session with one new handshake and sends the original request once more, encrypted under the new key. A second refusal is returned to the caller; no other problem is retried, and the request is never sent in plain text.

Bypass Conditions: the interceptor wraps every request that carries a body except those listed under Excluded Endpoints; the OAuth 2.0 token grant is wrapped.

4. AnkaSecureOpenApiClient Integration

The SDK's high-level API methods integrate seamlessly with the PQC transport layer.

Integration Points:

OkHttp Configuration:

The OpenAPI client configures OkHttp with the PqcTransportInterceptor during initialization. This ensures all HTTP requests pass through the PQC encryption layer before reaching the network.

Authentication Invalidation:

The SDK automatically invalidates PQC sessions at the start of authentication (authenticateApplication()). This ensures:

• Fresh session keys for each authentication
• No stale sessions after credential changes
• Clean separation between authentication phases

Transparency:

All SDK methods (encrypt(), encryptFileStream(), signFile(), etc.) work identically whether PQC transport is enabled or disabled. The encryption happens at the HTTP layer, completely transparent to business logic.

Session Lifecycle

sequenceDiagram
    participant SDK as AnkaSecureSdk
    participant Inter as PqcTransportInterceptor
    participant Sess as PqcSessionManager
    participant API as AnkaSecureOpenApiClient
    participant Server

    SDK->>API: authenticateApplication()
    API->>Sess: invalidateSession()
    API->>Inter: HTTP POST /authenticate (body JSON)
    Inter->>Sess: lease() → no session
    Sess->>API: performHandshake(pubKeyB64)
    API->>Server: POST /api/pqc-handshake/init
    Server-->>API: {ciphertextBase64, sessionId}
    API-->>Sess: decapsulate + cache AES key
    Sess-->>Inter: session + private copy of its key
    Inter->>Server: encrypted request (AES-GCM)
    Server-->>Inter: encrypted response
    Inter->>SDK: decrypted JSON response
    Note over Inter,Server: Later, the server no longer holds the session
    Inter->>Server: encrypted request (same session)
    Server-->>Inter: 400 problem+json, extensions.code pqc.session.notfound
    Inter->>Sess: replaceIfCurrent(session)
    Sess->>API: performHandshake(pubKeyB64), once for every request on that session
    Sess-->>Inter: new session + private copy of its key
    Inter->>Server: original body, encrypted under the new key
    Server-->>Inter: encrypted response

Lifecycle States:

1. First Encrypted Call: No active session → handshake triggered
2. Subsequent Calls (within TTL): Active session reused; each request leases a private copy of the session key and wipes it when the request ends
3. Session Expiry or Manual Invalidate: Next call triggers new handshake
4. Lost Session: A plain-text 400 carrying pqc.session.notfound evicts the session (its key is wiped) and triggers exactly one handshake, shared by every request that failed on that session; the original request is sent once more under the new key
5. Auth Calls: SDK explicitly invalidates to ensure fresh key material

For server-side session storage details, see Redis Architecture: PQC Transport Sessions^*

Request/Response Encryption

Request Encryption Process

When the PqcTransportInterceptor processes an outbound request:

Step 1: Encrypt Payload

The original JSON request body is encrypted using AES-GCM with the session's shared AES key. The encryption produces binary ciphertext that includes an initialization vector (IV) and authentication tag for integrity validation.

Step 2: Modify Request

The interceptor creates a new HTTP request with:

• Content-Type: Changed from application/json to application/octet-stream
• X-PQC-Transport: Header set to true (signals server to decrypt)
• X-PQC-Session: Header containing the session ID for key lookup
• Body: Binary encrypted payload (replaces original JSON)

Step 3: Send to Server

The encrypted request is sent to the server, where the PQC handshake API decrypts it using the session's AES key stored in Redis.

Response Decryption Process

When the server returns an encrypted response:

Step 1: Classify the Response

Only a non-empty application/octet-stream body is decrypted; an application/problem+json body on a 4xx or 5xx status is a plain-text refusal handed to the caller as it is, and any other body, a problem on any other status included, is refused (see Phase 2 above).

Step 2: Decrypt Payload

The binary ciphertext is decrypted using AES-GCM with the session's shared AES key. The authentication tag is verified to ensure integrity and authenticity.

Step 3: Reconstruct JSON Response

The interceptor creates a new HTTP response with:

• Content-Type: Restored to application/json
• Headers: X-PQC-Transport and X-PQC-Session headers removed
• Body: Decrypted JSON payload

Step 4: Return to Application

The decrypted JSON response is returned to the SDK method caller, completely transparent to the application code.

Configuration & Extension

Enabling/Disabling PQC Transport

Enable PQC transport (recommended for production):

pqc.transport.enabled=true

Disable PQC transport (for testing or debugging):

pqc.transport.enabled=false

When disabled, all API calls use standard JSON over TLS without the additional PQC encryption layer.

Algorithm Configuration

Configure the KEM algorithm used for key encapsulation:

# Available options: ML-KEM-512, ML-KEM-768, ML-KEM-1024,
#                    HQC-128, HQC-192, HQC-256, FRODO-976, FRODO-1344
pqc.kem.algorithm=ML-KEM-768

Algorithm selection:

The set is fixed in the SDK (PqcTransportConfig.supportedKemAlgorithms()), not read from the server's GET /api/v3/pqc-handshake/supported-algorithms. Classic McEliece is not offered: the BSI TR-02102 advisory recommends against it for new developments, and its public keys (up to about 1 MB) do not suit a handshake that is repeated on every session.

Each KEM uses its own Bouncy Castle route: ML-KEM and FrodoKEM on the BC provider, HQC on BCPQC (requested by its family name with its parameter set). The handshake sends the catalogue name, and the server must offer that KEM.

Session TTL Tuning

Adjust session lifetime based on your requirements:

# 600 seconds = 10 minutes -- the SDK default (PqcTransportConfig)
pqc.session.ttl.seconds=600

# 1 hour - fewer handshakes, keys live longer
pqc.session.ttl.seconds=3600

# 4 hours - the longest value in common use
pqc.session.ttl.seconds=14400

Considerations:

• Shorter TTL: More frequent handshakes, better security (fresh keys)
• Longer TTL: Fewer handshakes, better performance
• Default (600 seconds): the SDK's own default, and a deliberately short one

Endpoint Configuration

Customize the handshake endpoint if server path changes:

# Default
pqc.handshake.endpoint=/api/pqc-handshake/init

# Custom endpoint (if the server relocates it)
pqc.handshake.endpoint=/api/v3/pqc-handshake/init

Excluded Endpoints

The following requests bypass PQC encryption; every other request that carries a body is wrapped, the OAuth 2.0 token grant on /api/v3/auth/token included (Auth API decrypts it like Core API and Admin API do).

Always excluded:

• Handshake endpoint: the configured pqc.handshake.endpoint
• Streaming endpoints: /api/v3/crypto/stream/**
• Interoperability endpoints: /api/v3/interoperability/** (they take a caller-supplied key)

Excluded request types:

• Requests with no body (GET, DELETE)
• Streamed (duplex) bodies, whatever the path: a streamed body cannot be read whole without truncating it, so /api/v3/migration/stream/** travels under TLS alone, and the SDK logs one warning per process the first time it skips one while PQC transport is enabled

Under PQC transport, streaming is an explicit choice the caller makes. A streaming route is protected by TLS only, so while PQC transport is enabled the SDK never moves an operation there on its own. The auto-selecting producer operations (encryptFile, signFile) do not route by the compact limit, which is read through a request PQC transport does not wrap, and they fall back to streaming only on a 413 whose body decrypted under the session key: the service's own answer. Every other 413 is unauthenticated (a plain-text problem, an HTML error page, a body with no media type, an empty or undecryptable encrypted body) and could come from anything between the client and the service. A payload at or above the compact limit, or any unauthenticated 413, is refused with PqcTransportException (STREAMING_NOT_CHOSEN), whose message names the streaming operation to call explicitly (for example encryptFileStream); nothing is retried and the cached compact limit is kept. A streaming artifact passed to decryptFile or reencryptFile, recognised by its content rather than its file name, is processed over the TLS-only streaming route, as it was when it was encrypted; a compact envelope stays on the PQC-protected compact route. Without PQC transport, the SDK selects the route by the compact limit and falls back to streaming on a 413.

These exclusions ensure the transport layer doesn't interfere with endpoints that require specific content types or have no payload to encrypt.

Error Handling & Logging

Concurrency Protection

PqcSessionManager uses ReentrantLock to prevent concurrent handshakes when multiple threads try to establish sessions simultaneously. Replacing a lost session is single-flight: when several requests fail on the same session, one handshake replaces it and the others reuse the new one. Each request works from its own copy of the session key, wiped when the request ends, and a replaced session's key is wiped when it is evicted.

Error Types

Logging Levels

Troubleshooting Tips

"Tag mismatch" on second login

Cause: Multiple interceptor instances or stale session managers.

Resolution: Ensure only one PqcSessionManager and interceptor are registered per AnkaSecureOpenApiClient.

Large-file streaming errors

Cause: Multipart headers misconfigured.

Resolution: Verify client-side multipart builder matches controller's @RequestPart("header") and @RequestPart("file").

Debugging Handshakes

Enable debug logging:

Configure your logging framework to show DEBUG level for the SDK logger:

Logback example:

<logger name="co.ankatech.cli.dev" level="DEBUG"/>

What to inspect:

• Session ID generation and caching
• Base64 encoded KEM ciphertext
• Session expiration and renewal
• Handshake success/failure events

Verification steps:

1. Confirm sessionId cycles correctly after invalidateSession()
2. Check that session is reused for multiple requests
3. Verify new handshake occurs after TTL expiry
4. Monitor handshake frequency vs expected TTL

Upgrading KEM Algorithms

Selecting another supported KEM needs no code change: set pqc.kem.algorithm and the next handshake uses it.

Adding a KEM to the supported set is an SDK change:

1. Add its route (JCA name, provider and, for a family name, its parameter set) to the SDK's AsymmetricAlgorithmRegistry
2. Confirm the PQC Handshake API offers it (GET /api/v3/pqc-handshake/supported-algorithms)
3. Test a handshake and a session-key unwrap with it against a deployed environment

Coordination:

Both client and server must support the selected algorithm. If there's a mismatch, handshake will fail with HTTP 400 Bad Request.

Related Documentation

Architecture Documentation

PQC Transport Layer:

• PQC Transport Layer Architecture^* - Complete architecture, security model, and protocol specification
• Redis Architecture: PQC Sessions^* - Server-side session storage
• PQC Handshake API^* - Server-side handshake implementation

System Architecture:

• Component Architecture^* - System component overview
• PQC Transport Architecture^* - PQC transport layer details

SDK Documentation

Getting Started:

• SDK Overview - General SDK documentation and features
• SDK Usage Guide - Setup, authentication, and usage patterns
• Integration Flows - 21 complete integration examples

Configuration:

• OpenAPI Client - Underlying HTTP client configuration

Security & Compliance

Cryptographic Standards:

• Algorithm Catalog - 89 simple algorithm profiles and 41 composite pairings, including the ML-KEM parameter sets
• FIPS 140-3, Random Bit Generation and Approved Algorithms - ML-KEM and the other NIST-approved algorithms

Last Updated: April 2026

Target Audience: SDK Developers, Integration Engineers, Security Architects

© 2026 AnkaTech. All rights reserved.