Monitoring & Validation

Purpose: Health monitoring, success criteria, and migration validation procedures

Version: 3.0.0

Migration Health Monitoring

Key Import Rate Tracking

Purpose: Monitor progress of key import operations

Metrics to Track:

Metric	Description	Target	Alert Threshold
Keys imported/day	Import throughput	20-50 keys/day	< 10 keys/day
Import success rate	Successful imports / total attempts	> 98%	< 95%
Average import time	Time per PKCS#12 import	< 5 seconds	> 15 seconds
Failed imports	Count of failed imports	< 2%	> 5%

Monitoring Dashboard:

graph TD
    Metrics["Collect Metrics"] -->|Aggregate| Dashboard["Migration Dashboard"]

    Dashboard -->|Display| Panel1["Keys Imported<br/>Progress: 150/200"]
    Dashboard -->|Display| Panel2["Success Rate<br/>98.5%"]
    Dashboard -->|Display| Panel3["Average Time<br/>3.2 seconds"]
    Dashboard -->|Display| Panel4["Failed Imports<br/>3 failures"]

    Panel4 -->|If > 5| Alert["🔴 Alert:<br/>High failure rate"]

    style Panel1 fill:#51cf66,stroke:#333,stroke-width:2px
    style Panel2 fill:#51cf66,stroke:#333,stroke-width:2px
    style Panel3 fill:#51cf66,stroke:#333,stroke-width:2px
    style Alert fill:#ff6b6b,stroke:#333,stroke-width:2px

Certificate Expiration Alerting

Purpose: Proactive alerts for upcoming certificate expirations

Alert Thresholds:

graph TD
    Cert["Certificate"] -->|Check| Days{"Days until<br/>expiration"}

    Days -->|< 7 days| Critical["🔴 CRITICAL<br/>Expire within week"]
    Days -->|7-30 days| High["🟠 HIGH<br/>Expire within month"]
    Days -->|31-90 days| Medium["🟡 MEDIUM<br/>Expire within quarter"]
    Days -->|> 90 days| Low["🟢 LOW<br/>Monitor only"]

    Critical -->|Action| Immediate["Immediate migration required"]
    High -->|Action| Urgent["Schedule migration this month"]
    Medium -->|Action| Planned["Include in next sprint"]

    style Critical fill:#ff6b6b,stroke:#333,stroke-width:2px
    style High fill:#ff9933,stroke:#333,stroke-width:2px
    style Medium fill:#ffd43b,stroke:#333,stroke-width:2px

Alert Notification:

{
  "alertType": "CERTIFICATE_EXPIRATION",
  "severity": "HIGH",
  "kid": "rsa-encryption-legacy",
  "expiresAt": "2025-01-15T23:59:59Z",
  "daysRemaining": 22,
  "recommendation": "Migrate to ML-KEM within 2 weeks",
  "affectedApplications": ["Payment Gateway", "Document Storage"]
}

Validation Failure Monitoring

Purpose: Track and resolve import validation failures

Common Validation Failures:

Failure Type	Frequency	Resolution Time	Prevention
Expired certificate (STRICT mode)	High (40%)	< 1 hour	Use IMPORT_ONLY mode
Invalid password	Medium (20%)	< 1 day	Verify passwords before import
Corrupt PKCS#12 file	Low (5%)	1-3 days	Validate files with openssl first
Trust anchor missing	Low (10%)	< 1 hour	Configure trust anchors directory
File size exceeded	Low (5%)	< 1 hour	Increase MIGRATION_MAX_PKCS12_SIZE

Failure Trend Analysis:

graph LR
    Week1["Week 1<br/>25 failures"] -->|Improve| Week2["Week 2<br/>15 failures"]
    Week2 -->|Improve| Week3["Week 3<br/>8 failures"]
    Week3 -->|Improve| Week4["Week 4<br/>3 failures"]

    Week4 -.->|Target| Target["Target: < 2%<br/>failure rate"]

    style Week1 fill:#ff6b6b,stroke:#333,stroke-width:2px
    style Week4 fill:#51cf66,stroke:#333,stroke-width:2px
    style Target fill:#4dabf7,stroke:#333,stroke-width:2px

Performance Metrics

Purpose: Ensure migration operations meet performance expectations

Key Performance Indicators (KPIs):

Operation	Metric	Target	Current	Status
PKCS#12 import	Latency	< 5s	3.2s	✅
Certificate validation	Latency	< 2s	1.5s	✅
Data re-encryption	Throughput	> 100 MB/s	150 MB/s	✅
Signature conversion	Throughput	> 1,000 docs/min	1,500 docs/min	✅

Success Criteria Definition

Technical Success Metrics

Criteria for Migration Success:

graph TD
    Migration["Migration Complete"] -->|Validate| Technical{"Technical<br/>Criteria"}

    Technical -->|Check| Imported{"All keys<br/>imported?"}
    Imported -->|Yes: 200/200| Analyzed{"All keys<br/>analyzed?"}
    Imported -->|No| FailImport["❌ Incomplete import"]

    Analyzed -->|Yes: 100%| Converted{"All data<br/>converted?"}
    Analyzed -->|No| FailAnalyze["❌ Analysis incomplete"]

    Converted -->|Yes: > 99%| Performance{"Performance<br/>acceptable?"}
    Converted -->|No| FailConvert["❌ Conversion incomplete"]

    Performance -->|Yes: Within 10%| Security{"Security<br/>incidents?"}
    Performance -->|No| FailPerf["❌ Performance degraded"]

    Security -->|No: 0 incidents| TechSuccess["✅ Technical Success"]
    Security -->|Yes| FailSec["❌ Security issue"]

    style TechSuccess fill:#51cf66,stroke:#333,stroke-width:2px
    style FailImport fill:#ff6b6b,stroke:#333,stroke-width:2px
    style FailConvert fill:#ff6b6b,stroke:#333,stroke-width:2px

Technical Success Checklist:

[ ] 100% of keys imported successfully
[ ] > 99% of data re-encrypted/re-signed
[ ] Performance within 10% of baseline (or better)
[ ] Zero security incidents during migration
[ ] All validation tests passed
[ ] Rollback procedure tested

Business Success Metrics

Criteria for Business Success (the Actual column is an illustrative example of a completed report, not a measured customer result):

Metric	Target	Actual (example)	Status
Timeline adherence	Complete within 6 months	5 months	✅ Ahead of schedule
Budget adherence	Within $100K budget	$85K	✅ Under budget
Stakeholder satisfaction	> 80% approval	92%	✅ Exceeded
Compliance evidence produced	Evidence for each applicable framework (NIST, PCI DSS, HIPAA)	All delivered to the assessor	✅ Delivered
Zero downtime	99.9% uptime maintained	99.95%	✅ Exceeded
Cost savings vs. traditional	> 80% reduction	87.5%	✅ Significant savings

Compliance Checkpoints

Regulatory Validation:

graph LR
    Migration["Migration Complete"] -->|Validate| NIST{"NIST SP 800-208<br/>Compliant?"}

    NIST -->|Yes| EO{"Executive Order 14144<br/>Compliant?"}
    NIST -->|No| FixNIST["Fix NIST compliance"]

    EO -->|Yes| NSA{"NSA CNSA 2.0<br/>Ready?"}
    EO -->|No| FixEO["Fix EO compliance"]

    NSA -->|Yes| Industry{"Industry Requirements<br/>PCI, HIPAA, etc."}
    NSA -->|No| FixNSA["Fix NSA compliance"]

    Industry -->|Yes| ComplianceSuccess["✅ Full Compliance<br/>Achieved"]
    Industry -->|No| FixIndustry["Fix industry compliance"]

    style ComplianceSuccess fill:#51cf66,stroke:#333,stroke-width:2px

Validation Procedures

Pre-Cutover Testing Checklist

Perform before final cutover:

[ ] Functional Testing:
[ ] Decrypt all sample encrypted data with PQC keys
[ ] Verify all sample signatures with PQC keys
[ ] Test all API endpoints with PQC keys
[ ] Validate error handling (invalid ciphertexts, signatures)
[ ] Performance Testing:
[ ] Measure encryption/decryption latency (target: < 10ms)
[ ] Measure signing/verification latency (target: < 5ms)
[ ] Load test with concurrent operations (target: 1,000 ops/sec)
[ ] Stress test with peak load (target: no errors under 2x normal load)
[ ] Security Testing:
[ ] Verify HSM envelope encryption active (all keys wrapped)
[ ] Test tenant isolation (no cross-tenant key access)
[ ] Validate JWT scope enforcement (unauthorized ops rejected)
[ ] Check audit trail completeness (all operations logged)
[ ] Integration Testing:
[ ] Test cross-API workflows (Admin → Core → License)
[ ] Validate external system compatibility (partners can verify PQC signatures)
[ ] Test rollback procedure (switch back to legacy keys)
[ ] Verify monitoring dashboards and alerts working
[ ] Compliance Testing:
[ ] NIST algorithm compliance (ML-KEM, ML-DSA, SLH-DSA)
[ ] Audit trail meets regulatory requirements
[ ] Data retention policies enforced
[ ] Incident response procedures documented

Production Validation

Post-cutover validation (first 24 hours):

graph TD
    Cutover["Cutover Complete<br/>T-0"] -->|Hour 1| H1["Health Check<br/>All services green?"]

    H1 -->|Yes| H4["Hour 4<br/>Monitor error rates"]
    H1 -->|No| Rollback1["⚠️ Consider rollback"]

    H4 -->|Error rate < 1%| H12["Hour 12<br/>Monitor performance"]
    H4 -->|Error rate ≥ 1%| Rollback2["⚠️ Investigate errors"]

    H12 -->|Latency acceptable| H24["Hour 24<br/>Full validation"]
    H12 -->|Latency degraded| Investigate["Investigate performance"]

    H24 -->|All metrics green| Success["✅ Cutover Successful<br/>Continue monitoring"]
    H24 -->|Issues detected| Rollback3["⚠️ Rollback decision"]

    style Success fill:#51cf66,stroke:#333,stroke-width:2px
    style Rollback1 fill:#ff6b6b,stroke:#333,stroke-width:2px
    style Rollback2 fill:#ffd43b,stroke:#333,stroke-width:2px

Rollback Decision Criteria

Trigger rollback if:

Criterion	Threshold	Action
Error rate	> 2% for > 1 hour	Immediate rollback
Performance degradation	> 30% latency increase	Investigate, rollback if not resolved in 2 hours
Security incident	Any PQC-related incident	Immediate rollback
Data corruption	Any data loss detected	Immediate rollback
Compliance violation	Regulatory requirement not met	Immediate rollback

Operational Monitoring

Dashboard Setup

Migration Dashboard Components:

graph TD
    Dashboard["Migration Dashboard"] -->|Panel 1| Progress["Import Progress<br/>Gauge: 150/200 keys"]
    Dashboard -->|Panel 2| Success["Success Rate<br/>Line chart: 98.5%"]
    Dashboard -->|Panel 3| Performance["Performance<br/>Bar chart: Latency by operation"]
    Dashboard -->|Panel 4| Errors["Error Log<br/>Table: Recent failures"]

    Progress -->|Alert| ProgressAlert{"< 95%<br/>completion expected?"}
    Success -->|Alert| SuccessAlert{"< 98%<br/>success rate?"}
    Performance -->|Alert| PerfAlert{"> 20%<br/>degradation?"}
    Errors -->|Alert| ErrorAlert{"> 10<br/>errors/hour?"}

    ProgressAlert -->|Yes| Notify["Notify migration team"]
    SuccessAlert -->|Yes| Notify
    PerfAlert -->|Yes| Notify
    ErrorAlert -->|Yes| Notify

    style Dashboard fill:#4dabf7,stroke:#333,stroke-width:2px
    style Notify fill:#ff6b6b,stroke:#333,stroke-width:2px

Alert Configuration

Alert Severity Levels:

Severity	Condition	Response Time	Notification
Critical	Import failure rate > 10%	< 15 minutes	SMS + Email + Dashboard
High	Performance degradation > 30%	< 1 hour	Email + Dashboard
Medium	Certificate expiring < 7 days	< 4 hours	Email
Low	Certificate expiring < 30 days	< 24 hours	Dashboard only

Alert Example:

{
  "alertId": "ALERT-2025-001",
  "severity": "CRITICAL",
  "timestamp": "2025-01-15T14:30:00Z",
  "type": "IMPORT_FAILURE_RATE",
  "message": "Import failure rate: 12.5% (25 failures / 200 attempts)",
  "affectedKeys": ["rsa-key-1", "rsa-key-2", "..."],
  "recommendedAction": "Check certificate validation mode, review error logs",
  "escalationPath": "[email protected] → [email protected]"
}

Capacity Planning

Purpose: Ensure infrastructure can handle migration load

Resource Monitoring:

Resource	Current Usage	Max Capacity	Headroom	Status
CPU (Core API)	45%	100%	55%	✅ OK
Memory (Core API)	6 GB	8 GB	2 GB	✅ OK
PostgreSQL connections	120	200	80	✅ OK
Redis memory	2 GB	4 GB	2 GB	✅ OK
Disk I/O	400 MB/s	1 GB/s	600 MB/s	✅ OK

Scaling Triggers:

CPU > 70% sustained: Scale horizontally (add Core API replicas)
Memory > 80%: Increase container memory limits
PostgreSQL connections > 150: Increase connection pool
Redis memory > 3 GB: Increase Redis memory or add replicas

Audit & Compliance

Migration Audit Trail

Purpose: Complete audit trail for compliance and troubleshooting

Audit Events Captured:

graph TD
    Event["Migration Event"] -->|Log| Type{"Event Type"}

    Type -->|Import| ImportEvent["Key Import Event"]
    Type -->|Analyze| AnalyzeEvent["Analysis Event"]
    Type -->|Convert| ConvertEvent["Conversion Event"]
    Type -->|Validate| ValidateEvent["Validation Event"]

    ImportEvent -->|Record| AuditDB["(Audit Database)"]
    AnalyzeEvent -->|Record| AuditDB
    ConvertEvent -->|Record| AuditDB
    ValidateEvent -->|Record| AuditDB

    AuditDB -->|Query| Reports["Audit Reports"]

    Reports -->|Generate| Compliance["Compliance Report"]
    Reports -->|Generate| Security["Security Review"]
    Reports -->|Generate| Operations["Operational Metrics"]

    style AuditDB fill:#4dabf7,stroke:#333,stroke-width:2px
    style Compliance fill:#51cf66,stroke:#333,stroke-width:2px

Audit Log Entry Example:

{
  "eventId": "EVT-2025-001234",
  "timestamp": "2025-01-15T10:30:00Z",
  "eventType": "KEY_IMPORT",
  "tenantId": "enterprise-tenant",
  "userId": "[email protected]",
  "correlationId": "a1b2c3d4-e5f6-g7h8-i9j0-k1l2m3n4o5p6",
  "operation": "POST /api/v3/admin/tenants/{tenantId}/keys/import-keystore",
  "kid": "rsa-encryption-legacy",
  "algorithm": "RSA-2048",
  "validationMode": "IMPORT_ONLY",
  "certificateExpiry": "2023-12-31T23:59:59Z",
  "result": "SUCCESS",
  "status": "expired",
  "origin": "IMPORTED",
  "restricted": true,
  "keyOps": ["decrypt", "verify"]
}

Key Lineage Tracking

Purpose: Track origin and transformation of keys

Lineage Chain:

graph LR
    Origin["Origin:<br/>VeriSign PKCS#12<br/>2020-01-01"] -->|Imported| Imported["Imported Key:<br/>rsa-encryption-legacy<br/>2025-01-15"]

    Imported -->|Generated successor| PQC["PQC Key:<br/>mlkem-encryption-2025<br/>2025-01-20"]

    Imported -.->|Rotated to| PQC

    PQC -->|May rotate to| Future["Future PQC<br/>Next-gen algorithm<br/>20XX"]

    Origin -.->|Lineage trail| AuditTrail["Audit Trail:<br/>Full key history"]
    Imported -.->|Lineage trail| AuditTrail
    PQC -.->|Lineage trail| AuditTrail

    style Origin fill:#ffcccc,stroke:#333,stroke-width:2px
    style Imported fill:#ffd43b,stroke:#333,stroke-width:2px
    style PQC fill:#51cf66,stroke:#333,stroke-width:2px

Lineage Record:

{
  "kid": "mlkem-encryption-2025",
  "lineage": [
    {
      "generation": 0,
      "kid": "rsa-encryption-legacy",
      "algorithm": "RSA-2048",
      "origin": "PKCS#12 import from VeriSign CA",
      "importedAt": "2025-01-15T10:00:00Z",
      "status": "rotated"
    },
    {
      "generation": 1,
      "kid": "mlkem-encryption-2025",
      "algorithm": "ML-KEM-768",
      "origin": "Rotated from rsa-encryption-legacy",
      "createdAt": "2025-01-20T15:00:00Z",
      "status": "active"
    }
  ]
}

Regulatory Reporting

Purpose: Generate reports for regulatory compliance

Report Types:

Report	Frequency	Recipient	Content
Migration Progress	Weekly	Management	Keys imported, data converted, timeline adherence
Compliance Status	Monthly	Compliance Officer	NIST, PCI-DSS, HIPAA alignment
Security Review	Quarterly	CISO	Security incidents, audit findings, recommendations
Annual Certification	Yearly	External Auditors	Complete migration audit trail, key lineage

Compliance Report Example:

# Migration Compliance Report - Q1 2025

## Summary

- Total applications migrated: 200
- Keys imported: 200
- Data re-encrypted: 1.2 TB
- Compliance frameworks met: NIST SP 800-208, Executive Order 14144, NSA CNSA 2.0

## NIST Compliance

- ✅ ML-KEM-768/1024 deployed (NIST FIPS 203)
- ✅ ML-DSA-65/87 deployed (NIST FIPS 204)
- ✅ All legacy RSA < 2048 retired

## Security Incidents

- Zero security incidents during migration
- Zero data loss events
- Zero unauthorized access attempts

## Audit Trail

- 100% of operations logged with correlation IDs
- Full key lineage tracked
- Retention period: 7 years per PCI-DSS 4.0

Overview & Strategy - Migration capabilities and pillar alignment
Complete Enterprise Migration^* - End-to-end migration timeline
API Integration - Cross-service monitoring patterns
Error Reference - one page per failure type URI; the decision matrices are delivered with a licensed deployment

Version: 3.0.0
Last Updated: 2025-12-24 Status: Production Ready

* restricted content — see higher-tier documentation