hexasync.dataprotection

Envelope encryption (AES-256-GCM) with pluggable KEK custody and key rotation — the HexaSync at-rest data-protection library. Self-describing hxdp: wire format, code-registry algorithm versioning, per-service DEK keyring with in-memory cache, EF Core integration ([Encrypted]), and a self-healing DEK/KEK lifecycle driver.

The Azure Key Vault KEK adapter ships separately as hexasync.dataprotection.azurekeyvault.

How it works (30 seconds)

A per-service KEK (key-encryption key) wraps per-generation DEKs (data-encryption keys). Your data is encrypted with the active DEK; the wrapped DEKs live in a data_encryption_keys table in your database. On boot the library unwraps the keyring into an in-memory cache (one KEK round-trip), then encrypt/decrypt are local. It is hxdp:-only and fails closed — it never falls back to plaintext or a legacy format.

Install

dotnet add package hexasync.dataprotection
dotnet add package hexasync.dataprotection.azurekeyvault   # production KEK custody (optional in dev)

Configuration (flat DATA_PROTECTION_* env vars)

House convention: config via flat UPPER_SNAKE_CASE environment variables (no appsettings.json). Each maps to a property via [ConfigurationKeyName], bound as IOptions<DataProtectionConfig>.

DATA_PROTECTION_KEK_PROVIDER=Local              # "Local" (dev/low envs) or "AzureKeyVault" (prod); also the default KEK_ACTIVE
DATA_PROTECTION_KEK_ACTIVE=Local                # which registered provider WRAPS new/re-wrapped DEKs (default = KEK_PROVIDER).
                                                 # Register BOTH providers and flip this for a seamless Local↔Azure cutover
                                                 # (see "Multiple KEK providers" below). Unwrap always routes to whoever wrapped a DEK.
DATA_PROTECTION_MASTER_KEY_BASE64=<b64 32 bytes> # REQUIRED when Local is active/fallback (`openssl rand -base64 32`)
DATA_PROTECTION_KEK_ID=local                    # bookkeeping id stamped on locally-wrapped DEKs (optional)
DATA_PROTECTION_ACTIVE_SCHEME=v1                # optional; default = newest registered scheme; unknown → boot throws
DATA_PROTECTION_EXPECTED_MAX_PODS=16            # must be >= your HPA maxReplicas; drives the nonce early-rotation trip
DATA_PROTECTION_STARTUP_MAX_ATTEMPTS=5          # optional; bounded startup-warm retries (KEK may be transiently down)
DATA_PROTECTION_STARTUP_RETRY_DELAY=00:00:02    # optional

Local is for dev / low environments. The master key sits in config; a KV-wrapped DEK cannot be unwrapped locally (by design — prod data is not decryptable off-cluster). Production should use AzureKeyVault (see that package's README).

KeyringMaintenance is not a JSON section — it is configured in code via the AddKeyringMaintenance(cfg => …) lambda (below).

Wiring (EF-backed, in order)

// 1. Your DbContext factory — the library needs a FRESH context per keyring call (the provider is a singleton;
//    a captured scoped DbContext would throw ObjectDisposedException under load).
services.AddDbContextFactory<MyDbContext>(o => o.UseNpgsql(connectionString));

// 2. Compose the library (store + provider + IDataProtector + startup warm); binds the DATA_PROTECTION_* env vars.
services.AddHexaSyncDataProtectionWithEf<MyDbContext>(configuration);   // bundles AddEfDekKeyringStore<MyDbContext>()

// 3. Azure Key Vault KEK custody. May be registered ALONGSIDE Local (both coexist — DATA_PROTECTION_KEK_ACTIVE picks
//    which one wraps; unwrap routes to whichever wrapped a given DEK). This is the seamless-cutover path (below).
services.AddAzureKeyVaultKeyWrapper(configuration);

// 4. Health check (readiness): after AddHexaSyncDataProtection.
services.AddHealthChecks().AddHexaSyncDataProtectionHealthCheck();

// 5. Opt-in self-healing maintenance (DEK bootstrap/rotation + KEK re-wrap). Requires a leader lock (below).
services.AddSingleton<IDataProtectionLeaderLock, RedisMedallionLeaderLock>();
services.AddKeyringMaintenance(cfg => { cfg.MaxDekAge = TimeSpan.FromDays(180); });

A custom audit sink (optional) must be registered before AddHexaSyncDataProtection (it wins a TryAdd over the default no-op): services.AddSingleton<IDataProtectionAuditSink, MySink>();.

Leader-lock adapter (Medallion.Threading Redis)

AddKeyringMaintenance needs an IDataProtectionLeaderLock so exactly one replica per service runs the cycle. The library ships no Redis dependency — adapt your existing distributed lock (the shape is a single blocking acquire returning a handle held until disposed):

public sealed class RedisMedallionLeaderLock(IDistributedLockProvider locks) : IDataProtectionLeaderLock
{
    public async Task<IAsyncDisposable> AcquireAsync(string key, CancellationToken ct)
        => await locks.AcquireLockAsync(key, cancellationToken: ct);   // Medallion handle is already IAsyncDisposable
}

AcquireAsync should block until acquired (so a dead leader's lock is picked up as soon as it releases).

Your DbContext

Implement IDataProtectionDbContext (exposes the keyring DbSet), inject the resolved IDataProtector into the ctor, and call ApplyDataProtection in OnModelCreating:

public class MyDbContext(DbContextOptions options, IDataProtector protector)
    : DbContext(options), IDataProtectionDbContext
{
    public DbSet<DataEncryptionKeyRecord> DataEncryptionKeys => Set<DataEncryptionKeyRecord>();
    public DbSet<UserSecret> UserSecrets => Set<UserSecret>();

    protected override void OnModelCreating(ModelBuilder mb) => mb.ApplyDataProtection(protector);
    // ApplyDataProtection wires the [Encrypted] converter, guards (no index/non-string [Encrypted]), and maps the
    // data_encryption_keys table by construction.
}

[Table("user_secrets")]
public class UserSecret
{
    [Key] public int Id { get; set; }
    [Encrypted] public string TotpSeed { get; set; } = "";   // stored as an hxdp: blob, transparently en/decrypted
    public string Label { get; set; } = "";                  // plaintext, queryable
}

Migration (you own it — the library ships none)

The keyring table your migration must create:

CREATE TABLE data_encryption_keys (
    key_id               text        PRIMARY KEY,
    wrapped_dek          bytea       NOT NULL,
    kek_id               text        NOT NULL,
    state                text        NOT NULL,   -- pending | active | retired | destroyed  (plain text, not an enum)
    created_under_scheme text        NOT NULL,   -- e.g. 'v1'
    created_at           timestamptz NOT NULL
);

Each [Encrypted] business column is text (it holds the ASCII hxdp: blob — not bytea). If an entity implements IEncryptedEntity (for the future 1.9 re-encrypt sweep), also add a nullable dek_reconciled_version text column. Do not put an index / WHERE / ORDER BY / FK on an [Encrypted] column (see the ops runbook).

Health check

AddHexaSyncDataProtectionHealthCheck() reports Unhealthy until the keyring warms (IsInitialized) — wire it to readiness so a pod that can't reach the KEK at boot doesn't take traffic. When AddKeyringMaintenance is wired, an overdue rotation reports Degraded (never Unhealthy — the pod still serves).

Migrating off a legacy format

The library is hxdp:-only; migrating legacy AES-CBC (or any prior format) is the consumer's job. Read the raw legacy value with your own code (raw SQL, or a temporary non-[Encrypted] mapping — you can't read it through the [Encrypted] property, which would try to decrypt it as hxdp: and fail closed), decrypt it with your legacy code, then write the plaintext through the [Encrypted] property so the converter re-encrypts it on save. Use HxdpEnvelope.IsProtected on the raw value to skip already-migrated rows:

foreach (var (id, raw) in ReadRawLegacyColumn())          // your code — bypasses the [Encrypted] converter
{
    if (HxdpEnvelope.IsProtected(raw)) continue;          // already "hxdp:" → migrated; leave it

    // Attach a stub and mark ONLY the [Encrypted] property modified. This emits an UPDATE that encrypts the
    // assigned plaintext — WITHOUT a prior SELECT, so the legacy value is never read through the converter
    // (which would fail closed). Do NOT db.Find/materialize the row here.
    var stub = new UserSecret { Id = id };
    db.Attach(stub);
    stub.TotpSeed = DecryptLegacy(raw);                   // plaintext in → converter encrypts on save
}
db.SaveChanges();

Non-EF callers use IDataProtector directly — note it is bytes-in / bytes-out (the result is the ASCII hxdp: blob): var blob = Encoding.ASCII.GetString(await protector.EncryptAsync(Encoding.UTF8.GetBytes(plaintext)));.

No-break rollout checklist

The failure mode of any migration is a pod (or a restored backup) that can't read what another pod already wrote. Order the rollout so that can't happen:

1. Additive schema first. Add the new [Encrypted]/v2 column; never repurpose the legacy column in the same deploy — legacy readers keep working.
2. Decrypt-before-encrypt. Deploy the version that can decrypt hxdp: to 100% of replicas before any version writes hxdp:. For the length of a rolling update the old and new versions are both Ready and serving; the old pod must not choke on a blob the new pod just wrote. (A config-flag cutover is the same — the flip lands pod-by-pod.)
3. Backfill idempotently. Skip already-migrated rows with HxdpEnvelope.IsProtected(raw); the job must be safe to re-run and to interrupt mid-flight.
4. Drop the legacy column only after prod backfill is confirmed — production-gated, a separate later deploy, never bundled with the write cutover.

For a KEK provider cutover (Local ↔ Azure), the same discipline is re-wrap-before-drop — see the next section.

Sensitive-data-logging prohibition

EnableSensitiveDataLogging would write decrypted plaintext + key material to logs. Call context.WarnIfSensitiveDataLoggingWithEncryptedProperties(logger) at startup — it logs a warning if the trap is on for a context with [Encrypted] properties. (It warns; it does not block.)

Observability

Metrics ship on a Meter named HexaSync.DataProtection (tag-free counters: encrypt/decrypt/decrypt_miss/ rotation/rewrap/kek_unwrap; gauges: active-DEK nonce consumption + age). Subscribe your OpenTelemetry exporter to that meter name. For per-event auditing (rotate / re-wrap / destroy / decrypt-miss / kek-version-change), implement IDataProtectionAuditSink and register it before AddHexaSyncDataProtection.

Crypto inventory

EncryptedFieldInventory.ToMarkdown(ctx.Model) renders the declared [Encrypted] fields as a Markdown table for a CRYPTO-INVENTORY.md artifact (emit it at boot or from a CLI dump mode). Schema metadata only — never key material.

Name note: IDataProtector

This library's hexasync.dataprotection.IDataProtector shares its simple name with Microsoft.AspNetCore.DataProtection.IDataProtector. If a consumer uses both, namespace-qualify or alias (using IHxDataProtector = hexasync.dataprotection.IDataProtector;). They are unrelated types.

Multiple KEK providers & seamless Local→Azure cutover

Both KEK providers can be registered at once (a Local master and AddAzureKeyVaultKeyWrapper). They compose behind a single IKeyWrapper:

• Wrap (new + re-wrapped DEKs) always uses the active provider (DATA_PROTECTION_KEK_ACTIVE).
• Unwrap tries the active provider first; on a wrong-key/wrong-format failure it falls back to the other provider (so a DEK wrapped by either is always decryptable through a migration). A transient failure (KV throttle/timeout/5xx) retries the active provider and never falls through — an outage is never masked. A KEK outage degrades rotation, not decryption of already-cached DEKs.

Cutover (Local → Azure), reversible until the last step:

// 0. Register BOTH: Local master present + AddAzureKeyVaultKeyWrapper(...); DATA_PROTECTION_KEK_ACTIVE=Local.
// 1. Prove Azure works while Local is still active — the self-test endpoint (below), green across the fleet.
// 2. Flip DATA_PROTECTION_KEK_ACTIVE=AzureKeyVault  → new wraps use Azure; existing DEKs still unwrap via Local.
// 3. Re-wrap the existing DEK(s) onto the active provider (Local → Azure), no data re-encryption:
await provider.ReWrapAllToActiveAsync(ct);        // provider = the resolved KeyringDekProvider
// 4. Soak. Rollback (before step 5) = flip active back + ReWrapAllToActiveAsync again.
// 5. Drop Local ONLY after the fail-closed gate passes (else a still-Local-wrapped DEK becomes unrecoverable):
await provider.AssertNoneWrappedByAsync("Local", ct);   // throws if any DEK is still Local-wrapped
//    → then remove DATA_PROTECTION_MASTER_KEY_BASE64.

Both providers stay registered throughout, so a pod restart mid-cutover always unwraps. The gate classifies each DEK's actual recorded kek_id (not the literal "local"), so a custom DATA_PROTECTION_KEK_ID still gates correctly.

KEK self-test endpoint (hexasync.dataprotection.endpoints)

An admin can confirm a named provider's full path (auth/network/RBAC/wrap/unwrap) on demand — e.g. prove Azure works while Local is still active — without touching the keyring or real data:

builder.Services.AddDataProtectionSelfTestRateLimiter();           // fixed-window, 5/min per caller
app.UseRateLimiter();
// Authorization is CONSUMER-supplied via the configure callback — this package depends on no auth middleware
// (it is upstream of every consumer), so you apply your own capability gate here:
app.MapDataProtectionSelfTest("/api/v3/admin/dataprotection/self-test",
    endpoint => endpoint.RequireCapability(DataProtectionCapabilities.KekSelfTest));
// POST .../self-test?provider=AzureKeyVault  →  { provider, ok, kekIdFingerprint, latencyMs, errorKind? }

DataProtectionCapabilities.KekSelfTest ("dataprotection:kek:self_test") is a library-owned string constant — mirror it into your capability catalog and grant it to your admin/super-admin role. The response returns a fingerprint of the kek_id (never the vault URI) and a classified errorKind (never raw provider exception text). The self-test calls the adapter directly — it writes nothing to the keyring.

Operations

KEK-retention, compromise/rotation, encrypted-column rules, and the KV rotation burst budget are in the ops runbook: docs/data-protection/OPS_RUNBOOK.md. The KEK-retention rule is data-loss-class — read it before rotating a KEK in production.