The 34-Engine Registry — Design and Metadata

The Problem

A compliance officer at a law firm using the legal SaaS platform opened a ticket flagging that one specific scoring engine, legal-risk-posture, was producing output that read less like a behavioral signal and more like legal advice for a subset of matter types — language a non-lawyer platform should never be generating, and language that, left running, exposed both the firm and the platform to an unauthorized-practice-of-law complaint. The fix was obvious and small: stop that one engine from running for that one matter type. Shipping it was not small. With every engine hard-wired directly into the scoring handler, "stop this engine" meant a hotfix branch, a fast-tracked review, and a deployment pipeline that took eleven minutes end to end on a good day. Eleven minutes is a long time to keep generating output legal has just told you is a liability risk, for every request that arrives while the deploy is in flight.

That incident is the reason the engine registry exists, and it is worth stating plainly before any of the schema or code below: the business problem was never "how do we organize 34 functions" — it was "how do we guarantee that a compliance-driven decision takes effect in seconds, not in a deployment window," because the cost of the gap between those two is measured in real legal and regulatory exposure, not developer inconvenience. Everything in this article is the engineering answer to that one operational requirement, generalized to the other cases (removing an engine safely, scoping one to a specific team) that turned out to need the identical fix. That default state — behavior changes gated behind a deployment — shows up constantly across the Technology and Artificial Intelligence sectors specifically, in any team building infrastructure to run more than a handful of scoring functions in production, which is why the pattern below generalizes well past this one incident.

The Shape of the Problem Before There Was a Registry

Early prototypes of the behavioral AI platform did not have a registry at all. The Bayesian confidence engine, the behavioral entropy engine, and the state machine runtime were three functions imported directly into a single scoreLead() handler. That handler looked, roughly, like this:

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata

// Pre-registry era: engine calls hard-wired into the request handler.
// This is the code the registry replaced. It is included here because
// almost every team that adopts a registry pattern is migrating away
// from exactly this shape.
async function scoreLead(signals, context) {
  const entropy    = await scoreBehavioralEntropy(signals);
  const confidence = await scoreBayesianConfidence(signals, entropy);
  const stateTransition = await scoreStateTransitions(signals, confidence);

  // Ad-hoc gating logic, duplicated in three other handlers elsewhere
  // in the codebase, each slightly different.
  if (context.team === 'legal-beta') {
    const legalRisk = await scoreLegalRiskPosture(signals);
    return { entropy, confidence, stateTransition, legalRisk };
  }

  return { entropy, confidence, stateTransition };
}

This is not bad code by the standards of a two-week prototype. It is bad code by the standards of a platform that intends to run 34 engines across four product adapters with different governance requirements, different compute budgets, and different rollout schedules for each engine. Three specific failure modes emerged within the first quarter of running this pattern in production, and each one directly motivated a piece of the registry schema described later in this article.

Failure Mode One — Deployment-Coupled Activation

This is the incident from the opening of this article, generalized: turning any engine on or off meant editing scoreLead(), running the test suite, and shipping a deployment, regardless of how urgent the reason. That eleven-minute window is not just an engineering annoyance under this framing — every request served during it is a request where the platform kept doing the exact thing a compliance review had just flagged as a liability, which is precisely the kind of gap a regulator or opposing counsel asks about after the fact: not "did you fix it," but "how long did you knowingly keep serving it after you knew." The honest answer, before the registry, was "as long as our deployment pipeline takes." That is not an answer any legal-tech vendor wants to give a client's general counsel.

Failure Mode Two — Untraceable Removal

Deleting an engine that was no longer needed required finding every call site. In a codebase where scoreBehavioralEntropy was imported into four different handlers (the lead-scoring handler, the batch re-scoring cron job, an internal debugging endpoint, and a one-off script a data scientist wrote and forgot to delete), "delete this engine" became an archaeology exercise. The team's actual practice, for a while, was to never delete engines — only to stop calling them from the handlers they knew about, which left dead imports, dead compute cost, and, worse, orphaned schedule entries in the cron job that kept invoking a function nobody remembered existed.

Failure Mode Three — Scattered Restriction Logic

The if (context.team === 'legal-beta') line above is the third failure mode in miniature. Restricting an engine to a specific team, a specific product adapter, or a specific risk tier meant writing a conditional at every call site, and those conditionals drifted. Six months in, an audit found four different spellings of essentially the same gating condition across the codebase, two of which had a stale team ID that no longer existed. One of those stale conditionals was silently permissive — it evaluated to true for every team, because a refactor had removed the variable it originally checked and nobody noticed the guard had become a no-op.

The registry does not make these problems merely more convenient to deal with. It removes the class of bug entirely, because activation, restriction, and dependency information move out of scattered conditionals and into a single declarative object that the control plane reads once, at the top of every pipeline run.

A registry is not a nice-to-have abstraction layer. It is the difference between "change requires a deployment" and "change requires a database update that takes effect on the next request." Every design decision in this article follows from that distinction.

What Counts as an "Engine," Precisely

Before describing the schema, it is worth being precise about what the registry actually registers, because "engine" is used loosely elsewhere on this site and loosely in the industry generally. In the behavioral AI platform, an engine is a specific, narrow thing: a pure-ish scoring unit that accepts behavioral signals and context, and returns a score, a label, and a confidence value, without direct knowledge of any other engine's internals.

"Pure-ish" is doing real work in that sentence. Engines are allowed to read from shared reference data (a knowledge graph snapshot, a domain ontology, a set of pre-trained embeddings) and they are allowed to consult prior scores that were emitted onto the event bus by engines that ran in an earlier execution batch. What they are not allowed to do is call another engine's function directly, hold a reference to another engine's internal state, or make network calls to services that are not declared in their metadata. That constraint is what makes the registry's dependency graph trustworthy: if engines could reach around the registry and call each other directly, the declared dependencies array would just be documentation, not an enforced contract.

The Eleven Engine Categories

The 34 engines currently registered fall into eleven categories, each covered in its own article in this series. The registry does not hard-code these categories — category is a string field, not an enum baked into the schema — but the control plane's dashboard and the compute budget allocator both use category as a grouping key, so in practice the set is stable and changes rarely. The table below is the canonical list as of this writing.

Two things fall out of this table immediately. First, compute cost roughly tracks category, but not perfectly — digital-twin-simulation is expensive because it runs a Monte Carlo sweep over scenario space, not because "simulation" is inherently costly as a category. Second, risk level is not the same axis as compute cost. The kernel and governance engines are cheap to run (a few milliseconds of rule evaluation) but critical in risk level, because a bug in governance-safety does not produce a wrong score — it produces an unsafe output reaching a product adapter. Conflating cost and risk is a mistake the schema deliberately avoids by giving them separate fields, discussed next.

What an Engine Is Not

It is worth being explicit about the boundary. The registry does not register: product adapters (which consume engine output, but are not themselves scoring units), the event bus (infrastructure, not a scoring unit), the governance wrapper (a post-processing stage that runs after every engine batch, not an engine itself, even though it is sometimes informally called "the 35th engine" in team chat), or reference datasets like the shared ontology (data, not a scoring unit, even though several engines depend on it being loaded before they can run). Keeping this boundary sharp matters because the registry's guarantees — validated metadata, dependency ordering, budget enforcement — only apply to things that are actually registered. Anything that reaches into the pipeline without going through registerEngine() is invisible to the control plane's scheduler, invisible to the audit log, and invisible to the compute budget. That is precisely the failure mode the registry exists to prevent, so the discipline of "if it scores something, it registers" is treated as close to a hard rule on this platform.

The Engine Definition Schema

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata

interface EngineDefinition {
  engineId:         string;          // Unique, kebab-case. e.g. 'bayesian-confidence'
  displayName:      string;          // Human-readable
  category:         EngineCategory;  // 'foundational'|'cognitive'|'temporal'|'memory'|...
  domain:           string;          // 'cross-domain'|'legal'|'sales'|'trust'
  riskLevel:        'low'|'medium'|'high'|'critical';
  computeCost:      number;          // 1–10; used by scheduler for resource budgeting
  dependencies:     string[];        // engineIds that must run before this one
  emitsTopics:      string[];        // events this engine publishes to the bus
  subscribesTopics: string[];        // events this engine consumes from the bus
  activationState:  ActivationState; // 'active'|'restricted'|'research-only'|'deprecated'
  researchOnly:     boolean;         // if true, output is logged but never returned to adapters
  restrictedToTeams?: string[];      // only relevant when activationState === 'restricted'
  version:          string;          // semver — used for audit logs
}

Every field is mandatory except restrictedToTeams. The registry validates the definition at registration time and throws if any required field is missing or if a declared dependency does not exist. What follows is a field-by-field explanation of why each one exists, because a schema without rationale is just a list of names, and the rationale is what tells you how to extend it correctly when engine 35 needs a field that engine 1 through 34 did not.

engineId — the only identity that matters

Every other identifier attached to an engine — its file path, its export name, its class name if it happens to be implemented as a class — is allowed to change freely. engineId is the one identifier that is treated as permanent, because it is the value stored in audit logs, in the bc_explainability_traces table described in Part 5 of this series, and in every downstream dependency declaration. Renaming an engineId is, in practice, equivalent to deprecating the old engine and launching a new one, because every historical trace that references the old ID becomes orphaned. The convention is kebab-case, and the registry's validator rejects anything that does not match /^[a-z][a-z0-9-]*[a-z0-9]$/ — no leading digits, no trailing hyphens, no underscores, no camelCase. This is a small thing, but small naming conventions enforced mechanically are what keep a registry with 34 (and eventually well over 100) entries legible at a glance in log output.

displayName — for humans, never for logic

Nothing in the control plane, the scheduler, or the governance wrapper ever branches on displayName. It exists purely for the admin dashboard and for audit log messages that a human will read. This separation matters because display names change — marketing wants "Behavioral Entropy Engine" to read as "Signal Variability Engine" in a client-facing report, and that rename should be a one-line metadata edit, not a grep-and-replace across the codebase. If any logic ever keys off displayName, that rename becomes a breaking change, which defeats the purpose of separating the human label from the machine identity in the first place.

category and domain — two different axes, often confused

category answers "what kind of computation is this?" (temporal, cognitive, memory). domain answers "which product surface does this matter to?" (legal, sales, trust, or cross-domain for engines like bayesian-confidence that every product adapter uses). These are orthogonal. The batna-calculator engine is category negotiation and domain legal, because it is only meaningful in the context of the legal SaaS platform's settlement-posture feature; the bayesian-confidence engine is category cognitive and domain cross-domain, because every product adapter needs a confidence figure. Early versions of the schema tried to fold these into a single type field with values like legal-negotiation, and that fell apart almost immediately — the moment the sales adapter wanted its own negotiation scoring for deal terms, the team needed negotiation engines in two different domains, and a single combined field could not express that without turning into an unbounded string enum. Splitting the axis into two fields was a two-line schema change once the problem was diagnosed, but it took a full quarter of awkward workarounds before the team recognized that the field was conflating two questions.

riskLevel — drives review cadence, not scoring behavior

riskLevel never touches the actual computation an engine performs. It exists entirely to drive process: engines at critical risk level require two independent reviewer sign-offs before any version bump ships, engines at high require one, and low/medium follow the standard single-review process every other change goes through. The governance and kernel engines are all critical, not because they run expensive computation, but because a defect in one of them means an unsafe output can reach an end user without any downstream check catching it — there is no engine after the governance wrapper. That asymmetry between "how expensive is this to compute" and "how bad is it if this is wrong" is exactly why riskLevel and computeCost are separate fields rather than one derived from the other.

The four-tier scheme is deliberately legible against the external frameworks a compliance review will actually cite. A critical engine maps cleanly onto the highest-scrutiny tier the EU AI Act's risk-based approach describes; the review-gate discipline itself is the platform's concrete implementation of the AI-management-system controls ISO/IEC 42001:2023 asks an organization to demonstrate; and the act of assigning riskLevel in the first place — weighing likelihood and severity before an engine ever reaches production — is standard ISO/IEC 23894:2023 and NIST AI Risk Management Framework practice, just implemented as a required PR field instead of a spreadsheet.

computeCost — a declared estimate, checked against reality

The 1–10 scale is deliberately coarse. Early drafts of the schema used milliseconds as the unit, and that fell apart because milliseconds vary by hardware, by input size, and by cache warmth, and an engine author writing the definition at registration time cannot know the production percentile latency their code will actually exhibit. The coarse scale is instead an ordinal ranking: a 2 should be roughly an order of magnitude cheaper than an 8. The scheduler (covered in the compute budgeting section below) uses the declared cost to build an execution plan, but a background job separately measures actual wall-clock cost per engine per day and flags any engine whose measured cost percentile has drifted more than two buckets away from its declared computeCost. That drift check has caught real regressions — a change to the digital-twin-simulation engine that accidentally increased its Monte Carlo sample count by 10x shipped with no test failures (the outputs were still correct, just slower) and was caught within a day by the drift monitor rather than by a latency alert three weeks later.

dependencies — a promise the control plane enforces

This array is the input to the dependency graph described later in this article. It declares "these engineIds must have already produced output, on the event bus, before this engine is scheduled." It does not declare "these engines must be active" — a subtlety that trips up new engine authors constantly. If engine B depends on engine A, and engine A is deprecated, engine B does not silently skip its dependency; registration fails at startup with a clear error, because a dependency on a deprecated engine is almost always a sign that B itself needs to be updated or deprecated alongside A. The one exception is research-only: a dependency on a research-only engine is allowed, because research-only engines still run and still emit their topics — they just don't have their output exposed to adapters. This distinction between "does it run" and "is its output exposed" is why activationState and researchOnly are separate fields rather than folded into one, discussed below.

emitsTopics and subscribesTopics — the pub/sub contract

These two arrays describe the engine's interface to the event bus, covered in depth in Part 6 of this series. They exist as separate metadata from dependencies because dependency and topic subscription are related but not identical: an engine can subscribe to a topic emitted by an engine it has no ordering dependency on (useful for optional enrichment data that, if late, simply results in a lower-confidence score rather than a blocked pipeline), and an engine can depend on another engine's completion without actually consuming any of its emitted topics directly (useful when the dependency exists purely to guarantee ordering for a side effect, like a cache warm). The registry validates that every topic in subscribesTopics is emitted by at least one currently registered engine, which catches a whole class of "I renamed a topic and forgot to update the eleven subscribers" bugs at startup instead of in production.

activationState and researchOnly — deliberately not the same field

It would be tempting to fold researchOnly into activationState as a fifth value. The reason they are kept separate is that researchOnly is a data-governance property (does this output ever leave the system boundary?) while activationState is an operational property (does this engine run at all, and for whom?). An engine can be active and researchOnly: true simultaneously — it runs on every request, for every team, but its output never crosses into a product adapter response, only into the internal metrics store used to evaluate whether it should graduate. Folding these into one enum would have required inventing states like active-research-only and restricted-research-only, doubling the state count for no benefit and making the state transition diagram (covered later in this article) considerably harder to reason about.

restrictedToTeams — optional, and only meaningful in one state

The registry's validator enforces a cross-field rule that a plain type system cannot express on its own: restrictedToTeams must be present and non-empty when activationState === 'restricted', and must be absent (or ignored with a startup warning if present) otherwise. This is implemented as an explicit check in the validator function, shown in the validation section below, rather than as a discriminated union in the TypeScript type, because the team found that discriminated unions on this particular shape made the object literals engine authors write at the call site considerably more awkward, for a compile-time guarantee that the runtime validator already provides more helpfully (with an actionable error message, not a type error at a call site three files away from the actual mistake).

version — the unit of audit-log truth

Every entry in the audit log references an engineId and a version together, never just an engineId alone. This is what lets a compliance review answer the question "what exact scoring logic produced this output on March 14th" months after the fact, even if the engine has been updated four times since. The versioning discipline this implies — every change to scoring logic bumps at least the patch version, no exceptions — is covered in its own section further down, because it turns out to be one of the two or three things new engine authors get wrong most often.

Runtime Validation, Not Just Compile-Time Types

TypeScript's interface EngineDefinition gives editor autocomplete and catches obvious mistakes at build time, but it does nothing at runtime, and the registry cannot trust that every engine definition object passed to registerEngine() was actually type-checked — plugins, dynamically loaded engines, and engines registered from configuration-driven code paths can all bypass the compiler entirely. The registry therefore re-validates every definition at runtime using a schema library, and the runtime schema is intentionally stricter than the TypeScript type in several places where the type system's expressiveness runs out.

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata

import { z } from 'zod';

const EngineCategorySchema = z.enum([
  'foundational', 'cognitive', 'emotional', 'temporal', 'memory',
  'narrative', 'negotiation', 'motivation', 'simulation',
  'epistemic', 'kernel', 'governance', 'standards', 'adaptive',
]);

const ActivationStateSchema = z.enum([
  'active', 'restricted', 'research-only', 'deprecated',
]);

const EngineIdSchema = z
  .string()
  .regex(/^[a-z][a-z0-9-]*[a-z0-9]$/, 'engineId must be kebab-case, no leading/trailing hyphens');

export const EngineDefinitionSchema = z
  .object({
    engineId:           EngineIdSchema,
    displayName:         z.string().min(3).max(120),
    category:            EngineCategorySchema,
    domain:              z.string().min(2),
    riskLevel:           z.enum(['low', 'medium', 'high', 'critical']),
    computeCost:         z.number().int().min(1).max(10),
    dependencies:        z.array(EngineIdSchema).default([]),
    emitsTopics:         z.array(z.string()).default([]),
    subscribesTopics:    z.array(z.string()).default([]),
    activationState:     ActivationStateSchema,
    researchOnly:        z.boolean(),
    restrictedToTeams:   z.array(z.string()).optional(),
    version:             z.string().regex(/^\d+\.\d+\.\d+$/, 'version must be semver: MAJOR.MINOR.PATCH'),
  })
  .superRefine((def, ctx) => {
    if (def.activationState === 'restricted' &&
        (!def.restrictedToTeams || def.restrictedToTeams.length === 0)) {
      ctx.addIssue({
        code: z.ZodIssueCode.custom,
        path: ['restrictedToTeams'],
        message: `Engine "${def.engineId}" has activationState "restricted" but no restrictedToTeams entries.`,
      });
    }
    if (def.dependencies.includes(def.engineId)) {
      ctx.addIssue({
        code: z.ZodIssueCode.custom,
        path: ['dependencies'],
        message: `Engine "${def.engineId}" cannot depend on itself.`,
      });
    }
  });

export type EngineDefinition = z.infer<typeof EngineDefinitionSchema>;

Two things about this schema are worth calling out explicitly because they were not present in the first version the team shipped, and both were added after a specific production incident.

Why the self-dependency check exists

An engine author refactoring state-machine-runtime accidentally left 'state-machine-runtime' in its own dependencies array after a copy-paste from a similar engine definition. Because the dependency graph builder used Kahn's algorithm (described below), a self-loop like this produces a graph where the node's in-degree never reaches zero, which means the topological sort silently drops that single node from every execution batch rather than throwing an obvious error. The engine registered successfully, appeared in the registry's engine list, and simply never ran, for six days, before anyone noticed its output was missing from a downstream report. The superRefine self-dependency check was added the same day the root cause was found, specifically to turn that silent omission into a loud startup failure.

Why version is validated as strict semver, not just "any string"

The original schema accepted any non-empty string for version, and engine authors used everything from "v2" to "2024-03-14" to "final-final-2". This made the audit log's version-comparison logic (used to detect whether a version bump happened at all between two deployments) unreliable, because string comparison and semver comparison disagree on ordering in enough cases to matter — "v10" sorts before "v9" lexicographically, which is exactly backwards. Enforcing strict MAJOR.MINOR.PATCH at the schema layer, rather than trying to parse and warn about malformed versions after the fact, closed that entire class of problem in one change.

Self-Registration Pattern

Engines do not need to be manually listed anywhere. Each engine file calls registerEngine() when it is first imported. Node.js module loading handles the rest.

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata

// engines/temporal/state-machine.engine.js
import { registerEngine } from '../../core/registry.js';
import { scoreStateTransitions } from './state-machine.logic.js';

registerEngine({
  engineId:         'state-machine-runtime',
  displayName:      'Behavioral State Machine Runtime',
  category:         'temporal',
  domain:           'cross-domain',
  riskLevel:        'medium',
  computeCost:      4,
  dependencies:     ['bayesian-confidence'],        // needs confidence scores first
  emitsTopics:      ['state.transition.detected'],
  subscribesTopics: ['confidence.updated'],
  activationState:  'active',
  researchOnly:     false,
  version:          '1.2.0',
});

export async function score(signals, context) {
  return scoreStateTransitions(signals, context);
}

This pattern — register as a side effect of import — is convenient, and it is also the single most common source of subtle bugs new contributors to the engine codebase run into, because it makes registration order depend on import order, which is not the same thing as file order, alphabetical order, or the order the engines appear in any documentation. The rest of this section is about making that implicit ordering explicit and safe.

The Entry Point That Actually Controls Load Order

Nothing about ES module semantics guarantees that engine files are imported in any particular sequence unless something explicitly imports them in that sequence. The behavioral AI platform uses a single barrel file, engines/index.js, whose only job is to import every engine module in a fixed, deliberately chosen order, category by category:

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata

// engines/index.js — the ONLY file allowed to import engine modules directly.
// Order here is deliberate: foundational and cognitive engines register first
// because most other engines declare a dependency on bayesian-confidence,
// and registerEngine() will throw immediately if a declared dependency
// does not yet exist in the registry.

// Foundational
import './foundational/behavioral-entropy.engine.js';
import './foundational/information-theory.engine.js';
import './foundational/geometric-topological.engine.js';
import './foundational/meta-learning.engine.js';

// Cognitive
import './cognitive/bayesian-confidence.engine.js';
import './cognitive/bias-detection.engine.js';

// Emotional
import './emotional/emotional-regulation.engine.js';

// Temporal
import './temporal/state-machine.engine.js';
import './temporal/causal-graph.engine.js';
import './temporal/prediction-horizon.engine.js';

// ... remaining categories, same pattern ...

// Kernel & governance last: some kernel engines validate the *entire*
// registered set (e.g. checking that every 'critical' riskLevel engine
// has a corresponding governance rule), so they must load after
// everything else.
import './kernel/behavioral-vm-kernel.engine.js';
import './governance/governance-safety.engine.js';

export { engineRegistry } from '../core/registry.js';

No other file in the codebase is permitted to import an individual engine file directly — that rule is enforced by an ESLint restricted-import rule, not just convention, because a single stray direct import from, say, a test file, can cause that engine to register twice if the barrel file is also loaded in the same process, and double-registration is a bug the registry explicitly guards against (see below) but would rather engine authors never trigger in the first place.

Idempotent Registration — Guarding Against Double-Import

Module caching in Node.js means a given engine file's top-level code runs exactly once per process under normal circumstances, but "normal circumstances" excludes hot module reloading in development, certain test runner configurations that reset the module cache between test files, and worker-thread pools where each worker gets its own fresh module cache. All three of those situations are routine on this platform, so registerEngine() is written to be idempotent rather than to assume it will only ever be called once per engineId:

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata

function registerEngine(rawDef) {
  const def = EngineDefinitionSchema.parse(rawDef); // throws ZodError on invalid shape

  const existing = engines.get(def.engineId);
  if (existing) {
    if (deepEqual(existing, def)) {
      // Re-import of the same module with an unchanged definition.
      // This happens routinely under HMR and certain test runners — not an error.
      return existing;
    }
    // Re-registration with a DIFFERENT definition for the same engineId is
    // always a bug: either a copy-paste of engineId across two files, or a
    // stale cached module racing a freshly reloaded one.
    throw new EngineRegistrationError(
      `Engine "${def.engineId}" is already registered with a different definition. ` +
      `Existing version: ${existing.version}, incoming version: ${def.version}. ` +
      `This usually means two files declare the same engineId.`
    );
  }

  validateDependenciesExist(def);
  engines.set(def.engineId, def);
  registryIndex.addToIndices(def);
  auditLog.recordRegistration(def);
  return def;
}

The distinction between "re-registration with an identical definition" (silently accepted, because it is a harmless artifact of module reloading) and "re-registration with a different definition" (a hard failure) is what makes this function safe to call from hot-reload paths without either silently masking real bugs or making local development painful. Getting this distinction wrong in either direction was tried first: initially any re-registration threw, which made local development with file-watching nearly unusable because every saved file triggered a crash; the fix was not to relax the check generally, but to add the equality comparison so that only meaningfully different re-registration is treated as an error.

Testing Self-Registration Without a Full Server Boot

Because registration is a side effect of import, testing an individual engine's metadata does not require standing up the control plane or the event bus — importing the engine module in isolation, in a fresh module registry, is sufficient:

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata

import { describe, it, expect, beforeEach } from 'vitest';
import { engineRegistry } from '../../core/registry.js';

describe('state-machine-runtime engine registration', () => {
  beforeEach(() => {
    engineRegistry.reset(); // test-only helper, throws if called outside NODE_ENV=test
  });

  it('registers with the expected metadata', async () => {
    // Register its declared dependency first, or registration will throw —
    // this is intentional: the test documents the real dependency contract.
    await import('../../engines/cognitive/bayesian-confidence.engine.js');
    await import('../../engines/temporal/state-machine.engine.js');

    const def = engineRegistry.get('state-machine-runtime');
    expect(def.category).toBe('temporal');
    expect(def.dependencies).toEqual(['bayesian-confidence']);
    expect(def.riskLevel).toBe('medium');
  });

  it('rejects registration if a declared dependency is missing', async () => {
    await expect(
      import('../../engines/temporal/state-machine.engine.js')
    ).rejects.toThrow(/depends on unregistered engine "bayesian-confidence"/);
  });
});

The second test in that block is arguably more valuable than the first: it verifies the registry's fail-fast behavior, not just the happy path. This kind of negative test — "does the system correctly refuse to start in a known-bad configuration" — catches an entire category of bug (a missing import in the barrel file, an engineId typo in a dependencies array) that would otherwise only surface as a mysterious "engine X never runs" report weeks after the change shipped.

Internal Registry Data Structures

The registry is described so far as if it were a single flat store keyed by engineId, and at its core, it is — but a flat Map<string, EngineDefinition> alone does not answer the queries the control plane, the admin dashboard, and the compute budget allocator all need answered many times per request or per page load: "give me every active engine in domain X," "give me every engine whose activationState is research-only," "give me every engine that depends on bayesian-confidence." Answering those by scanning all 34 (soon, well past 100) entries on every call is wasteful, so the registry maintains a small set of secondary indices alongside the primary map.

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata

class EngineRegistry {
  #engines        = new Map();   // engineId -> EngineDefinition
  #byCategory     = new Map();   // category -> Set<engineId>
  #byDomain       = new Map();   // domain -> Set<engineId>
  #byState        = new Map();   // activationState -> Set<engineId>
  #dependents     = new Map();   // engineId -> Set<engineId that depend on it>
  #topicEmitters  = new Map();   // topic -> Set<engineId that emit it>
  #topicSubscribers = new Map(); // topic -> Set<engineId that subscribe to it>

  set(def) {
    this.#engines.set(def.engineId, def);
    this.#indexInto(this.#byCategory, def.category, def.engineId);
    this.#indexInto(this.#byDomain, def.domain, def.engineId);
    this.#indexInto(this.#byState, def.activationState, def.engineId);

    for (const depId of def.dependencies) {
      if (!this.#dependents.has(depId)) this.#dependents.set(depId, new Set());
      this.#dependents.get(depId).add(def.engineId);
    }
    for (const topic of def.emitsTopics) {
      this.#indexInto(this.#topicEmitters, topic, def.engineId);
    }
    for (const topic of def.subscribesTopics) {
      this.#indexInto(this.#topicSubscribers, topic, def.engineId);
    }
  }

  #indexInto(index, key, value) {
    if (!index.has(key)) index.set(key, new Set());
    index.get(key).add(value);
  }

  getByCategory(category) {
    return [...(this.#byCategory.get(category) ?? [])].map(id => this.#engines.get(id));
  }

  getActive() {
    return [...(this.#byState.get('active') ?? [])].map(id => this.#engines.get(id));
  }

  // Everything that would break if this engine were deprecated right now.
  getDependents(engineId) {
    return [...(this.#dependents.get(engineId) ?? [])];
  }
}

getDependents() is the index that makes the deprecation case study later in this article safe to perform without spelunking through every engine's dependencies array by hand. It is a small addition — a reverse index built alongside the forward one — but it converts "will deprecating this engine break anything?" from an O(n) scan a human has to remember to run into an O(1) lookup the deprecation tooling runs automatically and refuses to proceed past if the answer is non-empty.

Persistence: the In-Memory Registry Is Not the System of Record

Everything above describes the in-memory structure the control plane actually queries during a request, and it is deliberately in-memory: a hash map lookup measured in nanoseconds is not something the platform is willing to trade for a database round-trip on every single pipeline execution. But the in-memory registry is rebuilt from scratch on every process start, purely from the self-registration side effects of importing engines/index.js, which raises an obvious question: where does activation-state change live, if flipping an engine from active to restricted is supposed to happen without a deployment?

The answer is a small PostgreSQL table, bc_engine_registry_overrides, that stores only the fields an operator is allowed to change at runtime — activationState, restrictedToTeams, and a reason free-text column required on every write for audit purposes. Everything else about an engine (its dependencies, its computeCost, its emitsTopics) is fixed at code-review time and can only change by shipping new code, because those fields describe the engine's actual behavior, and behavior changes belong in version control and code review, not in a database row an operator can edit from a dashboard at 2 a.m.

-- (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
-- License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
-- Article published: 22 May 2026.
-- URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata

CREATE TABLE bc_engine_registry_overrides (
  engine_id          VARCHAR(80)  PRIMARY KEY REFERENCES bc_engine_registry(engine_id),
  activation_state   VARCHAR(20)  NOT NULL,
  restricted_to_teams TEXT[],     -- NULL unless activation_state = 'restricted'
  reason             TEXT         NOT NULL,
  changed_by         VARCHAR(80)  NOT NULL,
  changed_at         TIMESTAMPTZ  NOT NULL DEFAULT now()
);

-- Every override write also appends an immutable row here — the overrides
-- table above holds *current* state; this table holds *history*.
CREATE TABLE bc_engine_registry_override_history (
  id                 BIGSERIAL PRIMARY KEY,
  engine_id          VARCHAR(80)  NOT NULL,
  activation_state   VARCHAR(20)  NOT NULL,
  restricted_to_teams TEXT[],
  reason             TEXT         NOT NULL,
  changed_by         VARCHAR(80)  NOT NULL,
  changed_at         TIMESTAMPTZ  NOT NULL DEFAULT now()
);

On startup, after the in-memory registry is built from code (self-registration), the control plane makes exactly one query against bc_engine_registry_overrides, joins the overrides onto the in-memory definitions, and applies them. From that point until the process restarts or a poller picks up a fresh override (the platform polls every 15 seconds rather than pushing changes, a deliberate simplicity trade-off discussed in the operational runbook section), the effective activationState for any engine is override.activationState ?? codeDefinition.activationState.

This split — behavior in code, activation in a database row — is the entire trick that makes "restrict this engine without a deployment" possible. Everything else in the registry design exists in service of making that one operational capability safe.

Dependency Resolution & the DAG

The control plane builds a directed acyclic graph (DAG) from the dependencies field, then runs a topological sort to produce execution batches. Engines in the same batch have no inter-dependencies and run in parallel.

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata

function buildExecutionPlan(engines) {
  const graph  = buildDAG(engines);
  const sorted = topologicalSort(graph);  // Kahn's algorithm
  return groupIntoBatches(sorted);        // engines with in-degree 0 form a batch
}
// If a cycle is detected, topologicalSort throws — caught at startup, never at runtime.

That is the summary version. The actual implementation, and the reasons behind several of its non-obvious choices, are worth walking through in full, because "topological sort" is one of those phrases that sounds like a solved problem until you need it to behave correctly under partial engine activation, changing team-based restriction, and a compute budget that might drop engines mid-plan.

Kahn's Algorithm, In Full

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata

function buildDAG(engines) {
  const nodes = new Map(engines.map(e => [e.engineId, { def: e, inDegree: 0, edges: [] }]));

  for (const engine of engines) {
    for (const depId of engine.dependencies) {
      const dep = nodes.get(depId);
      if (!dep) {
        throw new EngineRegistrationError(
          `Engine "${engine.engineId}" depends on "${depId}", which is not ` +
          `active in the current registry snapshot. If "${depId}" is ` +
          `deprecated or restricted for this context, "${engine.engineId}" ` +
          `must be as well, or its dependency must be relaxed.`
        );
      }
      dep.edges.push(engine.engineId); // edge: dep -> engine (dep must run first)
      nodes.get(engine.engineId).inDegree += 1;
    }
  }
  return nodes;
}

function topologicalSort(graph) {
  const queue = [...graph.values()].filter(n => n.inDegree === 0).map(n => n.def.engineId);
  const sorted = [];
  const remaining = new Map(graph);

  while (queue.length > 0) {
    const id = queue.shift();
    sorted.push(id);
    const node = remaining.get(id);
    remaining.delete(id);

    for (const dependentId of node.edges) {
      const dependent = remaining.get(dependentId);
      dependent.inDegree -= 1;
      if (dependent.inDegree === 0) queue.push(dependentId);
    }
  }

  if (sorted.length !== graph.size) {
    // Whatever's left in `remaining` has inDegree > 0 with no path to zero —
    // that subset forms at least one cycle.
    const stuck = [...remaining.keys()];
    throw new CyclicDependencyError(
      `Cycle detected among engines: ${stuck.join(', ')}. ` +
      `Check dependencies[] on each — one of them depends (directly or ` +
      `transitively) on itself.`
    );
  }
  return sorted;
}

function groupIntoBatches(sortedIds, graph) {
  // Re-derive batches from the sorted order + edge structure so that
  // engines with no dependency relationship to each other, even if they
  // appear far apart in `sortedIds`, still end up in the same parallel batch.
  const batchOf = new Map();
  for (const id of sortedIds) {
    const node = graph.get(id);
    const deps = node.def.dependencies;
    const batch = deps.length === 0
      ? 0
      : Math.max(...deps.map(d => batchOf.get(d))) + 1;
    batchOf.set(id, batch);
  }

  const batches = [];
  for (const [id, batch] of batchOf) {
    (batches[batch] ??= []).push(graph.get(id).def);
  }
  return batches;
}

Three details in that implementation are the product of real incidents, not theoretical caution, and are worth calling out individually.

Why the Error Message on a Missing Dependency Names the Likely Cause

The first version of this error simply said Dependency not found: bayesian-confidence. That is technically correct and practically useless, because the actual cause is almost always one of two things — either a genuine typo in the dependencies array, or (much more commonly, once the registry supported restriction) the dependency exists in the full registered set but is not active in the current execution context, because it is restricted to a different team than the one whose request is currently being scored. The second case is not a bug in either engine's definition; it is a modeling gap: an engine that depends on a team-restricted engine needs to either declare the same restriction itself, or explicitly tolerate the dependency being absent and degrade gracefully. Making the error message spell out that distinction cut the average time to diagnose this class of failure from what used to be a 20–30 minute confused Slack thread down to, in most cases, an immediate fix, because the on-call engineer reading the error message already knows which of the two categories they are looking at.

Why Batches Are Rebuilt Per Execution Context, Not Cached Globally

It might seem wasteful to run buildExecutionPlan() on every single pipeline invocation rather than once at startup and caching the result. In fact the platform does both: a global plan is built once at startup from every currently active, unrestricted engine, and that plan is cached and reused for the overwhelming majority of requests. But because restrictedToTeams means the effective active set differs by team, a per-team plan is computed lazily on first request for that team and cached with a key of (teamId, registryGeneration), where registryGeneration is a monotonically increasing counter bumped every time the 15-second override poller (mentioned above) detects any change. This gives the platform the performance of a cached plan for the steady state while still correctly invalidating every cached plan — global and per-team — the moment an operator changes an engine's activation state.

Why Cycle Detection Runs at Startup and in CI, Not Just at Startup

Catching a cycle at startup is far better than catching it at request time, but "far better than at request time" still means a broken deployment reaches production servers before anyone notices the process is crash-looping. The team added a CI check that imports engines/index.js in an isolated process and asserts that buildExecutionPlan(engineRegistry.getActive()) succeeds, as a required check on every pull request that touches any file under engines/. This moved cycle detection from "the first production deploy after the change" to "the pull request that introduced the change," which is a difference of, typically, hours to days versus the minutes a CI run takes.

Compute Budgeting & Adaptive Scheduling

Thirty-four engines, each with a declared computeCost between 1 and 10, sum to a meaningful number if every engine ran on every single request regardless of context. In practice, most requests only need a subset — a sales-domain request never triggers the batna-calculator negotiation engine, because that engine's domain is legal and the sales product adapter never asks for it. But even within a single domain, running every applicable engine on every request is sometimes more compute than the platform wants to spend, particularly for low-value, high-volume traffic like bulk lead re-scoring jobs.

The Budget Model

Each execution context (roughly: each product adapter, sometimes further split by request type) declares a computeBudget, an integer on the same 1–10-per-engine scale as computeCost. The scheduler sums the computeCost of every engine that would run for a given request, and if that sum exceeds the budget, it does not fail the request — it drops the lowest-priority engines from the plan until the remaining set fits, where priority is a separately declared field (not shown in the core schema above, because it is context-dependent rather than intrinsic to the engine, and lives in a per-adapter priority override table rather than on the engine definition itself).

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata

function enforceComputeBudget(batches, budget, priorityOverrides) {
  const flat = batches.flat();
  const totalCost = flat.reduce((sum, e) => sum + e.computeCost, 0);
  if (totalCost <= budget) return batches; // fits — nothing to drop

  // Sort by priority ascending (lowest priority dropped first), and within
  // equal priority, by computeCost descending (biggest wins first, since
  // dropping one expensive engine is preferable to dropping several cheap
  // ones that together add up to the same savings but cost more auditing
  // overhead to explain later).
  const droppable = flat
    .filter(e => e.riskLevel !== 'critical') // critical engines are NEVER dropped for budget
    .sort((a, b) => {
      const pa = priorityOverrides[a.engineId] ?? 5;
      const pb = priorityOverrides[b.engineId] ?? 5;
      return pa - pb || b.computeCost - a.computeCost;
    });

  let remaining = totalCost;
  const dropped = [];
  for (const engine of droppable) {
    if (remaining <= budget) break;
    dropped.push(engine.engineId);
    remaining -= engine.computeCost;
  }

  if (remaining > budget) {
    // Even dropping every non-critical engine wasn't enough — this means
    // critical engines alone exceed the budget, which is a configuration
    // error, not a runtime condition to silently tolerate.
    throw new ComputeBudgetExceededError(
      `Critical-risk engines alone (${remaining}) exceed budget (${budget}). ` +
      `Raise the budget or re-evaluate riskLevel assignments.`
    );
  }

  auditLog.recordBudgetDrop(dropped, { totalCost, budget });
  return rebatch(flat.filter(e => !dropped.includes(e.engineId)));
}

The rule that critical risk-level engines are never dropped for budget reasons, full stop, is the single most important line in that function. It means the governance wrapper's input engines — the ones responsible for safe-language mapping and harm detection — are structurally protected from ever being silently omitted because a traffic spike pushed a request over budget. The tradeoff is exactly the ComputeBudgetExceededError case: if critical engines alone exceed the budget, the system fails loudly rather than quietly under-governing output, and an operator has to either raise the budget or reconsider why a critical engine got so expensive.

Adaptive Throttling Under Load

Static per-request budgets handle steady-state traffic, but do not respond to platform-wide load — if every product adapter is simultaneously experiencing a traffic spike, per-request budgets alone do not protect the shared compute pool the engines run on. A separate, coarser mechanism monitors aggregate engine execution latency across the whole platform (p95 batch execution time, sampled every 10 seconds) and, if it crosses a threshold, temporarily lowers every context's effective budget by a shared multiplier until latency recovers.

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata

function currentBudgetMultiplier(p95BatchLatencyMs) {
  if (p95BatchLatencyMs > 4000) return 0.5;  // halve every budget
  if (p95BatchLatencyMs > 2000) return 0.75;
  return 1.0;
}

This is deliberately coarse and reactive rather than predictive, on the philosophy that a simple mechanism the on-call team can reason about during an incident beats a more sophisticated predictive scaler nobody can debug at 3 a.m. The multiplier is logged on every change, and reverting to 1.0 requires latency to stay below the lower threshold for a full sustained window (60 seconds), not just a single sample, to avoid oscillation.

Compute Cost Accounting in Practice

Declared computeCost feeds budgeting decisions, but the platform also tracks measured cost — actual milliseconds consumed per engine per execution — separately, aggregated daily per engine into a small reporting table. This measured data serves two purposes beyond the drift-detection use mentioned earlier: it feeds a monthly compute cost report broken down by product adapter (so the sales team can see, concretely, what running behavioral scoring costs per lead scored, informing pricing decisions elsewhere on this site), and it is the primary input when an engine author proposes revising their engine's declared computeCost during a version bump — the review checklist for any PR that touches computeCost requires a link to at least two weeks of measured data supporting the new value, precisely so that declared cost does not drift into pure guesswork over time.

Activation States in Depth

An engine can be in one of four states. This lets you control what runs in production without code deployments: active — runs, output exposed to adapters; restricted — runs only for specific team IDs; research-only — runs, output logged but never exposed; deprecated — registered but not executed.

What is not obvious from that four-item list is that these are not four independent settings an operator can jump between freely — they form a state machine with a specific, deliberately narrow set of legal transitions, enforced by the override-write endpoint, not left to operator discretion.

# (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
# License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
# Article published: 22 May 2026.
# URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata


        ┌─────────────────┐
        │  research-only   │◀──────────────┐
        └────────┬─────────┘               │
                  │ graduate                │ demote
                  │ (min. 14 days data)      │ (regression found)
                  ▼                         │
        ┌─────────────────┐        ┌────────┴─────────┐
        │      active       │───────▶│    restricted    │
        └────────┬─────────┘  scope  └────────┬─────────┘
                  │            down            │
                  │                            │ scope up
                  │                            │ (re-approve for all teams)
                  │◀───────────────────────────┘
                  │
                  │ sunset
                  ▼
        ┌─────────────────┐
        │   deprecated     │  (terminal — no transitions out)
        └─────────────────┘
    

Every arrow in that diagram corresponds to a specific, named transition function in the override-write endpoint, each with its own preconditions. There is deliberately no arrow directly from research-only to deprecated without passing through active or being explicitly force-deprecated with a documented reason (the one exception path, used rarely, for research engines whose approach turned out to be a dead end and are being retired without ever having graduated).

Transition: research-only → active (Graduation)

This is covered in full as a worked case study later in this article, but the precondition worth stating here is the hard rule: a research-only engine must have accumulated at least 14 days of production traffic, with its outputs logged but not exposed, before it is eligible for graduation, and the graduation request must attach a comparison report showing the engine's research-mode outputs against whatever downstream decision the platform actually made without it, to demonstrate the engine would have changed outcomes in a beneficial direction. Engines that would not have changed any outcomes are candidates for being deprecated directly from research-only, rather than graduated, on the reasoning that an engine that never disagrees with the status quo is not adding information.

Transition: active → restricted (Scoping Down)

This is the fast path, and it is fast by design — this is the transition the compliance-engineer incident described in the opening section needs to complete in seconds, not minutes. The override-write endpoint accepts this transition with a single required reason field and no additional approval gate, on the philosophy that scoping an engine down is safe to make easy, because it can only reduce the set of contexts an engine's output reaches, never expand it.

Transition: restricted → active (Scoping Up)

Deliberately asymmetric with the transition above: scoping an engine up, from restricted-to-a-few-teams to fully active, requires the same review gate as a fresh graduation from research-only, because expanding exposure is exactly the direction of change that needs a second set of eyes. An engine that was restricted because of a legal-team-flagged concern does not get to quietly become fully active again just because the flag was addressed for one team; it needs an explicit re-approval covering the original concern.

Transition: active/restricted → deprecated (Sunset)

Covered as a full case study below. The precondition enforced by tooling, not just process, is that getDependents(engineId) (the reverse index described earlier) must return an empty set before a deprecation request is accepted — every engine that depends on the one being deprecated must itself already be deprecated, or migrated to depend on something else, first.

Demotion: active → research-only

The escape hatch for when an engine that graduated turns out, in production, to have a problem serious enough that its output should stop reaching adapters immediately, but not serious enough (or not yet understood well enough) to warrant full deprecation. This is functionally similar to the active → restricted fast path in urgency, but demotes to research-only rather than restricted, because the underlying concern is usually about the engine's accuracy or behavior in general, not about which specific teams should see it.

Versioning & the Audit Trail

Version drift — always bump the engine version when scoring logic changes. Audit logs use it to explain why the same input produced different outputs on different dates. That single sentence, from the "what to watch for" list in earlier drafts of this article, undersells how central versioning is to the platform's ability to answer a question that comes up constantly in both customer support and legal review: "why did this person get a different score today than they got last month, given what looks like the same input?"

None of this is exotic — it is MLOps discipline applied to a scoring function instead of a full model, with the same non-negotiables: every deployed change is versioned, every historical output is traceable to the exact version that produced it, and rollback is a first-class operation rather than an afterthought.

The Version Bump Discipline

Every change to an engine's scoring logic — not its metadata, its actual computation — requires at minimum a patch version bump, enforced by a CI check that diffs the engine's logic file against the previous commit and fails the build if the logic changed but version in the adjacent registration call did not. Whether a given change warrants a patch, minor, or major bump follows ordinary semver reasoning adapted to scoring behavior specifically:

• Patch (1.2.0 → 1.2.1) — a bug fix that corrects clearly wrong output; a performance change with no behavioral difference on correct input.
• Minor (1.2.1 → 1.3.0) — a new signal is incorporated, or an existing weight is tuned, such that outputs shift for some inputs but the overall scoring model and its interpretation are unchanged.
• Major (1.3.0 → 2.0.0) — the scoring model itself changes shape: new fields in the output, a changed scale, or a fundamentally different algorithm. A major bump requires the graduation-style review gate even for an engine that is already active, because a major version is, for audit purposes, functionally a new engine wearing the old engine's engineId.

The Explainability Trace Schema

Every scored output is stored, at write time, alongside the exact engineId+version pair that produced it, in the bc_explainability_traces table introduced in Part 5:

-- (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
-- License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
-- Article published: 22 May 2026.
-- URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata

CREATE TABLE bc_explainability_traces (
  id            BIGSERIAL PRIMARY KEY,
  request_id    UUID         NOT NULL,
  engine_id     VARCHAR(80)  NOT NULL,
  engine_version VARCHAR(20) NOT NULL,
  input_hash    CHAR(64)     NOT NULL,   -- SHA-256 of the normalized input signals
  score         NUMERIC(6,4) NOT NULL,
  confidence    NUMERIC(6,4) NOT NULL,
  evidence      JSONB        NOT NULL,
  created_at    TIMESTAMPTZ  NOT NULL DEFAULT now()
);
CREATE INDEX idx_traces_engine_version ON bc_explainability_traces (engine_id, engine_version);
CREATE INDEX idx_traces_request        ON bc_explainability_traces (request_id);

A compliance query of the form "did this specific engine's scoring logic change between when this person was first scored and when they complained about the score" reduces to a single join between two rows in this table filtered by engine_id and ordered by created_at, comparing engine_version. Without the version discipline enforced at registration time, that column would either be missing entirely or unreliable, and the query would be unanswerable — which, for a platform whose entire pitch is auditability, would be close to fatal.

Deprecation Does Not Delete History

When an engine is deprecated, its rows in bc_engine_registry and its historical rows in bc_explainability_traces are never deleted. The engine simply stops being scheduled by the control plane. This is a deliberate retention decision: the audit trail's value is largely retrospective, and a compliance review conducted two years after an engine was deprecated still needs to be able to answer "what did engine X, version Y, actually compute for this input," which means both the registry entry describing what the engine was and the trace rows describing what it produced must remain queryable indefinitely, subject only to the platform's general data retention policy, not to the engine's own operational lifecycle.

Restricted Engines & Team Scoping

The restrictedToTeams field looks simple — an array of team IDs — but enforcing it correctly touches three different layers of the platform, and getting any one of them wrong reopens exactly the "stale, silently-permissive conditional" failure mode described in the opening section, just moved from application code into a different layer. It is also the mechanism that answers a related but distinct question — the one GDPR Article 22 raises about a person's right to know which automated system scored them — which is why the ADV governance mirror of this article treats the registry's restriction layer as a right-to-know question, not only an operational one.

Layer One: the Execution Plan Itself

The per-team execution plan cache mentioned in the dependency resolution section is the first enforcement point: when building a plan for a specific team, any engine whose activationState is restricted and whose restrictedToTeams does not include that team's ID is excluded from the candidate engine set before the DAG is even built. This means a restricted engine is not merely hidden from output — it does not run at all for teams outside its allow-list, which matters for compute cost and for avoiding any possibility of its output leaking through a side channel (a log line, a debug endpoint) that the governance wrapper does not cover.

Layer Two: the RBAC Cross-Check

Team ID in restrictedToTeams is validated, at override-write time, against the platform's actual team registry (covered from the legal-product angle in Role-Based Access Control for Legal Teams) — an operator cannot restrict an engine to a team ID that does not exist, which prevents the specific stale-ID bug from the opening section's failure-mode-three example from recurring in the registry's own restriction mechanism.

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata

async function setRestriction(engineId, teamIds, reason, actor) {
  const validTeams = await teamsService.getExistingTeamIds(teamIds);
  const invalid = teamIds.filter(id => !validTeams.includes(id));
  if (invalid.length > 0) {
    throw new ValidationError(`Unknown team IDs: ${invalid.join(', ')}`);
  }
  return overridesStore.write({
    engineId,
    activationState: 'restricted',
    restrictedToTeams: teamIds,
    reason,
    changedBy: actor,
  });
}

Layer Three: the Governance Wrapper's Defense in Depth

Even though layer one means a restricted engine's scoring function never executes for a disallowed team, the governance wrapper independently re-checks restriction on every output it processes, comparing the requesting team against the engine's current restrictedToTeams a second time immediately before output leaves the system boundary. This is deliberate redundancy: if a bug in the execution-plan cache ever served a stale plan (for instance, a caching key collision between two teams, which did happen once, briefly, during a refactor of the cache key format), the governance wrapper's independent check is what actually prevented leaked output in that incident, not the plan-building logic that had the bug. Two independent checks that both have to be correct, rather than one check trusted everywhere downstream, is the deliberate defense-in-depth posture applied throughout the governance layer.

Testing the Registry

The registry's test suite is organized in three tiers, and the distinction between them matters because they catch different classes of bug and run at different points in the development cycle.

Tier One: Schema and Validator Unit Tests

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata

describe('EngineDefinitionSchema', () => {
  it('rejects a non-kebab-case engineId', () => {
    expect(() => EngineDefinitionSchema.parse({ ...validDef, engineId: 'BayesianConfidence' }))
      .toThrow(/kebab-case/);
  });

  it('requires restrictedToTeams when activationState is restricted', () => {
    expect(() => EngineDefinitionSchema.parse({
      ...validDef, activationState: 'restricted', restrictedToTeams: undefined,
    })).toThrow(/restrictedToTeams/);
  });

  it('rejects a non-semver version string', () => {
    expect(() => EngineDefinitionSchema.parse({ ...validDef, version: 'v2' }))
      .toThrow(/semver/);
  });

  it('rejects self-dependency', () => {
    expect(() => EngineDefinitionSchema.parse({
      ...validDef, engineId: 'foo', dependencies: ['foo'],
    })).toThrow(/cannot depend on itself/);
  });
});

Tier Two: Full-Registry Integration Tests

These import the real engines/index.js barrel file in an isolated process and assert properties of the entire registered set — not any single engine.

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata

describe('full engine registry, real import', () => {
  it('builds a valid execution plan with no cycles', async () => {
    await import('../../engines/index.js');
    const plan = buildExecutionPlan(engineRegistry.getActive());
    expect(plan.batches.flat().length).toBe(engineRegistry.getActive().length);
  });

  it('every critical-risk engine has zero unmet dependencies among active engines', async () => {
    await import('../../engines/index.js');
    const critical = engineRegistry.getActive().filter(e => e.riskLevel === 'critical');
    for (const engine of critical) {
      for (const dep of engine.dependencies) {
        expect(engineRegistry.get(dep)?.activationState).not.toBe('deprecated');
      }
    }
  });

  it('every emitted topic that is subscribed to actually has an emitter', async () => {
    await import('../../engines/index.js');
    for (const engine of engineRegistry.getActive()) {
      for (const topic of engine.subscribesTopics) {
        const emitters = engineRegistry.getTopicEmitters(topic);
        expect(emitters.length).toBeGreaterThan(0);
      }
    }
  });
});

Tier Three: Contract Tests Against Real Product Adapters

The final tier does not test the registry in isolation at all — it runs each product adapter's real request-handling code path against a snapshot of the registry, asserting that specific, business-critical engines are present in the resulting plan for specific request shapes. For example, a contract test asserts that any request tagged matterType: 'litigation' to the legal SaaS platform adapter always includes governance-safety and compliance-benchmarking in its execution plan, regardless of what other engines are active or restricted at the time. This tier is what actually catches the scenario where a well-intentioned restriction change (perfectly valid at the registry layer) accidentally removes an engine a specific product adapter has a hard business requirement to always run.

Registry Introspection: the Admin API

Everything described so far is invisible unless something exposes it to the humans operating the platform. The registry ships a small, read-mostly REST API, mounted under /internal/registry, that backs the admin dashboard and is also used directly by the on-call runbook during incidents.

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata

// GET /internal/registry/engines
// Returns every registered engine with its EFFECTIVE state (code definition
// merged with any active override), not just the code-defined defaults.
router.get('/engines', requireRole('platform-admin'), async (req, res) => {
  const engines = engineRegistry.getAll().map(def => ({
    ...def,
    ...overridesStore.getEffectiveOverride(def.engineId),
    measuredComputeCostP50: await metricsStore.getP50Cost(def.engineId),
    lastVersionBump: await auditLog.getLastVersionBump(def.engineId),
  }));
  res.json({ engines, generation: engineRegistry.generation });
});

// GET /internal/registry/engines/:engineId/dependents
// The reverse-dependency lookup used before any deprecation.
router.get('/engines/:engineId/dependents', requireRole('platform-admin'), (req, res) => {
  res.json({ dependents: engineRegistry.getDependents(req.params.engineId) });
});

// POST /internal/registry/engines/:engineId/restrict
// The fast-path scope-down transition described in the state machine section.
router.post('/engines/:engineId/restrict', requireRole('platform-admin'), async (req, res) => {
  const { teamIds, reason } = req.body;
  if (!reason || reason.length < 10) {
    return res.status(400).json({ error: 'reason is required and must be descriptive' });
  }
  const result = await setRestriction(req.params.engineId, teamIds, reason, req.user.id);
  res.json(result);
});

Every write endpoint under this router requires a non-trivial reason string and records req.user.id as the actor, because the override history table described earlier is only as useful as the discipline that populates its reason and changed_by columns, and making those fields required at the API layer, rather than merely encouraged by convention, is what keeps six-months-later audit queries answerable rather than full of reason: "fixing" placeholder text.

Failure Modes & Operational Runbook

Three failure modes recur often enough in the registry's operational history to warrant a standing runbook entry each, rather than being rediscovered from scratch during every incident.

Runbook: "An Engine Is Missing From Output That Should Include It"

1. Check GET /internal/registry/engines/:engineId — confirm effective activationState. The most common cause, by a wide margin, is an active restriction the requester did not know about.
2. If activationState is active and unrestricted, check the compute budget drop log (auditLog.recordBudgetDrop entries) for the relevant time window — the second most common cause is the engine being dropped under an adaptive-throttling multiplier during a load spike.
3. If neither of the above explains it, check for a silent DAG-drop caused by a self-dependency or a dependency on a deprecated engine — this should be impossible given the validators described earlier, but the runbook keeps this step because it was the actual root cause once, before the self-dependency check existed.
4. If still unexplained, check the per-team execution plan cache for a stale generation number — confirm engineRegistry.generation matches what the cached plan was built against.

Runbook: "Registry Fails to Start"

A startup failure is, definitionally, the validators and the DAG builder doing their job — the failure message itself (a ZodError, an EngineRegistrationError, or a CyclicDependencyError) names the specific engine and the specific problem in essentially every case, because that specificity was a deliberate design goal of every validator shown in this article. The runbook step here is short: read the error message fully before doing anything else, because in the overwhelming majority of cases the fix is exactly what the message says, and the temptation to start debugging via console.log insertion before reading a already-specific error message is the single biggest source of wasted time in this particular failure mode.

Runbook: "Override Change Is Not Taking Effect"

Given the 15-second polling interval mentioned earlier, the first check is simply whether enough time has passed. If it has, the second check is whether the write actually succeeded — bc_engine_registry_override_history is append-only and cheap to query, so confirming the intended row landed is faster than reasoning about the poller. The one recurring real bug in this category was a request that updated the bc_engine_registry_overrides row (current state) successfully but failed to insert into the history table due to a transaction being committed out of order across the two writes; this is why, in the current implementation, both writes happen inside a single database transaction rather than as two sequential statements.

Case Study: Promoting bias-detection From Research-Only to Active

To make the graduation transition concrete rather than abstract, this section walks through an actual (lightly anonymized) graduation of the bias-detection cognitive engine, from the day it was first registered as research-only to the day its state flipped to active.

Day 0 — Initial Registration

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata

registerEngine({
  engineId:         'bias-detection',
  displayName:      'Cognitive Bias Detection Engine',
  category:         'cognitive',
  domain:           'cross-domain',
  riskLevel:        'high',           // deliberately conservative at first registration
  computeCost:      3,
  dependencies:     ['bayesian-confidence'],
  emitsTopics:      ['bias.flag.raised'],
  subscribesTopics: ['confidence.updated'],
  activationState:  'research-only',
  researchOnly:     true,
  version:          '0.1.0',
});

Note the 0.1.0 starting version — engines below 1.0.0 are, by team convention, understood to be pre-graduation and subject to breaking changes without a major bump, mirroring ordinary semver convention for pre-1.0 software.

Days 1–14 — Shadow Traffic

For the full minimum research period, the engine ran on every eligible request, its output written to bc_explainability_traces like any active engine, but never surfaced to any product adapter's response. During this window, the engine's own version bumped twice (0.1.0 → 0.2.0 after a weighting adjustment; 0.2.0 → 0.3.0 after a false-positive-rate fix identified by comparing its flags against a manually reviewed sample), both routine patch-cycle activity that the research-only state allowed without any downstream consequence to a real decision.

Day 15 — the Graduation Report

The graduation request, submitted through the same override API described above but routed to a review queue rather than applied immediately, attached a report comparing 3,412 shadow-scored requests against the actual downstream decisions the platform made without the engine's input. The headline finding: for 71 requests, the bias-detection engine's flag would have changed the recommended action surfaced to a human reviewer, and a manual audit of a random sample of those 71 confirmed the flag was directionally correct in 64 of them (90%), with the remaining 7 being borderline cases neither clearly correct nor clearly wrong. That 90% figure, combined with the low absolute volume of affected requests (71 out of 3,412, roughly 2%), was the basis for the review board approving graduation with riskLevel kept at high rather than lowered, on the reasoning that a 2%-of-requests impact rate on a bias-related signal warranted continued conservative process even after graduation.

Day 15 — the Transition Itself

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata

await graduateEngine('bias-detection', {
  reason: 'Graduation review approved 2026-06-05, ref GOV-1142. ' +
          '71/3412 shadow requests would have changed outcome, 90% ' +
          'directional accuracy on manual audit sample.',
  actor: 'platform-admin:review-board',
  newVersion: '1.0.0',  // graduation itself is treated as a major version event
});

The version bump to 1.0.0 at the moment of graduation, even though no scoring logic changed on that specific day, is deliberate: it gives the audit trail a clean marker separating every trace row with engine_version LIKE '0.%' (shadow-mode data, never exposed) from every row at 1.0.0 and above (data that actually influenced a real decision), which is exactly the distinction a future compliance review is most likely to need.

Every graduation like this one is, functionally, a small act of AI adoption enablement — the whole point of the research-only stage and the graduation report is to let a genuinely useful engine earn production trust on evidence, rather than either shipping untested or being blocked indefinitely by caution.

Case Study: Deprecating an Engine Safely

The mirror case — retiring an engine — is walked through here using a composite example based on the platform's actual deprecation of an early, superseded version of the geometric-topological foundational engine, referred to here as geometric-topological-v1 to distinguish it from its eventual replacement.

Step One: the Dependents Check

# (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
# License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
# Article published: 22 May 2026.
# URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata

curl -H "Authorization: Bearer $ADMIN_TOKEN" \
  https://internal.example/internal/registry/engines/geometric-topological-v1/dependents
# {"dependents": ["meta-learning"]}

The reverse index immediately surfaces that meta-learning depends on it. Deprecation cannot proceed until that dependency is resolved — either meta-learning is deprecated alongside it (not appropriate here, since meta-learning is still useful), or it is migrated to depend on the replacement engine instead.

Step Two: Migrate the Dependent

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata

// engines/foundational/meta-learning.engine.js — before
registerEngine({
  engineId: 'meta-learning',
  dependencies: ['geometric-topological-v1'],
  // ...
});

// after — dependency swapped, version bumped (minor: no output shape change,
// but the input source changed, which the team treats as at least minor)
registerEngine({
  engineId: 'meta-learning',
  dependencies: ['geometric-topological'], // the v2 replacement's real engineId
  version: '2.4.0', // was 2.3.1
  // ...
});

Step Three: Confirm Zero Dependents, Then Deprecate

# (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
# License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
# Article published: 22 May 2026.
# URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata

curl -H "Authorization: Bearer $ADMIN_TOKEN" \
  https://internal.example/internal/registry/engines/geometric-topological-v1/dependents
# {"dependents": []}

curl -X POST -H "Authorization: Bearer $ADMIN_TOKEN" \
  -d '{"reason": "Superseded by geometric-topological v2, GOV-1180. Last dependent (meta-learning) migrated in PR #2291."}' \
  https://internal.example/internal/registry/engines/geometric-topological-v1/deprecate

Only once the dependents list returns empty does the deprecation endpoint accept the request — this ordering, dependents-check before deprecation-write, is enforced server-side, not just by runbook discipline, specifically because a runbook step that a human can skip under time pressure is not a guarantee, and this particular guarantee (never break a live dependent) is one the team decided was worth encoding in code rather than trusting to process.

Performance & Scaling Past 100 Engines

Everything described in this article was validated against a registry of 34 entries, and every data structure and algorithm choice was made with an explicit eye toward the platform's near-term roadmap of well over 100 engines as new product adapters and new behavioral domains come online. It is worth stating plainly where the current design holds up and where it will need to change.

What Scales Without Modification

The primary Map-based registry, its secondary indices, and Kahn's algorithm are all linear or near-linear in the number of engines and edges. At 100 engines with an average of 2–3 dependencies each, the full DAG build and topological sort completes in well under a millisecond on ordinary server hardware — this was benchmarked directly by synthetically generating a 300-engine registry with realistic dependency density and measuring buildExecutionPlan() repeatedly; it stayed under 2ms even at nearly triple the current roadmap size. None of the core algorithms need to change for the platform's foreseeable growth.

What Will Need Attention

The 15-second polling interval for override changes was chosen as a simple, adequate mechanism at 34 engines with a handful of restriction changes per week. At significantly higher change volume — anticipated once more teams have self-service access to restrict engines relevant to their own product surface — polling every 15 seconds against a Postgres table will remain cheap for the database, but the up-to-15-seconds staleness window may become operationally uncomfortable for teams used to instant effect. The team's current plan, not yet implemented, is to move override propagation to a lightweight pub/sub notification (using the same event bus infrastructure the engines themselves use) that pushes changes to every control-plane process immediately, with the 15-second poll retained only as a fallback safety net rather than the primary mechanism.

What Will Need Rethinking

The admin dashboard's engine list view, which currently renders all 34 engines in a single flat table, will not remain usable at 100+ without filtering, search, and probably a category-grouped default view rather than a flat list — a straightforward UI problem, not an architectural one, but worth naming here because it is the first piece of this system likely to visibly strain under growth, well before the underlying data structures do.

Security Considerations

The registry sits close enough to the platform's core decision-making that its own security posture deserves explicit treatment, separate from the governance wrapper's job of securing engine output. This is governance support work in the most literal sense: making sure the infrastructure a compliance decision depends on is itself trustworthy.

Registration Is Not an Externally Reachable Action

registerEngine() is only ever called from code that ships through the platform's normal code review and deployment pipeline — there is no API endpoint, internal or external, that accepts an arbitrary engine definition and registers it at runtime. This is a deliberate constraint: allowing dynamic, request-time registration of new scoring logic would mean the registry's dependency graph, compute budget, and risk-level review process could all be bypassed by anyone with write access to whatever mechanism accepted the registration, which defeats essentially every governance property this article has described. New engines exist exclusively as code, reviewed exclusively through the same pull-request process as everything else in the codebase.

The Override API Is the Actual Attack Surface

Because the override endpoints (restrict, graduate, deprecate) are reachable at runtime by design, they are the part of the registry that receives the most security scrutiny. Every override endpoint requires the platform-admin role, which is a small, explicitly provisioned group, not a broad internal role that accumulates members over time by default; access to it is reviewed quarterly as part of the platform's standard access-review cycle. Every write is logged with actor identity, and the audit history tables are append-only at the database level (no UPDATE or DELETE grants exist on bc_engine_registry_override_history for the application's database role, only INSERT and SELECT), so even a fully compromised application server cannot rewrite history to cover its tracks, only add new (attributable) rows.

Why the Registry Never Trusts Client-Supplied Engine Selection

No product adapter, and no external API caller, is ever able to specify which engines run for their request. The execution plan is derived entirely from the requester's authenticated team ID and domain, matched against the registry's activation and restriction state — there is no engines: [...] parameter anywhere in any public or adapter-facing API. This closes off an entire category of potential abuse where a caller might attempt to force a restricted or research-only engine to run, or force an expensive engine to run repeatedly as a denial-of-service vector against the compute budget.

Comparing the Registry Pattern to Alternatives

Before settling on the design described in this article, the platform team evaluated three existing patterns from adjacent problem spaces, on the theory that "engine orchestration with runtime-toggleable activation" is not a novel problem category in the abstract, even if the specific behavioral-AI application is. None of the three turned out to be a drop-in fit, but each contributed something, and it is worth explaining why each was rejected as a wholesale solution, because engineers joining the team consistently propose one of these three first.

Feature Flags

The most obvious comparison is a feature-flag system — LaunchDarkly, a homegrown flag service, or similar. Feature flags solve exactly one piece of the registry's job: toggling a boolean (or a small enum) at runtime without a deployment. What they do not solve is dependency ordering, compute budgeting, or the structured metadata (risk level, compute cost, emitted topics) that downstream systems like the compliance drift monitor and the admin dashboard depend on. A pure feature-flag approach was actually the platform's very first iteration — each engine's active/inactive state lived behind a flag — and it broke down almost immediately for a reason specific to this domain: flags are typically evaluated independently, per-flag, with no awareness of one flag depending on another. The team found itself manually keeping flag states consistent across dependent engines by convention, which is precisely the scattered-logic failure mode the registry exists to eliminate. Feature flags remain in use elsewhere on the platform, for things like UI rollouts and non-engine operational toggles, but engine activation state moved entirely into the registry once the dependency problem became apparent.

Service Mesh / Sidecar Patterns

A service mesh (Istio-style) solves routing, retries, and traffic shaping between independently deployed services, and it was considered seriously during an early design review, on the reasoning that if each engine were its own microservice, mesh-level traffic policies could handle restriction (route requests from restricted teams away from the engine) and even something resembling compute budgeting (via rate limiting). This was rejected for a straightforward operational reason: the 34 engines are not, and are not planned to become, independently deployed services. They are functions within a single Node.js process (or a small number of worker processes), chosen deliberately because the actual per-engine compute cost is small enough that the network overhead of inter-service calls between 34 separately deployed services would dwarf the computation itself for the majority of engines, and because a request that fans out to a dozen network calls introduces a dozen new failure modes (partial failures, timeout tuning per hop, distributed tracing overhead) that a single-process design avoids entirely. A service mesh remains the right tool for genuinely independent services elsewhere in the broader platform — the audio conversion microservice discussed in Building a Python HTTP Microservice for Audio Conversion is one such case — but not for engines whose entire value proposition includes being cheap enough to run a dozen of on every request.

Plugin Systems

Plugin architectures (the WordPress hook system, VS Code's extension model, or a homegrown equivalent) solve the self-registration and discovery problem well, and the registry's self-registration pattern is, honestly, closer in spirit to a plugin system than to either of the two alternatives above. Where a typical plugin system falls short for this use case is governance: plugin systems are generally designed to be permissive by default, optimizing for third-party extensibility, with sandboxing (if any) focused on preventing a plugin from crashing the host process, not on enforcing structured metadata like risk level or dependency contracts between plugins. The registry borrows the self-registration ergonomics of a plugin system while adding the strict, schema-validated, fail-fast-at-startup posture that a plugin system aimed at trusted, internally-authored code can afford to have and that a system aimed at untrusted third-party plugins typically cannot.

What the Registry Actually Is

In the vocabulary of distributed systems literature, the closest existing description is probably "a typed, dependency-aware plugin registry with policy-based runtime gating" — which is a mouthful precisely because it borrows one property from each of the three patterns above (self-registration from plugins, runtime toggling from feature flags, and a notion of declared contracts from service mesh policy) while deliberately not adopting any of the three wholesale. This is mentioned explicitly in the patent-defensibility discussion referenced from Part 1 of this series: the specific combination of properties, applied to behavioral scoring engines with the governance requirements this platform has, is the part that is more than the sum of any single existing pattern.

Migrating an Ad-Hoc Pipeline to the Registry Pattern

The opening section of this article showed the pre-registry scoreLead() handler as a cautionary example. This section shows the actual migration path the team followed to move away from it, because "just use a registry" is not, by itself, an actionable instruction for a codebase that already has production traffic flowing through hard-wired function calls. It is, in miniature, what digital transformation actually looks like at the engineering layer — not a rebrand, but a sequenced, low-risk migration off load-bearing legacy code.

Step One: Inventory Before Touching Any Code

The first step was not code at all — it was a spreadsheet. Every existing hard-wired scoring call across the codebase was found via a project-wide grep for the scoring functions' names, and each call site was recorded with its file, its calling context, and a first guess at that engine's category, domain, and risk level. This inventory step surfaced two engines nobody had remembered existed (called from the "one-off script a data scientist wrote and forgot to delete" mentioned earlier), both of which turned out to be dead code safe to delete outright rather than migrate — a small but real win from the inventory step alone, before a single line of registry code was written.

Step Two: Introduce the Registry Alongside the Old Code, Not Instead of It

Rather than a big-bang rewrite, the registry was introduced as an additive layer first. Every engine got a real registerEngine() call and a real entry in engines/index.js, but the old hard-wired call sites in scoreLead() and elsewhere were left completely untouched and continued to call the underlying scoring functions directly. This meant the registry existed, was fully populated, and could be validated (dependency graph, schema) in production without changing behavior for a single real request — a deliberately low-risk first deployment.

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata

// Transitional state: registry exists and is populated, but scoreLead()
// still calls functions directly. This shipped and ran in production for
// two weeks with zero behavioral change, purely to validate the registry
// itself under real traffic patterns (import order, registration timing).
async function scoreLead(signals, context) {
  const entropy    = await scoreBehavioralEntropy(signals);       // unchanged
  const confidence = await scoreBayesianConfidence(signals, entropy); // unchanged
  const stateTransition = await scoreStateTransitions(signals, confidence); // unchanged

  if (context.team === 'legal-beta') {
    const legalRisk = await scoreLegalRiskPosture(signals);
    return { entropy, confidence, stateTransition, legalRisk };
  }
  return { entropy, confidence, stateTransition };
}

Step Three: Cut Over One Product Adapter at a Time

Once the registry had run in production, side-by-side with the old code path, for two weeks with no discrepancies found by a comparison job (which ran both paths for a sampled percentage of real traffic and diffed the outputs), the team cut each product adapter over to the control-plane-driven execution plan one at a time, starting with the lowest-traffic, lowest-risk adapter (an internal debugging tool) and ending with the highest-traffic one (the chatbot platform's lead-scoring path). Each cutover was a single, small, easily revertible change: replace the hard-wired call sequence with a call to runPipeline() (shown in Part 1 of this series) for that adapter's context.

Step Four: Delete the Old Path

Only after every adapter had been cut over, and had run on the registry-driven path for a further two weeks without incident, were the original hard-wired functions and their direct call sites deleted. This last step was, in the team's retrospective notes, the single most satisfying commit in the whole migration — a large, purely subtractive diff, made safe specifically because the inventory from step one gave confidence that nothing was being silently orphaned by the deletion, unlike the pre-registry era where "can we delete this" was never a question anyone could answer with confidence.

What Would Be Done Differently

The comparison job in step three — running both paths and diffing outputs — was added partway through the migration, after the second product adapter's cutover produced a discrepancy that a simpler side-by-side deployment without automated diffing had missed for four days. In hindsight, the team's clear recommendation for anyone doing a similar migration is to build that comparison tooling first, before the first cutover, not as a reactive addition after the first near-miss.

Common Anti-Patterns in Engine Design

Beyond the schema-level validation described earlier, a handful of design mistakes recur across engine pull requests that no validator catches automatically, because they are shape-valid but poorly designed. These are flagged in code review by convention, and are collected here because a new contributor to the engine codebase benefits from seeing them named explicitly rather than discovering each one independently.

Anti-Pattern: the God Engine

An engine whose scoring function grows, over successive PRs, to compute several logically distinct things and return them bundled in one output object, because it was easier to add a field to an existing engine than to register a new one. The bayesian-confidence engine nearly fell into this trap early on, when a contributor proposed adding contradiction-detection logic directly into it rather than creating a separate engine, on the reasoning that contradiction detection "needs the same confidence inputs anyway." The fix applied in review was to keep bayesian-confidence narrow and instead have the new contradiction-detection engine declare a dependencies: ['bayesian-confidence'] relationship and consume its output via the event bus — exactly the mechanism the registry exists to make easy. The tell-tale sign of a god engine in review is an output object with more than three or four top-level fields that are not obviously related, or a scoring function whose name uses "and" ("scoreConfidenceAndContradiction").

Anti-Pattern: Hidden Cross-Engine Coupling

An engine that reads a value that another engine wrote to a shared cache, global variable, or database row — outside the declared dependencies/subscribesTopics contract — because it was faster to write during a deadline crunch than to properly declare the dependency. This is worse than it sounds: it means the dependency exists in reality but is invisible to the DAG builder, so the control plane can (and eventually will) schedule the two engines out of the order the hidden coupling actually requires, producing an intermittent bug that only manifests when execution timing happens to disfavor it. This exact anti-pattern caused a two-day production investigation early in the platform's history, tracked down eventually by a contributor who noticed an engine's output varied between two runs with byte-identical input, which should be structurally impossible for a pure scoring function and was the tell that hidden state was involved. The team's response was to add a lint rule that flags any engine file importing anything from another engine's directory other than through the registry/event-bus interfaces.

Anti-Pattern: Overclaiming Confidence

An engine that returns a high confidence value regardless of input completeness, because the author did not implement the low-evidence degradation described in Part 4 of this series. This is caught, in part, mechanically — a CI check runs every registered engine against a battery of deliberately sparse synthetic inputs and flags any engine whose reported confidence does not drop meaningfully as input completeness decreases — but the mechanical check is a backstop, not a substitute for engine authors internalizing that confidence is not optional decoration on a score, it is load-bearing information the governance wrapper and downstream adapters actually use to gate human review.

Anti-Pattern: Silent Fallback to a Default Score

An engine that, on encountering unexpected or malformed input, returns a plausible-looking default score (commonly, something near the middle of the scale) rather than throwing or explicitly signaling low confidence. This is dangerous specifically because it looks identical, from the output shape alone, to a genuine mid-range score, and the governance wrapper has no way to distinguish "the model genuinely computed 0.5" from "the model crashed internally and 0.5 is a hardcoded fallback" unless the engine is honest about which happened via the confidence field or an explicit error signal. The team's standing rule, stated in the engine-authoring guidelines: an engine that cannot score its input must either throw (if the control plane should treat this as a hard failure for the whole request) or return an explicit near-zero confidence with a labeled reason (if the pipeline should degrade gracefully) — it must never fabricate a plausible-looking score to paper over an internal problem.

Anti-Pattern: Dependency Sprawl

An engine that declares dependencies on far more upstream engines than it actually needs, "just in case," because the author was not sure exactly which upstream signals mattered and included everything available. Beyond the obvious cost implication (every declared dependency widens the engine's batch position in the DAG, potentially delaying when it can run), dependency sprawl makes the reverse-dependency index less useful for deprecation planning, since it inflates the apparent blast radius of deprecating any of the over-declared upstream engines. Code review for new engines specifically asks the author to justify each declared dependency by naming the specific field or topic it actually consumes, and PRs with unjustified dependencies are a routine source of review comments.

Code Review Checklist for New Engine Pull Requests

New engine PRs go through the platform's standard review process, but engine PRs specifically are checked against an additional list, kept as a living document alongside the codebase and reproduced here because it is the most concrete distillation of everything discussed so far in this article.

None of these nine checks is individually sophisticated. Their value is cumulative and mechanical — a reviewer working through the same list on every engine PR, rather than relying on remembering everything discussed in an article like this one from memory, is what actually keeps a registry with dozens of contributors from slowly reaccumulating the exact problems the registry was built to eliminate in the first place.

Multi-Tenant & Multi-Region Considerations

The registry as described so far is single-process and single-region in its simplest deployment. Two extensions matter once the platform runs across multiple regions and serves genuinely multi-tenant traffic at scale, and both are worth covering because they are the most common source of "does the registry pattern actually work at our scale" questions from engineers evaluating it for adoption elsewhere in the broader product suite.

Multi-Region: Registry Generation Consistency

Each region runs its own control-plane processes, each with its own in-memory registry built from the same engines/index.js code (identical across regions, since it ships through the same deployment pipeline) but polling its own regional replica of bc_engine_registry_overrides. Because the override tables are read-replicated from a single primary (writes are only accepted in the primary region, with cross-region override writes routed there), there is an inherent replication lag between when an override is written and when every region's poller picks it up — bounded, in practice, by replica lag (typically under a second) plus the up-to-15-second poll interval. For the fast-path restriction transition described in the activation-state section, this means a compliance-driven restriction can take up to roughly 16 seconds to take effect in the furthest region, which the team has accepted as tolerable given the incident scenario it serves (stopping an engine from running is still dramatically faster than a deployment, even with this bound), but it is an explicit, documented limit rather than an accidental gap discovered during an actual incident.

Multi-Tenant: Per-Tenant Compute Budget Overrides

The compute budgeting section above described a budget per execution context (roughly, per product adapter). At full multi-tenant scale, a single product adapter serves many distinct customer tenants, some on lower-cost pricing tiers that intentionally include a lower behavioral-scoring compute budget (tying back to the pricing model referenced elsewhere on this site, where lower seat tiers include fewer active engines running per scored interaction). This is implemented as a further override layer on top of the adapter-level budget: a per-tenant budget multiplier, defaulting to 1.0, stored alongside tenant billing configuration rather than in the registry's own override tables, and applied by the scheduler as an additional multiplication step immediately after the adaptive-throttling multiplier described earlier. Keeping tenant-tier budget configuration in the billing system rather than duplicating it into the registry's override schema was a deliberate choice to avoid two systems of record disagreeing about what a given tenant is entitled to.

Where Tenant Isolation Actually Lives

It is worth being explicit that tenant isolation for behavioral scoring is not primarily a registry concern at all — the registry's restriction mechanism operates at the team level (internal platform teams, like "legal-beta"), not at the level of individual customer tenants, which are a product-adapter-layer concept sitting above the registry entirely. A customer tenant's data never becomes visible to another tenant because each pipeline execution is scoped to a single request with a single tenant's signals as input; there is no shared mutable state between concurrent executions for different tenants at the engine level, since engines are pure(-ish) functions of their input, not stateful objects that could leak state across concurrent invocations.

Appendix: the Full Engine Catalog

The table below lists every engine referenced across this article and its sibling articles in the series, with the metadata fields most relevant to registry design. It is reproduced here as a single reference point, since the individual category articles (Parts 9 through 22 of this broader documentation set) each cover only their own category in depth.

This is 27 of the platform's 34 registered engines — the remaining 7 are product-adapter-specific variants (domain-scoped copies of a handful of the engines above, registered under distinct engineIds such as power-dynamics-sales for the chatbot platform's own narrative scoring needs) and are omitted here for brevity, but follow the identical registration pattern shown throughout this article.

Frequently Asked Questions

Can an engine belong to more than one category?

No — category is a single string, not an array, by design. An engine whose logic genuinely spans two categories is a sign it should probably be split into two engines with a declared dependency between them, per the god-engine anti-pattern discussion above, rather than tagged with multiple categories.

What happens if two engines emit the same topic?

This is explicitly allowed. The #topicEmitters index (shown in the internal data structures section) maps a topic to a set of engineIds, not a single one, precisely to support this. A subscriber to that topic receives events from every emitter; the event payload itself carries the emitting engineId, so a subscriber that cares can distinguish sources, and one that doesn't care can simply aggregate.

Is there a limit to how many engines can depend on a single engine?

No hard limit is enforced, but in practice, an engine with a very large number of dependents (bayesian-confidence currently has the most, at over a dozen) is treated as a de facto platform primitive, and changes to it go through a heightened review process regardless of its own declared riskLevel, because the blast radius of a regression is a function of the dependent count, not just the engine's own risk classification.

Can restrictedToTeams reference a team that only exists in one region?

Yes, and this is the normal case for region-specific pilot programs — a new engine is frequently restricted to a single region's beta team before wider rollout, entirely independent of the multi-region replication mechanics described in the multi-tenant section above.

What happens to in-flight requests when an engine transitions state mid-request?

Each request's execution plan is built once, at the start of that request's pipeline run, from a single consistent snapshot of the registry's effective state (tagged with the registryGeneration counter). A state change that occurs after a plan has been built does not retroactively alter that in-flight request — it only affects the next request's plan-building. This is a deliberate consistency guarantee: a single request never sees a mix of pre-change and post-change engine behavior partway through its own execution.

Why is there no owner or team field on the engine definition itself, given how central ownership is to the review process?

Ownership is tracked in the repository's code-ownership file (a standard CODEOWNERS-style mechanism mapping directory paths to teams), not duplicated into the registry schema, on the reasoning that ownership is a property of the code, changes independently of engine behavior, and would otherwise need its own change-review process separate from an ordinary code change if it lived in the schema. This is a deliberate instance of the same principle that keeps displayName out of any logic path: metadata that humans need but machines never branch on lives in the place best suited for humans to maintain it, which is not always the registry schema.

Does every engine need a corresponding ADV governance mirror article?

In practice, yes for any engine at high or critical risk level — this is a content-process convention tracked in the content plan rather than a registry-enforced rule, but it is checked as part of the code review checklist's cross-reference item described above, and engines shipped without their governance counterpart are a recurring content-debt item the editorial process tracks separately from engineering debt.

What is the actual performance cost of running the full validator (schema plus DAG build) on every process start?

Under 5ms for the full 34-engine registry on ordinary server hardware, measured directly, which is immaterial against typical process startup time dominated by other initialization (database connection pooling, and so on). This is one of the figures cited in the performance section above, restated here because it is the single most common question raised by engineers concerned that "validate everything at startup" implies a slow boot.

Glossary

Timeline: How the Registry Evolved

The version history of the registry's own design, distinct from any individual engine's version, is a useful closing lens on this article, because almost every feature described above was added in response to a specific, dated need rather than designed in from a blank slate on day one.

• Quarter 1 — the pre-registry era described in the opening section: three engines, hard-wired calls, no metadata.
• Quarter 2 — first registry implementation: flat Map<engineId, definition>, no validation beyond "does this object have the required keys," no dependency graph. Engines still called each other directly in a few places (this is when the hidden-coupling anti-pattern incident occurred).
• Quarter 2, later — Kahn's-algorithm-based dependency resolution added, directly motivated by the hidden-coupling incident; the lint rule preventing direct cross-engine imports added the same sprint.
• Quarter 3 — activationState and researchOnly introduced as separate fields, replacing an earlier single boolean enabled flag that could not express "run but don't expose," which the team had been working around with a hacky silent: true parameter passed at call sites.
• Quarter 3, later — runtime schema validation (the Zod schema shown earlier) added after a malformed engine definition with a missing riskLevel shipped to production and was only caught because the compute budget allocator crashed trying to read a field that did not exist.
• Quarter 4 — restrictedToTeams and the override-table split (code-defined behavior versus runtime-writable activation state) introduced, directly motivated by the compliance-engineer incident described in this article's opening section.
• Quarter 4, later — compute budgeting and adaptive throttling added ahead of the platform's first major traffic spike (a product launch that drove roughly 6x normal request volume for two days), specifically to ensure critical-risk engines would never be silently dropped under that kind of load.
• Present — the design described throughout this article, with multi-region override replication and per-tenant budget overrides as the most recent additions, and the event-bus-based override propagation mentioned in the performance section as the next planned change.

The throughline across every entry in that timeline is the same one stated in the opening section: each addition removed a class of bug that had already happened at least once in production, rather than anticipating a hypothetical one. That is a deliberate editorial choice for how this platform's engineering culture approaches infrastructure work generally, and the registry is simply the clearest example of it within this series.

Worked Example: Tracing a Single Request End to End

Every mechanism described so far — the DAG, the budget enforcer, the governance handoff — is easier to hold in your head as a concrete trace through real numbers than as abstract algorithm descriptions. This section follows one synthetic-but-realistic request from the legal SaaS platform adapter through the entire registry-driven pipeline, with actual intermediate values at each step.

The Request

A paralegal on a litigation matter uploads a client intake call transcript. The legal SaaS platform adapter calls the control plane with { domain: 'legal', matterType: 'litigation', teamId: 'team_442' } as context, alongside the extracted behavioral signals from the transcript.

Step 1 — Candidate Engine Selection

The control plane asks the registry for every engine where domain is legal or cross-domain, and whose effective activationState (code definition merged with any override) is active, or restricted with team_442 present in restrictedToTeams. For this trace, that candidate set is 19 engines: the full cross-domain set (behavioral-entropy, information-theory, geometric-topological, meta-learning, bayesian-confidence, bias-detection, emotional-regulation, state-machine-runtime, causal-graph, prediction-horizon, knowledge-graph, embedding-index, provenance-tracker, epistemic-intelligence, behavioral-vm-kernel, governance-safety, compliance-benchmarking) plus the two legal-domain negotiation engines relevant to litigation matters (power-dynamics, authority-mapping). batna-calculator and tactical-negotiation are excluded — their metadata (not shown in the catalog table for brevity) restricts them to matterType: 'settlement', not litigation, via an adapter-level matter-type filter applied on top of the registry's own domain filter.

Step 2 — DAG Build and Batching

Running buildExecutionPlan() against those 19 engines, using the dependency edges from the catalog table earlier in this article, produces five batches:

# (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
# License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
# Article published: 22 May 2026.
# URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata

Batch 0 (no dependencies, run immediately):
  behavioral-entropy, geometric-topological, bayesian-confidence,
  knowledge-graph, embedding-index, behavioral-vm-kernel

Batch 1 (depend only on batch 0):
  information-theory, meta-learning, bias-detection,
  emotional-regulation, state-machine-runtime, provenance-tracker,
  epistemic-intelligence, governance-safety

Batch 2:
  causal-graph, narrative-arc-dependency-chain-not-selected...
  power-dynamics (via narrative-arc, itself in batch 1 for legal domain),
  compliance-benchmarking

Batch 3:
  prediction-horizon, authority-mapping

Batch 4:
  (governance wrapper runs here — not itself a batch, but the pipeline's
  final stage, consuming every batch's output)

Total declared compute cost across all 19 engines: 78. The legal SaaS platform adapter's configured budget for a litigation-matter transcript scoring request is 90, comfortably above 78, so no budget-driven dropping occurs on this particular request — a deliberately "boring" first pass through the numbers, precisely because most real requests do fit within budget, and the interesting case (dropping) is a minority path worth seeing separately.

Step 3 — Batch Execution and Event Emission

Batch 0 runs entirely in parallel via Promise.all. bayesian-confidence completes in 4ms and emits onto confidence.updated. behavioral-entropy completes in 6ms and emits onto entropy.computed. By the time batch 1 begins, both events have already been published to the bus, so state-machine-runtime (which subscribes to confidence.updated) and bias-detection (same subscription) both have their required upstream data available the instant they start. This is the concrete payoff of the batching approach: batch 1's six engines all start at effectively the same wall-clock instant, rather than being serialized behind each other.

Step 4 — a Budget-Constrained Variant of the Same Request

Now suppose the same paralegal's firm is on a lower pricing tier with a per-tenant budget multiplier of 0.6 applied (the multi-tenant mechanism described earlier), dropping the effective budget from 90 to 54. Total declared cost of 78 now exceeds budget by 24. The budget enforcer sorts the 13 non-critical engines (all except behavioral-vm-kernel, governance-safety, and compliance-benchmarking, which are exempt) by priority, and in this trace, prediction-horizon (cost 6, lowest priority for this matter type per the adapter's priority overrides) and digital-twin-simulation-adjacent engines are not even in the candidate set to begin with, so the drop instead falls on epistemic-intelligence (cost 4), provenance-tracker (cost 3), meta-learning (cost 3), authority-mapping (cost 5), causal-graph (cost 5, but this would orphan prediction-horizon's dependency — the budget enforcer's actual implementation, elided from the simplified version shown earlier in this article, re-runs the dependents check from the deprecation tooling before finalizing any drop set, and re-batches around whichever engines survive), continuing until the running total falls to 54 or below. The resulting audit log entry for this request records exactly which five engines were dropped and why, queryable later if the paralegal's professional services firm ever asks why a particular signal was missing from their result.

Step 5 — the Governance Handoff

Whichever variant ran, the final batch's output — a flat object keyed by engineId, each value containing { score, confidence, label, evidence } — is handed to wrapWithGovernance(). In this trace, bias-detection returned a moderately high score (0.71) with high confidence (0.88) on one specific transcript segment. The safe-language map converts its internal label anchoring_bias_pattern to the client-facing phrase "reasoning pattern that may benefit from a second perspective", and because 0.71 exceeds the review threshold configured for this engine, requiresHumanReview: true is set on that specific field before the response reaches the legal SaaS platform adapter, which surfaces it as a flagged item in the paralegal's review queue rather than an automated action.

Appendix B: the Registry's Own Database Mirror

Earlier sections described bc_engine_registry_overrides, the table that holds runtime-writable activation state. There is a second, related table, bc_engine_registry, that deserves its own explanation because its existence is easy to misunderstand: it is not the runtime source of truth (that remains the in-memory registry, rebuilt from code on every process start, as emphasized earlier) — it is a queryable mirror, written once per deployment, that exists purely to make the registry's code-defined shape available to tools that cannot easily import a running Node.js process's memory: the reporting dashboards, the quarterly compute-cost report mentioned in the budgeting section, and ad-hoc analyst queries from the data team.

-- (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
-- License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
-- Article published: 22 May 2026.
-- URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata

CREATE TABLE bc_engine_registry (
  engine_id           VARCHAR(80)  PRIMARY KEY,
  display_name        VARCHAR(120) NOT NULL,
  category             VARCHAR(40)  NOT NULL,
  domain               VARCHAR(40)  NOT NULL,
  risk_level           VARCHAR(20)  NOT NULL,
  compute_cost         SMALLINT     NOT NULL,
  dependencies         TEXT[]       NOT NULL DEFAULT '{}',
  emits_topics         TEXT[]       NOT NULL DEFAULT '{}',
  subscribes_topics    TEXT[]       NOT NULL DEFAULT '{}',
  default_activation_state VARCHAR(20) NOT NULL,  -- the code-defined default, NOT the effective state
  research_only        BOOLEAN      NOT NULL,
  version               VARCHAR(20)  NOT NULL,
  deployed_at           TIMESTAMPTZ  NOT NULL DEFAULT now()
);

This table is written by a small deployment-pipeline step, not by application code at request time: immediately after a successful deployment, a one-shot script imports engines/index.js in an isolated process (the same mechanism the CI cycle-detection check uses), reads every registered definition out of the in-memory registry, and upserts each one into bc_engine_registry. This means the table is always, by construction, in sync with whatever code is actually running in production, without requiring engine authors to remember to also write SQL alongside every registerEngine() call — a manual-sync requirement that the team tried briefly early on and abandoned within a month, after the table drifted out of sync with the actual code twice in three weeks purely from authors forgetting the second step.

Why Effective State Lives Only in the Overrides Table, Not Here

Note that bc_engine_registry.default_activation_state is explicitly the code-defined default, and the column name says so, precisely to prevent anyone querying this table directly from mistaking it for the engine's current, effective state — that requires joining against bc_engine_registry_overrides, exactly as the in-memory registry itself does at runtime. Every dashboard and report built on top of this table performs that join rather than reading default_activation_state alone, and a lint-style check on new SQL added to the reporting codebase flags any query against bc_engine_registry that does not also reference the overrides table, specifically to prevent a report from silently showing stale "as originally coded" state instead of "as currently operating" state.

A Representative Migration

When the domain field's cardinality grew (originally just legal, sales, trust, cross-domain; later widened as new product adapters launched), the migration that added support for arbitrary domain strings rather than a fixed check constraint looked like this:

-- (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
-- License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
-- Article published: 22 May 2026.
-- URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata

-- Migration 0047: relax domain to accept any non-empty string, not a
-- fixed set — new product adapters were being blocked on a schema PR
-- every time a new domain value was needed, which defeated the point
-- of the registry being extensible without deployment friction.
BEGIN;

ALTER TABLE bc_engine_registry DROP CONSTRAINT IF EXISTS bc_engine_registry_domain_check;
ALTER TABLE bc_engine_registry
  ADD CONSTRAINT bc_engine_registry_domain_check CHECK (domain <> '');

-- Overrides table had no such constraint to begin with — this migration
-- exists purely to relax the primary table, and is safe to run without
-- downtime since it only removes a restriction, never adds one.

COMMIT;

This migration is deliberately minimal — one dropped constraint, one looser one added in its place — and is included here as a representative example of how the database mirror evolves independently of, and considerably more conservatively than, the in-memory registry's own logic, which changes on every code deployment rather than requiring a schema migration at all.

Design Rationale: Questions From Engineers Evaluating the Pattern

The registry pattern described in this article has been proposed, by engineers familiar with it from this platform, as a fit for unrelated problems elsewhere — most recently, for orchestrating the multi-channel adapters described in Multi-Channel Adapters — WhatsApp, Widget, Mobile. The questions that come up in those evaluation conversations are collected here, phrased as a dialogue, because they surface design tradeoffs that a purely expository description of the finished system tends to gloss over.

"Why not just use a workflow engine like Temporal or Airflow instead of building this yourself?"

Workflow engines are built for a different latency and durability profile than this problem has. Temporal and Airflow both assume individual steps can be long-running, need durable state that survives a process crash mid-workflow, and often involve human-in-the-loop waits measured in hours or days. Every engine in this registry runs in single-digit milliseconds to low tens of milliseconds, an entire pipeline run completes in well under a second, and if a process crashes mid-request, the correct behavior is simply for the caller to retry the whole request from scratch — there is no meaningful partial state worth persisting and resuming. Bringing in a workflow engine's durability machinery for a problem that does not need durability would add operational complexity (a separate service to run, a new failure mode to reason about) without buying anything the simpler in-process approach does not already provide.

"Doesn't building your own registry mean building your own bugs that an off-the-shelf tool would have already fixed?"

This is a fair challenge, and the honest answer is that the registry has had real bugs — several are described as incidents throughout this article, including the self-dependency silent-drop bug and the hidden-coupling incident. The counter-argument is not that building it yourself avoids bugs; it's that the specific properties this problem needs (sub-millisecond dependency resolution for dozens of in-process functions, tight integration with a bespoke compute-budget and governance model, and an audit trail schema shaped around this platform's specific compliance needs) do not map cleanly onto any existing open-source tool's abstractions, and the team's assessment, revisited periodically, has been that adapting an existing tool to fit would end up being comparably complex to what was built, while also inheriting an external project's own release cadence, bug backlog, and abstraction choices that were not designed with this problem in mind.

"How much of this could a small team realistically reuse for a much smaller registry — say, five engines instead of thirty-four?"

The core loop (self-registration, a Zod schema, Kahn's-algorithm batching) is genuinely small — well under a thousand lines of code total, excluding tests — and is exactly as useful at five engines as at thirty-four; nothing about it assumes a particular scale. What would reasonably be deferred at five engines: the override-table split (a five-engine system's operators can probably tolerate a deployment to change activation state, at least initially), the adaptive-throttling multiplier (not worth building until a real traffic spike has actually caused a problem), and the multi-region replication concerns. The recommended starting point for a smaller system is the schema, the self-registration pattern, and the DAG builder — roughly the first third of this article — with everything after the versioning section treated as "build when the specific need actually arises," which is, not coincidentally, exactly the order in which this platform's own registry actually grew, as the timeline section above lays out.

"What's the biggest thing you'd change if you were starting over?"

The near-unanimous answer, when this question comes up internally, is the initial enabled: boolean field that later had to be replaced by activationState and researchOnly as separate fields — not because that migration was especially painful (it wasn't; the flat structure made it a mechanical, low-risk change), but because it cost a full quarter of workaround code (the silent: true parameter hack mentioned in the timeline) before the team recognized the boolean was the wrong shape. In hindsight, spending an extra day in the initial design discussion enumerating "what are all the states an engine can meaningfully be in" — active, restricted, research-only, deprecated — before writing the first version of the schema would have avoided that entire detour. The general lesson the team draws from this, and applies now to new schema design elsewhere in the platform: state fields deserve disproportionate up-front thought relative to how simple they look, because they are the fields hardest to widen later without a migration that touches every call site.

Rollback Playbook: When a Version Bump Goes Wrong

Every mechanism covered so far describes the registry working as intended. This section covers the specific, practiced procedure for the case where it doesn't — a new engine version ships, passes CI, passes the compute-cost drift check, and then produces materially wrong output in production, discovered only after real requests have already been scored by it.

Detecting the Problem

Two independent signals typically surface this before a human notices manually. First, the meta-learning engine (described in the foundational engines category) continuously compares each engine's recent output distribution against its trailing 30-day baseline and raises an internal alert if a distribution shift exceeds a statistical threshold, which is exactly the kind of drift a bad version bump produces — scores that are individually plausible but collectively shifted in a way that does not match the historical pattern for that engine. Second, the contract tests described in the testing section run on a schedule against live traffic samples, not just at CI time, and a bad version bump that changes an engine's output shape (rather than just its values) will fail a contract test within the hour rather than waiting for the next deployment's CI run.

Immediate Containment: Demote, Don't Deprecate

The first response is almost never full deprecation — it is the demotion transition described in the activation-states section, moving the engine from active back to research-only. This is deliberately the fast, low-ceremony path (the same single-reason-field API call as the restriction fast path) specifically so that an on-call engineer does not need to reason through a full deprecation's dependents check under incident pressure — demotion keeps the engine running and logging, which preserves the exact data needed to diagnose the bad version, while immediately stopping its output from reaching any product adapter.

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata

await demoteEngine('bias-detection', {
  reason: 'v1.4.0 producing distribution-shifted scores per meta-learning ' +
          'alert ALERT-3391. Demoting to research-only pending investigation. ' +
          'Last known-good version: 1.3.2.',
  actor: 'oncall:jsingh',
});

Diagnosing With the Audit Trail

Because every trace row in bc_explainability_traces is tagged with the exact engine_version that produced it, the investigation query is a straightforward comparison: pull a sample of traces from 1.4.0 and an equally sized sample from the preceding 1.3.2 for structurally similar inputs (matched by input_hash where possible, or by matter type and rough signal-volume bucket otherwise), and diff the score distributions directly. In the incident that motivated this playbook being written down formally, this comparison took under twenty minutes and pointed straight at a specific change in the version diff — a normalization constant that had been updated for a different reason (to align with a new upstream data source) but had an unintended side effect on the score's scale for the majority of existing input shapes that did not use the new data source at all.

Fixing Forward vs. Reverting

Two options exist once the cause is understood, and the team's stated preference, in order, is: revert the specific offending change and ship 1.4.1 as a corrected version if the fix is small and well-understood; only revert the entire engine to the prior 1.3.2 code if the fix is not immediately obvious and the engine needs to stay stable while the real fix is developed more carefully. Reverting to a prior version's code is deliberately treated as a version bump in its own right — the redeployed old code registers as 1.3.3, not a re-registration of 1.3.2, specifically so the audit trail never has two different periods both labeled with the identical version string producing potentially different behavior (since a literal reinstatement of 1.3.2's tag would be ambiguous about which of the two deployment periods a given historical trace under that version actually reflects, if the rollback itself needed to be reverted again later).

Re-Graduation, Not Automatic Reactivation

Once a corrected version has run in research-only mode long enough to build confidence (the team's practice is a minimum of 72 hours of shadow traffic for a rollback-driven re-graduation, shorter than the 14-day bar for a first-time graduation, on the reasoning that this engine has a substantial prior track record and the fix is narrowly scoped, not a wholesale new capability), it goes through the same scope-up review gate as any restricted/research-only to active transition — there is no fast path back to full activation, even for an engine that was active moments before the incident, because the review gate exists precisely to catch exactly the kind of regression that just happened.

The Postmortem Artifact

Every demotion-triggered incident produces a postmortem that is explicitly linked from the engine's entry in the admin dashboard, permanently, not just filed away in an incident tracker disconnected from the registry itself. This is a small piece of tooling — a postmortemUrl field on the audit log entry for the demotion event — but it means a future engineer looking at bias-detection's version history six months later, and noticing the jump from 1.3.2 to 1.3.3 with no corresponding 1.4.0–1.3.9 gap explained anywhere else, can find the full incident writeup in one click from the exact place they are already looking, rather than needing to know the incident happened at all in order to go searching for it.

What to Watch For

• Version drift — Always bump the engine version when scoring logic changes. Audit logs use it to explain why the same input produced different outputs on different dates. The CI check described in the versioning section catches the mechanical case (logic changed, version didn't) but cannot catch the inverse mistake — a version bumped without any real logic change, which pollutes the audit trail with noise. Treat version bumps as meaningful signals, not a box to tick on every PR.
• Compute budget — Sum computeCost for all active engines and set a hard cap. The scheduler can drop low-priority engines if the total exceeds the budget for a request. Remember that critical risk-level engines are structurally exempt from budget-based dropping — if your budget cannot accommodate every critical engine, that is a configuration problem to fix immediately, not a runtime condition the scheduler will quietly absorb.
• Research-only to active promotions — Treat them as a release event. Run the engine in research mode for at least 2 weeks before activating it, and require a graduation report comparing shadow-mode output against actual historical decisions, not just an accuracy number in isolation.
• Restriction is not deletion — a restricted engine still runs, still emits topics, and still gets scored in bc_explainability_traces for the teams it applies to. Do not use restriction as a substitute for deprecation when an engine genuinely needs to stop existing.
• The dependents check before deprecation is not optional — even though the API enforces it server-side, understand why it exists before you find yourself needing to migrate a dependent under incident pressure. Plan dependency migrations ahead of a deprecation request, not during one.
• Stale execution plan caches — if an override change does not appear to take effect, check the registry generation number before assuming the override write itself failed; the two failure modes look identical from the outside but have different fixes.
• God engines and hidden coupling — the two anti-patterns most likely to reintroduce the exact scattered-logic problem the registry was built to remove, just relocated inside an engine's own file rather than across call sites. Review new engine PRs for both, not just for schema validity, since a schema-valid engine can still violate the isolation the whole design depends on.
• Multi-region propagation lag — a restriction change is fast, but not instantaneous, across regions. Do not assume a compliance-driven restriction has taken effect everywhere within the same second it was written; budget for the documented replication-plus-poll bound described in the multi-region section, and communicate that bound honestly when an incident's timeline depends on it.

Summary

The engine registry's job is narrow and specific: hold a validated, queryable description of every scoring unit in the platform, enforce the dependency contract between them, and let an operator change what runs — for whom — without shipping code. Every piece of the design described in this article, from the split between activationState and researchOnly, to the reverse dependency index, to the requirement that critical-risk engines can never be dropped for budget reasons, traces back to a specific operational need or a specific incident that happened when the design did not yet account for it.

The next article in this series, The Control Plane, picks up exactly where this one leaves off: given a validated execution plan produced by the registry's dependency resolver, how does the control plane actually run it, collate the results, and hand them to the governance wrapper before anything reaches a product adapter.

The compliance officer's ticket that opened this article was resolved, permanently, the week the registry shipped. The same class of request — stop this specific engine from running for this specific matter type, right now — now takes the time it takes to submit one database row through the override API and wait for the next poll cycle: under a minute, most of it spent on the human writing a clear reason field, not waiting on a deployment pipeline. That is the actual return on the schema, the validators, and the dependency graph described above — not architectural elegance for its own sake, but a compliance team's ability to act on a legal-exposure finding before it compounds across another few thousand requests.

If there is a single idea worth carrying forward from everything above, it is this: a registry is not a data structure you add once a system has grown complicated enough to need one. It is a decision, made early or made late, about where the truth of "what runs, for whom, in what order, at what cost" is allowed to live. Every scattered conditional, every hard-wired call site, every silently stale team-ID check described in the opening section was, in its own way, a small claim about that truth made somewhere other than a single, validated, queryable place. The registry's entire value is that it collapses all of those small, drifting claims into one — and then makes that one place strict enough, at startup and at every write, that it can be trusted the way the rest of this platform's governance and audit story depends on it being trusted.

Appendix C: Related Reading in This Series

This article assumes familiarity with, and is best read alongside, the following companion pieces, each of which covers one adjacent piece of the same overall architecture in depth rather than in the summary form referenced here.

• What is a Behavioral Intelligence OS? — the architectural overview this article's registry section builds directly on top of.
• The Control Plane — how the execution plan this article describes building is actually run, batch by batch.
• Confidence Propagation in Multi-Engine Systems — the full formula behind the confidence field referenced throughout this article's worked example and anti-patterns sections.
• Explainability Traces as First-Class Objects — the complete schema and design rationale for bc_explainability_traces, only summarized here in the versioning section.
• The Behavioral Event Bus — the publish/subscribe mechanics behind emitsTopics and subscribesTopics, covered here only from the registry's validation perspective.
• Governance Wrappers — Enforcing Safe Language — what actually happens to engine output in the final stage referenced throughout this article's worked example.
• Role-Based Access Control for Legal Teams — the team registry that restrictedToTeams validates against, from the product side rather than the registry side.

Appendix D: Reading a Real Startup Log

Closing this article with an actual (lightly redacted) startup log is deliberate — every mechanism described above has a corresponding, legible line in the log output the control plane produces when a process boots, and being able to read that output at a glance is, in practice, the single most-used diagnostic skill for anyone operating this system day to day.

# (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
# License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
# Article published: 22 May 2026.
# URL: https://govindpreetsingh.com/article.php?slug=the-34-engine-registry-design-and-metadata

[registry] importing engines/index.js …
[registry] behavioral-entropy@2.1.0 registered (foundational, cost=2, risk=low)
[registry] information-theory@1.4.2 registered (foundational, cost=3, risk=low)
[registry] geometric-topological@2.0.1 registered (foundational, cost=4, risk=medium)
[registry] meta-learning@3.2.0 registered (foundational, cost=3, risk=low)
[registry] bayesian-confidence@4.1.0 registered (cognitive, cost=2, risk=low)
[registry] bias-detection@1.3.3 registered (cognitive, cost=3, risk=high)
  … 28 more lines omitted …
[registry] behavioral-vm-kernel@1.0.4 registered (kernel, cost=2, risk=critical)
[registry] governance-safety@2.2.1 registered (governance, cost=3, risk=critical)
[registry] 34 engines registered in 41ms
[registry] building dependency graph …
[registry] graph valid — 34 nodes, 61 edges, no cycles
[registry] topological sort → 6 batches (sizes: 6, 9, 8, 6, 3, 2)
[registry] execution plan cached, generation=1
[overrides] polling bc_engine_registry_overrides …
[overrides] 2 active overrides loaded: bias-detection→restricted(team_119), prediction-horizon→research-only
[overrides] generation=2 (2 overrides applied over code defaults)
[control-plane] ready — listening for pipeline requests

Every line traces to a specific mechanism covered in this article: the per-engine registration lines are the self-registration pattern's side effects firing in barrel-file order; "graph valid" is the Kahn's-algorithm cycle check from the dependency-resolution section confirming no self-loops or circular references slipped past code review; the batch-size list is the direct output of groupIntoBatches(); and the final two [overrides] lines are the very first poll of the override table, which is why the reported generation jumps from 1 (code-only) to 2 (code plus overrides) within the same boot sequence, before the process has served a single real request. An operator who can read this log and immediately say "two engines are running under non-default state, here's which ones and why the generation counter is 2 and not 1" has, in a real sense, internalized the entire design this article set out to explain.