The Control Plane — Orchestrating an AI Pipeline

What a Control Plane Does That a Model Cannot

A client's data processing agreement flatly prohibited one specific category of behavioral inference — their contract with the platform said the emotional-regulation engine could never run against their staff's communications, full stop, no exceptions. That is not a preference to accommodate when convenient; it is a contractual term, and running that engine against their data even once, even briefly during an incident, is a breach with real financial and reputational consequences on both sides. A language model can't honor a restriction like that on its own — it processes a prompt and returns a completion, with no concept of "never run this specific capability for this specific client." Something has to sit above the model and enforce that boundary, for every request, without fail, and without needing a code change every time a new client signs a contract with a different set of restrictions.

That something is the control plane. It treats each engine as a unit of work with declared dependencies, and decides at request time — based on the registry state — what runs. Concretely, this is what let the platform honor that specific client's DPA the same day the contract was signed, not after a development sprint: the engine's activation state was scoped to exclude that client's team ID, and the control plane has enforced it on every request since, the same mechanism that also lets the team:

• Disable the emotional-regulation engine for a client that has a DPA restriction, without any code change.
• Run prediction-horizon in research-only mode for six months, collecting data without exposing outputs.
• Add a new tactical-negotiation engine and promote it to active — again, no deployment.

The distinction is worth dwelling on because it is the single most common point of confusion for engineers who arrive on this platform from a background in classical model-serving infrastructure. A model server — a TorchServe instance, a vLLM deployment, an API call to a hosted model — has exactly one job: take an input, run it through a fixed computation graph, and return an output. Every question about "what runs" was already answered when the model was trained and deployed. The control plane's job starts from the opposite premise: the set of things that will run for a given request is not fixed at deployment time, it is computed fresh from the registry's current state on every single request (or, more precisely, on every registry-generation change, as the plan-caching section below explains). That single architectural choice is what makes the rest of this article necessary — none of the machinery described here (batching, timeouts, research-only shadow runs, event-bus handoffs) has an analogue in a classical model-serving stack, because a classical model-serving stack never has to answer the question "given what's active right now, what should actually execute for this request?"

This is also, incidentally, a problem that shows up constantly across the Technology and Artificial Intelligence sectors specifically, in any organization running more than a handful of models or scoring functions behind a single product surface — the control plane described here is this platform's specific answer, not a claim that the problem itself is unique to it.

The Kernel Analogy, Taken Seriously

Calling the control plane "the kernel of the platform" in the lead paragraph above is not just a flourish — it is the design metaphor the team returns to whenever a new orchestration decision needs to be made, and it is worth making the analogy explicit rather than leaving it as decoration.

An operating system kernel does a small number of things well: it schedules processes onto available CPU time, it isolates one process's memory from another's, it mediates access to shared resources (disk, network) so that one process cannot starve another, and it provides a small set of primitives (signals, IPC) that let independent processes coordinate without knowing about each other's internals. The control plane does the direct analogue of every one of those things for engines instead of processes:

The value of taking this analogy seriously, rather than treating it as marketing language, is that it gives the team a fast heuristic for evaluating a proposed change: "is this something an OS kernel would do, or is this something a userspace process would do?" A proposal to have one engine directly call another engine's scoring function, bypassing the event bus, is instantly recognizable as "a userspace process reaching into another process's memory" — obviously wrong, for exactly the reasons it would be obviously wrong in a real kernel. A proposal to let the control plane defer a low-priority engine when the system is under load is instantly recognizable as ordinary process scheduling — obviously fine, and in fact exactly what a kernel scheduler is for.

Building the Execution Plan

The plan builder runs once per request (or, as the caching section below covers, can be reused across many requests and invalidated only when the registry actually changes). It uses Kahn's topological sort algorithm to group engines into parallel batches:

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-control-plane-orchestrating-an-ai-pipeline

// core/control-plane.js

function buildExecutionPlan(activeEngines) {
  // Step 1: build adjacency list
  const inDegree = new Map();
  const graph    = new Map();

  for (const engine of activeEngines) {
    inDegree.set(engine.engineId, 0);
    graph.set(engine.engineId, []);
  }

  for (const engine of activeEngines) {
    for (const dep of engine.dependencies) {
      if (!graph.has(dep)) throw new Error(`Missing dependency: ${dep}`);
      graph.get(dep).push(engine.engineId);
      inDegree.set(engine.engineId, inDegree.get(engine.engineId) + 1);
    }
  }

  // Step 2: Kahn's algorithm — produce topological batches
  const batches = [];
  let   queue   = activeEngines
    .filter(e => inDegree.get(e.engineId) === 0)
    .map(e => e.engineId);

  while (queue.length > 0) {
    batches.push([...queue]);
    const next = [];
    for (const id of queue) {
      for (const dependent of graph.get(id)) {
        const newDeg = inDegree.get(dependent) - 1;
        inDegree.set(dependent, newDeg);
        if (newDeg === 0) next.push(dependent);
      }
    }
    queue = next;
  }

  if (batches.flat().length !== activeEngines.length) {
    throw new Error('Cycle detected in engine dependency graph');
  }

  return batches.map(ids => activeEngines.filter(e => ids.includes(e.engineId)));
}

This is deliberately the same algorithm described from the registry's point of view in the registry article, and that overlap is intentional rather than duplicated effort: the registry validates, at startup and in CI, that a cycle-free plan can be built from whatever is currently registered. The control plane calls the same class of function at request time to build the plan that actually runs. The two call sites exist for different reasons — one is a build-time correctness guarantee, the other is a runtime scheduling decision — but they share the same underlying graph algorithm because there is no reason for them not to.

Why Batches, Not a Flat Ordered List

A naive topological sort produces a single ordered list of engines. The control plane needs batches — groups of engines with no ordering dependency on each other — because the entire performance case for this architecture rests on running independent engines concurrently rather than serially. A pipeline with 19 active engines and an average dependency depth of 4 batches, where each batch takes roughly the same wall-clock time as its slowest member, completes in roughly a quarter of the time a fully serial execution of the same 19 engines would take. Kahn's algorithm produces batches as a natural byproduct of its queue-based structure — every engine dequeued in the same iteration of the outer loop has, by construction, no unresolved dependency on any other engine still in that iteration's queue — which is why it was chosen over other topological-sort formulations (like a straightforward depth-first-search-based sort) that produce a valid ordering but not a natural batch grouping.

Running the Pipeline

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-control-plane-orchestrating-an-ai-pipeline

async function runPipeline(inputSignals, context) {
  const activeEngines = engineRegistry.getActive();
  const plan          = buildExecutionPlan(activeEngines);
  const results       = {};

  for (const batch of plan) {
    // All engines in a batch run in parallel
    const batchOutputs = await Promise.all(
      batch.map(engine => runEngineWithTimeout(engine, inputSignals, context))
    );

    batch.forEach((engine, i) => {
      const output = batchOutputs[i];
      results[engine.engineId] = output;

      // Publish events to the bus for engines in later batches
      for (const topic of engine.emitsTopics) {
        eventBus.emit(topic, { engineId: engine.engineId, output });
      }
    });
  }

  // Research-only engines run separately — their outputs are logged, never returned
  await runResearchEngines(inputSignals, context);

  return governanceWrapper.wrap(results, context);
}

async function runEngineWithTimeout(engine, signals, context) {
  const timeoutMs = computeTimeout(engine.computeCost);
  return Promise.race([
    engine.score(signals, context),
    new Promise((_, reject) =>
      setTimeout(() => reject(new Error(`Engine ${engine.engineId} timed out`)), timeoutMs)
    ),
  ]);
}

This is the simplified version, and it is the version that shipped first. It has one significant gap that surfaced within the first month of production traffic: Promise.all rejects as soon as any promise in the array rejects, which means a single engine timing out or throwing aborts the entire batch immediately — including the outputs of every other engine in that batch that would have succeeded fine, milliseconds later. The next section covers the fix.

Handling Partial Batch Failure

The first production incident this architecture produced was not a governance failure or a data problem — it was Promise.all behaving exactly as documented and the team not having thought through what that meant for a batch of independent, unrelated engines. A batch of six foundational and cognitive engines ran together; one of them (a since-fixed bug in an early version of geometric-topological) threw on a specific malformed input shape roughly once per few thousand requests. Every time it did, the other five engines' outputs — computed successfully, sitting right there in already-resolved promises — were discarded, because Promise.all doesn't hand back partial results on rejection. The pipeline returned nothing for the entire request instead of five good engine outputs and one flagged failure.

The fix replaces Promise.all with Promise.allSettled, and treats a single engine's failure as data about that engine, not as a reason to fail the whole batch. The distinction between the two methods is a single word in the API name, and it is easy to see why the original implementation reached for Promise.all without a second thought — it is the more familiar, more commonly reached-for method, and its behavior is entirely correct and desirable in the far more common case where every promise in an array genuinely does need to succeed for the combined result to mean anything (fetching several required pieces of a single coherent response, for instance). The mistake here was not misusing Promise.all; it was applying a combinator designed for "all of these must succeed together" to a batch of engines whose entire premise, per the dependency graph, is that they are independent of one another and each individually valuable on its own.

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-control-plane-orchestrating-an-ai-pipeline

async function runBatch(batch, inputSignals, context) {
  const settled = await Promise.allSettled(
    batch.map(engine => runEngineWithTimeout(engine, inputSignals, context))
  );

  const batchResults = {};
  for (let i = 0; i < batch.length; i++) {
    const engine = batch[i];
    const outcome = settled[i];

    if (outcome.status === 'fulfilled') {
      batchResults[engine.engineId] = { ok: true, output: outcome.value };
      for (const topic of engine.emitsTopics) {
        eventBus.emit(topic, { engineId: engine.engineId, output: outcome.value });
      }
    } else {
      batchResults[engine.engineId] = { ok: false, error: outcome.reason.message };
      auditLogger.logEngineFailure(engine.engineId, outcome.reason, context);
      // Downstream engines that depend on this one will see ok:false and
      // must degrade — never treat a missing dependency output as zero
      // or as "no signal," which silently overstates confidence.
    }
  }
  return batchResults;
}

The comment in that snippet points at the second half of the fix, which is easy to miss: making the batch resilient to one engine's failure only solves half the problem. Any downstream engine that declared a dependency on the failed one now needs to run with a missing input, and the platform's standing rule (covered in the confidence-propagation article) is that a missing upstream signal must reduce confidence, not silently be treated as if the missing engine had returned a neutral or zero score. Every engine's score() function is required to check its declared dependencies' ok flag before reading their output, and to apply the platform's low_evidence reducer when a dependency it needs is missing. This is enforced, not just documented: the same synthetic-input CI battery mentioned in the registry article's anti-patterns section includes a case that simulates a missing upstream dependency for every registered engine and asserts that reported confidence drops.

What Still Fails the Whole Request

Not every failure is handled by graceful per-engine degradation. Three categories still abort the request entirely, by design: a failure in any critical-risk-level engine (there is no safe way to serve a response if the governance-safety engine itself couldn't run), a failure that occurs before the plan can even be built (a malformed registry snapshot, caught by the same validators described in the registry article), and a failure in the governance wrapper itself, which sits after every engine batch and has no further stage to hand a degraded response to. This is a deliberate three-tier failure model — most engine failures degrade gracefully, critical-path failures abort loudly — and it maps directly onto the riskLevel field's actual operational purpose: it isn't just a review-process trigger, it's also the exact list of engines whose failure the control plane treats as fatal rather than degradable.

Timeout Calibration

Set engine timeouts proportional to computeCost. A cost-8 engine gets 8× the base timeout. Never let one slow engine block the whole pipeline. The actual formula, and the reasoning behind its specific shape, is worth spelling out because the naive linear version undersells how much tail latency varies engine to engine.

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-control-plane-orchestrating-an-ai-pipeline

const BASE_TIMEOUT_MS = 40;    // baseline for a computeCost:1 engine
const TIMEOUT_FLOOR_MS = 150;  // no engine gets less than this, regardless of cost
const TIMEOUT_CEILING_MS = 2500; // no engine gets more than this, regardless of cost

function computeTimeout(computeCost) {
  const scaled = BASE_TIMEOUT_MS * Math.pow(computeCost, 1.4); // super-linear, not linear
  return Math.min(TIMEOUT_CEILING_MS, Math.max(TIMEOUT_FLOOR_MS, Math.round(scaled)));
}
// computeCost 1 -> 150ms (floor)   computeCost 4 -> ~294ms
// computeCost 7 -> ~730ms          computeCost 9 -> ~1102ms
// computeCost 10 -> ~1440ms

The exponent of 1.4, rather than a straight linear scale, came out of a deliberate look at measured latency data rather than a guess: plotting actual p99 latency against declared computeCost across all 34 engines showed a distribution that curves upward faster than linear, because higher-cost engines (Monte Carlo simulation, large knowledge-graph traversals) don't just do proportionally more work — they also have proportionally more variance in how much work a given input requires, which pushes their tail latency out further than their median latency alone would suggest. A straight linear timeout formula, calibrated to the median, was clipping roughly 3% of digital-twin-simulation's legitimate slow-but-correct runs as timeouts before this fix; the super-linear formula, recalibrated against p99 rather than median, brought that down to a fraction of a percent while barely changing timeouts for the cheap, low-variance foundational engines.

The Floor and Ceiling Both Matter

The floor exists because even a computeCost:1 engine needs enough slack to survive ordinary network/scheduler jitter — without it, the cheapest engines would have the tightest timeouts and, counterintuitively, the highest timeout-failure rate purely from noise, not from actually being slow. The ceiling exists for the opposite reason: without it, a single misbehaving high-cost engine (one that has genuinely regressed, not just hit a slow day) could hold up a batch for multiple seconds, and the control plane's own per-request latency budget (covered in the backpressure section below) needs a hard upper bound on how long it will wait for any single engine, no matter how expensive its declared cost.

The Event Bus Handoff Between Batches

The pipeline runner publishes to the Redis Streams-backed event bus after every batch completes, not after every individual engine completes — this ordering detail matters more than it looks like it should. If publication happened per-engine rather than per-batch, an engine in the next batch that subscribes to two different topics from two different engines in the current batch could start running after seeing only one of the two events, on whichever one happened to resolve first, and would then be operating on an incomplete view of the batch it actually depends on. Per-batch publication (after the whole Promise.allSettled resolves, not per-promise) guarantees that any engine in batch N+1 which starts running has a fully consistent view of every event batch N produced, including the failure/success outcome of every engine in it.

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-control-plane-orchestrating-an-ai-pipeline

async function runPipeline(inputSignals, context) {
  const plan = getCachedOrBuildPlan(context);
  const results = {};

  for (const batch of plan.batches) {
    const batchResults = await runBatch(batch, inputSignals, context);
    Object.assign(results, batchResults);

    // Publish once, after the whole batch settles — not per-engine.
    for (const [engineId, result] of Object.entries(batchResults)) {
      if (!result.ok) continue;
      const engine = batch.find(e => e.engineId === engineId);
      for (const topic of engine.emitsTopics) {
        await eventBus.publish(topic, {
          engineId, output: result.output, batchIndex: plan.batches.indexOf(batch),
        });
      }
    }
  }

  await runResearchEngines(inputSignals, context);
  return governanceWrapper.wrap(results, context);
}

Full detail on the bus's own delivery guarantees, consumer groups, and topic-naming conventions lives in the event bus article — the relevant fact for the control plane's own design is narrower: the control plane treats the bus as at-least-once, not exactly-once, and every engine's score() function is required to be safe to call twice with the same event (idempotent with respect to its own output, even if it isn't idempotent with respect to side effects like metrics counters). This requirement exists because a batch that partially fails and is retried at the control-plane level (see below) can, in rare timing windows, cause a downstream engine to observe the same upstream event more than once.

Research-Only Execution

Research-only engines run the full scoring pipeline but their outputs are written to a separate audit log and never returned to product adapters. This lets you collect real-world data on a new engine's behaviour before trusting it with live decisions.

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-control-plane-orchestrating-an-ai-pipeline

async function runResearchEngines(signals, context) {
  const researchEngines = engineRegistry.getResearchOnly();
  for (const engine of researchEngines) {
    try {
      const output = await engine.score(signals, context);
      auditLogger.logResearchOutput(engine.engineId, output, context);
    } catch (err) {
      auditLogger.logResearchError(engine.engineId, err, context);
    }
  }
}

Two design choices in this function are easy to overlook and both were deliberate. First, research-only engines run after the main pipeline's batches complete and are awaited separately, rather than being folded into the main batch structure — this means a slow or failing research-only engine can never add latency to the response the caller actually receives, because the function that returns the governed response (governanceWrapper.wrap(results, context) in the pipeline runner above) doesn't wait on runResearchEngines at all; it's fired and forgotten from the caller's perspective, with its own independent completion tracked only in the audit log. Second, the loop runs research engines sequentially (a plain for loop, not Promise.all), not in parallel batches — research-only engines are explicitly exempt from the compute-budget and timeout discipline that governs active engines, precisely because they are not on the response-latency critical path, and giving them their own generous, unhurried execution window is a deliberate choice to avoid research-mode engines needing the same tight timeout tuning active engines do before they've even been evaluated.

Why Research-Only Failures Are Logged, Not Alerted

An active engine's failure triggers the audit log entry shown in the partial-batch-failure section above, which feeds into on-call alerting if the failure rate crosses a threshold. A research-only engine's failure is logged to the same audit infrastructure but deliberately does not feed into the same alerting path. This was a specific fix after a research-only engine with an unhandled edge case in its early, still-being-developed logic paged on-call every few minutes for a full weekend before someone realized the "failing" engine was one that had never been active in the first place and whose failures had zero user-facing consequence. The distinction the team draws now: alerting exists to protect users from a degraded experience; a research-only engine, by construction, cannot degrade any user's experience, so its failures are a data-quality concern for whoever owns that engine's graduation, not an operational incident.

Plan Caching and Registry-Generation Invalidation

The execution plan is deterministic for a given registry state. Cache it and invalidate only when the registry changes. Re-building it on every request is wasteful — and, as the registry article's performance section already established, building a 34-to-100-engine plan from scratch costs well under 2ms, so "wasteful" here is relative rather than an emergency, but at the request volumes this platform actually serves, avoiding tens of thousands of redundant graph builds per minute is easy value to capture.

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-control-plane-orchestrating-an-ai-pipeline

const planCache = new Map(); // cacheKey -> { plan, generation }

function getCachedOrBuildPlan(context) {
  const cacheKey = planCacheKey(context); // usually just context.teamId, see below
  const currentGeneration = engineRegistry.generation;

  const cached = planCache.get(cacheKey);
  if (cached && cached.generation === currentGeneration) {
    return cached.plan;
  }

  const activeEngines = engineRegistry.getActiveFor(context); // team-scoped, per the restriction rules
  const plan = { batches: buildExecutionPlan(activeEngines) };
  planCache.set(cacheKey, { plan, generation: currentGeneration });
  return plan;
}

This is the same registryGeneration counter introduced in the registry article's internal-data-structures section, reused here for a second purpose: there, it keys the per-team plan cache the registry itself is aware of; here, it's the invalidation signal the control plane's own cache checks on every request, at the cost of one integer comparison, before deciding whether to reuse a cached plan or rebuild one. The two caches are, in the current implementation, actually the same cache — the registry article describes it from the registry's point of view, this article describes the exact same mechanism from the control plane's point of view, because the control plane is the code that reads and writes that cache. There is deliberately no second, separate plan cache anywhere else in the codebase; a second cache would be a second place invalidation could go stale.

Concurrency Limits and Backpressure

The compute-budget mechanism described in the registry article decides which engines run for a given request. It says nothing about how many requests' worth of engines the control plane will try to run at the same time — that is a separate concern, handled entirely within the control plane, and conflating the two was a design mistake the team made early on and had to unwind.

The Mistake: Unbounded Concurrency

The first production version of runPipeline had no concurrency limiter at all — every incoming request simply called it, and every batch's Promise.all/Promise.allSettled spun up however many concurrent engine invocations the current traffic implied. Under ordinary load this was fine. Under a traffic spike (the same product-launch spike referenced in the registry article's timeline), the number of concurrently in-flight engine invocations grew fast enough to exhaust the Node.js process's practical concurrency ceiling — not a hard limit, but a point past which event-loop scheduling overhead and per-invocation memory overhead started measurably degrading every single in-flight request's latency, including ones that had nothing to do with the spike's originating traffic.

The Fix: a Bounded Worker Pool

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-control-plane-orchestrating-an-ai-pipeline

import pLimit from 'p-limit';

const globalConcurrencyLimit = pLimit(400); // tuned from load-test data, not a guess

async function runEngineWithTimeout(engine, signals, context) {
  return globalConcurrencyLimit(async () => {
    const timeoutMs = computeTimeout(engine.computeCost);
    return Promise.race([
      engine.score(signals, context),
      new Promise((_, reject) =>
        setTimeout(() => reject(new Error(`Engine ${engine.engineId} timed out`)), timeoutMs)
      ),
    ]);
  });
}

Every individual engine invocation, across every concurrent request being served by the process, now competes for one of 400 concurrency slots. When the limit is saturated, new engine invocations queue rather than starting immediately — which sounds like it just moves the problem, but the queueing itself is the fix: a queued invocation adds latency to the specific requests waiting on it, in a predictable, boundable way, rather than degrading the throughput of the entire process by over-subscribing the event loop. The number 400 is not a principled constant — it came from load-testing the process against increasing concurrency until p99 latency started degrading, then setting the limit comfortably below that knee point, and it gets re-validated any time the underlying server hardware changes.

Per-Request Latency Budgets Feed Load Shedding

On top of the global concurrency limiter, every request carries its own latency budget (distinct from the compute-cost budget) — typically 2.5 seconds for a synchronous product-adapter request. If the control plane detects, partway through executing a plan's batches, that remaining time in the budget is insufficient to complete the remaining batches even at their timeout floor, it short-circuits: it stops starting new batches, runs the governance wrapper against whatever results have already been collected, and returns a partial response with a degraded: true flag rather than blowing through the caller's own timeout and returning nothing at all. This is the request-level analogue of the registry article's compute-budget dropping — the same "protect the response over protecting completeness" philosophy, just triggered by elapsed wall-clock time instead of a compute-cost sum.

Observability: Tracing a Pipeline Run

Every pipeline run is assigned a requestId at entry, and every engine invocation within it emits a structured trace span tagged with that requestId, the engine's engineId and version, its batch index, and its start/end timestamps. This is deliberately the same request_id column used in the bc_explainability_traces table described in the registry article's versioning section — the observability trace and the compliance audit trail are, by design, two views over the same underlying event stream, not two independently-maintained systems that could drift apart.

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-control-plane-orchestrating-an-ai-pipeline

async function runEngineWithTimeout(engine, signals, context) {
  const span = tracer.startSpan('engine.execute', {
    attributes: {
      'engine.id': engine.engineId,
      'engine.version': engine.version,
      'engine.computeCost': engine.computeCost,
      'request.id': context.requestId,
    },
  });
  try {
    const result = await globalConcurrencyLimit(() =>
      Promise.race([
        engine.score(signals, context),
        timeoutPromise(engine),
      ])
    );
    span.setStatus({ code: SpanStatusCode.OK });
    return result;
  } catch (err) {
    span.setStatus({ code: SpanStatusCode.ERROR, message: err.message });
    throw err;
  } finally {
    span.end();
  }
}

With this in place, "why was request X slow" stops being a question that requires reproducing the request — it's a query against the trace store, filtered by request.id, sorted by span start time, and the answer is almost always visible immediately as either one abnormally long-running engine span, an unusually large number of batches (meaning the dependency graph for that request's active engine set was deeper than typical), or time spent queued behind the concurrency limiter rather than time spent actually executing.

Testing the Control Plane

Unlike the registry's tests (schema validation, DAG correctness — covered in the registry article), the control plane's test suite is built around simulating exactly the failure conditions the sections above describe, because the whole point of this layer is behavior under adverse conditions, not just behavior on the happy path.

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-control-plane-orchestrating-an-ai-pipeline

describe('control plane — partial batch failure', () => {
  it('returns the other engines\' results when one engine throws', async () => {
    const engines = [
      fakeEngine('good-a', { dependencies: [] }),
      fakeEngine('bad-b',  { dependencies: [], throws: new Error('boom') }),
      fakeEngine('good-c', { dependencies: [] }),
    ];
    const result = await runBatch(engines, {}, {});
    expect(result['good-a'].ok).toBe(true);
    expect(result['bad-b'].ok).toBe(false);
    expect(result['good-c'].ok).toBe(true);
  });

  it('degrades a dependent engine\'s confidence when its dependency failed', async () => {
    const upstream = fakeEngine('upstream', { throws: new Error('boom') });
    const downstream = fakeEngine('downstream', { dependencies: ['upstream'] });
    const plan = buildExecutionPlan([upstream, downstream]);
    const results = await runPlan(plan, {}, {});
    expect(results['downstream'].output.confidence).toBeLessThan(0.5);
  });

  it('never lets a research-only engine\'s latency affect the response', async () => {
    const slowResearch = fakeEngine('slow-research', {
      activationState: 'research-only', delayMs: 5000,
    });
    const start = Date.now();
    await runPipeline({}, { requestId: 'test-1' });
    expect(Date.now() - start).toBeLessThan(1000); // did not wait on the 5s research engine
  });
});

The third test in that block is the one the team considers non-negotiable in code review for any change to the pipeline runner: research-only isolation is a governance promise (their output never influences a live decision) and a performance promise (their latency never influences response time), and the second half of that promise is exactly as easy to accidentally break — one refactor that moves runResearchEngines before the return statement instead of after it, and suddenly every research-only engine's latency is back on the critical path. This test exists specifically to make that refactor mistake fail CI immediately rather than surface as a latency regression weeks later.

Failure Modes & Operational Runbook

Runbook: "Pipeline Latency Has Spiked"

1. Check the concurrency limiter's queue depth first — if requests are spending most of their time queued rather than executing, the fix is capacity (more processes, a higher limit if hardware allows) not code.
2. If queue depth is normal, use the trace store (observability section above) to find whether one specific engine's span duration has drifted — this is almost always a computeCost drift the registry article's drift monitor should also have caught; cross-check its alerts.
3. If neither of the above explains it, check whether the active engine set has grown (a new engine promoted to active) without a corresponding registry-generation-triggered cache invalidation actually taking effect — confirm the plan cache's cached generation matches engineRegistry.generation for the affected context.

Runbook: "Requests Are Returning degraded: true"

This is the per-request latency budget's load-shedding path firing, by design, not a bug. Check whether it's isolated to one product adapter (likely a locally expensive active engine set for that context) or platform-wide (likely a genuine capacity or downstream-dependency problem, e.g. a slow database backing one of the memory-category engines). A small, steady baseline rate of degraded responses under peak load is expected and acceptable; a sudden jump is the signal worth paging on.

Runbook: "A Downstream Engine's Output Looks Wrong, But Its Own Code Didn't Change"

Check the trace for that requestId for an upstream engine failure the response didn't surface as an error (because the pipeline degraded gracefully per the partial-batch-failure design) — this is, by a wide margin, the most common root cause of "this engine is producing weird output" reports that turn out not to be that engine's fault at all.

Runbook: "The Research-Only Audit Log Shows Errors, But Nobody Got Paged"

This is expected behavior, not a gap — the research-only-execution section above explains why research-only failures deliberately don't feed the same alerting path active-engine failures do. The correct response is not to page anyone retroactively; it's to route the finding to whoever owns that specific engine's graduation review, as a data point about its current reliability, exactly the way the section describes. If this keeps coming up as a source of confusion for a specific team, the actual fix is making sure that team's on-call runbook links here, not changing the alerting behavior itself.

Runbook: "Two Regions Are Returning Different Results for What Looks Like the Same Request"

First confirm the requests are actually identical — a surprising fraction of apparent cross-region discrepancies turn out to be genuinely different input (a client retried with slightly different data, or routed to a different region on a subsequent, not-actually-identical call). Once genuine identical-input divergence is confirmed, check each region's effective registryGeneration and its most recent override poll timestamp — a brief window of disagreement immediately after an override write is the documented, accepted behavior described in the multi-region section above, not a bug. Divergence that persists well past the documented replication-plus-poll bound is the actual bug signal worth escalating; the bound itself is not.

Case Study: Diagnosing a Slow Pipeline Run

A concrete walkthrough, lightly anonymized from an actual investigation, ties the observability and backpressure mechanisms above together.

Symptom: a specific product adapter's p95 latency crept from ~180ms to ~640ms over the course of a week, with no corresponding deployment on that adapter's own code.

Step 1 — trace store, not guesswork. Pulling a sample of the adapter's slow requests from the trace store (observability section above) showed the same pattern in every one: a single engine, knowledge-graph, had a span duration around 400ms, roughly 8× its historical baseline of ~50ms.

Step 2 — was this a computeCost drift, or something else? The registry article's drift monitor had not flagged knowledge-graph, which ruled out "the engine's own algorithm got slower." That pointed at an external dependency instead — knowledge-graph reads from a Neo4j-backed graph store, not from the engine's own process memory.

Step 3 — the actual cause. A separate, unrelated team had started running a nightly bulk-import job against the same graph database instance, and the import job's write load was contending for the same disk I/O the engine's read queries needed during business hours (the import job's schedule had been set assuming "nightly" meant no daytime overlap, which turned out to be wrong once the import job's own data volume grew past its original estimate).

Why this took under an hour to diagnose instead of days: without the per-engine trace spans, this would have looked like "the whole platform got slower" with no obvious starting point. With them, the investigation had a specific engine and a specific dependency to look at within the first ten minutes, which is the entire value case the observability section above is making — the cost of instrumenting every engine invocation is trivial next to the cost of a multi-day latency investigation with no starting point.

Step 4 — the fix, and why the fix was cheap once the cause was known. The immediate fix was purely operational, not architectural: the import job's schedule moved to a window with genuinely no daytime overlap, agreed with the owning team the same day. No control-plane or engine code changed at all — the entire incident, once correctly diagnosed, resolved itself as a scheduling conversation between two teams. This is worth stating explicitly because it is the common shape of a large fraction of this platform's real latency incidents: the expensive part is almost always diagnosis, not remediation, which is exactly why the observability investment described throughout this article is weighted so heavily toward cutting diagnosis time rather than toward building more elaborate automatic-remediation machinery.

The follow-up that outlasted the incident itself: because knowledge-graph's dependency on a shared, externally-writable database had turned out to be invisible to the platform's own dashboards until someone went looking by hand, a standing dashboard panel was added specifically tracking query latency for every engine with an external data dependency, broken out by dependency, not just by engine — so that the next time a shared resource gets contended, the relevant graph is already on a dashboard someone glances at daily, rather than requiring a fresh investigation to construct.

Performance and Scaling

The concurrency-limiter figure (400) and the timeout formula's constants were both tuned against the platform's current traffic profile and current server sizing. Two things are worth stating plainly about how this scales. First, the plan-building and batching machinery itself scales the same way the registry article's own performance section describes — it is not a bottleneck at any engine count the platform's roadmap anticipates. Second, and less obviously, the concurrency limit and timeout constants are not the kind of thing that should be treated as scaling automatically with traffic growth — they are calibrated to specific hardware and specific measured latency distributions, and the team's practice is to re-run the load test that originally produced the 400 figure every time either traffic volume or server specs change meaningfully, rather than assuming the old number still holds. A concurrency limit that was correct at last year's traffic level and hasn't been revisited is a plausible, easy-to-miss source of unnecessary load-shedding once traffic has genuinely grown to justify a higher limit.

Security Considerations

The control plane's isolation guarantees are what actually enforce the tenant-isolation claims made in the registry article's multi-tenant section: because every pipeline run operates on its own context object with no shared mutable state between concurrent requests, and because engines are pure(-ish) functions of their input rather than stateful objects, there is no code path by which one tenant's in-flight request could read or influence another's, even under the heavy concurrency the worker pool described above is explicitly designed to allow. This isolation property is tested directly, not just asserted: a dedicated test suite runs many concurrent synthetic requests with deliberately distinguishable payloads and asserts that no response ever contains data traceable to a different request's input, specifically to catch any future refactor that might accidentally introduce shared state (a module-level cache keyed incorrectly, for instance) that would violate this guarantee silently.

Framed against the NIST Cybersecurity Framework's five functions, this isolation-testing discipline sits squarely under "Protect" — a control specifically aimed at preventing one tenant's data from becoming accessible through another's request path — while the trace-and-audit machinery covered earlier in this article sits under "Detect." Neither label changes anything about how the code is written; the value of the mapping is that it gives a compliance reviewer, who may never read this article's code samples directly, a way to locate this section's guarantees within a framework they already use to structure a review, without the platform team needing to re-explain its architecture from scratch for every audit.

Comparing the Control Plane to Other Orchestration Patterns

As with the registry pattern, the control plane's design was evaluated against existing orchestration tooling before being built in-house, and the same question comes up whenever a new engineer encounters this layer for the first time: why not just use an existing scheduler?

A Kubernetes-Style Scheduler

Kubernetes' scheduler solves a genuinely similar-sounding problem — deciding what runs where, subject to resource constraints and dependency-like affinity rules. It was ruled out for the same fundamental reason a service mesh was ruled out in the registry article: Kubernetes schedules pods, with a scheduling latency budget measured in hundreds of milliseconds to seconds, onto a cluster of nodes. The control plane schedules function calls within a single process, with a scheduling latency budget measured in microseconds, dozens of times per incoming HTTP request. Asking Kubernetes to make a fresh scheduling decision for every engine invocation in every pipeline run — potentially thousands of decisions per second at the platform's traffic volume — would mean the scheduling overhead alone dwarfs the actual engine computation it's trying to schedule.

A Durable Workflow Engine (Temporal, Step Functions)

This comparison is worth revisiting from the control plane's specific angle, distinct from the registry article's version of the same question. Temporal's core value proposition is that a workflow survives a process crash partway through — step 4 of a 7-step workflow resumes from step 4, not step 1, because Temporal persists execution state after every step. The control plane deliberately does not want this property for engine execution: if a process crashes mid-pipeline-run, the correct behavior is for the caller to retry the entire request, which completes in well under a second even from scratch, because there is no meaningful partial state worth the durability machinery's overhead to preserve. Adopting a durable workflow engine here would mean paying a real latency cost (durable workflow engines typically add tens of milliseconds of persistence overhead per step, explicitly to buy the crash-resumption property) for a guarantee this specific workload doesn't need.

A Message-Queue-Based Actor Model (BullMQ, Akka-style actors)

An actor-model approach — each engine as a long-lived actor with its own mailbox, messages routed between them — was prototyped briefly during the same design phase that produced the event bus. It was set aside for a subtler reason than the two comparisons above: actors are naturally good at modeling long-lived, stateful entities that persist between messages, and every engine in this platform is explicitly stateless between invocations (a deliberate property, covered in the security-considerations section above, that is what makes the tenant-isolation guarantee possible). Building an actor-model layer to host inherently stateless computations adds the actor runtime's own overhead (mailbox management, message serialization even for in-process actors in some implementations) without using the property actors are actually good at. The Promise.allSettled-based batch runner shown throughout this article is, in effect, the stateless-specific simplification of what an actor system would otherwise need significant extra machinery to express.

The GraphQL DataLoader Pattern — the Closest Real Analogue

Of everything evaluated, the closest actual cousin to the batch-execution design in this article is not an orchestration system at all — it's the DataLoader pattern from the GraphQL ecosystem, which batches many individual, seemingly-independent data-fetch requests that occur within a single GraphQL resolution pass into one grouped underlying query, deferred to the end of the current event-loop tick. The structural similarity is genuine: both patterns exist to convert what looks like N independent requests into the minimum number of grouped operations the underlying dependency structure actually requires, both rely on the JavaScript event loop's microtask queue to know when it's safe to say "everyone who was going to ask for something in this round has now asked," and both trade a small amount of added complexity in the batching layer for a large reduction in either query count (DataLoader) or wall-clock serial execution time (this platform's batches). The genuine difference is what triggers a batch boundary: DataLoader's boundary is implicit and time-based (the current microtask tick), discovered dynamically as resolvers happen to call it; this platform's batch boundaries are explicit and structure-based, computed up front from the declared dependency graph via Kahn's algorithm, precisely because engine-to-engine dependencies are known ahead of time and don't need to be discovered by observing calls as they happen. Anyone who has worked with DataLoader already has real intuition for why this platform batches the way it does — the underlying motivation is the same, even though the batch-boundary-detection mechanism differs for a good, structural reason.

What Was Actually Borrowed From Each

Despite ruling out all three wholesale, each contributed a specific idea that shows up in the design above: the concurrency-limited worker pool is a direct, deliberately simplified analogue of a Kubernetes node's resource quota; the per-request latency budget and graceful degradation under it borrows the "fail fast and return partial results" philosophy that durable workflow engines apply at the workflow level, just applied here at the single-request level instead; and the at-least-once, idempotency-required contract for event bus consumption is standard message-queue practice, adopted directly rather than reinvented.

Postmortem: The Promise.all Incident, in Full

The partial-batch-failure section above summarizes the incident that motivated switching from Promise.all to Promise.allSettled. The full postmortem is reproduced here because the timeline itself is instructive — the gap between "the bug existed" and "the bug was found" is where the real lesson lives, not in the one-line fix.

Timeline

• Day 0 (deploy): the initial Promise.all-based batch runner ships. All tests pass — every test used well-formed synthetic inputs, and the specific malformed-input shape that triggered geometric-topological's bug did not exist in the test corpus.
• Days 1–11: the bug fires silently, roughly once every few thousand requests, on a rare but real malformed-input shape arriving from one specific upstream data source. Each occurrence discards five good engine outputs along with the one bad one. No alert fires, because the pipeline still returns a (technically valid, just empty) response — there is no error path to alert on.
• Day 11: a downstream product team notices their aggregate confidence-score dashboard has an unexplained dip in data volume for one specific engine category, files a low-priority ticket.
• Day 14: the ticket gets picked up; the investigation, using the trace store described in this article's observability section (which, notably, had not yet been built at incident time — its absence is a big part of why this took 14 days instead of hours), eventually correlates the missing data with a single upstream engine's rare failure.
• Day 14, same day: root cause identified once someone actually read the Promise.all documentation's rejection semantics side by side with the batch runner code. The fix — Promise.allSettled plus the per-engine ok flag — shipped within hours of diagnosis.

What Actually Changed as a Result

The code fix (Promise.allSettled) is one line different from Promise.all. The organizational fixes that came out of the same postmortem were larger and are the reason this incident is still referenced in onboarding material years later: the observability tracing described earlier in this article was built specifically because this incident took 14 days to even notice, let alone diagnose, with nothing better than a downstream team's dashboard anomaly; the synthetic-malformed-input CI battery (referenced in the registry article's anti-patterns section) was expanded specifically to include the input shape that triggered this bug, plus a general "throw an error partway through a batch" fixture that every new engine's tests must pass; and the alerting philosophy shifted to explicitly ask, for every new failure-handling code path, "if this fails, does anything actually notify a human," rather than assuming a failure that doesn't crash the process will surface itself eventually.

None of that organizational response is exotic — it is standard DevOps / DevSecOps incident-response discipline (blameless postmortem, a concrete action-item list, at least one of those items landing as an automated CI check rather than a written policy) applied to an AI orchestration layer instead of a conventional web service, and treated with exactly the same seriousness. The same underlying discipline — every deployed change versioned, every failure traceable, rollback treated as a first-class operation — is what the versioning section of the registry article calls MLOps applied to a scoring function; this postmortem is the control-plane-side evidence that the discipline extends past individual engines to the orchestration layer that runs them.

Worked Example: One Request, Batch by Batch

A concrete trace through the control plane's execution, using the same litigation-transcript-scoring request the registry article's worked example builds, but followed here from the control plane's own point of view rather than the registry's.

Plan: 6 Batches, 19 Engines

The registry article's worked example already derived the batch structure for this request. Picking up from there: getCachedOrBuildPlan(context) is called first. Assuming this is not the first request for this team since the last registry-generation change, the plan cache hits — no DAG build, just a Map lookup and a generation-number comparison, both sub-microsecond.

Batch 0 (6 engines) — the Concurrency Limiter Barely Notices

All 6 batch-0 engines request a slot from the 400-slot global concurrency limiter. Under normal load, with perhaps a few dozen concurrent requests in flight platform-wide, this request's 6 slots are granted immediately — no queueing. Each engine's runEngineWithTimeout wrapper starts its own trace span; bayesian-confidence (computeCost 2, timeout floor 150ms) completes in 4ms; the slowest of the six, geometric-topological (computeCost 4), completes in 19ms. The batch as a whole resolves in 19ms — the slowest member's time, since Promise.allSettled waits for every promise in the array regardless of how fast the others finished.

Batch 1 (8 engines) — the Event Bus Handoff in Action

bias-detection, one of batch 1's members, declared a dependency on bayesian-confidence and subscribes to its confidence.updated topic. Because batch 0's results were published to the event bus only after the entire batch settled (the ordering guarantee from the event-bus-handoff section above), bias-detection's invocation, which starts at the top of batch 1, can immediately read a fully-resolved confidence.updated event rather than racing against a same-batch sibling's in-flight computation.

A Simulated Failure Mid-Run

Suppose, for this trace, that provenance-tracker (batch 1, computeCost 3) throws on a malformed field in the transcript's metadata. Promise.allSettled still resolves the batch; provenance-tracker's entry in batchResults is { ok: false, error: '...' }; auditLogger.logEngineFailure records it, tagged with the same requestId the trace spans use. Batch 3's authority-mapping engine, which does not depend on provenance-tracker, is entirely unaffected. No other engine in this request's plan declares a dependency on provenance-tracker in this particular trace, so the failure's blast radius is exactly one engine's missing output in the final response — the governance wrapper receives 18 successful results and 1 explicit failure marker, not a discarded response.

Research-Only, Running Alongside, Invisible to the Caller

If a research-only engine happens to be active for this team at the time (say, a not-yet-graduated successor to epistemic-intelligence), it starts executing only after all 6 active-engine batches have resolved and governanceWrapper.wrap() has already been called — its own completion, whenever it happens, has zero effect on when the caller receives their response, per the research-only-execution section's latency-isolation design.

Total Wall-Clock Time

Summing each batch's slowest-member time across all 6 batches for this trace comes to roughly 210ms of actual engine execution time, comfortably inside the 2.5-second per-request latency budget, with the concurrency limiter never becoming a factor at this traffic level. The degraded: true load-shedding path, and the slow-pipeline runbook above, are both for the tail of the distribution this specific trace sits nowhere near.

Code Review Checklist for Control-Plane Changes

Changes to the files described in this article — the plan builder, the batch runner, the concurrency limiter, the research-only execution path — go through a stricter review bar than an ordinary engine PR, because a bug here has platform-wide blast radius rather than being scoped to one engine's own output.

Multi-Region Control Planes

Each region runs its own control-plane processes, independently, with no cross-region coordination for the actual pipeline-execution path — this is a deliberate simplicity choice, distinct from the registry's cross-region override-propagation mechanism described in the registry article. A pipeline run is entirely local to the region that received the request: it reads a local (replicated) copy of the registry state, builds or reuses a locally-cached plan, and executes entirely within that region's process pool. There is no scenario in which one region's control plane invokes an engine running in another region's process — the latency cost of a cross-region call would violate the sub-second pipeline-execution budget this entire design is built around.

The one place cross-region consistency matters is exactly the same registry-generation mechanism discussed in the plan-caching section: because each region polls its own replica of the override tables independently, two regions can briefly disagree about whether a given engine is active, restricted, or deprecated, bounded by the replication-plus-poll window the registry article documents. During that window, it is possible (though rare in practice) for the same logical request, retried and routed to a different region, to receive a plan built from a very slightly different active-engine set. This is treated as an accepted, documented limitation rather than a bug to eliminate — eliminating it entirely would require exactly the kind of cross-region synchronous coordination the sub-second latency budget cannot afford.

Each region's concurrency limit and timeout constants are also tuned and re-validated independently, not shared as a single global figure — a region with different hardware, or serving a structurally different traffic mix (a region skewed heavily toward one product adapter's engine set, say), can legitimately need a different concurrency-limit value than another region's. The quarterly load-test cadence described in the concurrency-limit case study runs per region for exactly this reason; a single global figure, tuned against one region's traffic profile and blindly applied everywhere, would risk being simultaneously too conservative in a lightly-loaded region and too permissive in a heavily-loaded one.

Frequently Asked Questions

Why is the concurrency limiter global to the process rather than per-request or per-team?

A per-request or per-team limiter would protect fairness between tenants but do nothing to protect the process itself from being over-subscribed in aggregate — the original incident that motivated the limiter was exactly an aggregate over-subscription problem, not a single tenant monopolizing resources. A global limiter is the direct fix for the actual failure mode observed; per-team fairness, if it becomes a real problem, would be an addition on top of the global limit, not a replacement for it.

What happens if the plan cache and the registry genuinely disagree for a moment?

By construction, they can't disagree for more than the interval between an override write and the next poll cycle, because the plan cache's only invalidation signal is the registry's own generation counter — there is no independent staleness window on the control-plane side beyond what the registry article already documents for the registry itself.

Does a slow batch ever get retried automatically?

No. A batch that partially or fully fails is not automatically retried by the control plane — the caller (the product adapter, or ultimately the end-user's client) is responsible for deciding whether to retry the whole request. Automatic retry at the batch level was considered and rejected specifically because it complicates the idempotency contract described in the event-bus-handoff section for no clear benefit — a whole-request retry from the caller is simpler to reason about and just as fast, given typical pipeline execution times.

Could the control plane run engines speculatively, before all of a batch's dependencies are confirmed, to save time?

This has been proposed (start a downstream engine's likely-required upstream fetch before the upstream engine's batch has technically completed) and rejected so far, on the reasoning that the actual time saved is small relative to the correctness risk — a downstream engine acting on a not-yet-final upstream value, if the "final" value later differs even slightly, reopens exactly the kind of hidden-timing-dependent bug the registry article's anti-patterns section warns engine authors away from creating within a single engine. The batch boundary is a correctness guarantee, not just a scheduling convenience, and the team's stated bar for giving it up is "a demonstrated, not hypothetical, latency problem it would solve" — which hasn't materialized yet.

Glossary

Anti-Patterns in Control-Plane Extensions

Just as the registry article catalogs anti-patterns specific to individual engine design, the control plane has its own recurring set of mistakes, distinct from those, because the failure surface here is orchestration behavior rather than any single engine's scoring logic. Every one of these has actually happened in a real pull request against this codebase, not a hypothetical.

Anti-Pattern: Reaching Into the Registry Mid-Batch

A tempting shortcut, when an engine needs to know something about another engine's metadata (its riskLevel, say, to adjust its own behavior conditionally), is to call engineRegistry.get(otherEngineId) directly from inside a score() function, rather than having that information passed in through context or consumed via the event bus. This looks harmless — the registry is, after all, just an in-memory object, cheap to read — but it quietly reintroduces a hidden coupling between two engines that the dependency graph doesn't know about, exactly the anti-pattern the registry article warns against from the registry's side. The control-plane-specific consequence is worse than the registry article's version: because the control plane's batching decisions are made entirely from the declared dependencies array, an engine that secretly reads another engine's registry entry can end up scheduled in the same batch as (or even before) the engine whose metadata it's reading, with no guarantee that engine's metadata reflects anything meaningful about that specific request's execution. The fix, enforced by the same lint rule that blocks direct engine-to-engine imports, extends to blocking any engine module from importing engineRegistry at all — only the control plane itself is allowed to touch the registry directly.

Anti-Pattern: Swallowing Timeout Errors Silently

An engine's score() function that wraps its own internal logic in a broad try/catch and returns a default value on any error, including a timeout, defeats the entire partial-batch-failure design described earlier in this article. The control plane's ok: false / audit-logging / confidence-degradation machinery only activates when an engine's promise actually rejects; an engine that catches its own timeout internally and returns something that looks like a successful score bypasses all of it, and produces exactly the overclaiming-confidence anti-pattern the registry article warns about, just triggered by a different root cause (a timeout, not malformed input). Engine authors are told, explicitly, in the same authoring guidelines referenced in the registry article: let it throw. The control plane's failure handling exists specifically so individual engines don't need their own bespoke failure-recovery logic.

Anti-Pattern: Adding a Second, Local Concurrency Limiter

More than once, an engine with an unusually expensive external call (a network request to a third-party enrichment API, for instance) has had a contributor add its own local concurrency limiter — reasonable-sounding in isolation ("don't overwhelm this specific third-party API"), but invisible to, and uncoordinated with, the control plane's global limiter. The result is a second queue the control plane's own latency-budget accounting doesn't know about, which means the per-request degradation logic can't correctly predict whether remaining batches will fit inside the latency budget, because it doesn't know a hidden second queue exists. The correct pattern, when an engine genuinely needs to protect a specific external dependency, is to express that as a lower per-engine timeout and let the existing machinery handle backpressure, or, if genuine separate rate-limiting is needed, to make it visible to the control plane rather than hidden inside the engine.

Timeline: How the Control Plane Evolved

As with the registry's own version history, most of what this article describes was added in response to a dated, specific incident rather than designed in from the start.

• Quarter 2 — first control plane: the simplified runPipeline shown early in this article, Promise.all-based, no concurrency limiting, no tracing, no research-only isolation (research-only engines were, at this point, simply not run in production at all — the concept didn't exist yet).
• Quarter 2, later — the research-only execution path is added, alongside the registry's activationState/researchOnly split described in that article's own timeline; the two changes shipped together since neither was independently useful without the other.
• Quarter 3 — the Promise.all incident (full postmortem above) and its fix, Promise.allSettled, plus the first version of per-engine trace spans, built directly because the incident took 14 days to notice with nothing better than a downstream dashboard anomaly.
• Quarter 3, later — the concurrency limiter, added ahead of (and validated against) the same major traffic spike referenced in the registry article's own timeline, after early load testing revealed the unbounded-concurrency degradation described in the backpressure section.
• Quarter 4 — per-request latency budgets and graceful degraded: true responses, added once the concurrency limiter's queueing behavior made it clear that some requests, under sustained peak load, would need a defined worst-case behavior rather than an unbounded wait.
• Present — the design described throughout this article, with multi-region independence (no cross-region pipeline execution, only registry-state replication) as the most recent addition, mirroring the registry article's own multi-region section.

Appendix: Timeout Reference Table

The exact timeout, in milliseconds, that computeTimeout() produces for every integer computeCost value, useful as a quick reference when reasoning about a specific engine's failure budget without running the formula by hand.

Note that the ceiling constant (2500ms) never actually clamps any value on this scale — at the current computeCost range of 1–10, the formula's natural output never reaches it. The ceiling exists as a safety bound against a future engine being registered with a computeCost outside the documented 1–10 range (which the registry's own schema validation should reject, per the registry article, but the control plane's timeout formula does not want to depend on that validation never having a gap) rather than as a bound that's expected to bind in ordinary operation today.

Design Rationale: A Short Dialogue

"Why bound concurrency at the process level instead of just running more, smaller processes?"

Both are done, in practice — the platform runs multiple control-plane processes behind a load balancer, and each individual process also bounds its own concurrency. The two are complementary, not alternatives: horizontal scaling (more processes) increases total platform capacity; the per-process concurrency limit protects each individual process from being over-subscribed regardless of how many processes exist, which matters because the degradation the original incident exposed was specifically about a single process's event-loop and memory pressure, not about total platform capacity being insufficient.

"Could the per-request latency budget just be the sum of every engine's timeout?"

No, and this is a common first assumption that turns out to be wrong: summing every engine's individual timeout would produce a wildly pessimistic worst case, because most batches complete in far less than their slowest member's timeout, and most requests involve far fewer total batches than the full active engine set's dependency depth would suggest in the worst case. The per-request budget is instead a separate, empirically-set figure (2.5 seconds) chosen to comfortably exceed real p99 end-to-end latency while still being short enough that a caller waiting on a synchronous response doesn't experience it as a hang.

"What would have to be true for the team to reconsider the no-cross-batch-speculation stance?"

A demonstrated latency problem specifically attributable to strict batch sequencing, measured from the trace store, not a hypothetical one — the same evidentiary bar the registry article's graduation process applies to promoting an engine from research-only to active. So far, the observability data has never shown batch-boundary waiting as a meaningful contributor to overall pipeline latency at current traffic and engine-count levels, which is exactly why the proposal has stayed shelved rather than being built and then found unnecessary.

What a New Product Adapter Integration Actually Requires

Every section so far describes the control plane from the inside. It's worth closing the technical portion of this article with the view from a new product adapter team's side, since "how do we actually plug into this" is the practical question that eventually follows from everything above, and the answer is deliberately narrow.

A new product adapter needs exactly three things from the control plane, and nothing more: a domain value registered against whichever engines are relevant to it (a registry-side change, not a control-plane one — new engines or new domain tags on existing engines go through the registry's own review process described in that article); a call to runPipeline(inputSignals, context) with a correctly populated context object (the shape documented in the "what actually lives in context" section above); and adherence to the three-part adapter-side contract from the degraded-response section — never treat a missing engine as a negative signal, surface high/critical-risk gaps to a human reviewer, and retry with backoff rather than immediately. Everything else described in this article — batching, timeouts, concurrency limiting, tracing, research-only isolation — is entirely invisible to an adapter team and requires zero adapter-side code to benefit from. This narrowness is deliberate: the whole point of the control plane sitting between engines and adapters is that an adapter team should never need to understand batching order, timeout formulas, or concurrency limits to safely consume behavioral scoring output. If a new adapter integration ever seems to require understanding any of those internals to work correctly, that is treated as a signal that the contract itself has a gap worth fixing, not that the adapter team needs deeper platform knowledge.

One More Thing: Why This Article Exists Separately From the Registry Article

A fair question, raised more than once during review of this series' outline, is whether the registry article and this one should simply have been a single, longer article — after all, both cover Kahn's algorithm, both reference the same registryGeneration counter, and a plan built by one is executed by the other in the same request. The decision to keep them separate came down to audience and failure-mode overlap being smaller than they first appear. An engineer adding a new engine reads the registry article and, in the overwhelming majority of cases, never needs to touch anything described in this one — the registry's self-registration pattern, schema validation, and activation-state machine are the entire surface area a typical engine author interacts with. An engineer diagnosing a slow pipeline, tuning a timeout, or investigating a partial failure reads this article and, just as often, never needs to open the registry's source at all. Splitting them lets each document stay focused on the actual questions its actual readership brings to it, at the cost of the two small sections of genuine overlap (the DAG algorithm itself, the generation counter) being described twice, once from each side. That redundancy was judged worth paying, deliberately, rather than forcing every reader through a single combined document twice as long, most of which would be irrelevant to whichever specific problem brought them here.

Appendix: Related Reading

• What is a Behavioral Intelligence OS? — the architecture overview this article's kernel analogy builds on directly.
• The 34-Engine Registry — where the execution plan's input (the active, dependency-validated engine set) actually comes from.
• Confidence Propagation in Multi-Engine Systems — what happens to a degraded or missing engine output once it reaches the confidence-combination stage this article only summarizes.
• The Behavioral Event Bus — full detail on the publish/subscribe mechanics this article's event-bus-handoff section only covers from the control plane's side.
• Governance Wrappers — what happens to results after governanceWrapper.wrap() is called, the exact handoff point every code sample in this article ends at.

How Product Adapters Should Handle a Degraded Response

Everything so far describes the control plane's own behavior. It is worth spending real space on the other half of the contract: what a product adapter — the chatbot platform, the legal SaaS platform, the deal intelligence platform — is supposed to do when it receives a response with degraded: true, because the control plane's graceful degradation is only actually graceful if the caller handles it correctly, and the first version of this contract did not specify that clearly enough.

The Response Shape

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-control-plane-orchestrating-an-ai-pipeline

// What governanceWrapper.wrap() actually returns to a product adapter.
{
  degraded: false,        // true if the per-request latency budget cut this short
  results: {
    'bayesian-confidence': { score: 0.82, confidence: 0.91, label: 'high-trust', requiresHumanReview: false },
    'bias-detection':      { score: 0.71, confidence: 0.88, label: 'reasoning pattern...', requiresHumanReview: true },
    // ...
  },
  missingEngines: [],      // engineIds that were planned but never ran (degraded case)
  failedEngines: [],       // engineIds that ran but threw (see partial-batch-failure)
  requestId: 'a1b2c3d4-...',
}

The first version of this contract only had results and an implicit assumption that every planned engine would be present in it. Once graceful degradation and partial batch failure both existed, product adapters started receiving responses with silently missing keys, and each adapter team wrote its own ad hoc "is this key present" check, inconsistently. The explicit degraded, missingEngines, and failedEngines fields were added specifically to make "what happened to this response" a first-class, structurally guaranteed part of the contract rather than something every consuming team had to reverse-engineer from key absence.

The Adapter-Side Contract

Every product adapter is required to implement three behaviors, checked in the adapter's own integration test suite against a control-plane stub that can simulate each condition on demand:

1. Never treat a missing engine's absence as a negative or zero score. A missing bias-detection result means "we don't know," not "no bias detected." Adapters that get this wrong (one early version of the deal-intelligence adapter did) end up silently underreporting risk exactly when the platform was too loaded to fully evaluate it — the worst possible time for that mistake.
2. Surface degraded: true to a human reviewer when the missing engines include anything at high or critical risk level. A degraded response missing only low-risk foundational engines is usually fine to serve as-is; a degraded response missing a critical governance engine should never reach an end user without a flag, and in practice the governance wrapper's own critical-engine-exemption from budget dropping (registry article) means this case is rare, but the per-request latency budget's degradation path is a separate mechanism that isn't subject to that same exemption, so adapters cannot assume it away. Getting this contract right end to end — not just at the control plane, but all the way through every adapter that consumes its output — is governance support work as concretely as any policy document: a human reviewer is only actually protected if the flag reaches them.
3. Retry with backoff, not immediately, and cap retries. A degraded response is frequently a symptom of platform-wide load, and an adapter that immediately retries every degraded response adds more load to an already-loaded system — precisely the wrong response. The standard adapter retry policy is one retry after a short jittered delay, then serve the degraded response as-is with the appropriate human-review flag rather than retrying indefinitely.

Choosing a Concurrency-Limiting Mechanism

The concurrency-limiter section above shows p-limit as the implementation, without explaining why that specific library over the alternatives. It is a small enough decision that it doesn't warrant its own top-level design-rationale question, but it came up often enough in code review on related PRs that it's worth documenting here rather than leaving it as tribal knowledge.

Option 1: A Hand-Rolled Semaphore

A minimal semaphore is genuinely not much code:

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-control-plane-orchestrating-an-ai-pipeline

class Semaphore {
  #available;
  #queue = [];
  constructor(limit) { this.#available = limit; }

  async acquire() {
    if (this.#available > 0) { this.#available--; return; }
    return new Promise(resolve => this.#queue.push(resolve));
  }

  release() {
    if (this.#queue.length > 0) { this.#queue.shift()(); }
    else { this.#available++; }
  }
}

This was, in fact, the very first implementation, before the team switched to p-limit. It works, but it pushes the burden of correct acquire/release pairing onto every call site — miss a release() in an error path (easy to do, and exactly the kind of mistake a rushed incident-response hotfix is prone to making) and the semaphore permanently leaks a slot, silently reducing effective concurrency until the next process restart.

Option 2: Worker Threads With a Fixed Pool

Node's worker_threads module gives true parallelism (bypassing the single-threaded event loop entirely) rather than the concurrency (interleaved, still single-threaded) that both the semaphore and p-limit provide. This was evaluated and rejected for engine execution specifically because the vast majority of engine computation in this platform is not CPU-bound enough to benefit from true parallelism — most engines spend their time waiting on I/O (a database query, an external API call) or doing lightweight in-process computation, and the serialization overhead of passing input/output across the worker-thread boundary (structured-clone serialization, not free) would cost more than the parallelism gains for the platform's actual workload profile. Worker threads remain the right tool elsewhere on the platform for genuinely CPU-bound work — the audio-conversion microservice referenced earlier is a Python process for exactly this class of reason — but not for engine scoring specifically.

Why p-limit Won

p-limit's API returns a wrapped function that automatically handles the acquire/release pairing via the promise it wraps resolving or rejecting — there is no call site that can forget to release a slot, because release isn't a separate call at all, it's implicit in the wrapped promise settling. This eliminates the entire class of bug the hand-rolled semaphore was vulnerable to, for a small, well-maintained, single-purpose dependency, which is exactly the trade the team is comfortable making for infrastructure code sitting this close to the platform's core request path — a small, auditable dependency instead of hand-rolled code that has to get several subtle things right on every call site, forever.

Chaos Testing the Control Plane

The unit and integration tests shown earlier assert specific, known failure conditions. A separate, periodic chaos-testing exercise exists to find failure conditions nobody thought to write a specific test for, by deliberately injecting randomized faults into a staging environment running realistic traffic replay.

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-control-plane-orchestrating-an-ai-pipeline

// chaos/inject-faults.js — run against staging only, gated behind an
// explicit environment flag that refuses to start if it detects a
// production database connection string.
const FAULT_TYPES = ['timeout', 'throw', 'slow-10x', 'malformed-output'];

function wrapWithChaos(engine, faultRate = 0.02) {
  return {
    ...engine,
    async score(signals, context) {
      if (Math.random() < faultRate) {
        const fault = FAULT_TYPES[Math.floor(Math.random() * FAULT_TYPES.length)];
        switch (fault) {
          case 'timeout':          await sleep(10_000); break;
          case 'throw':            throw new Error('chaos: injected failure');
          case 'slow-10x':         await sleep(engine.computeCost * 400); break;
          case 'malformed-output': return { score: 'not-a-number', confidence: null };
        }
      }
      return engine.score(signals, context);
    },
  };
}

Running staging traffic through a registry where every engine is wrapped this way, at a low but nonzero fault rate, has surfaced real bugs that no hand-written unit test anticipated — most notably, an early version of one product adapter's client code that handled a failedEngines entry correctly but crashed outright on a malformed-output case (a score field that was present but not the expected type), because every existing test had only ever exercised "engine missing" and "engine threw," never "engine returned successfully but with a nonsensical value." The malformed-output fault type was added to the chaos harness specifically after that gap was found in a different, unrelated investigation, and it's since caught two more similar issues before they reached production. Chaos runs are scheduled weekly against a shadow copy of a recent day's real traffic, not continuously, on the reasoning that continuous chaos injection in staging makes staging too noisy to use for its other normal purpose (validating ordinary feature work before release).

Appendix: Comparing This Design Against a Naive Sequential Pipeline

It is easy to lose track, after this many sections of failure-handling detail, of how much the batching-and-parallelism design actually buys over the simplest possible alternative — running every active engine strictly one after another. The table below uses the registry article's worked-example request (19 active engines, 6 dependency-depth batches) to make the comparison concrete.

The gap between the first and second rows — roughly 2–3× faster from batching alone, for this specific request shape, and considerably more for requests with a shallower dependency graph relative to their total engine count — is the entire performance case for building the DAG-and-batch machinery in the first place, independent of every reliability mechanism layered on top of it since.

A Longer Look at the Kernel Analogy's Limits

The kernel analogy earlier in this article is useful, and it is worth being equally clear about where it stops being accurate, because stretching an analogy past its useful range is its own source of design mistakes. A real OS kernel provides strong memory protection between processes, enforced by hardware (the MMU) — one process genuinely cannot read another's memory even if its code tries to. The control plane's "isolation" between concurrent requests, by contrast, is a property of how the code is written (no shared mutable state, fresh context objects per request), not a hardware-enforced guarantee — it is closer to convention backed by tests (the isolation test suite mentioned in the security-considerations section) than to a guarantee the runtime itself makes impossible to violate. This is a meaningfully weaker guarantee than a real kernel's memory protection, and it's why that isolation test suite exists and is treated as load-bearing rather than a nice-to-have: it is the only thing standing between "isolation by convention" and "isolation by construction," and the team is honest internally that this is a real, accepted gap relative to the analogy rather than something to paper over.

Similarly, a real kernel's process scheduler can preempt a running process mid-execution — forcibly pause it and resume something else. The control plane's engine execution has no equivalent: once an engine's score() function starts running (synchronously, or via an awaited promise chain), the control plane can only wait for it to settle or hit its timeout; it cannot forcibly interrupt genuinely CPU-bound synchronous JavaScript mid-execution, because JavaScript's single-threaded event loop doesn't offer that capability the way an OS scheduler's preemptive multitasking does. This is the underlying reason the engine-authoring guidelines (referenced from the registry article) insist engine logic stay asynchronous and avoid long synchronous computation blocks — a misbehaving engine that blocks the event loop synchronously for hundreds of milliseconds affects every other concurrently-executing request's ability to make progress, in a way the timeout mechanism cannot protect against, because the timeout's own setTimeout callback needs the event loop to be free to fire in the first place.

Case Study: Scaling the Concurrency Limit for a Product Launch

The registry article references a major product-launch traffic spike, roughly 6× normal request volume for two days, as the event that motivated the compute-budget and adaptive-throttling mechanisms described there. That same launch is worth revisiting here in detail, because it is also the event that put the control plane's concurrency limiter through its first genuine stress test, and the way the team approached raising the limit ahead of the launch — rather than reactively during it — is a template the team still follows for every subsequent capacity-planning exercise.

Three Weeks Before Launch: Establishing a Baseline

The marketing and product teams had a firm date and a rough traffic estimate from a similar past launch on an adjacent product. Rather than guessing at a new concurrency-limit value, the platform team's first step was purely observational: instrument the existing 400-slot limiter to log queue depth and queue wait time continuously, not just on request, and let that instrumentation run against ordinary daily traffic for a full week to establish exactly how close to saturation the existing limit ran under real peak-hour conditions, not synthetic load. The finding was reassuring on its own — ordinary peak traffic used, at most, around 140 of the 400 available slots, meaning there was substantial headroom before the limiter itself would become the binding constraint at all.

Two Weeks Before Launch: Synthetic Load Testing at Six Times Volume

Headroom under ordinary traffic doesn't tell you what happens at 6× that traffic, so the next step was a synthetic load test, replaying a recorded day of real (anonymized) traffic at 6× speed against a staging environment sized identically to the then-current production fleet. This is where the first real problem surfaced: at simulated 6× load, queue depth on the 400-slot limiter grew essentially without bound over the course of the test, and p99 latency climbed past the per-request latency budget within the first few minutes of sustained load, which meant a meaningful fraction of requests would have started returning degraded: true responses well before the platform was anywhere near its actual hardware ceiling. The 400-slot figure, chosen against a very different traffic profile a year earlier (the original incident referenced in the backpressure section), had simply never been revisited as traffic organically grew in the intervening months, and had quietly become the binding constraint before hardware capacity was.

One Week Before Launch: Re-Tuning, Not Just Raising the Number

The naive fix — raise the limiter to, say, 1200 slots, three times the old figure — was tried first and rejected by the same synthetic load test: past a certain concurrency level, per-invocation latency itself started degrading (more concurrent work competing for the same CPU and memory, even though no individual engine's own algorithm had changed), which meant simply raising the ceiling didn't scale throughput linearly — it just moved where the knee in the latency curve sat, and moved it to a point uncomfortably close to where actual hardware saturation began. The eventual fix combined two changes: the concurrency limit was raised to 700 (found, again empirically via the load test, to be the point where per-invocation latency degradation started becoming noticeable, kept as the ceiling with margin below it), and, more consequentially, an additional two control-plane processes were added to the fleet specifically for the launch window, spreading total concurrent capacity across more processes rather than trying to push a single process's limiter arbitrarily high. This is the same "horizontal scaling plus a sane per-process limit, not one or the other" principle referenced in the earlier design-rationale dialogue, validated here under real stress-test conditions rather than stated as abstract preference.

During Launch: What Actually Happened

Real launch-day traffic peaked at roughly 5.3× normal volume, slightly under the 6× the synthetic test had targeted, comfortably inside the re-tuned capacity. Queue depth on the concurrency limiter stayed bounded throughout, p99 latency stayed under the per-request budget for the overwhelming majority of the launch window, and the small number of degraded: true responses that did occur clustered predictably around the single highest-traffic ten-minute window, exactly matching what the pre-launch load test had predicted for that specific load level — which the team treats as the actual validation that the load-testing methodology itself, not just this specific capacity number, was sound.

What Changed as a Standing Practice

The concurrency limit is no longer treated as a constant set once and forgotten. It is now re-validated against a synthetic load test on a standing quarterly cadence, independent of whether a specific launch is scheduled, specifically because this incident demonstrated that traffic growth can silently erode a previously-adequate limit's headroom well before anyone notices via ordinary monitoring — ordinary peak-traffic queue depth looked completely fine right up until the synthetic 6× test revealed how little margin actually remained.

Extended Design Rationale: More Questions From Engineers

"Why doesn't the control plane just prioritize which engines to run, the way an OS scheduler prioritizes processes, instead of running everything active and dropping under budget pressure?"

This is close to what actually happens, just implemented as two separate mechanisms operating at two different layers rather than one unified priority scheduler. The registry's compute-budget dropping (covered in the registry article) is effectively priority-based admission control, deciding which engines are even included in a given request's plan before execution starts. The control plane's per-request latency budget and graceful degradation is a second, later-stage mechanism that can still shed work mid-execution if reality diverges from the plan's expectations (a batch running slower than its timeout-based worst case would suggest, for instance, due to genuine load rather than any single engine misbehaving). A single unified scheduler that combined both concerns was considered during the original design phase and set aside because the two mechanisms have meaningfully different inputs — the registry's decision is about declared, static metadata (computeCost, priority overrides) known before any request-specific execution begins; the control plane's decision is about real, observed, request-specific timing — and conflating them into one mechanism seemed likely to produce something harder to reason about than two smaller mechanisms with clearly separated responsibilities.

"Has the team ever considered moving engine execution to a separate process pool, away from the HTTP-request-handling process?"

Yes, and the honest answer is that it remains an open question the team revisits periodically rather than a settled no. The case for it: it would let engine execution scale independently of HTTP-connection-handling capacity, and would isolate a misbehaving engine's resource consumption from the process actually accepting new connections. The case against, so far decisive: it would reintroduce a network hop (or at minimum an IPC hop) between the control plane and every engine invocation, undermining exactly the low-per-invocation-overhead property that ruled out a Kubernetes-style scheduler and a durable workflow engine earlier in this article. The current stance is that this tradeoff would only be worth making once engine-execution CPU load and HTTP-connection-handling load can be shown, from real production metrics, to actually be competing for capacity in a way the current single-process model measurably suffers from — and that evidence hasn't materialized yet, mirroring the same evidentiary bar the batch-speculation question above was held to.

"Why is the trace-span mechanism built in-house rather than using a standard tracing library like OpenTelemetry directly?"

It is, in fact, built on top of OpenTelemetry's SDK — the tracer.startSpan() call shown in the observability section is a thin, platform-specific wrapper around the standard OpenTelemetry API, not a bespoke tracing system built from scratch. The wrapper exists only to enforce the platform-specific convention of always attaching engine.id, engine.version, and request.id as span attributes consistently, and to route spans to the same backing store the compliance audit trail reads from — the tracing standard itself is not something the team saw any reason to reinvent.

"What's the single biggest thing that would break if traffic grew 100× from today?"

Almost certainly not the algorithmic pieces — the registry article's own performance section already validated the DAG-build machinery well past any traffic-driven engine-count growth the roadmap anticipates, and Kahn's algorithm's cost is a function of engine count and edge count, not request volume. The honest answer is the concurrency-limiter-and-process-count capacity question covered in the case study above: 100× today's traffic would require a capacity-planning exercise of a similar shape to the one described there, just at a scale the current quarterly re-validation cadence hasn't been stress-tested against. The team's stated position is that the process (measure, don't guess; load-test synthetically before committing to a number; treat capacity figures as living, not fixed) would scale to that traffic level even if the specific numbers it currently uses would not, unmodified.

Onboarding Checklist: Your First Change to the Control Plane

New contributors to the control-plane codebase specifically (as distinct from contributing a new engine, which follows the registry article's own onboarding path) are pointed at this checklist before their first PR against any file described in this article.

1. Read this article in full, plus the registry article's dependency-resolution and compute-budgeting sections — the control plane is not a self-contained system; a meaningful fraction of its behavior is a direct consequence of registry-level decisions.
2. Run the full control-plane test suite locally, including the partial-batch-failure and research-only-isolation tests shown earlier, and read through at least the three tests reproduced in this article's testing section closely enough to explain, from memory, what each one is actually protecting against.
3. Read the Promise.all postmortem in full. It is deliberately kept as required reading rather than summarized away, because the one-line diff (Promise.all → Promise.allSettled) undersells how easy the underlying mistake is to reintroduce elsewhere in a similar shape — a new batch-like construct added anywhere in this codebase should be checked against the same failure mode from day one.
4. Before proposing any change to the concurrency limit, the timeout formula's constants, or the per-request latency budget, read the concurrency-limit case study above and expect to be asked for load-test data, not a reasoned guess, in review — this is treated as a hard requirement, not a suggestion, specifically because "reasoned guess" is exactly how the original 400-slot figure went stale before the product-launch stress test caught it.
5. Any new failure path added anywhere in the pipeline runner needs an explicit, written answer (in the PR description, not just implied by the code) to "does anything notify a human if this fires in production" — directly descended from the alerting-philosophy fix in the postmortem, and checked as a standing item on the code-review checklist earlier in this article.
6. Before touching anything in the research-only execution path specifically, write (or find and re-run) the latency-isolation test from the testing section and confirm it still passes against the unmodified code first — knowing the baseline passes before you start is the only way to be confident your change is what broke it later, rather than discovering a pre-existing gap and wrongly attributing it to your own diff.

None of these five steps is optional in practice, and none of them is enforced purely by good faith — the test suite itself, the required load-test-data attachment on capacity-related PRs, and the code-review checklist earlier in this article are the actual mechanisms that make this list something more than aspirational documentation. A new contributor who skips straight to writing code without this onboarding path tends to rediscover, the hard way and usually in production, some subset of the incidents this article has already documented in detail.

Reference: Every Function in control-plane.js

A consolidated reference, gathering every function shown piecemeal across this article into one table, useful when reading the actual source file for the first time and wanting to know at a glance what each piece is responsible for without re-reading every section.

A Technical Detour: Why Promise.race, Not AbortController

Every timeout implementation shown in this article uses Promise.race against a rejecting setTimeout, rather than the more modern AbortController/AbortSignal pattern many newer Node.js APIs support natively. This is a deliberate choice worth explaining, because it looks, at first glance, like the platform is using an older, less idiomatic pattern than it should.

AbortController is the more correct tool when the underlying operation actually supports cancellation — an AbortSignal passed into fetch(), for instance, genuinely stops the in-flight network request, freeing the underlying socket immediately rather than just abandoning a promise while the real work continues unseen. The critical problem for engine execution specifically: the vast majority of the platform's 34 (soon 100+) engines do not accept an AbortSignal anywhere in their internal implementation, because their internals are a mix of synchronous computation, third-party library calls, and database queries via drivers that, in several cases, don't support abort signals at all. A Promise.race-based timeout does not stop the losing promise's underlying work — it just stops waiting for it; the engine's score() function keeps running to completion in the background, its eventual result simply discarded once the timeout promise has already won the race and rejected.

This is a real, accepted limitation, not an oversight: a timed-out engine invocation still consumes CPU and holds its concurrency-limiter slot (released only when the underlying promise actually settles, not when the timeout race resolves) for however long its real execution actually takes, even though the caller stopped waiting on it. The team's stated position is that this is the correct tradeoff for now, given how few of the platform's actual dependencies (database drivers, third-party SDKs) support genuine cancellation today — building genuine abort-propagation into every engine's internals would be a large undertaking for a benefit (freeing resources marginally sooner on the relatively rare timeout path) that hasn't yet been shown to matter at the platform's current timeout-failure rate, which per-engine dashboards show sitting at a small fraction of one percent of invocations platform-wide. If that rate were to grow substantially, or if a specific expensive engine's timeout-but-still-running behavior were shown to meaningfully contribute to the concurrency-limiter saturation the case study above describes, genuine abort-propagation would become the next thing worth building — but it would be built in response to that evidence, following the same "measure before you build" discipline this article has described repeatedly, not preemptively.

Adopting This Pattern in a Smaller System

Not every reader of this article is operating at 34-plus-engine, multi-region scale, and the registry article's own version of this question (what to adopt first, what to defer) applies here too, with a control-plane-specific answer.

The minimal viable version of everything in this article is genuinely small: a topological-sort-based batch builder (the buildExecutionPlan function shown early on, essentially unchanged from a first implementation), a Promise.allSettled-based batch runner (never Promise.all, from the very first line of code — this is the one piece of advice in this entire article worth adopting even in the smallest possible system, given how expensive the alternative proved to be here), and nothing else. No concurrency limiter, no per-request latency budget, no distributed tracing, no chaos testing — all of those exist here because they fixed a specific, dated production problem at a specific scale, and a system with a handful of engines and modest traffic is unlikely to hit any of those problems before it has the traffic volume to justify the investment.

The recommended order to add the rest, if and when the need actually materializes, mirrors the order they were actually built here, per the timeline section above: research-only execution and its latency-isolation guarantee first (as soon as there is any concept of "not yet trusted" logic that needs to run without affecting production decisions); basic trace spans next (cheap to add, and the single highest-leverage addition for cutting future incident diagnosis time, per the postmortem's own conclusion); a concurrency limiter only once real load-testing — not intuition — shows unbounded concurrency degrading shared resources; and a per-request latency budget with graceful degradation only once the concurrency limiter's queueing behavior has, in practice, started producing occasional slow outliers worth bounding explicitly. Building any of these before the problem they solve has actually been observed is exactly the premature-complexity trap this platform's engineering culture has otherwise tried hard to avoid, as both this article and the registry article repeatedly note. Adopted in this deliberately staged order, this is what digital transformation looks like at the infrastructure layer for a smaller team: not copying a 34-engine platform's full machinery wholesale, but sequencing the same underlying ideas against evidence of actual need.

What Actually Lives in context

Every code sample in this article passes a context object through the pipeline without ever showing its shape. It's worth documenting explicitly, since almost every engine's score(signals, context) function reads from it, and getting its contents wrong (reading a field that isn't guaranteed to exist, or worse, writing to it and expecting the write to be visible to a later batch) is a recurring source of new-contributor confusion.

// (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
// License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
// Article published: 22 May 2026.
// URL: https://govindpreetsingh.com/article.php?slug=the-control-plane-orchestrating-an-ai-pipeline

interface PipelineContext {
  requestId:        string;    // UUID, assigned once at pipeline entry, immutable
  teamId:           string;    // drives registry restriction checks and plan-cache keying
  tenantId:         string;    // drives per-tenant budget multipliers (registry article)
  domain:           string;    // 'legal' | 'sales' | 'trust' | ... — filters candidate engines
  matterType?:      string;    // adapter-specific filter, e.g. 'litigation' | 'settlement'
  latencyBudgetMs:  number;    // remaining time before graceful degradation kicks in
  startedAt:        number;    // Date.now() at pipeline entry, used to compute remaining budget
  traceContext:     TraceContext; // OpenTelemetry propagation context for this request
}

context is deliberately immutable once constructed at pipeline entry — no engine, and no part of the control plane itself past the entry point, ever mutates a field on it. Any information a batch needs to pass to a later batch goes through the event bus instead, exactly as the event-bus-handoff section describes, never by attaching a new field to context and hoping every later reader knows to look for it. This rule exists for the same reason the registry article insists engines never read another engine's registry entry directly: an object that anything can silently mutate becomes, over enough contributors and enough time, a second, undeclared communication channel that the dependency graph has no visibility into — precisely the hidden-coupling anti-pattern this series keeps returning to in different guises.

More Frequently Asked Questions

Can a batch contain engines from different risk levels?

Yes, routinely — batching is purely a function of dependency structure, not risk level. A critical-risk governance engine can sit in the same batch as a low-risk foundational engine if neither depends on the other. The exemption from budget-based dropping (registry article) and the never-fails-the-whole-request-gracefully treatment (this article's partial-batch-failure section) both key off each individual engine's own riskLevel, not its batch-mates'.

What happens if the registry itself fails to build a plan (a cycle, say) mid-request?

This should be structurally impossible by the time a request reaches the control plane, because the registry's own startup and CI validation (registry article) already guarantee the currently-registered set is cycle-free. If buildExecutionPlan somehow still throws at request time — which would indicate a validator gap rather than an expected condition — the request fails loudly with a 500-equivalent error rather than attempting any kind of partial or degraded response, on the reasoning that a cycle indicates the platform's own configuration is in a state no amount of graceful runtime handling can meaningfully paper over.

Is there ever a reason to run the same engine twice within one pipeline run?

No — every engine appears at most once in a given execution plan, because the plan is built from the deduplicated, currently-active engine set, keyed by engineId. An engine that genuinely needs to reconsider its output after seeing a later batch's results is a design smell the platform doesn't have a sanctioned pattern for; the closest sanctioned equivalent is splitting the "reconsider" logic into its own downstream engine with an explicit dependency, rather than looping the original engine.

Does the control plane ever run engines for a request that never sees the response — for logging or metrics purposes only?

Research-only execution is the only sanctioned version of "runs but the caller never sees it," and it's fully documented above. There is no separate "fire and forget for metrics" execution path distinct from that mechanism — if a team wants to collect data on a not-yet-trusted engine's behavior, the answer is always "register it as research-only," never a bespoke one-off logging hook, specifically to keep every engine invocation's existence traceable through the one registry-and-audit-log mechanism rather than accumulating parallel, undocumented execution paths over time.

What's the difference between a batch timing out and the whole request timing out?

A batch doesn't have its own timeout as a unit — each engine within it has its own individual timeout, from the computeTimeout formula, and the batch as a whole simply takes as long as its slowest surviving member (or fails that specific member, per the partial-batch-failure design, without failing the batch itself). The per-request latency budget is the only timeout that operates at a level above individual engines, and it doesn't "time out" a batch already in flight — it prevents the next batch from starting if there isn't realistically enough remaining budget to run it. A batch that has already started always runs to completion (success, individual-engine timeout, or individual-engine failure); the request-level budget only ever affects whether a not-yet-started batch gets to begin.

If two engines in the same batch both emit to the same topic, does the second one's event overwrite the first's?

No — the registry article's #topicEmitters index and the event bus both treat a topic as having potentially many emitters, and a subscriber receives every emission as a distinct event carrying its own engineId, not a single overwritten value. This is deliberate: it means two engines can legitimately contribute independent signal to the same downstream topic without either one clobbering the other, and a downstream engine that only cares about one specific emitter can filter on engineId within the topic rather than the platform needing separate topics per emitter to achieve the same isolation.

Circuit Breaking: Should a Chronically Failing Engine Be Skipped?

Every mechanism in this article treats each engine invocation as an independent event, evaluated fresh on its own merits — a timeout or failure on one request has no memory carried forward to the next. This is a deliberate simplification, and it is worth being explicit about the case it does not handle well: an engine that is not intermittently flaky (the partial-batch-failure design handles that fine) but is persistently broken — every invocation for the last several minutes has failed or timed out, perhaps because a downstream dependency it needs (a database, a third-party API) is entirely down.

Without any circuit-breaking, the control plane keeps dutifully invoking that engine on every eligible request, each invocation running to its full timeout before failing, each one consuming a concurrency-limiter slot for the duration. At any individual-request level this is harmless — the partial-batch-failure handling degrades gracefully exactly as designed. At a platform level, under sustained failure, it is a real (if usually modest) drag on total capacity: every request touching that engine pays its full timeout cost for a guaranteed failure, right as the concurrency limiter may already be under pressure from whatever caused the downstream outage in the first place.

A simple circuit breaker — after N consecutive failures within a window, stop attempting the engine entirely for a cooldown period, treating it as if it failed instantly rather than after a full timeout — was prototyped and is not yet in production, deliberately. The reasoning: the platform's actual incidence of a genuinely persistent, all-requests engine failure (as opposed to the much more common transient, low-rate failures the existing machinery already handles well) has been rare enough in practice that the team has not yet had a real incident where circuit breaking would have measurably helped beyond what the existing timeout-and-degrade path already provides. This sits in the same category as the batch-speculation and separate-process-pool questions raised earlier in this article — a change the team has a concrete design for, is not opposed to on principle, and is deliberately not building ahead of clear evidence that the problem it solves is actually costing the platform something today.

Appendix: a Real Trace, Annotated

Closing with an actual (lightly redacted) trace-span sequence for one pipeline run — the same litigation-transcript request for a professional services law firm this article's worked example builds — in the same spirit as the registry article's annotated startup log; every mechanism described in this article has a corresponding, legible entry here.

# (c) Govind Preet Singh -- govindpreetsingh.com. All rights reserved.
# License required for use -- contact govindpreetsingh.com. No unapproved use permitted.
# Article published: 22 May 2026.
# URL: https://govindpreetsingh.com/article.php?slug=the-control-plane-orchestrating-an-ai-pipeline

[pipeline] requestId=a1b2c3d4 team=team_442 domain=legal matterType=litigation
[pipeline] plan cache HIT (generation=118) — 19 engines, 6 batches
[batch 0] starting 6 engines, concurrency-limiter slots available: 6/400 requested
  span engine.execute engineId=behavioral-entropy      dur=6ms   status=OK
  span engine.execute engineId=geometric-topological    dur=19ms  status=OK
  span engine.execute engineId=bayesian-confidence      dur=4ms   status=OK
  span engine.execute engineId=knowledge-graph          dur=11ms  status=OK
  span engine.execute engineId=embedding-index          dur=8ms   status=OK
  span engine.execute engineId=behavioral-vm-kernel     dur=2ms   status=OK
[batch 0] settled in 19ms (slowest: geometric-topological)
[event-bus] published 6 topics from batch 0
[batch 1] starting 8 engines
  span engine.execute engineId=bias-detection           dur=14ms  status=OK
  span engine.execute engineId=state-machine-runtime    dur=22ms  status=OK
  span engine.execute engineId=provenance-tracker       dur=—     status=ERROR "malformed metadata field: intake.callDuration"
  span engine.execute engineId=governance-safety        dur=3ms   status=OK
  ... 4 more, all OK ...
[batch 1] settled in 22ms — 7 ok, 1 failed (provenance-tracker)
[audit] logEngineFailure engineId=provenance-tracker requestId=a1b2c3d4
[event-bus] published 7 topics from batch 1 (provenance-tracker's topics NOT published)
[batch 2..5] — 5 more engines, all OK, cumulative 210ms total
[research] 1 research-only engine (epistemic-intelligence-v2) fired async, not awaited
[governance] wrap() called — 18 ok, 1 failed, 0 missing (no budget/latency drops on this request)
[pipeline] requestId=a1b2c3d4 complete in 214ms, degraded=false

Every line traces back to a mechanism this article covers in detail: the "plan cache HIT" line is the registry-generation check from the caching section; the per-engine spans with duration and status are exactly the OpenTelemetry instrumentation from the observability section; the batch-1 failure and the immediately following audit log line are the partial-batch-failure design in action, end to end, on a real (simulated) malformed-input case; the note that provenance-tracker's topics were not published is the direct consequence of the event-bus-handoff section's rule that only successful results get published; and the final degraded=false confirms this particular request never came close to its latency budget, landing at 214ms against a 2500ms ceiling. An engineer fluent in reading this trace format, the way the registry article's closing section asks readers to become fluent in its startup log, has genuinely internalized how every mechanism in this article behaves together on a single real request rather than in isolation.

What to Watch For

• Timeout calibration — Set engine timeouts proportional to computeCost, using a super-linear curve validated against real p99 latency data, not a linear guess. A cost-8 engine needs meaningfully more than 8× the base timeout once variance is accounted for. Never let one slow engine block the whole pipeline.
• Partial batch failure — Use Promise.allSettled, never Promise.all, for batches of independent engines. A single engine's failure must never discard every other engine's already-computed output in the same batch.
• Event bus backpressure — If a later-batch engine subscribes to a topic that was never emitted (because an earlier engine failed), it must handle the missing event gracefully — check the ok flag on every declared dependency before reading its output, and degrade confidence rather than treating a missing signal as zero.
• Plan caching — The execution plan is deterministic for a given registry state. Cache it, keyed by context, and invalidate only on a registryGeneration change. Re-building it on every request is unnecessary work at scale.
• Concurrency limits are a separate concern from compute budgets — a compute budget controls which engines run for one request; a concurrency limit controls how many requests' worth of engine invocations run at the same time, platform-wide. Conflating the two, or omitting the second entirely, is how a traffic spike degrades every in-flight request instead of just the ones actually causing the spike.
• Research-only latency isolation is a promise worth testing directly — don't rely on code review alone to catch a refactor that accidentally puts a research-only engine's execution back on the response-latency critical path; write the test that asserts it.
• Capacity constants go stale silently — a concurrency limit or timeout formula that was correct when it was set can quietly become the binding constraint as traffic grows, well before ordinary monitoring notices. Re-validate against a real synthetic load test on a standing cadence, not just before a known launch. Ordinary peak-traffic metrics looking healthy is not, by itself, evidence that real headroom remains — the product-launch case study above is a direct demonstration of exactly that gap.
• Don't confuse "the response returned" with "nothing went wrong" — check degraded, missingEngines, and failedEngines on every response a product adapter consumes; a structurally successful HTTP response can still represent a partially-failed pipeline run, and treating it as fully successful is how missing signals silently understate risk.

Appendix: Applying computeTimeout to Real Engines

The reference table earlier in this article shows the formula's output for every integer computeCost in the abstract. Grounding it against the actual catalog from the registry article makes the practical effect more concrete.

The spread is deliberate and visible here: the platform's two cheapest, most-depended-on engines get the floor value regardless of how low their formula-computed timeout would otherwise be, while digital-twin-simulation, the single most expensive registered engine, is allowed over a full second before the control plane gives up on it — more than seven times bayesian-confidence's allowance, reflecting a genuinely different computation, not an arbitrary difference in patience.

Could Engines Be Written in Other Languages?

Every engine and every control-plane code sample in this article is JavaScript/Node.js, and a question that comes up whenever a data-scientist contributor wants to write an engine using a Python ML library directly, rather than calling out to a separately-hosted Python service, is why engines can't simply be exposed as individual internal HTTP endpoints, letting each be implemented in whatever language its author prefers.

This was evaluated seriously, not dismissed out of hand, because the underlying need — genuinely wanting to use Python's ML ecosystem for a specific engine's internals — is real and recurring. The reason it isn't the platform's general answer is almost entirely about the batch-execution latency math worked through earlier in this article: a batch of six foundational engines currently completes in roughly 19ms because every invocation is an in-process function call. Making even one of those six engines a network call to a separately-hosted process changes that batch's floor from "bounded by the slowest in-process function" to "bounded by the slowest in-process function, plus at least one network round-trip" — typically a few milliseconds within the same datacenter, but no longer negligible against a 19ms baseline, and considerably worse if that network call has to cross an availability zone or contend with the called service's own load.

The platform's actual answer for genuine polyglot needs is narrower and more deliberate than "any engine can be any language": a small number of engines are explicitly designated as calling out to a separately-hosted service — this is precisely the pattern the audio-conversion microservice referenced earlier in this article uses, just applied to a scoring engine rather than a media-processing utility — and those specific engines carry a materially higher declared computeCost and a correspondingly longer timeout to account for the network hop honestly, rather than pretending the call is as cheap as an in-process one. This is treated as an explicit, reviewed exception per engine, not a general capability every engine author can reach for by default, specifically so that the platform's overall latency profile doesn't quietly erode one "just this once" cross-process call at a time.

The Full Test Matrix

Pulling together every testing mechanism referenced across this article into one table, since they were introduced piecemeal alongside the specific failure mode each one addresses.

No single layer in that table is sufficient on its own — the unit tests would never have caught the Promise.all incident's actual production impact (they tested the failure-handling logic once it existed, not the absence of it beforehand), and the chaos testing would never catch a subtle contract mismatch between two specific pieces of code the integration tests check directly. The six layers together are the platform's actual answer to "how do we know this works," and each one earns its place in this list by having caught, or being built specifically because of, a real incident referenced somewhere else in this article.

A Note on Naming

"Control plane" is a deliberate borrowing from networking and distributed-systems vocabulary, where it names the part of a system responsible for deciding what should happen, as distinct from the "data plane," which is the part that actually carries the traffic and does the work. The naming is worth a short note because two other names were seriously considered and rejected, and each rejection is informative about what this layer is and isn't.

"Orchestrator" was rejected because, in most of the systems that use that word (container orchestrators being the dominant example), it implies coordinating independently-deployed, independently-failing services across a network — exactly the model the service-mesh comparison earlier in this article rejected for engine execution. Calling this layer an orchestrator would invite exactly the wrong mental model in a new contributor's head before they'd read a single line of its actual code.

"Scheduler" was rejected for being too narrow — it accurately describes the batch-building half of this article (the DAG, the topological sort) but says nothing about the failure-handling, backpressure, research-only isolation, and observability responsibilities that make up the rest of it. A scheduler decides what runs when; this layer also decides what happens when something it scheduled doesn't behave as expected, which is a distinct and, in this codebase, larger body of responsibility than the scheduling decision itself.

"Control plane," borrowed with its meaning intact rather than repurposed, captures the actual scope accurately: this is the part of the system that decides what should happen and enforces that decision's boundaries, while the actual computation (each engine's score() function) is the data plane the control plane governs but does not itself perform. It is, ultimately, the same OS-kernel-adjacent naming instinct the kernel-analogy section made explicit earlier — kernels, in networking hardware and in operating systems both, are usually described in exactly this control-plane/data-plane split, and the platform's own terminology is a direct, intentional continuation of that convention rather than a coincidence.

Registry vs. Control Plane: Where Does a Given Concern Actually Live?

Because this article and the registry article cover closely related ground — both touch dependency graphs, both touch computeCost, both touch the generation counter — new contributors sometimes struggle to remember which file actually owns a given decision. This table is the fast answer.

The rule of thumb that falls out of this table: if the question is "what is true about an engine," the answer lives in the registry; if the question is "what happened when this specific request ran," the answer lives in the control plane. Most confusion about which article covers a given topic resolves immediately once a contributor asks which of those two questions they're actually asking.

Summary

The control plane's job is to take a validated dependency graph from the registry and actually run it: batch engines by dependency depth, run each batch concurrently but under a bounded worker pool, tolerate individual engine failures without discarding the rest of a batch, keep research-only engines fully off the response-latency critical path, publish state to the event bus in a way downstream batches can rely on, and hand the collected results to the governance wrapper. Every mechanism in this article — partial-batch-failure handling, super-linear timeout scaling, the concurrency limiter, per-request latency budgets — exists because the simpler version that preceded it broke in production in a specific, traceable way.

The next article in this series, Confidence Propagation in Multi-Engine Systems, picks up exactly where the partial-batch-failure section left off: given that any engine's output — including a degraded, missing, or low-evidence one — has to be combined into a single confidence figure, what does that combination formula actually look like, and why.

Two smaller, practical takeaways are worth restating outside the bullet list below because they're the ones most likely to transfer directly to a reader's own system regardless of scale: first, reach for Promise.allSettled by default for any batch of independent async operations, not just inside this specific platform — the failure mode Promise.all produces when used this way is silent and easy to miss in code review, exactly as it was here for the better part of two weeks. Second, instrument before you need to — the trace-span mechanism cost relatively little to add and was, by the team's own account, the single highest-leverage change described anywhere in this article, precisely because it converts every future "why is this slow" question from an open-ended investigation into a targeted query. Neither of these two lessons requires 34 engines, a registry, or a governance wrapper to be worth adopting on their own.

It is worth being honest, too, about what this article has not claimed: nowhere above is there an argument that this specific design is the only correct way to build an orchestration layer for a multi-engine AI system, or even necessarily the best one available today. It is the design that fit this platform's specific constraints — in-process function calls cheap enough to make network-hop orchestration wasteful, a request-latency budget tight enough to rule out durable-workflow-style persistence, a compliance requirement strict enough to make every trace span do double duty as an audit record — and every constraint that shaped it is named explicitly somewhere in the sections above, specifically so a reader building something different, under different constraints, can tell which parts of this design actually transfer and which parts were an answer to a problem their own system may not have.

Stepping back from any single mechanism: what makes this layer worth the amount of documentation in this article is that it is the one piece of the platform every request passes through unconditionally, regardless of product adapter, regardless of domain, regardless of which specific 19 or 25 or 34 engines happen to be active for that request. A bug in one engine affects that engine's output. A bug in the control plane affects every request, for every product, simultaneously — which is exactly why its test coverage, its onboarding checklist, and its code-review bar are all set noticeably higher than an individual engine's, and why incidents here (the Promise.all postmortem chief among them) get treated as organization-wide learning moments rather than filed away as one team's bug.

The client whose DPA opened this article has had that restriction enforced without a single exception, across every request, since the day their contract was signed — the control plane simply never schedules emotional-regulation into their execution plan, and there is no code path by which it could. The product launch this article's capacity case study describes was, in plain business terms, the platform's highest-traffic day to date staying fully available for every paying client on it, including several whose own contracts carried financial penalties for platform downtime during that specific week. Neither outcome required a single line of client-specific code; both are the direct, unglamorous payoff of the batching, failure-isolation, and capacity-planning discipline described above. Read end to end, the pattern across every section of this article is the same one the registry article closes on: almost nothing here was designed in advance of a problem. The batch-based execution plan came from a straightforward reading of what the dependency graph already implied about parallelism; Promise.allSettled replaced Promise.all only after fourteen days of silently discarded engine output made the alternative's cost impossible to ignore; the concurrency limiter exists because an unbounded worker pool degraded every concurrent request during a real traffic spike, not a hypothetical one; the per-request latency budget exists because the concurrency limiter's own queueing, once it existed, needed a defined worst case; and the trace-span instrumentation exists because a fourteen-day diagnosis was judged unacceptable and has not recurred since. None of this is presented as a finished, permanent design — the open question about moving engine execution to a separate process pool, discussed in the extended design-rationale section, is a live example of a tradeoff the team is still actively weighing rather than one it has settled. What is settled is the process: measure before building, keep the smallest version that solves the actual observed problem, and treat every constant — a timeout formula's exponent, a concurrency limit, a latency budget — as a number to be re-validated against real data on a standing cadence, not set once and trusted indefinitely. A control plane built by someone else, for a different platform, at a different scale, should probably not copy the specific numbers in this article. It should copy that process.