Routing Is Latency. Every Minute You Sit On a Lead, the Odds Decay.

A lead came in at 2:14pm and got assigned twice. Two reps called the same buyer within a minute of each other, the buyer got annoyed, and the deal never happened. Nobody wrote a bug that assigns leads twice. The form fired a webhook, the webhook timed out, the retry fired, and the second delivery hit before the first one finished writing the owner. Two deliveries, two assignments, one confused buyer. Most teams file that under “routing is messy” and move on. The old way of thinking about routing is that it is a fairness problem: whose turn is it, distribute evenly, round-robin whenever someone gets to it.

Here is the break. Routing is not a fairness problem, it is a latency problem, and latency is measured against a curve that punishes you by the minute. The 2:14pm double-call cost you nothing on a fairness ledger, both reps got a lead. It cost you everything on the latency curve, because while the two of them were untangling who owned the buyer, the buyer’s intent was decaying in real time. Every reliability failure in your routing path (a duplicate delivery, a dropped lead, a retry storm, a mid-chain crash) shows up as the same thing on the buyer’s side: minutes of silence. Watch what those minutes do to your odds.

The decay curve is why routing reliability is a revenue problem and not a hygiene problem. A system that double-assigns, drops, or stalls a lead does not just annoy a rep. It parks a hot buyer on the flat part of that curve, where the odds are already gone. So the build has two jobs that are the same job: get to first touch fast, and never let a failure between the form and the rep quietly add minutes. To do the second one you have to stop treating routing like a spreadsheet formula and start treating it like the thing it actually is.

Routing is a distributed system, and every hop is a failure domain

A lead crosses a web form, a message queue, a CRM, an enrichment API, a scoring service, and a Slack notifier before a rep ever sees it. Each of those hops can be up while the next is down, can succeed while the caller thinks it failed, and can be retried by something upstream that never learned the first attempt worked. That is the textbook definition of a distributed system, and it has been one the whole time.

The naive design treats this as one synchronous chain: the form calls enrichment, enrichment calls scoring, scoring writes the owner, all in a single request. When any link is slow, the whole chain blocks, and every second it blocks is a second on the decay curve. When the caller times out, it retries the whole chain and re-runs the writes that already succeeded, which is exactly how you get the 2:14pm double-assign. Synchronous chains do not fail gracefully. They fail by duplicating and by dropping, the two failure modes that either double-call a buyer or strand one in silence.

The queue is the seam that owns the failures

The fix is to stop routing inline and route through a durable queue. The form does one thing: write a routing job and return fast. A separate worker drains the queue, does the enrichment and scoring and assignment, and marks the job done. That seam between “event arrived” and “work happened” is where every reliability property lives: dedup, retry, ordering, and a dead-letter path for jobs that will never succeed. Returning fast also helps the latency curve directly, because the buyer-facing acknowledgment happens in milliseconds while the slow work drains behind it.

The artifact: a routing job table

The queue is a table. On Salesforce I build it as a custom object; on a warehouse stack it is a table with the same columns. The columns are what give you every guarantee, so here is the schema, not a description of it.

CREATE TABLE routing_job (
  id              BIGINT PRIMARY KEY,
  idempotency_key TEXT NOT NULL UNIQUE,   -- dedup: same key = same job, ever
  lead_id         TEXT NOT NULL,
  payload         JSONB NOT NULL,          -- the event as received
  status          TEXT NOT NULL DEFAULT 'queued', -- queued|working|done|dead
  attempts        INT  NOT NULL DEFAULT 0,
  max_attempts    INT  NOT NULL DEFAULT 5,
  next_run_at     TIMESTAMPTZ NOT NULL DEFAULT now(), -- backoff schedule
  locked_by       TEXT,                    -- worker id holding this job
  locked_at       TIMESTAMPTZ,
  ordering_key    TEXT,                     -- per-account FIFO if needed
  created_at      TIMESTAMPTZ NOT NULL DEFAULT now(),
  completed_at    TIMESTAMPTZ
);
CREATE UNIQUE INDEX ON routing_job (idempotency_key);

The idempotency_key column carries the whole dedup guarantee. Derive it from something stable in the event, not from a random ID the sender generates fresh on each retry. For a form fill I use a hash of email plus form ID plus a coarse time bucket, so two deliveries of the same submission collapse to the same key and the second insert simply violates the unique constraint. The double-assign that opened this piece cannot happen, because the second job was never created.

The worker claims work with an atomic lock so two workers never grab the same job:

UPDATE routing_job
SET status = 'working', locked_by = :worker_id, locked_at = now(), attempts = attempts + 1
WHERE id = (
  SELECT id FROM routing_job
  WHERE status = 'queued' AND next_run_at <= now()
  ORDER BY created_at
  FOR UPDATE SKIP LOCKED           -- two workers never claim the same row
  LIMIT 1
)
RETURNING *;

FOR UPDATE SKIP LOCKED is the line that lets you run ten workers in parallel with zero coordination and zero double-processing. Each worker skips rows another worker already locked. Ten workers draining in parallel is also how you keep median first-touch on the left side of the decay curve under a burst of inbound: the queue absorbs the spike, the workers fan out, and no lead waits on a single-threaded chain.

Idempotency at the write, not just the enqueue

Deduping the job is half the battle. The write to the CRM has to be idempotent too, because a worker can crash after it assigns the owner but before it marks the job done, and the retry will re-assign. Make the assignment a conditional write: only set the owner if it is still unassigned, or record the assignment decision keyed by the same idempotency key so replaying it is a no-op. On Salesforce that looks like a guarded update.

// Idempotent assignment: the worker can replay this safely.
Lead ld = [SELECT Id, OwnerId, Routing_Idem_Key__c FROM Lead WHERE Id = :leadId FOR UPDATE];
if (ld.Routing_Idem_Key__c == idemKey) {
    return; // already routed by this exact job; replay is a no-op
}
if (isUnassignedOrQueue(ld.OwnerId)) {
    ld.OwnerId = chosenOwnerId;
    ld.Routing_Idem_Key__c = idemKey;   // stamp so a replay short-circuits
    update ld;
}

Stamping the lead with the key that assigned it means a replay reads the stamp and returns. The buyer gets one call, from one rep, no matter how many times the queue redelivers. This is the same idempotency discipline that keeps an automation from running twice and double-charging you; routing is the highest-stakes place to apply it, because the duplicate is not a wasted credit, it is a lost deal.

Retry with backoff, then dead-letter

Not every failure should retry the same way, and no failure should retry forever. An enrichment API 500 is transient: retry it with exponential backoff. A malformed payload is permanent: retrying it a thousand times just burns the queue and, worse, keeps a routable lead behind it waiting. The attempts and next_run_at columns encode the policy.

When attempts hits max_attempts, the job moves to dead, and a human reviews the dead-letter queue. The critical design choice: a poison job lands in a review queue, never in a rep’s lap and never dropped silently. A lead that could not be routed is a lead someone has to look at, not a lead that vanished onto the flat part of the decay curve.

What ordering buys you, and when to pay for it

Sometimes order matters. Two events for the same account arriving out of order can assign the account to the wrong owner if the later event lost the race. The ordering_key column lets you enforce per-account FIFO: the worker will not start a job whose ordering key has an earlier unfinished job. You pay for this with throughput, and throughput is latency, so only turn it on for keys where order changes the outcome. Most lead events are independent and do not need it. Account-merge and owner-change events do.

A worked example that reconciles to the curve

Take 1,000 inbound demo requests a month and run them down both architectures. The synchronous chain assigns inline, blocks on every slow hop, and re-runs on every timeout. The queued system enqueues in milliseconds and drains with ten parallel workers. The qualify-odds column reads directly off the decay curve at the top of this piece, at whatever minute each path actually reaches first touch.

The synchronous chain reaches a median buyer at 47 minutes, deep into the cliff where the index has fallen to about 15, and it double-assigns or drops roughly 76 leads a month that never get a clean first touch at all. The queued system reaches the median buyer at 6 minutes, near the 95 mark on the curve, and its duplicate rate is rounding error. Same 1,000 leads, same reps, same enrichment. The only thing that changed is that one architecture keeps leads on the left of the decay curve and the other lets them slide down it while a retry storm sorts itself out. The 240-versus-38 meeting gap is the decay curve priced in dollars.

Where the duplicates actually come from

Before you build any of this, measure it. Most teams assume their routing is clean because nobody filed a ticket, but the double-assigns are quietly happening and the buyers are quietly leaving. This is roughly the breakdown I find when I audit a synchronous routing path.

Synchronous chain vs queued system

Same six services. Two architectures. One survives production, and it is the one that keeps leads off the flat part of the curve.

The build order: ship it in five steps

Five steps, and you can ship them incrementally. The queue and idempotency key alone kill the majority of the failures and the latency they add, so build those first and add the rest as the dead-letter queue fills. The order is not arbitrary: it is sorted by how much latency each failure injects into the curve, worst first.

The latency argument on the operations side makes the case that routing has to be fast. This is the other half: it has to be reliable while it is fast, because a fast system that double-assigns or drops leads just reaches the wrong outcome sooner, and the buyer still slides down the curve. Treat routing like the distributed system it is, borrow the queue, the idempotency key, and the dead-letter path from the people who solved this a decade ago, and the 2:14pm double-call stops happening. Start this week with one table and a unique index on the idempotency key, then let the dead-letter queue tell you what to build next. The decay curve is not waiting for your retries to settle.