Fan-out and consensus

Press ⌘⏎ in Polycode and the prompt does not go to one model. It goes to every model you’ve enabled from every un-muted provider, in parallel, at the same instant. That shape is the whole point of the app — fan-out gives you a comparison surface in the place where a single-vendor chat client gives you one opinion. This article walks through how that runs end-to-end and what each piece is actually doing.

What fan-out means

When you submit, the consensus engine inside Polycode dispatches one request per configured peer: a Claude call, a GPT call, a Gemini call, an Apple Intelligence call, an Ollama call, and so on for whatever set you built in Settings → Providers — every model you’ve enabled, minus any provider you’ve Muted on its Connections row. The peers run concurrently. None of them blocks on the others. The fastest reply lands first; the slowest closes the round when it finishes or hits its timeout.

The dispatch is per-peer, not per-provider — if you’ve configured two models from the same vendor (say, Claude Sonnet and Claude Opus), they each get their own dispatch slot with their own pricing and their own latency budget. Same prompt, different model, separate row in the trace.

Why parallel inference, not routing

A common alternative shape is to route: pick one provider per prompt based on classifier heuristics, send to that one. Routing optimizes for the average case; it hides the cases where the routing decision was wrong.

Fan-out optimizes for the times you actually wanted a second opinion. Three peers strongly agreeing on a fact is a signal you didn’t have when one peer said it. Three peers diverging on an interpretation is a signal you definitely didn’t have. The cost is more tokens per turn; the payoff is calibration — you know which answers to trust and which to dig into.

Per-peer streaming

Each peer streams its response on its own track. Above the reply the chat surface renders a flat FAN-OUT pill row — one pill per dispatched peer, each carrying a live status while the round runs, and a phase caption on the end of the row (fanning out…, then synthesizing…). As the fan-out round closes, each pill settles into its success or failure state — a status-toned track plus a small pip — while the primary is still writing the synthesis. When the whole exchange finalizes, the row’s label becomes CONSENSUS OF and the per-peer latency (e.g. 860ms) surfaces on the SOURCES pills beneath the reply, one for each peer the synthesis cited.

The per-peer timer records two values: first-token latency (when did this peer start saying anything) and total latency (when did it finish). The consensus engine stores both — first-token in a lock-protected map keyed by peer, total at finalize time — and the inspector reads them back later. First-token is the live-feel signal; total is the one you compare across peers in a trace.

Synthesis: one primary peer writes the reply

When the slowest peer either finishes or times out, the engine ends the fan-out round and hands the full set of responses to your primary peer — the model you trust most, set in Settings → Providers and overridable on a given send via the per-window model pick or a Mode. That peer reads every other peer’s reply, then composes a single synthesized answer that either agrees with the consensus, surfaces a dissent, or chooses between competing options.

The synthesis prompt instructs the primary peer to cite a peer only when that peer’s contribution is load-bearing — a fact only one model surfaced, a disagreement worth flagging, a peer that was unavailable. Universal-agreement sentences carry no citation; that’s the cleanest signal that the peers all said the same thing.

When the panel splits

Synthesis is not averaging. If one peer says one thing and three peers say another, the synthesizer is instructed to surface the disagreement, not paper over it — “surface disagreements; do not average them away” is in the prompt.

That disagreement is then addressable, not just visible. The answer carries a Split on row whose chips name each contested claim; the inspector grows a Where the panel split section with a card per claim, the providers on each side, and each side’s closest passage from its own reply; any provider’s full response can be diffed against the synthesized answer sentence by sentence; and you can adopt a provider’s reply as the displayed answer, reversibly.

All of that lives in disagreement and adoption.

Cost: roughly N× a single model, plus a synthesis pass

Every peer is a full inference call, billed by that provider at their listed price. A multi-peer fan-out costs roughly N× one model, where N is the number of peers — plus one synthesis pass billed at your primary model’s rate, since the primary runs a separate turn to compose the final reply (free if your primary is Apple Intelligence or a local endpoint). The math varies because pricing is per-token-per-model and response lengths differ. Each row of the inspector trace shows the exact tokens in / out and dollars for that peer — and the synthesis lane is metered too — so the per-turn spend is never a mystery.

A debate round roughly doubles it

A mode with Rounds set to Debate runs the fan-out twice: every peer sees the other peers’ first drafts and revises once before synthesis. Both rounds are real inference calls, so a debate exchange costs roughly 2× the fan-out plus the one synthesis pass, and every round is counted in the inspector’s aggregate Σ and in Settings → Usage. The mode editor states the multiplier where you turn it on, and its “This mode runs” card names the round setting before you save.

One consequence is worth knowing before you enable it. In a debate round each peer is shown the other peers’ drafts — so provider A’s text is sent to provider B. Outside a debate round that never happens: only the primary sees other peers’ replies, because only the primary synthesizes. The drafts are labelled Draft A, Draft B, … and Polycode attaches no provider names to them, but the text itself does cross between providers. If that matters for what you’re asking about, leave Rounds on Single pass.

Two practical mitigations sit inside the app. Apple Intelligence on-device is free and counts as a fan-out peer — the inspector marks it Free — so every fan-out with on-device enabled gets one free vote. Local OpenAI-compatible endpoints (Ollama, LM Studio, an internal gateway) carry no per-token cost either; they show as unpriced (—) in the trace rather than a labeled Free. The unpriced peers make the comparison surface workable for everyday questions; the paid peers stay in for the questions that warrant them.

Tracking spend over time

Per-turn cost lives in the inspector, but Settings → Usage rolls it up over a date range: a running total, broken down by provider and by project, plus a top-sessions list — all computed locally from the exchange-time costs already stored with each response. A Reset baseline control rebases the dashboard’s start date without deleting any history. See data lifecycle § Usage for exactly what’s counted.

How agreement is measured

Polycode reports agreement by reading the synthesized answer’s own citations.

That works because the synthesis prompt already constrains where citations go: a peer is cited only when its contribution is load-bearing, universal-agreement sentences carry no citation at all, and a divergence must be attributed to each side. So the citations in the finished answer are the primary’s own account of what the panel did — and they can be counted.

Every unit of the answer — a sentence, or a list item, heading, or table row — is classified by the citation clusters inside it:

A side is the set of peers named by one run of adjacent citations. Two peers cited together are one side, not two — that’s agreement, not a split. Two separate citation runs in one sentence are two sides, which is what a disagreement looks like when the primary writes it down: “take it off the merge gate [Anthropic] [Apple Intelligence] — or keep it gating and wrap it in a bounded retry [OpenAI].”

A unit whose every cited peer failed is a status note — “Gemini timed out before returning, so its position is not represented here” — and it is left out of the percentages entirely, so a provider failing can never inflate the agreement figure. Attendance is reported separately, on its own caption line.

What this does and doesn’t measure

It is the primary’s account, counted — not an independent semantic comparison of the replies. If the primary miscites, the classification inherits that. Polycode prefers a measurement you can audit (open the answer, look at the citations, count them yourself) over one that is more sophisticated and impossible to check.

It is also per claim, not per provider. A provider is not scored, ranked, or weighted. The unit of measurement is the sentence, and the question is whether the panel diverged on it.

The inspector renders these shares as a three-segment bar with a legend; see the inspector for the card, and disagreement and adoption for reading and acting on a split.

Tools live alongside fan-out

A peer can decide to call a tool — read a file, run a query, hit an MCP server — instead of (or before) producing its final response. The tool-call step happens inside the peer’s run, the result feeds back into that peer’s context, and the peer continues. Which tools a peer sees follows an audience rule. Native file tools split by what they do: reads reach every model answering in parallel, while Write, Edit, Delete, and Rename are reserved for the primary. MCP tools split by server instead — a server set to Offer tools to every model gives peers all of its tools, including any that write; one left primary-only gives them none. Within that set, different peers may still make different tool calls on the same prompt; the inspector shows which peer called what.

Tool spend is budgeted per answer, across all models. With the default Standard budget, a multi-model answer checks in with a single banner after 75 tool calls in total and stops at 250; the Focused and Extended presets move both numbers. Choosing Stop and summarize at the check-in ends every model’s tool use. The approval model is covered in tools and approvals.

What’s next

• The inspector — open the per-provider trace, read what each peer actually said and what it cost.
• Disagreement and adoption — read where the panel split, compare a peer against the answer, and adopt a peer’s reply.
• Setting up providers — the peers strip, the per-connection Mute control, and the per-model enablement that decide which peers participate in fan-out.
• Tools and approvals — connect MCP servers and manage the approvals that peers ask for during a run.