Federated learning offers a different deal: the data never moves. Instead of pulling records into a central store, you push the training to where the data lives — hospitals, banks, devices, branch offices — and only model updates come back. A coordinator aggregates those updates into a shared model.
Both are legitimate engineering choices. Neither is free. Here’s the honest version of each.
Centralized learning: the incumbent.
- Throughput and efficiency. Data locality is solved by construction. You can saturate GPUs, shard however you want, and use a decade of mature tooling.
- Simplicity. One cluster, one data store, one training loop. Debugging a loss spike means reading one set of logs, not forty.
- Predictable quality. You can see your training data. You can clean it, balance it, dedupe it, and hold out a test set you actually trust.
- The data has to move. That is the entire problem. For a growing class of data — patient records, financial transactions, defense telemetry, anything regulated or contractually siloed — “just centralize it” is a non-starter. Legal says no, customers say no, economics (data gravity, egress cost) sometimes says no.
- Centralized data is centralized risk. One breach or one insider, and every contributor’s data is exposed at once. Aggregation creates a target.
- Some datasets effectively cannot exist. Competing banks almost never pool raw transaction streams with each other. The exceptions — credit bureaus, payment utilities — were built by the industry itself out of shared losses rather than by regulatory mandate, and the oldest of them took a century to reach national coverage. Confidential computing and data clean rooms nibble at the edges, but for practical purposes, the centralized version of that dataset doesn’t exist.
Centralized training wins whenever the data can move. The question is what you do when it can’t.
Federated learning: the honest version.
- The data never moves. Training goes to the data; only model updates come back. Datasets that no one would ever pool become usable.
- Data residency is satisfied structurally, not contractually. Raw records stay behind each owner’s firewall and inside each jurisdiction — a necessary condition for a growing list of regulated workloads, though not a sufficient one: the updates that do cross borders can attract their own transfer rules, and privacy is not automatic.
- No central pile of raw records. The training data never concentrates in one breachable store. The updates that do move, and the model itself, are a smaller — but nonzero — exposure.
And now the part most FL marketing skips: federated learning, as practiced today, mostly doesn’t work well.
- Non-IID data is brutal. Real silos are not random samples of one distribution. Hospital A’s patients are not Hospital B’s. Naive federated averaging on heterogeneous data converges slowly, to a worse model, or — with aggressive local training — diverges outright. The mitigations (proximal terms, control variates, personalization layers) help, but they don’t erase the gap — they narrow it, at the cost of extra per-round state and communication, more hyperparameters, or per-client complexity.
- Communication is the bottleneck. Every round ships a full model update per participating client — hundreds of megabytes to gigabytes each way for large models, over real networks (smaller for adapter-style updates, but never free). Compression and partial-participation schemes trade this cost against yet more convergence quality.
- Privacy is not automatic. FL moves the data problem, it doesn’t delete it. Gradients leak information — membership inference and gradient-inversion attacks, which can reconstruct training examples from shared updates, are published, practical results. If you want an actual privacy guarantee you need differential privacy or secure aggregation on top, and both cost you accuracy, engineering, or both.
- Someone has to trust the aggregator. Vanilla federated averaging will happily average in a poisoned or broken contribution — a single malicious client is enough. Byzantine-robust aggregation exists, but almost nothing ships it by default.
- The systems work is the real work. Heterogeneous nodes that drop mid-round, version skew across clients, orchestrating rounds across organizations with different security postures — the research frameworks give you an aggregation function and reference orchestration; the production tail — your nodes, your consents, your auditors — is still on you.
This is why few cross-organization federations have survived past pilot into sustained production. Cross-device FL inside one company works — Google’s Gboard is the canonical example — because one owner controls both ends of the wire. Between organizations, the successes (a handful of pharma and hospital consortia) are real but rare — and they didn’t compound. The largest, a 71-site tumor-segmentation federation, was never reconstituted; its follow-up round ran as a single-machine benchmark. A ten-company pharma consortium finished its funded term and stopped, reporting modest gains against non-negligible cost. The idea is sound. The tooling, until recently, was research code.
So why keep pushing on it? Because the demand side never went away. The data that can’t move is also the data that’s most valuable to train on — cross-hospital diagnostics, cross-border pharma research, cross-agency intelligence. Every year the regulatory wall gets higher and the models get hungrier. Whoever makes FL boring — reliable, gated, observable, deployable — unlocks datasets no centralized pipeline can reach.
What “FL at scale” actually requires.
Getting federated learning from pilot to production is no longer only an algorithms problem. It’s a platform problem:
- A real pipeline. Training rounds as a DAG with quality gates — gradient health checks, capability-regression gates, held-out accuracy thresholds — that fail closed: a bad round blocks promotion instead of silently degrading the model, and anything already deployed rolls back automatically when its serving metrics breach. Not a script; a system with retries, resume-from-failure, and lineage. (The honest follow-up — held out by whom? In practice the consortium has to designate or jointly construct a trusted evaluation set; a platform can enforce the gate, but it can’t conjure the data.)
- Robust aggregation as a first-class option. Trimmed mean and Krum as selectable aggregation strategies, with a FoolsGold-style similarity detector running alongside in the consensus layer, aimed at sybil collusion rather than Byzantine faults — chosen per trust model, with differential privacy as a configurable layer, not an afterthought. One honest caveat: robustness and heterogeneity pull against each other — under heavily non-IID data, an honest minority silo can look like an outlier, so these knobs need per-consortium tuning, not blind defaults.
- Verification without disclosure. In an open contributor network you need to score every contribution and reward or penalize on that basis — but the strongest contributors are the least willing to hand over raw gradients so you can check. The resolution is cryptographic: zero-knowledge proofs that a contribution matched its commitment and was scored correctly, verifiable by any party in the network, revealing nothing beyond the score. (To be precise about who sees what: the coordinating aggregator still observes updates when running robust strategies like trimmed mean or Krum — the proofs are what let everyone else verify without seeing them.)
- A way out. No rational organization joins a training consortium it can never leave. When a data subject’s deletion right reaches data a member contributed — or a member contractually exits the consortium — “remove that influence from the model” has to mean something verifiable, and increasingly something provable.
- Observability. Per-round metrics, contribution scores, enforcement events, drift between silos. If you can’t see it, you can’t run it.
- A deployment path. Versioned, signed, content-addressed models with a promotion path from staging to production — the discipline software CI/CD converged on.
That is the bar CID was built to clear. CID runs training as a seven-stage pipeline — data_prep, train, aggregate, eval, benchmark, validate, deploy — with quality gates that fail closed, Byzantine-robust aggregation, differential privacy with per-round budget accounting, automatic rollback for deployed models on error-rate or latency breach, a signed model registry with full lineage, and an observability stack out of the box.
Proofs, not promises. CID can prove contribution scores, not just assert them. Its zkPoC circuits produce Groth16 zero-knowledge proofs that a submitted gradient matched its commitment and that its score was computed correctly — a verifier checks the proof in milliseconds and never sees the gradient. In the spirit of this post, the honest cost sheet: proving takes seconds per contribution plus a per-circuit trusted setup — one explicit trust assumption in the stack — while verification is milliseconds. The asymmetry is the point. And lineage compounds it: every round’s proof is folded into a tamper-evident hash chain, so the entire training history is auditable end to end — with succinct, single-proof verification of the whole history on the roadmap.
That machinery is also what makes leaving possible. CID tracks each contributor’s updates per round, SISA-style. When a removal is required, the platform subtracts that contributor’s historical influence from the deployed model — by direct gradient subtraction, Fisher approximation, or partial shard retrain, depending on the cost-precision tradeoff — and registers the new model version with a zkForget proof.
What the proof attests, precisely, is that the removal procedure was executed correctly; like all unlearning short of full retraining, it is a verifiable procedure, not a guarantee that no trace remains — and whether approximate unlearning discharges an erasure right is a legal question no vendor should answer for you. What CID provides is the engineering foundation: tracked influence, an executable removal path, and cryptographic evidence it ran. That’s what makes consortium formation practical at all — organizations join networks they can verifiably exit.
Same pipeline, two modes.
Here’s the part that matters for this comparison: CID is not federated-only. The same pipeline that coordinates a cross-organization federated run also runs an ordinary centralized training job on your own cluster — centralized is the default; federated is a mode you opt into per job.
| If… | Run on CID |
|---|---|
| Data can move | Centralized job — same gates, same registry |
| Data can’t move | Federated rounds — same gates, same registry |
Same quality gates, same deployment path, same audit trail. The choice becomes an architecture decision per workload, not a bet on a paradigm.
Federated learning’s rough present is an engineering gap, not a law of nature.
When it closes — and it is closing — FL will look less like a research framework and more like CI/CD. That’s the platform we’re building.