Your decisions, your model

A model that never writes a sentence

A System One model evaluates a state and returns typed answers and probabilities. It does not write replies, produce code or generate explanations. TypeSafe borrowed the name from Kahneman: System 1 is the fast, instinctive judgment, System 2 the slow deliberation. The pitch is to keep a large language model for open-ended reasoning and generation, and to hand the thousands of small decisions around it — routing, gating, triage, judging — to a model built only for decisions.

The contract is small. You send a state, a string or a JSON document, and a set of named questions. A choice question offers up to 255 options and returns one, with a probability for each. A score question offers two to ten ordered levels. A noul question asks whether a proposition holds and returns the probability that it does. Questions are evaluated in parallel and in isolation, so a request carrying forty questions costs one pass and no question contaminates another.

That is also why it is fast and cheap. Nothing is decoded. The model reads the state once and produces a distribution over the answers you declared, so a decision costs a prefill rather than a generation, and the answer cannot fall outside the schema. The mechanism is not new. Reading a classification off the logits of a language model is how MMLU has been scored since 2020, and encoder classifiers are older still. What is new is the product: questions defined at run time, probabilities that are meant to be believed, and a price that makes it reasonable to ask a model something a hundred times per request.

Five days to reproduce the contract

Jev shipped on a Monday. By Tuesday there was jevlike, a from-scratch option scorer under an MIT licence. By Thursday there were three serious open models: Convai's Laya, a 421-million-parameter ModernBERT with a marker-scoring head, trained with a re-implementation of TypeSafe's calibration objective; Jared Palmer's kev, a Qwen adapter with a pointer head trained on six public datasets rendered as questions; and trycua's CUA-S1-FORMS, a 706,048-parameter byte-level specialist that beat Jev on the one form-filling task it was built for and nothing else. By the weekend an awesome-list counted roughly a hundred projects.

Every one of them, independently, converged on the same architecture: encode the state once, then score each declared answer with a small head. The architecture is not the moat. Two things are: the training that makes the probabilities honest, and the breadth of decision data it was trained on. TypeSafe calls its training RLCD, reinforcement learning for calibrated decisions: the rewards are proper scoring rules against verifiable ground truth, so a model is paid for saying 0.7 only when it is right about seven times in ten. No paper, no weights and no reliability curve have been published for Jev, and that is exactly the gap the open models are racing to close.

What the independent numbers say

Vendor benchmarks are vendor-built, and TypeSafe says so: its four published workflows are labelled by LLM consensus. Third-party tests are more useful, and in the first week there were several with code and data.

Two lessons matter more than any single number. First, the model is strongest on narrow, well-posed questions and weakest on ordinal and emotional ones. Second, it is sensitive to how a question is asked. In one phishing test a single question scored 62.6%; the same task split into five questions and weighted with a logistic regression fitted on a thousand labels scored 95.0%. The decomposition helped Jev and hurt the LLM it was compared with. A System One model rewards question design and a small labelled set, which is another way of saying it rewards an evaluation harness.

Fifty hand-labelled rows cannot test whether 0.85 means 85%.

That is the third lesson, and the one we care about most. Calibration is a property you measure on your own decisions, with enough of them to see whether the confident ones are right more often than the hesitant ones. A third-party measurement put Jev's expected calibration error at 0.144 on one typed-decision set; TypeSafe has not published its own. Neither number tells you what Jev does on your data.

You cannot fine-tune Jev

Search for “fine-tune Jev” and the honest answer is that you cannot. Jev is a hosted model with no published weights, a waitlist and a vendor-run evaluation. You can prompt it, decompose questions for it and fit a logistic layer on top of it. You cannot change what it learned, run it inside your own network, or show an auditor what made a decision. For a routing hint that is fine. For a decision that gates a claim, a transfer or a clinical handoff, it is the wrong side of your boundary.

There are three ways to get a System One model you own, and CID supports all of them on the same contract.

Path 1 on CID today: register, serve, calibrate

CID has spoken the contract since 18 September. Register a model with model_type: probabilistic and CID exposes POST /v1/systemone for it, validated server-side and behind the same authentication, rate-limit, attribute-based access control and tenant chain as every other route. A chat turn on a probabilistic model becomes a typed decision, and the reply carries the decision next to the text. Every decision is a platform event, attributed to a caller and admitted by a policy.

POST /v1/systemone
{
  "model": "claims-triage",
  "state": { "claim_id": "C-4471", "amount": 18400, "notes": "…" },
  "questions": {
    "route":  { "type": "choice", "description": "Who should handle this claim?",
                "options": [ { "id": "auto_approve" }, { "id": "adjuster" }, { "id": "fraud_review" } ] },
    "urgent": { "type": "noul", "description": "Does this claim need a response today?" }
  }
}

{
  "decisions": {
    "route":  { "type": "choice", "value": "adjuster", "confidence": 0.81,
                "probabilities": { "auto_approve": 0.11, "adjuster": 0.81, "fraud_review": 0.08 } },
    "urgent": { "type": "noul", "value": false, "confidence": 0.77,
                "probabilities": { "true": 0.23, "false": 0.77 } }
  },
  "backend": "cid/model-studio-system-one/s1-v1"
}

The serving side is a scoring backend. Ours today is a llama.cpp fork that prefills the state once, evaluates every question against the cached prefix and reads a distribution off the answer slots, with thinking disabled so the first token is the answer. On a Qwen3.5-4B base it agrees with the OpenJev reference on 94.1% of its fixture. It is not calibrated, and it says so. Calibration is the afternoon's second task: score a thousand of your recorded decisions, fit a temperature per question shape, and you have a baseline every trained candidate has to beat.

Paths 2 and 3: Model Studio trains it

Model Studio is CID's training pipeline: data_prep -> train -> aggregate -> eval -> benchmark -> validate -> deploy, with signed artifacts, retained evaluation sets and readiness checks at every handoff. As of this week it has a decision-model recipe. Set training.method to system_one and the same pipeline trains, evaluates and publishes a System One candidate instead of a text adapter.

The data is your decisions. Each row carries a state, the evidence the decision saw; a question in the same shape the API accepts; and a gold answer, which may be an option, a yes or no, a probability, or a full distribution when the right answer is genuinely uncertain. Rows that share evidence and a question never straddle a split, so the held-out set is really held out.

Training runs in three stages. A supervised stage teaches a small scoring head, attached to a low-rank adapter on the base model, to put probability on the recorded answer. A reinforcement stage then does what TypeSafe describes: it samples perturbed versions of each decision, scores them with proper scoring rules — the logarithmic and spherical scores, plus a ranked score for ordered levels — and pushes the model toward the distributions that would have been rewarded, with the supervised loss kept in the objective so the reward cannot be gamed. A calibration stage fits one temperature per question type and answer count on the validation split, and records which buckets were fitted and which were left alone because the data could not support a fit.

"training": {
  "method": "system_one",
  "lora_rank": 16, "learning_rate": 0.0002, "epochs": 3,
  "system_one": {
    "head": "pointer", "head_hidden_size": 256, "head_learning_rate": 0.001,
    "rlcd_epochs": 1, "group_size": 8,
    "sigma_start": 0.4, "sigma_end": 0.1,
    "spherical_weight": 0.75, "rps_weight": 1, "ce_weight": 1
  }
},
"compute":    { "profile": "s1-v1" },
"evaluation": { "min_accuracy": 0.85, "max_ece": 0.05, "max_latency_ms": 100 }

Evaluation is where a decision model earns its name. The held-out set is scored for accuracy, for the loss of its probabilities, for expected calibration error and Brier score, and for coverage: the largest share of decisions the model can take on its own while keeping selective error under five percent. Two more numbers guard against fooling ourselves. The loss is compared with the untrained readout baseline from Path 1, so a candidate has to beat what you could have had without training. And every question is scored a second time against another row's evidence; if accuracy barely drops, the model is reading the question and not the state.

A candidate that passes is published to the registry as a probabilistic model, tagged for decision dispatch, signed, and carrying its scoring head and calibration sidecar next to the adapter. A serving script loads it behind the same /v1/systemone route, and refuses to load a candidate whose recorded trainer hash differs from the one retained with the run.

The decision stays inside your boundary

Hosted decision APIs put the decision on the wrong side of the line. The state you send is the most sensitive thing your application holds, because it is the thing the decision is about: the claim, the message, the account, the record. On CID the model runs where your data already is, the caller is authenticated and authorised by attribute policy before the question is asked, the tenant boundary holds, and the decision is written to the audit trail with the policy that admitted it and the version of the model that made it. When a regulator asks why a transfer was flagged, the answer is a record, not a support ticket.

The same boundary makes improvement safe. Outcomes become labels. Temperatures are refitted on a schedule. A retrained candidate has to pass the same gates as the first one, against a baseline that is now yesterday's model.

What we will publish next

We have said nothing here about how good a CID-trained System One model is, because we do not yet have a number worth printing. The recipe has been verified end to end on a small fixture, which proves the pipeline and nothing about quality. The next post will carry accuracy, calibration error and coverage on real decision families, measured the way we would ask a vendor to measure, and the public jev-benchmarks sets scored the same way, so the comparison is a fine-tuned model against a zero-shot API and labelled as such.

Questions we keep getting