We trained one in minutes. Here is how to keep it learning.

1. Why train your own decision model

The Jev launch and its first week of community reproductions showed one family of use: moderation queues, LLM-judge replacement, review-queue clearing, intent routing, tool-call gating for agents. They share a shape: a human or an LLM was already making a yes/no or pick-one call, and the System One model makes it faster and cheaper. Useful, and also the easy half of the problem.

The more interesting uses are the ones where nobody was making the call at all, because the only options were a hand-written rule or a full LLM round-trip that was too slow or too expensive to put in the path. A typed answer with a probability you can trust changes what is worth deciding, and a model trained on your own decisions is what makes the probability trustworthy.

Financial crime first, because it is where calibrated decisions are already the product. A risk team does not want a label; it wants a probability it can put a threshold on, at the natural fraud rate, with a coverage number that says how much of the queue can be decided without a person:

Beyond money, ten more places we keep seeing, none of them in the launch material:

Questions are defined at run time and several share one pass, so the deploy gate and the alert triage can be new questions against the same model, five of them for the price of one. What makes them safe has to be measured on your data: is 0.85 right 85% of the time, and how much of the queue can the model take on its own under an error budget. That is why the rest of this post is about training on your own decisions, and keeping the model trained.

2. Fine-tuning a System One model on CID

Four steps, each one something Model Studio already does for text adapters, plus one field that turns it into a decision trainer.

Step 1. Shape the data

A training row is the call you will make in production, plus the answer: {state, question, gold, family}. The state is the evidence, JSON or text; the question is a typed question (noul, choice or score) with its options; gold may be an option id, a boolean, a probability or a full distribution, so you can train toward the right amount of confidence rather than a hard label; family groups questions for reporting. Any labelled set becomes decisions in one command: to_system_one.py reads JSONL, CSV or Parquet. A real row from the dino-game model in §4.5:

{"state": "Dino runner game. Small cactus ahead, 364 px away.",
 "question": {"type": "choice", "description": "Choose the best safe action for the dinosaur.",
   "options": [{"id": "jump", "description": "jump: Safe. Jumps too early."},
               {"id": "duck", "description": "duck: Safe. Crouches; no benefit."},
               {"id": "run", "description": "run: Safe. Waits; acts later. Best."}]},
 "gold": "run", "family": "dino"}

Step 2. Write the recipe

Set training.method: system_one and add a system_one block; the rest is the recipe teams already use for text adapters. The dino recipe, trimmed to what matters:

{"model": {"name": "laya-cid-base", "version": 1},
 "dataset": {"id": "dino-labeled-s1", "version": 1, "format": "system_one"},
 "training": {"method": "system_one", "lora_rank": 16, "target_modules": ["Wqkv", "Wo"],
              "epochs": 1, "max_seq_length": 128,
              "system_one": {"head": "laya", "rlcd_epochs": 1}},
 "evaluation": {"min_accuracy": 0.5, "max_ece": 0.1, "max_regression": 0.1, "max_latency_ms": 2000}}

Bring your own base, from disk: POST /v1/model-studio/bases registers a directory inside the boundary as a base model. That is how Laya went in, with no Hub download.

Step 3. Run it

One Model Studio job runs data_prep → train → aggregate → eval → benchmark → validate → deploy, and every stage writes a receipt verified against the recipe and dataset hashes. The eval stage scores the candidate, the untrained base and a shuffled-evidence control on held-out rows, so you always see what training added and whether the model reads the evidence at all; the benchmark stage times it; the validate stage fails the run when a candidate misses its accuracy or calibration gate, so it never reaches deploy. What it cost on our runs:

Everything above ran on one desktop-class GPU, a DGX Spark, unless the row says otherwise. Not simplified, because it should not be: dependencies and the CUDA build are pinned and reviewed, and a candidate that misses its gate never reaches deploy.

Step 4. Serve it

Serving is the same route. Start the candidate's server (serve_system_one.py) and register it as a serving model's backend; CID then routes /v1/systemone calls, and typed questions in chat, to it by model name, and the serving shim refuses a candidate whose trainer hash differs from the one retained with the run. Several questions share one pass: 1, 4, 8 and 16 questions per request cost a p50 of 38, 91, 165 and 297 ms through CID, so the price per question falls from 38 to 19 ms. And you can score any endpoint the way the eval stage does, CID's or a vendor's: system_one_bench.py reports accuracy, cross-entropy, ECE, Brier, coverage and latency against any /v1/systemone URL.

3. Keeping it good: a fine-tuning loop

A decision model is trained on yesterday's decisions, and fraud patterns, intents and catalogues move. The answer is a loop, and every step of it is a CID API call. What CID does not do yet is run the loop for you, so here is which part is the platform and which part you script.

# nightly: retrain on the outcomes that came in, ship only if it beats what is serving
rows = join(decision_log, outcomes, on="request_id")          # your log, your labels
write_jsonl("rows.jsonl", rows)                                # {state, question, gold, family}

ref = POST("/v1/training-data/upload", file="rows.jsonl")["upload_ref"]
ver = POST(f"/v1/datasets/{DS}/versions", {"content_hash": sha256("rows.jsonl"), "row_count": len(rows)})["version"]
POST(f"/v1/model-studio/datasets/{DS}/versions/{ver}/bind", {"upload_ref": ref})

recipe["dataset"]["version"] = ver
rev = PATCH(f"/v1/model-studio/recipes/{RID}", {"config": recipe, "expected_revision": rev})["revision"]
pre = POST(f"/v1/model-studio/recipes/{RID}/preflight", {"expected_revision": rev})
job = POST(f"/v1/model-studio/recipes/{RID}/runs", {"expected_revision": rev,
           "preflight_digest": pre["digest"], "idempotency_key": uuid4()})["job_id"]

if wait(job) != "complete":             # a missed gate fails the run before deploy
    alert("candidate failed its gates"); exit()
if bench(candidate(job)) <= bench(SERVING):   # system_one_bench.py, same fixed holdout
    exit()                                    # keep what is serving
url = start_server(job)                  # serve_system_one.py --artifact <run>/adapter
POST("/v1/models", {"name": SERVING, "model_type": "probabilistic", "probabilistic_backend": url, ...})

This is the loop Tetris went through in §4.6. The first candidate failed its gate at 30% agreement and never reached deploy; a new dataset design, not a bigger model, produced the one that plays like its planner. The missing pieces are the ones you would expect: a decision id to join outcomes to, an outcome endpoint, scheduled recipe runs and gated promotion. Until they exist, the loop is about twenty lines around the API, and we would rather tell you that than draw a diagram that implies a button.

4. What it buys: the evidence

The previous post ended with a promise: accuracy, calibration error and coverage on real decision families, measured the way we would ask a vendor to measure, and the public sets scored the same way, a fine-tuned model against a zero-shot API and labelled as such. Here it is, condensed; every table, protocol and significance test is in the benchmark report.

4.1 Four fraud decision families

We fine-tuned Laya (Convai's open System One model, Apache-2.0, a ModernBERT-large encoder of 395 M parameters with a typed decision head) on 12,000 decisions over 6,000 user sessions: is this session fraud (noul), how to route it (approve, step-up, manual review, block), its risk level (low to critical) and the fraud type (none, account takeover, stolen card, friendly fraud). The evidence is what a risk system sees, from device and geo history to recent payout changes. The data is synthetic, from a documented generator, and we say so wherever the numbers appear; real logs replace it row for row.

On 1,200 held-out rows it scores 0.952 accuracy against 0.675 for the same weights untrained, with a calibration error of 0.033 and every decision under a 5% selective-error budget. A shuffled-evidence control, the evidence swapped between rows, drops it to 0.763, roughly the majority-answer prior; that gap is the only number that proves a model reads the evidence, and a vendor number without a control is a number about the prior. Through CID, on 300 fresh sessions the training never saw:

4.2 The public sets, against Jev

Labelled as promised: the left-hand column is a zero-shot API scored on 100 rows per set, about ±5 points of sampling error; ours are fine-tuned specialists on the full official test splits. Scoring Jev on the same splits would mean running TypeSafe's API on TypeSafe's terms, so we quote their numbers as published and let the sample sizes speak. A fine-tuned specialist should win on its own distribution, and on AG News it barely does: zero-shot Laya already scores 0.924, and training moves it to 0.930. On Emotion, the set where Jev put zero probability on the true label 16% of the time (a confident way to be wrong), fine-tuning takes Laya from 0.591 to 0.865 on 4,000 rows and 0.934 on the full split. On Banking77, Jev's zero-shot 0.870 beats our 4,000-row fine-tune at 0.829; given the full 10,003-row training split, which is the one thing you cannot give Jev, the same recipe reaches 0.893 with calibration error under 0.02. The gain is proportional to how far your decisions are from what the base already knows, and to how many rows per answer you can afford.

4.3 At scale: half a million card transactions

Recast as decisions with the same converter and trained on 24,000 rows, the card model was scored on all 555,719 transactions of the Sparkov test split at their natural fraud rate of 0.39%. On ranking it sits between logistic regression and gradient boosting trained on the same rows (AUROC 0.991), catching 87.5% of fraud at a 1% false-positive budget; on calibration it is what we built it for, an error of 0.0015 over half a million decisions after prior correction, with 99.5% of the split decidable under a 0.1% selective-error budget. Given each card's history, the same recipe reaches AUROC 0.997 and a calibration error of 0.0002, second to boosted trees with the same history by 1.4 points at the 1% budget. The account-opening benchmark (Bank Account Fraud, all 205,011 test applications) caps every model near AUROC 0.89, ours at 0.883. On the Fraud Dataset Benchmark the honest word is tied where the evidence is readable (Sparkov, malicious URLs) and behind where the evidence is a vector of anonymous numbers (the ULB card data) or the positives are few (fake job postings).

4.4 The standard suites

Every classifier paper reports the same public suites, so we ran them the way those papers do: MTEB's ten English classification tasks, GLUE and SuperGLUE, and the four remaining Fraud Dataset Benchmark sets, on full official evaluation splits (GLUE dev sets where test labels are hidden). Every trained system sees the same training rows, trains for a fixed number of epochs and is never tuned on the test set. The bar is a plain fine-tuned ModernBERT-large classifier (all 395M parameters), the baseline encoder papers use; ours trains about 31M (LoRA + decision head). In these first results, ours trails the plain classifier by 0.5 to 3.1 points of accuracy, significantly on MTOP and MASSIVE, and is as well or better calibrated on all four tasks. The rest is still training; we will add it as it lands. Full tables, protocol and significance tests are in the benchmark report.

Each row is one task: the same training rows and the same official test split, a standard fine-tuned ModernBERT-large classifier (the baseline these papers report) against our fine-tuned Laya.

Accuracy on each task's official test split (SST-2 on GLUE's dev set). The difference is ours minus ModernBERT-large in points of accuracy, with an exact McNemar p-value on identical rows; calibration error is ECE, where lower is better. The simpler baselines (the most common answer, TF-IDF with logistic regression, the untrained models) are in the benchmark report.

4.5 Real time: the dino game

Contrastive Language Models published a real-time test we liked enough to borrow: Chrome's offline dinosaur game, played live at 60 frames a second. A physics planner labels each move (jump, duck, run) safe or unsafe for the moment the answer will land, the model reads those labels and picks one, and a shield replaces any answer labelled unsafe. The harness keeps up to six questions waiting on the model at once, so a fast player gets a turn every frame or two; our server answers one question at a time, so we also ran it with two waiting, the rows marked tuned. We ran their harness unchanged on the same five seeded courses, 60 seconds each, with our model on the DGX Spark and the game on a Mac joined to it by a direct 10-gigabit cable. Ours is Laya fine-tuned for four minutes on 10,972 planner-labelled situations from 80 other courses; the five test courses were never seen in training. CLM and Jev played untrained in CLM's published run, and we ran CLM and CID's 4B readout untrained on the same Spark.

Recorded 24 Sep 2026 on a DGX Spark over a direct 10 GbE link with CLM's T-Rex harness, tuned setting (two questions waiting at a time). A 12-second excerpt of a real 60-second run, not a mock-up.

With the shield on, our model survived every course and picked the planner's best move on 100% of its answers. CLM, run untrained on the same Spark, agreed 63.4% of the time and needed the shield 5,612 times in five minutes, close to the 65.8% and 4,883 of its own published run, so the harness behaves on our hardware as it does on their RTX 4090. Jev, in that published run, agreed 98.7% of the time at a leisurely 149.8 ms from question to effect; ours took 16.7 to 16.9 ms and made 3,282 to 3,587 decisions a minute to Jev's 1,119. Jev did need the shield less, 28 times to our 182 to 248; all but two of ours were answers that were right when asked and stale when they landed, and more answers are more chances to land a frame late. The shield never overrode one of our choices. Our server answers a repeated situation from a cache, as CLM's does, which is where most of its 2.5 to 3.0 ms median model time comes from.

With the shield off, the answer stands. Our model still died, 8 times in five minutes at the harness default and 3 tuned, although it picked the planner's best move every time: the planner labels a move for the frame the answer is expected to land, and an answer that lands a frame late can be wrong by then. Over Wi-Fi, where round trips spiked past 100 ms, it died 13 times. CLM died 27 times, and CID's untrained 4B readout, more than ten times slower, 40. So without the shield, survival measures the timing of the whole system as much as the model, which is why the table reports agreement with the planner, and why the lockstep run takes timing out altogether.

Lockstep takes timing away: the game waits for every answer, the shield is off, and only a wrong choice can kill the dinosaur. Our model survived all five courses without a death, and so did CID's untrained 4B readout, both picking the planner's best move on every answer; CLM died 22 times in the same five minutes of play, agreeing with the planner 86% of the time. So the 4B's 40 real-time deaths are its speed, not its judgment, and CLM's are both.

4.6 Two more games, with the markers taken away

The dino protocol hands the model its answer in the options ("Safe. ... Best."), so it measures reading and timing, not judgment. For two more games we took the markers away. Each option now states the facts a move depends on and nothing else: for Snake, the distance to the food after the move, how many free cells stay reachable and whether the head can still reach its tail; for Tetris, the lines a placement clears, the holes it leaves, its landing height, its row and column transitions and its well depth. A planner played training courses (seeds 1000 and up), we fine-tuned Laya on its decisions in Model Studio, and the model then played five unseen courses with no safety net: its most probable move is executed, every time.

Recorded 24 Sep 2026 on a DGX Spark over a direct 10 GbE link; turn-based, the facts prompt, the model's most probable move executed every time. A 12-second excerpt of a real 900-step run, not a mock-up.

On Snake, trained on 12,000 of the planner's moves in 19 minutes, our model agreed with its planner on 99.4% of 4,312 moves, never hit a wall or itself, and finished with a mean length of 61.2 against the planner's 58.6 on the same five courses; each of them lost one course to the 250-step starvation rule.

Tetris took two tries. Asked to choose among every placement, 9 to 34 per piece, the model trained on 15,996 decisions picked the planner's placement 30% of the time on held-out boards, which Model Studio's quality gate refused to deploy; played anyway, it topped out within 27 pieces on every course without clearing a line, because each imperfect placement left a board messier than any the planner had shown it. So we did what game engines do and split the job: a search proposes the five placements a standard heuristic likes best, in shuffled order, and the model chooses one from the facts. Trained on 16,000 such decisions in 34 minutes, it picked the planner's placement 96.1% of the time on held-out boards and, on the five unseen courses, cleared 157 to 159 lines in 400 pieces, a mean of 158.2, exactly the planner's own, without topping out once. The search narrows; the model decides.

Recorded 24 Sep 2026 on a DGX Spark over a direct 10 GbE link; turn-based, the facts prompt over the search's top five placements, the model's most probable placement executed every time. A 12-second excerpt of a real 400-piece run, not a mock-up.

One caveat we owe you: both models score about the same with the board description shuffled between rows, so they decide from the option facts rather than from the state text. They learned which facts matter and by how much, which is the point of the exercise; they did not learn to see the board.

4.7 Head to head

Contrastive Language Models (CLM-8B; Kwok et al.) is the closest published system to ours: a System One model behind the same typed-question API, built on frozen Qwen3-8B embeddings and a two-tower head. We ran their trainer and published head on our benchmarks, on the same official test rows. Fine-tuned, CLM trails our models on every set, with an encoder about twenty times the size of ours; our lead is significant on JevBench (+13.0 points), Banking77 (+11.2), AG News (+6.7) and Emotion (+28.6), and not on typed decisions (+1.4, p = 0.21), where the two are level. On Jev's home ground, TypeSafe's own typed-decisions workflows, 43 minutes of the same recipe put us level with Jev 1.13 on accuracy (0.724 against 0.727) and twice as good on Brier (0.074 against 0.148): the same answers, with confidence that means something, and a few points short of Laya's own longer fine-tune, which we would rather say than let someone else find.

5. Limits worth knowing

6. Next

A real-log fraud family is still the number we most want and cannot produce ourselves. If you have the logs, we have the recipe, and now the loop to keep it current.