Your AI agent is writing essays to tick boxes

By Sqwish Labs · 2026-10-14

tl;dr Today we’re releasing kettei: blazing-fast decision models that choose tools, check answers and read charts in milliseconds. Our flagship, kettei fat, beats every listed decision model on the public Decision Index and JevBench (and just about everything else we could measure, really). It reads images as well as text, as does kettei chub; the smaller kettei core and kettei dot are open source and lead every model their size. On OneHot, our inference and fine-tuning platform, you can keep prompt-tuning and fine-tuning on your own data as new examples and outcomes come in. With a few clicks, you can have a tuned chub or core ready within 15 minutes, still running at close to base-model speed.

We kept building the same model

We started by squeezing prompts: our first model decided, span by span, what a long context could live without. Then came routers that chose a model and reasoning budget per request (one took #1 on RouterArena), calibrated judges, bandits over harness versions, and a context pruner.

Each was a separate small model with its own data and failure modes, yet they were all doing the same thing: taking a context and a short list of options and returning a probability for each, fast. They sat in the agent’s harness, the code that coordinates these steps. We wrote that down in May:

The same note called our router "the first concrete instance of a more general mechanism": stop a language model before it generates a word and read what it has computed. A compressor makes that decision per span, a router per model, a judge per answer, a reranker per passage. We wanted one model for all of them, so we trained it.

What a decision foundation model is

A decision foundation model takes your context (text, images or both), the questions you want to ask and the allowed answers, then returns a probability for each answer. Those answers might be yes and no, a list of possible actions, or points on a rating scale.

Because the question is part of the input, you can ask the same model to triage a ticket, rank passages, judge an answer, pick a tool or read a chart, just by describing the decision in plain language. Training across many domains and kinds of decision lets it follow a question about a task it has never seen before. That is the “foundation” part: you don’t need to train a new classifier every time your agent needs to make a different choice.

Most agents use an LLM for these choices. The LLM reads the options, then writes text naming one, which you parse. You pay for every token, wait for every decode step, and get one sample where you wanted a distribution, with a confidence that post-training is known to break.^[1] When the options are already on the table, generation is a tax.

In the support agent below, seven of the eight steps choose among options the agent already has; only writing the reply needs generation. A decision model can do those seven jobs, leaving the language model to write.

A decision model skips the tax. It scores every option in one prefill pass, so cost scales with what it reads, not what it says: about 8 ms of GPU time for kettei core on a short request, 26 ms for kettei fat. Use the probabilities to decide what happens next: accept the answer, escalate it, or weigh a mistake against its cost. That only works if 0.8 means right 80% of the time, which is why we spent so long on calibration.

The kettei family

Today we’re releasing the models we built for those decisions: kettei fat, kettei chub, kettei core and kettei dot. The two larger models, fat and chub, take images as well as text; core and dot are smaller, text-only models.

We’re open-sourcing kettei core (3.7B) and kettei dot (900M), built on the Institute of Foundation Models’ K2-Horizon-3.7B^[2] and K2-Horizon-0.9B^[3] respectively, so you can run them yourself. We’re also launching OneHot, our inference and fine-tuning platform and API, with kettei fat, kettei chub and kettei core available today. Try them on a decision from your own agent.

(kettei means “decision” in Japanese. The resemblance to “kitty” may have helped, especially once we had named the larger models fat and chub.)

A brief history of choosing

Nobody invented the decision model this September. Wenpeng Yin, Jamaal Hay and Dan Roth showed how an entailment model could score labels supplied at request time;^[4] Joe Davison made that a Hugging Face pipeline.^[5] In 2020, Rodrigo Nogueira and colleagues' monoT5 ranked documents using the probabilities of “true” and “false”.^[6]

Nils Reimers's cross-encoders made scoring passages routine;^[7] Kush Dubey's CAPPr^[8] and Knowledgator's GLiClass^[9] gave developers other ways to score choices without generating an answer. Nandakishor Mukkunnoth was predicting sales outcomes with a specialised model in March 2025, well before Jev;^[10] his later Laya brought open weights for typed decisions.^[11]

These systems differ in scope, calibration and how many passes they need to score the options. TypeSafe's Jev brought general-purpose, calibrated decisions to a typed API, and helped make the case to a much wider audience. What interests us is how much work remains after a model can pick an answer: making its probabilities reliable, serving it fast, and keeping it useful as your task changes.

How we trained kettei

Most of kettei's quality came from the data work. We organised training around a broad map of domains, kinds of decision and cognitive skills: understanding a banking request, checking evidence for a claim, comparing charts, reasoning about code, choosing an agent's next action. Training spans 78 languages and 24 writing systems, with tasks in text and images.

We trained in rounds on open datasets and targeted synthetic examples, measuring where each model was weak and using those failures to shape the next round. Once a model could read a chart, we asked it to compare two. Closely matched options taught finer distinctions, including the small change that should flip a decision.

We also trained for the cases where confidence goes wrong. Shuffling the options should not change an answer, and an ambiguous question should not produce a beautifully precise lie. The data work targeted both: robustness to how choices are presented, and probabilities that reflect how much the evidence actually supports an answer.

We filtered and rechecked the synthetic training data, paying particular attention to questions the models kept getting “wrong”.^[12] A generator can stamp the same mistake onto a whole family of questions while a random sample looks fine. Inspecting those disagreements helped us find faults in the generated answers as well as gaps in the model.

All benchmarks can be gamed, but some are useful

Here is how that training holds up against the field. George Box said all models are wrong, but some are useful; benchmarks deserve much the same treatment. Public questions find their way into training data through pretraining, synthetic generators and overlapping datasets, so we report the public scores alongside held-out tests and live play from pixels.

Text and images, against the field

kettei fat is the strongest decision model we know of, on text and on images, and each smaller kettei leads every model its size. Figure 4 compares the public scores and reported latency.

On text, we use the Decision Index’s public suite, scored with the maintainer’s kit on its stratified 12,000-request sample. We never used it to pick a checkpoint; the board’s private suites are open-model-only, so kettei fat and kettei chub cannot enter. fat leads every listed decision model, including vLLM Semantic Router’s new d3 flagship;^[13] chub matches Perplexity’s 27B Decider, and kettei core and kettei dot lead their size classes by wide margins. Frontier LLMs still score higher, taking seconds and costing 20 to 200 times more per answer than fat.

On the public half of the Decision Index vision board, kettei fat beats the new leader, vLLM SR d3, while kettei chub is close to Perplexity Decider. Jev is absent because it does not take images. We score identical items with the board’s formula and anchor the results to its published scale. On our own held-out image set (5,000 questions, 542 of them multi-image), fat scores 97.5% and chub 96.9%, against 73.5% for Cloudflare Clef and 79 to 86% for frontier LLMs.

On the 231 public JevBench items, kettei fat scores 93.1% at a median of 160 ms: more accurate and faster than every other decision model we ran. Those are end-to-end timings from the UK through each provider's public API. Frontier LLMs score almost perfectly, taking seconds per answer. We have submitted the kettei models to JevBench's private evaluation, and will update this chart when the scores are in.

Grouped by what developers build, kettei fat leads the decision models compared here on six of seven axes: tool calling, fact-checking, search and ranking, maths and code reasoning, and on images, documents and visual perception. The widest gaps are in fact-checking and visual perception. Intent routing is the exception: Cloudflare Clef leads on BANKING77 and CLINC150, with kettei fat second. Routing is the axis we are working on next, and the tuning section shows what kettei chub gains from learning your own data.

The same pattern holds beyond English: on SIB-200, the public topic benchmark covering 200 languages, kettei fat and kettei chub each score 92% on a 35-language sample, three to four points ahead of Cloudflare Clef, Liquid d1 and Perplexity Decider on the same questions. fat holds 93.8% across the 19 languages in that sample not written in the Latin alphabet.

Doom, Mario and Tetris, from pixels

Games are where decisions compound: one bad choice puts you in a pit. We never trained our models to play anything, with no trajectories, rewards or episodes. They get a frame, a few sentences of rules and the legal moves, and we take the top option at every step.

In ViZDoom’s "defend the centre",^[14] kettei chub averages 11.7 kills a game over 48 games, beaten only by a scripted bot reading the engine’s own enemy labels. Clef-flash spent two-thirds of its turns firing at nothing, despite a rule against it.

Mario taught us the benchmark-gaming lesson in miniature. Holding jump-right cleared every start in our first protocol, no model required. So we randomised the starts until the stuck key stopped winning so reliably. On that harder test, kettei chub beats it by about 300 pixels a run. Cloudflare Clef's 27B model, given the same frames, walked into the first Goomba on five of eight starts.

Tetris asks the model to follow a rule from a picture: clear the most lines, then leave the fewest holes, then keep the stack low. Each turn it chooses among lettered landing spots. kettei fat places 72.9 pieces per game over 48 games before the board fills. Even with the rule written above the board, GPT-6 Luna Decisions and Clef-flash did worse than random placement.

Each game is hundreds of decisions in a row, and every one changes the next frame; a lucky guess cannot carry a 48-game average.

When the model should be unsure

Before letting an agent act on those choices, we also want to know whether it can recognise when the evidence is missing. We tested confidence as hard as accuracy, starting with a question the model could not possibly answer.

Kanta Hayashi asked Jev to call a hidden fair die roll. It said "1" every time, at 83% average confidence, and was right 19% of the time.^[15] kettei’s probabilities stayed near chance, so a confidence threshold sent every roll for review. Once the result was supplied in text, the models answered confidently; kettei fat could also read it from a photo.

A less artificial version of the same problem is a question that careful human annotators disagree on, where the useful answer is the split. kettei fat’s average confidence almost matches the annotators’ share; Jev and the frontier LLMs overshoot it.

We also tried to shake those probabilities by reordering the same questions five ways and swapping in neutral labels: letters, numbers and random strings. The few answers that changed were ones the models were already unsure about. No shuffle flipped an answer held at 80% confidence or more.

Log loss penalises confident mistakes hardest, which makes it a useful check on a model whose probabilities will drive an agent’s actions. On our harder text validation set, kettei fat leads, with the whole kettei family well below Jev and Cloudflare Clef.

Reliable confidence also lets you choose where to spend more thinking time. One way is a System 1 / System 2 hybrid: let kettei answer first, and bring in a larger language model where it is most likely to help. With kettei fat in front of Gemini 3.8 Flash, the pair matches Gemini’s accuracy at 44% of its cost, and four in five requests never reach Gemini. Jev and Perplexity Decider start close to kettei fat on accuracy, but need to pass on far more requests to catch Gemini. The same pattern holds with Claude Opus 5.5 or GPT-6 Astra as the second model.

Start from the strongest base, then make it yours

Your own examples can teach kettei distinctions that public benchmarks never cover: what your team means by “good enough to send to a customer”, or which of two nearly identical support queues a ticket belongs in. We built OneHot so you can keep teaching it those distinctions as outcomes come in, check the result and put the better model to work. When the task changes or you find a new way for it to go wrong, you can run another round.

OneHot can start by tuning the question, the descriptions of the possible answers and the scoring rubric. Wording often moves a decision model’s confidence more than its knowledge, which makes the prompt a cheap lever for bias. In a production run, "Is this a good one?" leaned towards yes; prompt tuning found "Is this film a good one?", taking kettei core from 76.5% to 90.0% on held-out rows in 83 seconds.

Fine-tuning kettei chub on Banking77’s 77 banking intents, many of them maddeningly similar, takes it from 83.1% to 93.4% accuracy. On OneHot, a few clicks get you a tuned chub or core within 15 minutes. It runs on the same engines at close to base-model speed, so you can keep improving it for your task without slowing down the agent that calls it.

Finding good examples for each round took much of the work on kettei, and we’re bringing those lessons into OneHot too. In early preview, the platform flags suspect labels and helps you label unlabelled traffic for human review. We are also building synthetic data augmentation into fine-tuning: find the weak spots, fill gaps in your examples and generate near-misses where the model confuses two answers.

The API bills input tokens, with no generated output to charge for. You can ask multiple questions against a shared context in one request. Fine-tuned inference costs twice the base rate; training is billed per million trained tokens, with a $2.50 minimum per job.

Fast where you and your customers are

A model that takes milliseconds to decide can still spend most of its time waiting on the network. On a short request, kettei spends 11 to 33 ms scoring and our API adds about 25 ms. The other 95 ms of a call from the UK today is the round trip to a single region. At that point, moving the whole decision path closer to the agent (the engine, the API and the data it needs) buys more than squeezing another millisecond out of the model.

OneHot is therefore built as many small regional deployments, not one large one. Each region has its own API, its own copy of the data a decision needs, and its own GPUs, and the nearest region answers. Keys, credit and usage are checked locally and settled with the central ledger after the response: no request waits on a far database. The United States and Europe come first, and the same design extends to GPU clusters wherever our customers' agents run.

The cheapest, fastest way to choose is the smallest model that is accurate enough, in the nearest region, in a single pass. From the UK today the four kettei sizes answer a short request in 133 to 155 ms end to end, against 244 to 315 ms for every other decision model we timed on the same requests. Up to 2,000 tokens kettei dot, kettei core and kettei chub stay ahead of all of them; at 4,000 tokens dot and core still lead, and chub is level with OpenAI Decisions and Jev.

The self-improving harness

Back to Figure 1: every step except the one that writes is a decision, and each can run on a decision foundation model.

The reranker learns from which passages answers actually used. The router learns which model each request needs from cost and outcome together. The context pruner learns what the answer needed. Tool choice learns from which calls succeeded, and escalation from where the cheap path fails. The judge, the LLM-as-a-judge teams use to score their agents, becomes a decision model calibrated against your own reviews, so the signal driving everything else gets more trustworthy as well as cheaper.

This is the loop we wanted when we were building all those separate models: one family and API to handle the choices, with OneHot handling the tuning as useful examples come back. A corrected answer or a failed tool call becomes something you can learn from, instead of another entry in a log nobody opens.

Our manifesto argued that closed-loop AI will decide who lasts. We built kettei and OneHot to make that loop practical: models you can call throughout an agent’s run and keep tuning from what happens next.

References