Blog · Product
Teaching a Small Model Your Job
Fine-tuned on your own decisions, a small, fast model can take on a frontier LLM's job, or do your task far better than it could out of the box. On legal contracts, a $12 fine-tune matched Claude Opus 5.5, 12 times faster and at 1/200th of the cost per answer.
Sqwish Labs · Nov 13, 2026 · 7 min read
What a decision model is
Where generative LLMs write out an answer token by token, a decision model reads a situation, a question, and a fixed set of candidate options in one forward pass and returns a calibrated probability for each option with no text to parse. Because nothing is decoded word by word, calls return in a couple of hundred milliseconds, you pay only for the input tokens the model reads, and every option comes back with a probability you can set a threshold on.
Our kettei family comes in four sizes: fat, chub, core, and dot for on-device runs. Across every experiment in this post, we fine-tuned chub, our mid-sized hosted model, to measure how far a few thousand labelled decisions can push a fast model and where those gains stop.
Contracts: as good as a frontier LLM, for pennies
MAUD is the Atticus Project's benchmark of public-company merger agreements, where M&A lawyers answered dozens of standard American Bar Association deal-point questions about each contract, such as whether a material-adverse-effect definition carves out pandemics or when a target board can speak with rival bidders. We split the dataset by deal so no contract crossed between splits, fine-tuned chub on 1,640 excerpts from 122 agreements (a 44-minute single-GPU run in our research harness, or roughly 20 minutes on the hosted OneHot service), and tested it on 1,814 questions across 30 unseen agreements.
Fine-tuned chub matched the lawyers' answer on 94.5% of those test questions, rising 8 points over base chub (86.5%) and beating every off-the-shelf API decision model we called on OpenRouter by 7 to 17 points. Two frontier reasoning models came close: Claude Opus 5.5 scored 93.6% answering blind, and GPT-6 Sol with high reasoning effort scored 91.7%, while our largest model, kettei fat, reached 91.8% with no fine-tuning at all.
Answering all 1,814 questions cost about $6 with Opus 5.5 and $4.36 with Sol. Training the chub adapter cost $12 once at API list price, and scoring the entire test set afterward cost about 3 cents, roughly 1/200th the inference cost of Opus 5.5. Because chub scores every option in one pass rather than reasoning out loud for hundreds of tokens, its median latency on the OneHot API is 220 ms per question compared with 2.6 to 2.7 seconds for Sol and Opus 5.5, making it 12x faster in the request path.
Merger contracts (MAUD)30 agreements the model never saw
When we audited the remaining disagreements blind, roughly 4% of the benchmark's human lawyer annotations turned out to be mislabelled or cut off mid-clause before the deciding provision. Our adapter had faithfully copied more than half of those noisy labeller habits during training, and removing the broken test rows pushed fine-tuned chub above 96%.
Headlines: gains stay inside the distribution you train on
In the 2010s, Upworthy A/B-tested thousands of headline pairs on live readers to see which wording earned more clicks. Training chub for $2.50 on 3,535 pre-2015 pairs lifted accuracy on 1,000 later pairs from 2015 from 81.4% to 86.0%, matching kettei fat (86.1%) while outside decision APIs and a Claude Haiku 5.5 judge scored between 62.3% and 70.8%.
Picking the winner of a pairchance is 50%
When we asked that Upworthy adapter to predict which of two Reddit comments earned more upvotes, it scored 72.5%, virtually unchanged from base chub at 72.2%. Training a second $2.50 adapter on Reddit lifted Reddit accuracy to 79.0% while nudging Upworthy down to 80.8%, showing that each adapter learned the preferences of its own audience rather than generic persuasion.
Each fine-tune on both sitesgain over kettei chub, base, in percentage points
| Upworthy headlines | Reddit comments | |
|---|---|---|
| kettei chub, base | 81.4%baseline | 72.2%baseline |
| kettei chub, trained on Upworthy | +4.686.0% · trained on this | +0.372.5% |
| kettei chub, trained on Reddit | −0.680.8% | +6.879.0% · trained on this |
| kettei fat | +4.786.1% | +2.074.2% |
Chess: continual learning in miniature
Out of the box, chub played chess at 540 Elo, roughly a beginner who knows how the pieces move. We had Stockfish score every legal move across 50,000 positions and trained chub ($58.79 at API list price) to match Stockfish's win-probability targets, lifting the model to 966 Elo and a 71% win rate against base chub.
Watching the replays exposed a glaring flaw: the model kept building winning positions and then shuffling its pieces back and forth until the game ended in a threefold-repetition draw. Across 43 repetition draws against base chub and weakened Stockfish, our adapter was clearly winning when the loop began in 37 games, missing a forced checkmate within five moves in 25 of them and mate in one twice.
Both bugs came from how we shaped the first round's data. Win probability saturates near 100% once you are a queen ahead, which taught the model that aimless safe moves were just as good as checkmate. Worse, almost none of our 50,000 positions stood one move away from a third repeat, and because repeating a board for the second time does not draw the game yet, Stockfish rated the setup move just as highly as a winning advance.
Rather than throwing away a 966-Elo model and starting from scratch, we treated the bug as a continual update. Starting from the phase-one adapter, we mixed original middlegame positions with 27,500 boards from the model's own games, including its repetition loops, and relabelled won positions by pawn advantage and distance to mate so faster checkmates beat slower ones ($92 cumulative training cost across both phases). That warm-started second round lifted strength to 1001 Elo and turned draws into finishes, pushing the win rate against base chub from 71% to 95% and reaching 90% against both Liquid d1-ultra and TypeSafe Jev 1.14 without forgetting ordinary middlegame play.
Chess strengthElo
Share of games wonchess
Battleship: how unordered chess moves killed a grid bias
Our games arena includes other board games, and Battleship shares chess's prompt geometry: a text grid paired with one option for every open coordinate. When we played the chess adapter across 400 Battleship games (100 against each of four outside API decision models) with zero Battleship training, it won 72.5% of its matches, beating base chub (62.5%), kettei fat (69%), and our MAUD contracts adapter, which scored below base chub.
Inspecting the shot distributions revealed why only chess transferred. Base chub and the contracts adapter carry a heavy bias toward A1 and the top-left edges of a coordinate grid, whereas legal moves in chess arrive in an order that says nothing about which move is best, so soft probability targets across every legal move punished any habit of favouring A1 or the start of an option list. Freed from that corner bias, the chess adapter spreads its opening shots across the centre of the board and finds ships faster, though the advantage holds only while the prompt preserves a familiar grid layout. Changing how chess boards or coordinate lists are formatted erases that format-bound transfer just as quickly.
Battleship, never trained ongain over kettei chub, base, in percentage points
| Liquid d1-ultra | Perplexity Decider 1.2 | Nace Drex Max | TypeSafe Jev 1.14 | All four | |
|---|---|---|---|---|---|
| kettei chub, base | 66%baseline | 53%baseline | 72%baseline | 59%baseline | 62.5%baseline |
| kettei fat | +470% won | +1063% won | +981% won | +362% won | +6.569% won |
| kettei chub, chess fine-tune | +975% won | +1366% won | +981% won | +968% won | +1072.5% won |
Turning manual data surgery into continual learning
In both contracts and chess, the sharpest gains came from inspecting where version one stumbled, auditing noisy labels, and curating a targeted second mixture that fixed the blind spot without overwriting earlier skills. Most product teams do not want to run manual data surgery every time traffic shifts, which is why our research at Sqwish turns those exact steps into an automated continual loop.
At the weight level, our continual updates operate inside constrained low-rank subspaces anchored to the parent adapter rather than letting unconstrained fine-tuning drift across parameter space. Every candidate update is checked against a held-out gate spanning both newly observed failure cases and historical contexts before activation, so round two cannot quietly overwrite what round one already learned. Instead of retraining on 95% repetitive easy traffic, the system maintains a selective replay buffer over the live stream, scoring incoming decisions by their novelty against examples already in the buffer and their informative value where the current model was uncertain or outcomes proved it wrong.
Alongside that core loop, we are testing two early-preview capabilities that amplify sparse production feedback. Agentic data cleaning audits datasets for contradictory human labels, truncated contexts, and rubric bugs before an adapter memorises them, catching the kind of 4% annotation noise we found in MAUD. Failure-targeted data augmentation maps the clusters where the live model stumbles and generates hard counterfactuals right along those boundaries, automating the targeted loop repair that taught our chess model to finish won games. Because decision rows are compact and updates touch only a lightweight adapter, every full training run in this post cost between $2.50 and $92 at API list price.
What each fine-tune cost to trainonce, at API list price
We don't train on your data, but you can
Today, if you have a few thousand historical decisions with known answers, you can upload a JSONL dataset on OneHot and deploy a custom chub or core adapter in minutes. Static file uploads are only the bridge, though. Our goal across OneHot and LoopOne is to let you skip dataset exports altogether: you flip continual learning on for an endpoint, attach your outcome signal, and let the platform curate informative examples and promote gated adapter updates automatically behind the scenes while keeping serving latency flat.
Every gradient and replay example in that loop stays strictly inside your workspace. Live requests and outcomes update only your isolated private adapter and never flow into Sqwish's shared base models or anyone else's weights. We don't train on your data, but you can. Start a fine-tune today at console.sqwish.ai.