Blog · Product

Teaching a Small Model Your Job

Fine-tuned on your own decisions, a small, fast model can take on a frontier LLM's job, or do your task far better than it could out of the box. On legal contracts, a $12 fine-tune matched Claude Opus 5.5, 12 times faster and at 1/200th of the cost per answer.

Sqwish Labs · Nov 13, 2026 · 7 min read


What a decision model is

Where generative LLMs write out an answer token by token, a decision model reads a situation, a question, and a fixed set of candidate options in one forward pass and returns a calibrated probability for each option with no text to parse. Because nothing is decoded word by word, calls return in a couple of hundred milliseconds, you pay only for the input tokens the model reads, and every option comes back with a probability you can set a threshold on.

Our kettei family comes in four sizes: fat, chub, core, and dot for on-device runs. Across every experiment in this post, we fine-tuned chub, our mid-sized hosted model, to measure how far a few thousand labelled decisions can push a fast model and where those gains stop.

Contracts: as good as a frontier LLM, for pennies

MAUD is the Atticus Project's benchmark of public-company merger agreements, where M&A lawyers answered dozens of standard American Bar Association deal-point questions about each contract, such as whether a material-adverse-effect definition carves out pandemics or when a target board can speak with rival bidders. We split the dataset by deal so no contract crossed between splits, fine-tuned chub on 1,640 excerpts from 122 agreements (a 44-minute single-GPU run in our research harness, or roughly 20 minutes on the hosted OneHot service), and tested it on 1,814 questions across 30 unseen agreements.

Fine-tuned chub matched the lawyers' answer on 94.5% of those test questions, rising 8 points over base chub (86.5%) and beating every off-the-shelf API decision model we called on OpenRouter by 7 to 17 points. Two frontier reasoning models came close: Claude Opus 5.5 scored 93.6% answering blind, and GPT-6 Sol with high reasoning effort scored 91.7%, while our largest model, kettei fat, reached 91.8% with no fine-tuning at all.

Answering all 1,814 questions cost about $6 with Opus 5.5 and $4.36 with Sol. Training the chub adapter cost $12 once at API list price, and scoring the entire test set afterward cost about 3 cents, roughly 1/200th the inference cost of Opus 5.5. Because chub scores every option in one pass rather than reasoning out loud for hundreds of tokens, its median latency on the OneHot API is 220 ms per question compared with 2.6 to 2.7 seconds for Sol and Opus 5.5, making it 12x faster in the request path.

Merger contracts (MAUD)30 agreements the model never saw

kettei chub, contracts fine-tuneClaude Opus 5.5kettei fatGPT-6 Sol (high)Liquid d1-ultraPerplexity Decider 1.2kettei chub, baseNace Drex MaxTypeSafe Jev 1.14Accuracy75%80%85%90%95%kettei chub, contracts fine-tune: 94.5%94.5%Claude Opus 5.5: 93.6%93.6%kettei fat: 91.8%91.8%GPT-6 Sol (high): 91.7%91.7%Liquid d1-ultra: 87.3%87.3%Perplexity Decider 1.2: 87.2%87.2%kettei chub, base: 86.5%86.5%Nace Drex Max: 79.8%79.8%TypeSafe Jev 1.14: 77.7%77.7%Cost for all 1,814 questions$0$2$4$6kettei chub, contracts fine-tune: about 3¢about 3¢Claude Opus 5.5: about $6about $6kettei fat: 1.9¢1.9¢GPT-6 Sol (high): $4.36$4.36Liquid d1-ultra: 1.5¢1.5¢Perplexity Decider 1.2: 0.8¢0.8¢kettei chub, base: 1.5¢1.5¢Nace Drex Max: 2.9¢2.9¢TypeSafe Jev 1.14: 2.4¢2.4¢Accuracy75%80%85%90%95%kettei chub, contracts fine-tune94.5%kettei chub, contracts fine-tune: 94.5%Claude Opus 5.593.6%Claude Opus 5.5: 93.6%kettei fat91.8%kettei fat: 91.8%GPT-6 Sol (high)91.7%GPT-6 Sol (high): 91.7%Liquid d1-ultra87.3%Liquid d1-ultra: 87.3%Perplexity Decider 1.287.2%Perplexity Decider 1.2: 87.2%kettei chub, base86.5%kettei chub, base: 86.5%Nace Drex Max79.8%Nace Drex Max: 79.8%TypeSafe Jev 1.1477.7%TypeSafe Jev 1.14: 77.7%Cost for all 1,814 questions$0$2$4$6kettei chub, contracts fine-tuneabout 3¢kettei chub, contracts fine-tune: about 3¢Claude Opus 5.5about $6Claude Opus 5.5: about $6kettei fat1.9¢kettei fat: 1.9¢GPT-6 Sol (high)$4.36GPT-6 Sol (high): $4.36Liquid d1-ultra1.5¢Liquid d1-ultra: 1.5¢Perplexity Decider 1.20.8¢Perplexity Decider 1.2: 0.8¢kettei chub, base1.5¢kettei chub, base: 1.5¢Nace Drex Max2.9¢Nace Drex Max: 2.9¢TypeSafe Jev 1.142.4¢TypeSafe Jev 1.14: 2.4¢
Lawyers’ answers on merger-agreement deal points, on 30 agreements the model never saw.

When we audited the remaining disagreements blind, roughly 4% of the benchmark's human lawyer annotations turned out to be mislabelled or cut off mid-clause before the deciding provision. Our adapter had faithfully copied more than half of those noisy labeller habits during training, and removing the broken test rows pushed fine-tuned chub above 96%.

Headlines: gains stay inside the distribution you train on

In the 2010s, Upworthy A/B-tested thousands of headline pairs on live readers to see which wording earned more clicks. Training chub for $2.50 on 3,535 pre-2015 pairs lifted accuracy on 1,000 later pairs from 2015 from 81.4% to 86.0%, matching kettei fat (86.1%) while outside decision APIs and a Claude Haiku 5.5 judge scored between 62.3% and 70.8%.

Picking the winner of a pairchance is 50%

kettei chub, trained on Upworthykettei fatkettei chub, baseLiquid d1-ultraPerplexity Decider 1.2TypeSafe Jev 1.14Claude Haiku 5.5, LLM judgeUpworthy headlines50%60%70%80%90%kettei chub, trained on Upworthy: 86.0%86.0%kettei fat: 86.1%86.1%kettei chub, base: 81.4%81.4%Liquid d1-ultra: 70.8%70.8%Perplexity Decider 1.2: 69.3%69.3%TypeSafe Jev 1.14: 64.8%64.8%Claude Haiku 5.5, LLM judge: 62.3%62.3%Upworthy headlines50%60%70%80%90%kettei chub, trained on Upworthy86.0%kettei chub, trained on Upworthy: 86.0%kettei fat86.1%kettei fat: 86.1%kettei chub, base81.4%kettei chub, base: 81.4%Liquid d1-ultra70.8%Liquid d1-ultra: 70.8%Perplexity Decider 1.269.3%Perplexity Decider 1.2: 69.3%TypeSafe Jev 1.1464.8%TypeSafe Jev 1.14: 64.8%Claude Haiku 5.5, LLM judge62.3%Claude Haiku 5.5, LLM judge: 62.3%
kettei chub, trained on Redditkettei fatkettei chub, baseLiquid d1-ultraPerplexity Decider 1.2Claude Haiku 5.5, LLM judgeTypeSafe Jev 1.14Reddit comments50%60%70%80%90%kettei chub, trained on Reddit: 79.0%79.0%kettei fat: 74.2%74.2%kettei chub, base: 72.2%72.2%Liquid d1-ultra: 76.3%76.3%Perplexity Decider 1.2: 73.8%73.8%Claude Haiku 5.5, LLM judge: 73.0%73.0%TypeSafe Jev 1.14: 70.0%70.0%Reddit comments50%60%70%80%90%kettei chub, trained on Reddit79.0%kettei chub, trained on Reddit: 79.0%kettei fat74.2%kettei fat: 74.2%kettei chub, base72.2%kettei chub, base: 72.2%Liquid d1-ultra76.3%Liquid d1-ultra: 76.3%Perplexity Decider 1.273.8%Perplexity Decider 1.2: 73.8%Claude Haiku 5.5, LLM judge73.0%Claude Haiku 5.5, LLM judge: 73.0%TypeSafe Jev 1.1470.0%TypeSafe Jev 1.14: 70.0%
Share of pairs where the model picked the winner, on 1,000 Upworthy pairs from 2015 and 600 Reddit pairs. Each fine-tune cost $2.50. The LLM judge is Claude Haiku 5.5 with our own prompt.

When we asked that Upworthy adapter to predict which of two Reddit comments earned more upvotes, it scored 72.5%, virtually unchanged from base chub at 72.2%. Training a second $2.50 adapter on Reddit lifted Reddit accuracy to 79.0% while nudging Upworthy down to 80.8%, showing that each adapter learned the preferences of its own audience rather than generic persuasion.

Each fine-tune on both sitesgain over kettei chub, base, in percentage points

Upworthy headlinesReddit comments
kettei chub, base81.4%baseline72.2%baseline
kettei chub, trained on Upworthy+4.686.0% · trained on this+0.372.5%
kettei chub, trained on Reddit−0.680.8%+6.879.0% · trained on this
kettei fat+4.786.1%+2.074.2%
The large figure is the gain over kettei chub, base, in percentage points; the accuracy is under it. A tinted cell is the site that fine-tune trained on.

Chess: continual learning in miniature

Out of the box, chub played chess at 540 Elo, roughly a beginner who knows how the pieces move. We had Stockfish score every legal move across 50,000 positions and trained chub ($58.79 at API list price) to match Stockfish's win-probability targets, lifting the model to 966 Elo and a 71% win rate against base chub.

Watching the replays exposed a glaring flaw: the model kept building winning positions and then shuffling its pieces back and forth until the game ended in a threefold-repetition draw. Across 43 repetition draws against base chub and weakened Stockfish, our adapter was clearly winning when the loop began in 37 games, missing a forced checkmate within five moves in 25 of them and mate in one twice.

Both bugs came from how we shaped the first round's data. Win probability saturates near 100% once you are a queen ahead, which taught the model that aimless safe moves were just as good as checkmate. Worse, almost none of our 50,000 positions stood one move away from a third repeat, and because repeating a board for the second time does not draw the game yet, Stockfish rated the setup move just as highly as a winning advance.

Rather than throwing away a 966-Elo model and starting from scratch, we treated the bug as a continual update. Starting from the phase-one adapter, we mixed original middlegame positions with 27,500 boards from the model's own games, including its repetition loops, and relabelled won positions by pawn advantage and distance to mate so faster checkmates beat slower ones ($92 cumulative training cost across both phases). That warm-started second round lifted strength to 1001 Elo and turned draws into finishes, pushing the win rate against base chub from 71% to 95% and reaching 90% against both Liquid d1-ultra and TypeSafe Jev 1.14 without forgetting ordinary middlegame play.

Chess strengthElo

kettei fatkettei chub, basekettei chub, chess fine-tunefirst run · 50,000 positions · $58.79kettei chub, chess fine-tuneafter the fix · +27,500 positions · $924006008001000kettei fat: 540540kettei chub, base: 540540kettei chub, chess fine-tune: 966966kettei chub, chess fine-tune: 100110014006008001000kettei fat540kettei fat: 540kettei chub, base540kettei chub, base: 540kettei chub, chess fine-tunefirst run · 50,000 positions · $58.79966kettei chub, chess fine-tune: 966kettei chub, chess fine-tuneafter the fix · +27,500 positions · $921001kettei chub, chess fine-tune: 1001
Elo against Stockfish at low skill levels and with random moves mixed in. Each fine-tune shows its training data and its cumulative cost at list price.

Share of games wonchess

kettei chub, basekettei fatkettei chub, chess fine-tunefirst runkettei chub, chess fine-tuneafter the fixvs base chub0%50%100%same modelkettei fat: 19%19%kettei chub, chess fine-tune: 71%71%kettei chub, chess fine-tune: 95%95%vs Liquid d1-ultra0%50%100%kettei chub, base: 15%15%kettei fat: 15%15%kettei chub, chess fine-tune: 68%68%kettei chub, chess fine-tune: 90%90%vs TypeSafe Jev 1.140%50%100%kettei chub, base: 20%20%kettei fat: 20%20%kettei chub, chess fine-tune: 76%76%kettei chub, chess fine-tune: 90%90%vs base chub0%50%100%kettei chub, basesame modelkettei fat19%kettei fat: 19%kettei chub, chess fine-tunefirst run71%kettei chub, chess fine-tune: 71%kettei chub, chess fine-tuneafter the fix95%kettei chub, chess fine-tune: 95%vs Liquid d1-ultra0%50%100%kettei chub, base15%kettei chub, base: 15%kettei fat15%kettei fat: 15%kettei chub, chess fine-tunefirst run68%kettei chub, chess fine-tune: 68%kettei chub, chess fine-tuneafter the fix90%kettei chub, chess fine-tune: 90%vs TypeSafe Jev 1.140%50%100%kettei chub, base20%kettei chub, base: 20%kettei fat20%kettei fat: 20%kettei chub, chess fine-tunefirst run76%kettei chub, chess fine-tune: 76%kettei chub, chess fine-tuneafter the fix90%kettei chub, chess fine-tune: 90%
Share of games won. The first fine-tune drew games it had won; the fix made it finish them.

Battleship: how unordered chess moves killed a grid bias

Our games arena includes other board games, and Battleship shares chess's prompt geometry: a text grid paired with one option for every open coordinate. When we played the chess adapter across 400 Battleship games (100 against each of four outside API decision models) with zero Battleship training, it won 72.5% of its matches, beating base chub (62.5%), kettei fat (69%), and our MAUD contracts adapter, which scored below base chub.

Inspecting the shot distributions revealed why only chess transferred. Base chub and the contracts adapter carry a heavy bias toward A1 and the top-left edges of a coordinate grid, whereas legal moves in chess arrive in an order that says nothing about which move is best, so soft probability targets across every legal move punished any habit of favouring A1 or the start of an option list. Freed from that corner bias, the chess adapter spreads its opening shots across the centre of the board and finds ships faster, though the advantage holds only while the prompt preserves a familiar grid layout. Changing how chess boards or coordinate lists are formatted erases that format-bound transfer just as quickly.

Battleship, never trained ongain over kettei chub, base, in percentage points

Liquid d1-ultraPerplexity Decider 1.2Nace Drex MaxTypeSafe Jev 1.14All four
kettei chub, base66%baseline53%baseline72%baseline59%baseline62.5%baseline
kettei fat+470% won+1063% won+981% won+362% won+6.569% won
kettei chub, chess fine-tune+975% won+1366% won+981% won+968% won+1072.5% won
Never trained on Battleship. The large figure is the gain over kettei chub, base, in percentage points of games won against each API decision model; the win rate is under it.

Turning manual data surgery into continual learning

In both contracts and chess, the sharpest gains came from inspecting where version one stumbled, auditing noisy labels, and curating a targeted second mixture that fixed the blind spot without overwriting earlier skills. Most product teams do not want to run manual data surgery every time traffic shifts, which is why our research at Sqwish turns those exact steps into an automated continual loop.

At the weight level, our continual updates operate inside constrained low-rank subspaces anchored to the parent adapter rather than letting unconstrained fine-tuning drift across parameter space. Every candidate update is checked against a held-out gate spanning both newly observed failure cases and historical contexts before activation, so round two cannot quietly overwrite what round one already learned. Instead of retraining on 95% repetitive easy traffic, the system maintains a selective replay buffer over the live stream, scoring incoming decisions by their novelty against examples already in the buffer and their informative value where the current model was uncertain or outcomes proved it wrong.

Alongside that core loop, we are testing two early-preview capabilities that amplify sparse production feedback. Agentic data cleaning audits datasets for contradictory human labels, truncated contexts, and rubric bugs before an adapter memorises them, catching the kind of 4% annotation noise we found in MAUD. Failure-targeted data augmentation maps the clusters where the live model stumbles and generates hard counterfactuals right along those boundaries, automating the targeted loop repair that taught our chess model to finish won games. Because decision rows are compact and updates touch only a lightweight adapter, every full training run in this post cost between $2.50 and $92 at API list price.

What each fine-tune cost to trainonce, at API list price

Headlines3,535 headline pairsContracts (MAUD)1,640 contract excerptsChess50,000 positions, then 27,500 more$0$25$50$75$100Headlines: $2.50$2.50Contracts (MAUD): $12$12Chess: $92$92$0$25$50$75$100Headlines3,535 headline pairs$2.50Headlines: $2.50Contracts (MAUD)1,640 contract excerpts$12Contracts (MAUD): $12Chess50,000 positions, then 27,500 more$92Chess: $92
A one-off cost at our API list price: $0.80 per million training tokens, $2.50 minimum a job. It grows with the training data, so chess, trained on the most positions over two rounds, cost the most.

We don't train on your data, but you can

Today, if you have a few thousand historical decisions with known answers, you can upload a JSONL dataset on OneHot and deploy a custom chub or core adapter in minutes. Static file uploads are only the bridge, though. Our goal across OneHot and LoopOne is to let you skip dataset exports altogether: you flip continual learning on for an endpoint, attach your outcome signal, and let the platform curate informative examples and promote gated adapter updates automatically behind the scenes while keeping serving latency flat.

Every gradient and replay example in that loop stays strictly inside your workspace. Live requests and outcomes update only your isolated private adapter and never flow into Sqwish's shared base models or anyone else's weights. We don't train on your data, but you can. Start a fine-tune today at console.sqwish.ai.