Picking the Right Model for the Coding Task: Speed, Quality, and Cost

Most developers reach for one model for everything and then complain that it's either too slow or too dumb. The fix isn't finding the single best model; it's routing each kind of task to the tier that fits. Coding work spans a huge range — from "finish this line" to "refactor this module across twelve files" — and those extremes want very different things from a model. Here's how I think about the trade-offs and how I actually assign work.

The three axes, and why you can't max all of them

Every model choice trades among speed, quality, and cost. In 2026 the major providers each ship a tiered family — a small fast model, a mid "workhorse," and a large frontier model (Anthropic's Haiku/Sonnet/Opus split is the clearest example, and OpenAI and Google offer analogous tiers). The tiers exist precisely because you can't have all three at once. Bigger models reason better and hallucinate less on hard problems, but they cost more per token and respond more slowly. Small models are cheap and fast but fall apart on multi-step reasoning and large context.

Two cost mechanics are worth internalizing. First, output tokens usually cost several times more than input tokens, so a model that rambles is expensive twice over. Second, prompt caching — now standard across the major APIs — makes repeated context (your system prompt, a stable file set) dramatically cheaper on subsequent calls. For agentic loops that re-send the same context dozens of times, caching changes the economics enough to affect which model is actually cheapest per completed task.

Match the model to the loop

Inline completion is a latency game. You're typing; the suggestion has to appear in well under a second or it's worse than nothing because it breaks your flow. Quality matters less than you'd think here because you're reviewing every character as it lands. Use the fastest small model your tool offers. Frontier models are the wrong choice for tab-completion — you'll feel the lag and gain little.

Chat edits and explanations — "why is this test flaky," "refactor this to use the repository pattern" — are the workhorse tier's home. A mid model answers in a few seconds with strong quality and reasonable cost. Reach for the frontier tier only when the mid model visibly struggles: subtle concurrency bugs, tricky type-level programming, or reasoning across a lot of interacting code.

Agentic tasks — an assistant that plans, edits many files, runs tests, and iterates — are where model quality pays for itself. A weak model in an agent loop is a disaster: it takes wrong turns, burns tokens re-reading files, and produces changes you can't trust. Here the frontier or strong-mid tier is usually the right call even though per-token cost is higher, because the cost that matters is cost per correctly completed task, and a smarter model finishes in fewer, straighter steps. A cheap model that needs three correction rounds and still ships a bug is the expensive option.

The metric that actually matters: cost per completed task

Per-token price is a trap. What you care about is total tokens to get a working result times price, plus your own time. Work an example. Say a refactor takes a strong model one clean pass — read five files, produce a diff, tests pass. That might be, hand-wavingly, 40k input and 6k output tokens, once. A cheaper model at a third of the price might need three passes because it misreads the code, breaks a test, and re-reads everything each round: now you're at 150k+ input and 20k+ output, plus you've spent your own attention refereeing it. The "cheap" model cost more money and a lot more of your time.

This flips for high-volume, low-stakes work. Classifying log lines, generating boilerplate CRUD, writing docstrings across a repo — thousands of easy, independent calls — is exactly where the small fast model wins, because each task is trivial and the volume makes per-token price dominate. Route by difficulty, not by habit.

Practical routing you can implement today

Most tools now let you set a model per mode. A sane default configuration:

• Inline / tab: smallest, fastest model.
• Everyday chat and edits: mid workhorse model.
• Agent runs, hard bugs, architecture: frontier model, and accept the slower, pricier calls.
• Bulk mechanical transforms: small model, scripted through the API rather than the editor.

If you script your own calls, a lightweight router beats a fixed choice. Start with a cheap model and escalate on a signal — a failing test, low self-reported confidence, or a retry — rather than paying frontier prices for every request up front.

def solve_task(task):
    # Try the cheap tier first for easy work.
    result = call_model("small", task)
    if passes_checks(result):        # tests, type-check, lint
        return result
    # Escalate only when the cheap attempt fails verification.
    return call_model("frontier", task, hint=result.failures)

The escalation pattern captures most of the savings without the risk: easy tasks resolve on the cheap tier, and only the genuinely hard ones pay for the big model. The gate is passes_checks — automated verification, not vibes. Without a real check, escalation-on-failure degrades into "escalate whenever I feel unsure," which defeats the point.

Failure modes to watch

• Over-provisioning. Running the frontier model for tab-completion and simple edits quietly triples your bill and slows your loop for no quality gain. This is the most common waste.
• Under-provisioning agents. Putting a cheap model in a multi-step agent to "save money" usually costs more in tokens and produces changes you can't merge. False economy.
• Ignoring context cost. A million-token window doesn't mean you should fill it. You pay for every token sent, every call. Large context also degrades reasoning quality, so bloat costs you twice.
• Chasing benchmark leaderboards. Public coding benchmarks are a rough signal, contaminated by training-data leakage and gamed prompts. Trust a two-hour trial on your codebase over any leaderboard delta.
• Latency blindness on interactive work. A model that's 5% "smarter" but twice as slow is often a net loss for anything you sit and wait on. Measure wall-clock time to a working result, not just answer quality.

A default worth stealing

Fast small model for completion, a solid mid model as your daily driver, and a frontier model on standby for agent runs and the genuinely hard problems. Turn on prompt caching everywhere you can. Judge everything by cost and time to a verified result, not by sticker price per token or a leaderboard. Re-check your assignments every few months, because the tiers shift fast and today's frontier capability tends to become next year's mid-tier default. Get the routing right and you'll spend less, wait less, and trust the output more — all at once, which is the one place the three axes stop fighting each other.