← Research library

Small models, big jobs: matching model size to marketing tasks

AI-assisted research. This note was prepared with the assistance of AI tools. It may contain errors and is not professional advice. See our Legal Disclosures.

Agentic marketing systems now run many steps per task, and each step calls a model. Model providers typically offer a range, from large, highly capable models to smaller, faster, cheaper ones. Many teams default to the largest model for everything.

That default is reasonable. More capability feels safer, and for complex reasoning and nuanced writing it is often the right choice. For many routine steps, though, it is unnecessary, and it makes agents slower and more expensive than they need to be.

This post covers how to sort tasks by the capability they need, how to mix models in one system, how to test the choice, and what to watch after switching.

What makes a task suit a smaller model

The useful question is how much reasoning a task needs to reach a right answer, and how clear that answer is. Tasks with a well-defined output and little ambiguity tend to suit smaller models. We look for three properties:

  • A clear right answer: a reviewer would agree on the correct output most of the time.
  • Short, focused input: the task does not require reading and combining many sources.
  • Simple instructions: the rules fit in a few sentences without many exceptions.

When all three hold, a smaller model is worth testing. When one fails, test anyway, but expect a larger model to win more often.

Tasks that often suit smaller models

Several common marketing steps meet those properties. These are the ones we test first:

  • Classification: tagging leads by industry, sorting support requests, labeling content by topic.
  • Extraction: pulling names, dates, and fields from structured or semi-structured text.
  • Formatting: converting content between formats and applying templates.
  • Simple checks: confirming required elements are present and checking length and structure.
  • Routing: deciding which specialist should handle a clearly described request.

These tasks have clear right answers and limited need for deep reasoning. They also tend to be high volume, which is where savings in cost and latency compound. Routing is a good example; our post on the router agent covers how to keep routing decisions simple enough for a small model.

Tasks that usually benefit from larger models

Other steps reward capability. They involve judgment, synthesis, or writing that has to hold together across many constraints:

  • Planning: breaking complex goals into tasks with dependencies and risks.
  • Nuanced writing: long-form content, sensitive topics, and careful brand voice.
  • Synthesis: combining many sources into a coherent analysis.
  • Ambiguous judgment: deciding how to handle unusual or conflicting situations.
  • Review of complex work: judging whether a draft meets subtle criteria.

Review deserves a note. It is tempting to use a cheap model to check an expensive model's work. For mechanical checks that works well, but for subtle criteria the reviewer usually needs at least as much capability as the writer.

Mixing models in one system

In an orchestrated system, different steps can use different models. A content pipeline might use a smaller model to extract facts and check formatting, and a larger model to draft and edit. A lead workflow might use a smaller model to classify and route, and a larger one to write account briefs for the highest-priority accounts.

Consider a hypothetical lead workflow that processes 2,000 inbound leads a week. Every lead is classified and routed, and about 100 of them get a written account brief. If classification and routing move to a smaller model, the large model runs on 100 tasks instead of 2,100. The arithmetic is simple, but the savings depend entirely on how per-task costs compare for your providers, so measure them.

A second mixing pattern is escalation by confidence. The smaller model handles each case first and passes it to a larger model when its output fails a check or it reports low confidence. This keeps the large model for the cases that need it, as long as the small model's confidence signal is itself tested and trustworthy.

Mixing models adds a design cost. Each step needs its own test set and its own quality threshold, and handoffs between models need clear formats. Our post on cost per task covers how to attribute spend by step so the trade is visible.

Test, do not assume

Model capabilities change often, and smaller models can improve quickly. The dependable way to choose is to test on your own tasks:

  1. Run your golden dataset for the task on two or three model sizes.
  2. Compare quality scores, cost per task, and response time.
  3. Choose the smallest model that meets your quality threshold, with some margin.
  4. Retest when providers release new models or when your task changes.

Set the threshold before you run the test. Deciding what counts as good enough after seeing results makes it too easy to talk yourself into the cheaper option. Margin matters because a test set is a sample, and production will include cases the set does not.

Retesting also protects against drift in the other direction. A test set built last quarter may no longer reflect the inputs the step sees today. See evaluation drift for how to keep test sets current.

Watch the failure modes

Smaller models may fail differently from larger ones. They may follow complex instructions less reliably, handle edge cases worse, or be more easily confused by long context. Include difficult cases in your tests, not just typical ones.

After switching, monitor a few signals closely for the first weeks:

  • Edge-case accuracy: results on the hard cases in your test set, tracked separately.
  • Escalation rate: how often the step flags uncertainty or hands off to a person.
  • Downstream rework: whether later steps or reviewers fix more errors than before.

Downstream rework is the easiest to miss. A cheaper classifier that mislabels a small share of leads may look fine in isolation while sending the wrong accounts to the brief-writing step.

Load is another reason to consider smaller models. Under heavy traffic, faster models can help agents stay within rate limits and latency targets, as discussed in scaling agents for traffic spikes.

The question we are studying next is dynamic selection: routing each request to a model based on its difficulty instead of fixing one model per step. It promises better cost and quality together. It also adds a new decision that can itself go wrong, and we are testing whether the gain survives that added risk.

How to apply this: identify one high-volume, clearly defined task, such as lead classification, and test it on a smaller model against your current one. If quality holds on your test set with margin, switch and monitor edge cases and downstream rework.

Related research

Put this research to work

Marketix turns these patterns into software: agents, automated workflows, and a hub that keeps people in control. Sign up for early access.