← Research library

Q3 kickoff: three agentic experiments worth running this quarter

AI-assisted research. This note was prepared with the assistance of AI tools. It may contain errors and is not professional advice. See our Legal Disclosures.

Agentic marketing has reached the point where the open questions are mostly practical. An agent can draft a plan or a report. Whether it does that well inside your workflow, with your data and your reviewers, is still something each team has to find out.

For many B2B teams, July and August bring fewer launches and more people on leave. That makes the third quarter a good window for experiments that would be hard to fit around year-end work. This post sets out three we recommend, each sized to run inside the quarter and each designed to inform a specific decision for Q4 and next year.

Why a calm quarter suits experiments

Experiments need two things that busy quarters lack: stable conditions and attention. When campaigns launch every week, it is hard to tell whether a change in results came from the agent or from everything else that moved. Reviewers are also stretched, so the careful comparison an experiment needs tends to get skipped.

A quieter quarter gives you both. Volume is steadier, so before and after comparisons mean more. Reviewers have time to record what they changed and why. A failed experiment also costs less, because fewer customer-facing deadlines depend on it.

The trade-off is that summer conditions are not year-end conditions. An agent that performs well on July volume may behave differently under the traffic spikes of a major launch. Treat Q3 results as evidence about quality and fit, and plan a separate load check before your busiest weeks.

Experiment 1: plan review before execution

The question here is whether an agent can draft a useful campaign plan, and whether reviewing that plan before work starts saves time later. Planning is a good early candidate because a plan is cheap to change. A bad plan caught in review costs an hour, while the same mistake caught mid-campaign costs far more.

To run it, pick the next three campaigns on your calendar. For each one, ask a planner agent to produce a structured plan with tasks, owners, dependencies, assumptions, and risks. Review each plan as a team and record every change you make, with a one-line reason. Then execute as usual, and at the end note which problems the review caught and which it missed.

The change log is the real output. If most changes are about missing context, such as a budget limit or a sales commitment the agent could not know, the fix is better inputs. If most changes are about judgment, such as sequencing or channel choice, planning may need to stay human-led for now. For background on the pattern, see the planner-executor pattern.

Experiment 2: shadow mode for one recurring workflow

The second question is how an agent would perform on a recurring workflow if it were doing the work for real. Test sets show what an agent can do on curated cases. They cannot show how it handles the messy inputs, late changes, and edge cases of an ordinary week.

Choose one recurring workflow, such as lead research, weekly reporting, or first-draft nurture emails. Run an agent in shadow mode for four weeks, producing its own output alongside the human output without sending anything. Each week, compare the two on a sample and record where they differ.

A few checks keep the comparison honest:

  • Score blind where you can: have a reviewer rate outputs without knowing which came from the agent.
  • Use the same rubric each week: changing criteria mid-test hides drift.
  • Log the cases the agent declined or flagged: knowing when to stop is part of the result.
  • Record human effort: time saved only counts if review time does not cancel it out.

At the end of four weeks you have real-world performance evidence and a clear basis for deciding whether the agent should take on the work in Q4. If the results are promising, the shadow cases also make a strong start on a golden dataset for future testing.

Experiment 3: answer engine baseline

The third question is how your brand appears in AI-generated answers to the questions your buyers ask. A buyer can now ask an assistant to explain a software category or compare tools, and the answer may shape a shortlist before anyone visits your site. We treat this as a channel worth measuring before anyone knows how much it matters.

Build a list of 30 questions a buyer might ask at different stages, from defining the problem to comparing vendors. Ask each one across several major assistants and AI search experiences, three times each, at the start and end of the quarter. Record whether your brand appears, how it is described, whether the description is accurate, and which sources are cited.

Repeat runs matter because answers vary from one run to the next. A single run can make a brand look present or absent by chance. Three runs per question per assistant give a rough sense of consistency at little cost.

The output is a baseline and a list of content gaps: questions where you are absent, described inaccurately, or cited from outdated sources. See measuring share of answer for a fuller method.

Running experiments well

The three experiments differ, but the discipline around them is the same. Before you start each one, put the following in writing:

  • The question: one sentence, specific enough that a result can answer it.
  • The decision it informs: what you will do differently depending on the result.
  • One owner: a named person who runs it and writes it up.
  • A review date: set before the end of the quarter, so results arrive in time for Q4 planning.
  • A result threshold: the outcome that would change your plans, decided in advance.

The threshold is the easiest item to skip. Without it, almost any result can be read as encouraging. Deciding in advance that an edit rate above a level you choose means keeping a human in the loop turns the write-up into a decision instead of a discussion.

Write up every result, including the disappointing ones. A short note on what you tried, what you saw, and what you decided is often more useful to the next team than a polished success story. It is also the raw material for deciding which pilots become programs, a step we cover in from pilot to program.

Three experiments run well will teach you more than ten loosely tracked ones. If capacity is tight, run one.

What we are watching

The open question for us is how well summer results predict year-end behavior. An agent that does well on steady July volume may handle the pressure of a major launch differently, and we do not yet know how much of a Q3 result carries forward. We are studying which measures stay stable across those conditions and which need to be checked again.

How to apply this: choose one of these experiments for Q3, name an owner this week, and put the review date on the calendar. Write down, before you start, the result that would change your Q4 plans.

Related research

Put this research to work

Marketix turns these patterns into software: agents, automated workflows, and a hub that keeps people in control. Sign up for early access.