How to Measure the Lift From Switching to AI Bidding

To measure the ROI lift from switching to AI-driven bidding while controlling for seasonality, run the manual and the automated strategy at the same time on randomly split traffic, so that demand swings, promotions and competitor moves reach both arms equally. The season then cancels out of the comparison, and what remains is the effect of the bid strategy, reported as a range. Where the platform cannot split a campaign, regions stand in for users. The examples use Google Ads, where automated conversion bidding is called Smart Bidding and the help pages (read on 7 October 2026) spell out how these tests should run. Our guide to what ROAS is covers the ratio itself.

Why does a before and after comparison mislead in a seasonal period?

A before and after comparison credits the new bid strategy with everything that changed between the two periods, and in a seasonal stretch most of what changed is demand. Sales climb into a peak whatever the bids are doing, so a campaign moved to automated bidding in late October posts a better November under either strategy. Conversion delay pushes the other way: the latest days are missing conversions that have yet to be reported, which understates the newer period. Comparing against the same weeks last year removes the calendar but brings in twelve months of changes to prices, competitors and tracking.

How does a holdout test work for a bid strategy?

The holdout test methodology for Smart Bidding incrementality measurement starts with a randomized split of one campaign: a share of traffic is held out on the existing manual bids while the rest runs the automated strategy over the same dates. Google Ads Help says an experiment lets you "isolate your new value based strategy within the experiment (trial arm)" and compare it with the current strategy in the base arm "while keeping all other variables constant". In the table, the first three rows come from the Google Ads Help page on custom experiments and the rest from that value-based bidding guidance.

For Shopping campaigns, the same guidance points to a one-click experiment for Target ROAS. Check auto-apply before launch as well: Google Ads Help says the feature, offered on some experiment types, "is enabled by default" and shifts 100% of traffic to the trial campaign when results are favorable, which could end the holdout early.

How do learning periods change the readout?

The automated arm starts with a handicap the manual arm never carries. Google Ads Help says a changed automated strategy "needs some time to calibrate towards the new goal", that calibration typically takes 1-2 conversion cycles, and that the learning period does not apply to Manual CPC.

Google's guidance for value-based bidding experiments turns this into a timeline: a ramp-up of "2 weeks or 1-2 conversion cycles, whichever is longer", always excluded from the evaluation, then at least 30 uninterrupted days. The end of the window is trimmed too, by dropping recent days "where less than 90% of your conversions have been reported". The page adds that the interface "may determine that results are statistically significant before this timeline has been completed" and says to follow the timeline anyway. Report the trimmed window as the headline and the full window, ramp-up included, as the one-time cost of migrating a campaign. In a seasonal test, launch early enough for the ramp-up to finish before the peak, so the peak days are neither discarded nor graded on a strategy that is still calibrating.

What do seasonality adjustments and data exclusions change?

In incrementality testing with a holdout method, a Smart Bidding seasonality adjustment is an instruction to the bidder and leaves the measurement alone. Google Ads Help defines these adjustments as a tool "to inform Smart Bidding of expected changes in conversion rates for future events like promotions or sales", advises using them "only if you expect major changes to conversion rates, because Smart Bidding already manages seasonal events", and calls them ideal for short events of 1 to 7 days. Google lists them for Search, Shopping and Display campaigns on Target ROAS or Target CPA, and for Performance Max and App (beta) campaigns on any bid strategy. The concurrent holdout already controls for the season, so an adjustment adds nothing to the validity of the comparison. It does change one arm, since manual bids have no bidder to inform. Decide before launch whether adjustments are part of how you would run the strategy afterwards, and record the decision.

Data exclusions solve a different problem. Google Ads Help presents them as a way to "reduce the impact of conversion tracking issues in Smart Bidding", such as a removed tag or a site outage, and notes that they "only affect the data that Smart Bidding uses", so the excluded conversions still appear in reporting. If tracking breaks mid-test, apply the exclusion, remove the affected dates from both arms of your own analysis, and extend the test: where a week or more of clicks is affected, Google says fluctuations "may persist for 1-2 conversion cycles".

What if the campaign cannot be split?

An incrementality experiment with a geo holdout measures the causal impact of AI bidding by switching the bid strategy in some regions and holding matched regions on manual bids, so that seasonality reaches both groups. It is the fallback where no experiment type fits the campaign, or where a shared budget cannot be removed. In a 2011 paper, Google Research described geo experiments in which "non-overlapping geographic regions are randomly assigned to a control or treatment condition". For a bidding test, the treatment is the automated strategy, confined to the treatment regions through location targeting.

For the analysis, the CausalImpact R package takes the treated series plus a set of control series and "constructs a Bayesian structural time-series model" to predict "how the response metric would have evolved after the intervention if the intervention had never occurred". Its summary gives the estimated effect with a 95% interval by default. Two assumptions carry the result: the controls were "themselves not affected by the intervention", and their relationship to the treated series "remains stable throughout the post-period". Seasonality tests the second. The built-in seasonal option supports a single component (the documentation's example is day-of-week for daily data), so in the default setup the yearly peak has to be carried by the control regions. Random assignment and a pre-period containing a comparable peak are the practical defences.

Is the lift incremental or only attributed?

To measure incremental ROI lift from AI-driven bidding while controlling for seasonality, the outcome has to change along with the design, because a platform experiment compares attributed conversion value between two arms that both serve ads. A bidder optimizing toward attributed conversions earns the same credit for a sale it caused as for one it merely touched, so attributed and incremental lift can differ (our explainer on what incrementality is covers why). Google's holdout tool, Conversion Lift, compares people who see your ads with people who don't and reports incremental ROAS; Google Ads Help states that it is not available for all Google Ads accounts. It measures ads against no ads, a different question from one bid strategy against another. The geo design answers the bidding question when its outcome is first-party revenue by region: incremental revenue in the treatment regions divided by the change in spend.

A worked readout with a confidence range

The figures in this section are hypothetical and exist to show the arithmetic. A Search campaign runs a 50/50 cookie-based experiment, with manual CPC in the base arm and Target ROAS in the trial arm. The first 14 days are excluded as ramp-up; the 30 evaluated days include a peak sales week.

Assume the report shows the conversion value difference as +15.9% with a 95% range of +4.1% to +27.7%. On the base arm's $128,000, that is between $5,248 and $35,456 of extra conversion value, with $20,320 as the point estimate, for $1,200 of extra cost. At an assumed 40% gross margin, the low end returns $2,099 of margin against that $1,200 and the point estimate returns $8,128. The switch pays for itself across the whole range, which is the test a decision should pass. (Treating the cost difference as fixed is a simplification.) Had the range crossed zero, the right call would be to keep the experiment running.

Check the confidence level before you circulate a number like this. Google's monitoring page, cited above, says you can "pick your own confidence intervals (80% is the default confidence interval)" and that a statistically significant result carries a blue asterisk. An 80% range is narrower than a 95% range on the same data.

The before and after comparison would have told a different story. Suppose the campaign ran at a ROAS of 2.90 on manual bids in the 30 days before launch. Before against after reads 2.90 to 3.60, a 24.1% gain. The base arm shows manual bidding alone reaching 3.20 in the peak weeks, up 10.3% with no change to bidding, which leaves 12.5% for the strategy. The ROAS calculator is a further resource for rerunning these ratios, and lead generation accounts can follow the same steps with cost per conversion as the metric, feeding customer acquisition cost the way this readout feeds ROAS.

A result measured in peak weeks describes peak weeks at the tested spend level, so read our notes on how to scale paid ads without killing ROAS before raising budgets on the strength of it. And write the test plan before the launch date: the arms, the split, the excluded ramp-up days, the confidence level, the trimming rule for conversion delay, whether seasonality adjustments will run, and the result that would make you revert. Setting those rules in advance is the work our paid media team does with accounts moving to automated bidding.

Sources

• Google Ads Help, About Value-based bidding using campaign experiments for Search and Shopping
• Google Ads Help, Set up a custom experiment
• Google Ads Help, Monitor your experiments
• Google Ads Help, Duration of the learning period for campaigns and what affects it
• Google Ads Help, About seasonality adjustments
• Google Ads Help, About data exclusions
• Google Ads Help, About Conversion Lift
• Google Research, Measuring Ad Effectiveness Using Geo Experiments
• CausalImpact, An R package for causal inference using Bayesian structural time-series models