Working · 2025-10PrimaryGrowth opsPaid

Geo holdout incrementality tests

Measure what ads truly add by turning them off, or on, in randomly chosen regions or audience holdouts.

Also called: incrementality testing, holdout test, Geo-lift incrementality tests, geo experiments, GeoLift

Claim and evidence

The claimMeta advertising is very likely incremental.

Meta advertisers on Haus · haus.io · 2025-08-13

What the numbers show

No published number for this tactic has passed our word-for-word check yet, so there is no chart.

Why it works

Diagram, not data
Pick matchedregionsRandomlyassign regionsTreatment: adskeep runningControl: adsswitched offCompareoutcomes andrebalancebudget
Pick matched regionsRandomly assign regionsTreatment: adskeep runningControl: adsswitched offCompare outcomes and rebalancebudget

Randomised regions give a clean comparison, so you see lift instead of platform-attributed conversions.

How to run it

  1. Pick matched regions or holdouts
  2. Randomly assign treatment and control
  3. Run for a fixed period
  4. Compare outcomes
  5. Convert to incremental ROAS and rebalance budgets

Ways to run it

Design options to test, not promised results.

  1. Option AGeo go-dark test

    Turn a channel off in randomly chosen regions for a fixed period and measure the drop against the rest.

  2. Option BAudience holdout

    Withhold ads from a random share of the audience where the platform supports it.

  3. Option CPlatform-by-platform checks

    Test each major channel in turn, as advertisers testing Meta with Haus did, then convert lift into incremental return.

Timeline

  1. 12011 first documented: Jon Vaver and Jim Koehler, Google
  2. 22025 Meta advertisers on Haus — Haus ran 640 Meta experiments and reported averages.
  3. 32025-10 status checked: Working

Dated examples

  1. 2025

    Meta advertisers on Haus

    Haus ran 640 Meta experiments and reported averages.

    Average 19% lift; about $115 incremental revenue per $100 platform-attributed (7-day click).

    “Meta advertising is very likely incremental.”
    ✓ quote checked word-for-word on the page

    haus.io · 2025-08-13

Does it still work?

Workingas of 2025-10

Method published 2011; Haus (vendor) reports results from its own platform. source

Risks

  • Noisy tests can mislead
  • Needs scale
  • Haus results are vendor-reported
  • Spillover when people travel across regions
  • Plan 7 to 14 weeks from design to analysis per Google
  • Not usable with few regions

Sources

  1. research.google · Google Research · Jon Vaver, Jim Koehler · 2011
    “In these experiments, non-overlapping geographic regions are randomly assigned to a control or treatment condition”
    ✓ quote checked word-for-word on the page
  2. haus.io · Haus · 2025-10-09
    “The 80% audience segment is known as the treatment group, while the 20% audience segment is known as the holdout group.”
    ✓ quote checked word-for-word on the page
  3. haus.io · Haus · 2025-08-13
    “Meta advertising is very likely incremental.”
    ✓ quote checked word-for-word on the page
  4. github.com · GitHub: facebookincubator/GeoLift · undated
    “GeoLift is an end-to-end solution to measure Lift at a Geo-level using the latest developments in Synthetic Control Methods.”
    ○ page blocks automated checks; quote as relayed
  5. developers.google.com · Google Meridian GeoX docs · 2026-09-01
    “the KPI should be based on raw unfiltered conversion events”
    ✓ quote checked word-for-word on the page

Change log

  • 2026-11-20 — merged: acq-paid.jsonl:50 (geo-holdout-incrementality-test); ops-ai.jsonl:30 (geo-lift-incrementality-tests)
  • 2026-11-20 — harvest v0.1 (acq-paid, ops-ai)