Working · 2026-11Vendor claimGrowth opsAnalytics

CUPED variance reduction

Adjust each user's metric with their pre-experiment data to cut noise and reach conclusions faster.

Also called: pre-experiment adjustment

Claim and evidence

The claimAdjust each user's metric with their pre-experiment data to cut noise and reach conclusions faster.

No dated example yet. Treat this tactic as a hypothesis.

What the numbers show

No published number for this tactic has passed our word-for-word check yet, so there is no chart.

Why it works

Diagram, not data
Collectpre-experimentbehaviourPredict eachuser's baselineSubtract thepredictablepartSmaller effectsbecomedetectable
Collect pre-experiment behaviourPredict each user's baselineSubtract the predictable partSmaller effects becomedetectable

Part of each user's behavior is predictable from before the test. Removing it shrinks variance, so a smaller effect is detectable with the same traffic.

How to run it

  1. Enable pre-exposure data in the platform
  2. Choose a pre-exposure window (7 days is common)
  3. Handle users without history
  4. Check results with and without adjustment
  5. Document the setting

Ways to run it

Design options to test, not promised results.

  1. Option APlatform setting

    Turn on pre-exposure adjustment in the experimentation platform and document the window used.

  2. Option BNew-user handling

    Decide how to treat users with no history, since they gain nothing from the adjustment.

  3. Option CSide-by-side readout

    Report results with and without adjustment so reviewers can see what it changed.

Timeline

  1. 12013 first documented: Deng, Xu, Kohavi and Walker, as cited by Statsig
  2. 22026-11 status checked: Working

Dated examples

No dated example yet. Treat this tactic as a hypothesis.

Does it still work?

Workingas of 2026-11

On by default in Statsig Cloud; also in GrowthBook for both engines. source

Risks

  • Little gain for new users with no history
  • Needs reliable pre-period data

Sources

  1. docs.statsig.com · Statsig docs · 2025-09-18
    “reduces a metric's variance by adjusting each user's metric value with that user's pre-exposure data.”
    ✓ quote checked word-for-word on the page
  2. docs.growthbook.io · GrowthBook docs · undated
    “In fact, tools like CUPED are available for both engines.”
    ✓ quote checked word-for-word on the page

Change log

  • 2026-11-20 — harvest v0.1 (ops-ai)