Working · 2026-02PrimaryRevenuePricing

Cohort-based price test with fixed sample

Run price tests by assigning new users to variants, defining sample size and tolerable error first, then stopping assignment and watching cohorts.

Claim and evidence

The claimRun price tests by assigning new users to variants, defining sample size and tolerable error first, then stopping assignment and watching cohorts.

No dated example yet. Treat this tactic as a hypothesis.

What the numbers show

No published number for this tactic has passed our word-for-word check yet, so there is no chart.

Why it works

Diagram, not data
Set samplesize anderror firstAssign newusers tovariantsStopassignmentat targetWatchcohortsover weeksRecordresult anddecision
Set sample size and error firstAssign new users to variantsStop assignment at targetWatch cohorts over weeksRecord result and decision

Avoids peeking and noise; lets revenue and retention differences show over time rather than a few days.

How to run it

  1. Define sample size and error up front
  2. Assign users to variants
  3. Stop assignment at target
  4. Observe cohorts over weeks
  5. Record result and decision

Ways to run it

Design options to test, not promised results.

  1. Option AFixed-sample test

    Define sample size and acceptable error before launch and do not peek early, as Lovable describes.

  2. Option BNew users only

    Expose only new signups to price variants so existing customers see no change.

  3. Option CRevenue and retention readout

    Judge variants on revenue and retention per cohort, not just first-week conversion.

Dated examples

  1. 2026

    Lovable

    Describes defining sample size/error beforehand and observing cohorts.

    metronome.com · 2026-02-03

Does it still work?

Workingas of 2026-02

Described by Lovable's growth lead in a Feb 2026 Q&A. source

Risks

  • Needs volume
  • Legal/ethical review for different prices to similar users

Sources

  1. metronome.com · Metronome blog · Stephanie Keep (Q&A with Elena Verna, Lovable) · 2026-02-03
    “Before launching a test, the team defines sample size and acceptable error up front.”
    ✓ quote checked word-for-word on the page

Change log

  • 2026-11-20 — harvest v0.1 (revenue-expansion)