Claim and evidence
Run price tests by assigning new users to variants, defining sample size and tolerable error first, then stopping assignment and watching cohorts.
No dated example yet. Treat this tactic as a hypothesis.
Primary
The company or founder describes the tactic and its result themselves: own blog, talk, filing.
Working · 2026-02 Described by Lovable's growth lead in a Feb 2026 Q&A.
What the numbers show
No published number for this tactic has passed our word-for-word check yet, so there is no chart.
Why it works
Avoids peeking and noise; lets revenue and retention differences show over time rather than a few days.
How to run it
- Define sample size and error up front
- Assign users to variants
- Stop assignment at target
- Observe cohorts over weeks
- Record result and decision
Ways to run it
Design options to test, not promised results.
- Option AFixed-sample test
Define sample size and acceptable error before launch and do not peek early, as Lovable describes.
- Option BNew users only
Expose only new signups to price variants so existing customers see no change.
- Option CRevenue and retention readout
Judge variants on revenue and retention per cohort, not just first-week conversion.
Dated examples
- 2026
Lovable
Describes defining sample size/error beforehand and observing cohorts.
metronome.com · 2026-02-03
Does it still work?
Described by Lovable's growth lead in a Feb 2026 Q&A. source
Risks
- Needs volume
- Legal/ethical review for different prices to similar users
Sources
- metronome.com
“Before launching a test, the team defines sample size and acceptable error up front.”
✓ quote checked word-for-word on the page
Change log
- 2026-11-20 — harvest v0.1 (revenue-expansion)