# How long should an experiment run?

> Not "until it's significant". Sample size sets the floor, whole business cycles set the shape, and there's an upper bound most people never think about.

- Source: https://bixlow.com/en/blog/how-long-should-an-experiment-run
- Language: en
- Published: 2026-10-15T09:00:00.000Z
- Updated: 2026-10-06T09:28:18.400Z
- Author: fernando
- Topics: experiment design, sample size, statistical significance, A/B testing, guardrails, CRO, experimentation program
- Publisher: Bixlow

---
Three rules set the duration of an experiment, and they can conflict. Knowing which one binds in your case is most of the skill.

## Rule one: the sample size sets the floor

You need a pre-declared number of sessions per variant. Divide it by your daily traffic on the targeted page and you have a minimum in days.

The critical word is *pre-declared*. Running "until significant" isn't a duration policy, it's the peeking problem with extra steps: you're sampling repeatedly and stopping at the moment the noise happens to favour you.

## Rule two: whole business cycles set the shape

Even when your sample arrives in four days, run for at least one full week — and preferably two.

Your Tuesday traffic and your Saturday traffic are different populations. Weekday visitors may be researching from work; weekend visitors may be buying. If your experiment covers Monday to Thursday, your result describes weekday behaviour and you'll deploy it to a week that includes weekends.

Two full weeks is better than one, because a single week can be distorted by one payday, one campaign burst or one unusual Saturday. If your business has a monthly rhythm — payday spikes, monthly subscription cycles — consider a full month for anything you'll act on strategically.

In practice: take the larger of your sample-size floor and your cycle requirement, then round up to a whole number of weeks. Bixlow does this rounding by default rather than stopping mid-week, because a test that ends on a Wednesday has one and a half weekends in it and that asymmetry lands in your result.

## Rule three: there is an upper bound

The rule nobody states. Beyond roughly six to eight weeks, an experiment starts measuring a moving world rather than your change.

Cookie loss accumulates, so your persistence degrades and visitors start crossing between variants. Your traffic mix shifts as campaigns change. Seasonality arrives. Your own team ships other changes to the site. The control group you're comparing against isn't the same control group you started with.

If your sample-size calculation says you need fourteen weeks, the honest reading isn't "run it for fourteen weeks". It's "this hypothesis is not testable at my current traffic" — go back and pick a bigger change, a more frequent metric, or a broader population.

## What to do while it runs

Check two things, neither of which is the result.

**Health.** Are both variants rendering? Is traffic splitting roughly as configured? Are error rates normal? A variant that silently fails to load on Safari will show as a beautiful win for control.

**Guardrails.** If revenue per session collapses in the variant, stop. Stopping for harm is always legitimate — you're not making a positive claim, you're preventing damage. Stopping for a win is not.

That asymmetry is the whole discipline. You may end a test early to stop losing money. You may not end it early to start claiming victory.

## Extending a test that came back inconclusive

Tempting, and usually wrong. If you declared 20,000 sessions and got an inconclusive result, extending to 30,000 because you didn't like the answer is the peeking problem wearing a different hat.

The legitimate version: decide *before* extending that you're now running a new, longer test with a new declared sample, and accept that the data you already have contributed to that decision. Better still, treat inconclusive as the answer it usually is — the effect, if any, is smaller than you cared about — and move on to a bigger hypothesis.

## Key takeaways
- Duration is the larger of your declared sample floor and your business-cycle requirement, rounded to whole weeks.
- "Run until significant" is not a policy, it's peeking.
- Past six to eight weeks you're measuring a moving world. A fourteen-week requirement means the hypothesis isn't testable, not that you should wait.
- Stopping early to prevent harm is always allowed. Stopping early to declare a win never is.

---

## Meet our AI CRO agent

What you just read, put to work by an agent: it understands your visitors, decides what to test or personalize, builds it on your site without a developer and tells you when a result is real.

https://bixlow.com/en/ai-cro-agent