# Choosing your minimum detectable effect: a business decision, not a statistical one

> The MDE is the one input in a sample-size calculation that isn't a fact about your site. It's a statement about what would change your mind.

- Source: https://bixlow.com/en/blog/choosing-minimum-detectable-effect
- Language: en
- Published: 2026-10-19T09:00:00.000Z
- Updated: 2026-10-06T09:28:18.400Z
- Author: fernando
- Topics: minimum detectable effect, sample size, experiment design, A/B testing, low traffic, statistical significance, new brands
- Publisher: Bixlow

---
Every sample-size calculator asks for a minimum detectable effect and most people guess. Your conversion rate is a measurement. Your traffic is a measurement. The MDE is a choice, and it's the one with the most leverage over how long your test takes.

## What you're actually deciding

The MDE is the smallest improvement your experiment will be able to distinguish from noise. Set it at 5% and you need a lot of traffic. Set it at 25% and you need far less.

What it is *not* is a prediction of how much the variant will improve things. You're not guessing the effect; you're declaring the threshold below which you don't care.

That reframing is the whole article. The right question isn't "how much will this help?" — you don't know, that's why you're testing. It's: **how small an improvement would still change what I do?**

## Working it out from money

For a small brand this is a calculation you can do on the back of an envelope.

Take the annual revenue flowing through the page you're testing. Say a product page carries €200,000 a year. A 5% conversion improvement there is €10,000 a year. A 20% improvement is €40,000.

Now ask what the change costs to keep. If it's a permanent design change with no ongoing cost, €10,000 is clearly worth having, so a 5% MDE is justified — if you have the traffic for it. If the change requires maintaining two versions of a page forever, €10,000 a year may not survive the overhead, and you should only care about effects large enough to pay for the complexity.

The MDE falls out of that comparison rather than out of a rule of thumb.

## The trade you're making

Halving the MDE roughly quadruples the sample you need. That relationship is brutal and it's why this input matters more than the others.

Going from a 20% MDE to a 10% MDE on a page with 300 daily sessions takes your test from about three weeks to about three months. Three months crosses the upper bound where seasonality, traffic-mix shift and cookie loss start contaminating the comparison — so in practice you haven't bought a more sensitive test, you've bought an unreliable one.

This is why the honest answer for low-traffic brands is often to raise the MDE rather than lower it. A test that can only detect large effects, run properly, beats a test that claims to detect small ones and runs for a season.

## What raising it costs you

Be clear-eyed: a high MDE means you will miss real improvements. If you set it at 25% and the true effect is 8%, your test comes back inconclusive and you'll conclude the idea didn't work. It did — you just couldn't see it.

That's the actual trade, and it's why "inconclusive" should be recorded as *"no effect larger than 25%"* rather than as "didn't work". Those are very different notes to read back in a year.

Bixlow states the MDE on the result alongside the interval for this reason. An inconclusive test with a declared 25% threshold tells you something specific; an inconclusive test with no stated threshold tells you almost nothing.

## A workable default

If you don't want to do the revenue maths every time, this works for most small brands:

- **20–25% MDE** for your first tests. Large, structural changes only. Results in weeks.
- **10–15%** once your traffic supports it, for refinements to things that already won.
- **Below 10%** only if you genuinely have the volume, and only for changes on your highest-value page.

And whenever the calculator returns a duration over about eight weeks, don't lower your confidence — raise your MDE, or pick a different hypothesis.

## Key takeaways

- The MDE is a choice about what would change your decision, not a prediction of the effect.
- Derive it from the revenue on the page and the ongoing cost of keeping the change.
- Halving the MDE roughly quadruples the sample. This is the input with the most leverage.
- Record inconclusive results as "no effect larger than X%", not as "didn't work".

---

## Meet our AI CRO agent

What you just read, put to work by an agent: it understands your visitors, decides what to test or personalize, builds it on your site without a developer and tells you when a result is real.

https://bixlow.com/en/ai-cro-agent