3 min read
Statistical significance explained without any maths
What the number actually claims, what it doesn't, and the four sentences you need to read a result honestly without ever computing anything.
Topic
Articles on Experiment design
Everything we have published on Experiment design: guides, experiments and what we learned along the way.
15 articles
3 min read
More than two variants: when an A/B/C actually makes sense
Adding a third variant costs you a third of your sample and buys you a comparison you may not need. Three cases where it's worth it, and one very common case where it isn't.
3 min read
Traffic allocation: when to use 50/50 and when to use 90/10
An even split gets you the fastest answer. An uneven one limits how many customers see a change you're not sure about. You can't have both.
3 min read
Variant persistence: why a visitor must always see the same version
If someone sees variant B on Tuesday and variant A on Friday, you haven't run an experiment. You've run a bug that also produces numbers.
3 min read
Mutually exclusive tests: stopping two experiments from contaminating each other
Two experiments on the same funnel produce four combinations and one confused conclusion. Mutual exclusion is how you keep the answer readable.
3 min read
Supertargeting: combining country, device, source and behaviour
Stacking conditions makes the segment sharper and the sample smaller. Here's how to know when the trade is worth it — and when you've segmented yourself into noise.
3 min read
Mobile, desktop and tablet: segmenting by device without duplicating work
Responsive design made one page serve every screen. It didn't make one experiment serve every screen — and most flat results are a mobile win cancelling a desktop loss.
3 min read
Geographic targeting: why the same test doesn't work in two countries
Shipping expectations, payment habits, price sensitivity and trust signals vary by market. Averaging across them is how you get neutral results that hide real effects.
3 min read
How to prioritise hypotheses when you can only run one test a month
ICE and PIE were designed for teams running twenty tests a quarter. When you get twelve a year, the scoring model has to change.
3 min read
The five mistakes every brand makes in its first month of CRO
Stopping early, testing everything at once, no guardrail, changing the test mid-flight and treating a win as a fact. All five are avoidable.
4 min read
What to test first when your site doesn't have much traffic
Low traffic doesn't mean you can't experiment. It means you can only detect large effects — so you have to test the things capable of producing them.
5 min read
How to build your first experimentation program as a new brand
You don't need a CRO team, a data warehouse or a statistician. You need a hypothesis, a segment and a way to measure. Here's the whole thing, from zero.
2 min read
Primary and Secondary Metrics: What to Measure in Each Test
Pick the one metric that decides your A/B test before you launch, and use secondary and guardrail metrics so an apparent winner isn't a hidden problem.
2 min read
How to Avoid Bias in Your Experiments
An A/B test promises objectivity, but only when it's free of bias. Learn the hidden biases that quietly corrupt your results and how to keep every experiment honest.
2 min read
How to Write a Good Test Hypothesis
A clear, testable hypothesis is the difference between an experiment that teaches you something and one that just burns traffic. Here's how to write them.
KEEP EXPLORING
From reading to results
- AI CRO agentUnderstands your visitors, decides what to test or personalize, builds it on your site and measures whether it worked.Meet the agent
- Bixlow StudioA team of strategists and agents that runs your experimentation program end to end.Meet the Studio
- Book a demoWe show you Bixlow on your own website and propose your first three experiments.Book a demo