What is an A/B test
An A/B test is an experiment where you show two versions of a page (the original, A, and a variant, B) to randomly split visitors and compare which one gets more conversions, so you know from data rather than opinion whether the change works.
By Fernando Blanco Dosil · Updated November 2, 2026
How an A/B test works
The key is random assignment. If half your visitors see A and the other half see B at the same time, everything else (day of the week, the Instagram campaign, the weather) affects both groups equally. The only difference between them is the change you are testing.
Each visitor always sees the same version, even if they come back tomorrow. If they saw A one day and B the next, you wouldn't know what to credit their purchase to.
- You choose what to change and which metric decides (for example, completed orders).
- The tool splits visitors between A and B.
- Each version's conversions are counted.
- Statistics tells you whether the difference is real or chance.
A/B test versus A/B/n test
Same method. What changes is how many variants compete with the original.
| A/B test | A/B/n test | |
|---|---|---|
| Versions | Original and one variant | Original and two or more variants |
| Traffic needed | The minimum for your conversion rate | More, because it is shared across more versions |
| When to use it | You have one clear idea to test | You have several ways to solve the same problem |
| Example | "Buy now" button versus "Add to bag" | Three different labels for the same button |
Mistakes that ruin an A/B test
Stopping it as soon as the variant is ahead. Numbers swing a lot in the first days. You need enough sample and at least one full week, because people don't buy on a Monday the way they do on a Saturday.
Changing the test midway. If you edit the variant or the split while the test runs, you mix two different experiments into the same result.
Not checking the split. If you asked for 50/50 and one version gets noticeably more traffic than the other, something technical is broken and the result can't be trusted.
How Bixlow does it
In Bixlow, the AI agent builds the variants on your live site without a developer, and you can run A/B or A/B/n tests for all traffic or for a specific audience. Each visitor always sees the same version, you check every variant in a preview before publishing and nothing goes live without your approval. The verdict comes from a Bayesian statistics engine that requires a minimum sample, a minimum duration and a probability to be best of at least 95%, and that warns you if the split gets out of balance. It installs with one script on any website.