How to prioritise hypotheses when you can only run one test a month

Every CRO prioritisation framework you'll find — ICE, PIE, PXL — assumes abundance. They're ranking systems for teams with more testing capacity than good ideas, where the cost of picking a mediocre test is a week.

If you can run twelve experiments a year, the cost of picking a mediocre one is a twelfth of your entire learning budget. That changes the maths, and it changes which criteria matter.

Detectability comes first, not impact#

Standard frameworks score "impact" as a guess at how much the change will move the metric. For a low-traffic brand, there's a prior question: could you tell either way?

A hypothesis with a plausible 4% effect is worth zero to you, no matter how large 4% would be in absolute revenue, because you cannot detect 4% in a month. It isn't a bad idea. It's an unmeasurable one, and running it costs you a month and returns nothing.

So the first filter isn't a score, it's a gate: given my traffic on that page, what's the smallest effect I could detect in four weeks? Anything whose realistic effect is below that line comes off the list. Bixlow computes this from your own baseline when you configure, so the gate is a number rather than a feeling.

This usually removes 70% of a typical backlog on the first pass. That's the point.

Then rank by what the answer changes#

Of what survives, prioritise by decision value, not by expected lift.

Ask: if this test comes back positive, what do I do differently? If it comes back negative, what do I do differently? If the honest answer to both is "nothing much", the experiment is expensive trivia.

The best hypotheses for a small brand are the ones where the two branches diverge sharply. "Should paid traffic get a dedicated landing page?" is excellent: a yes rebuilds your acquisition funnel, a no saves you from building twelve landing pages. "Should the testimonial be above or below the fold?" is poor: either way, next week looks the same.

Weight reusability#

With twelve shots a year, you want each one to teach you something that applies beyond the page it ran on.

A test on whether your audience responds to social proof or to guarantees tells you something you can apply to every page, every ad and every email. A test on the specific wording of one button tells you about that button.

When two candidates score similarly, take the one whose result generalises. Over a year this compounds into an actual understanding of your customers, which is the thing that makes the thirteenth experiment better than the first.

Cost of being wrong without testing#

The last criterion inverts the question. Some changes you can simply ship: cheap to build, easy to reverse, low downside if wrong. Shipping those without an experiment frees your sample for something else.

Reserve experiments for changes that are expensive to build, hard to reverse, or where being wrong is costly and invisible. A pricing page restructure qualifies. A new footer link does not.

A workable scoring sheet#

For each candidate, three columns:

1. Detectable in four weeks? Yes or no. No means it's out — no score, no discussion.
2. Decision value, 1–5. How differently do you act depending on the outcome?
3. Reusability, 1–3. Does the learning apply elsewhere?

Rank by 2 × 3. That's it. Don't add confidence or effort columns — effort barely matters when the tool builds the variant for you, and confidence is what you're trying to find out.

Revisit the list every time an experiment finishes, because each result changes what you believe about the ones still on it. That's the part most teams skip, and it's the part that makes a small programme improve over time instead of just accumulating tests.

Key takeaways#

• Detectability is a gate, not a score. If you can't measure the effect in four weeks, it's off the list.
• Rank by how differently you'd act on a positive versus a negative result.
• Prefer hypotheses whose learning generalises beyond the page they ran on.
• Ship the cheap, reversible, obvious changes without an experiment and spend the sample elsewhere.