Black Friday for a new brand: what to test, and when to stop testing

For a small brand, peak season pulls in two directions. It's the only stretch of the year with enough traffic to detect effects quickly. It's also when a broken variant costs the most, and the results you collect describe a crowd you won't see again until next November. Most of the way out of that bind is getting the order of things right.

The weeks before peak are when testing pays back best. Traffic is climbing, a mistake is still survivable, and whatever you learn can be put to work for the weeks that matter. Three things are worth the sample.

Start with the promotional mechanic. A percentage off, a fixed amount off, free shipping or a gift with purchase tend to produce large effects that are easy to detect, and the answer sets the shape of the whole campaign.

Then urgency: countdowns, stock levels, "offer ends Sunday" messages. The effects are large here too, in either direction. You want to know whether your audience responds to it or backs away, because in some categories urgency does real damage to trust.

And the landing experience for paid traffic. You'll be buying more of it than usual, so knowing where it should land is worth more in November than in any other month.

Give all of them enough time to finish before the freeze.

About a week before peak starts, stop changing things, and don't start again until it's over.

The reason is risk, and statistics have little to do with it. A variant that breaks in one browser costs several times more during peak than it would in March, and your attention is on fulfilment and customer service, not on a dashboard. A bug you'd normally catch in two hours can run for a whole day.

That includes experiments that have already won. If a test concluded in October, ship the winner as a permanent change and switch the experiment off. A running experiment is one more moving part, and peak week is a bad week for moving parts.

Only personalisation that is clearly helpful, low risk, and was configured and checked well beforehand. Showing returning visitors what they were browsing, or putting the promotion in front of people who arrive from a campaign that mentioned it, has a small blast radius, and you'll have tested it in October. That's fine.

Anything that touches checkout, anything structural, or anything you set up that same week is not. A useful rule of thumb: would you be comfortable if it failed silently for six hours? When everyone is busy, that's the realistic worst case.

As a testing environment, peak traffic is dangerous. As data, it's extremely valuable.

Record conversion by segment, device, source and landing page. Watch sessions. Keep a note of where customer service questions pile up. You'll collect more observations in that one week than in the rest of the quarter.

Just don't generalise from them. Black Friday shoppers aren't your usual customers: they care more about price, are more often buying from you for the first time and are more likely to abandon. A change that wins during peak can easily lose in February. Treat what you see as a source of hypotheses for December and January and hold off on conclusions about your business.

The two weeks after the rush are the best testing window of the year for a small brand. Traffic is still well above your baseline, there's much less at stake, and you're carrying a pile of observations from peak week.

Most teams are worn out by then and let it pass. If you have two experiments chosen and ready to launch in early December, they'll reach a result faster than they would at any other point in the year, so it's worth picking them during peak week while the observations are fresh.