Heatmaps per variant: where attention actually went

Heatmaps have a bad reputation among people who take measurement seriously, and mostly they deserve it. A heatmap of your product page tells you people click the buttons and scroll down. You knew that.

What changes the picture is comparison. Two heatmaps of the same page — one per variant, same period, same device — turn a decorative image into a measurement of where your change moved attention.

Scroll depth. What percentage of visitors reached each vertical point. This is the one that most often explains a null result: if your change lives at 70% page depth and only 30% of visitors get there, you changed something most of your sample never saw. Comparing scroll curves between variants also tells you whether the change made people explore further or gave up earlier.

Click maps. Where people tap or click. Two things to look for. First, whether your new element gets interaction at all. Second, and more useful, clicks on things that aren't clickable — images people expect to zoom, headings they expect to expand, prices they expect to open a size chart. Dead clicks are direct evidence of an expectation you're not meeting.

Area/attention maps. Where people paused. Useful mainly for checking whether a block you added is being read or skipped, and whether it displaced something that used to get attention.

This is the whole discipline and it is routinely ignored.

"Only 22% of people reach the reviews" is close to meaningless on its own — it may be completely normal for that page, that device, that category. "22% in variant B versus 41% in control" tells you your change pushed reviews out of reach for half the people who used to see them.

So always open them in pairs. In Bixlow, a result links through to Clarity pre-filtered to each variant, which makes the side-by-side a couple of clicks rather than a data project.

A heatmap that combines mobile and desktop is genuinely useless, because the layouts are different and the two curves are being averaged into a shape that describes neither.

Same for new and returning visitors: returning visitors scroll faster and further because they know where things are, so mixing them flattens exactly the difference you're looking for.

Filter to one variant, one device, one visitor type. Then compare against the same slice of control. Anything less specific and you're looking at an average of populations that behave differently.

Did the change get seen? Scroll depth at the position of your change, both variants. If the answer is "by a third of people", your effect was diluted by two-thirds of the sample and the real effect on people who saw it is much larger than your headline.

Did it displace something? Adding a block pushes everything below it down. Compare scroll depth at the old elements. A trust block that lifts trust and pushes the add-to-cart below the fold can easily be net negative.

Are there new dead clicks? If your variant introduced an element that looks interactive and isn't, the click map shows it immediately, and it's usually a five-minute fix that turns a losing variant into a winning one.

Did rage clicks appear? Clarity flags these. A rage click cluster that exists in one variant and not the other is a bug, not a preference.

Heatmaps aggregate, which means they hide the individual path. They'll tell you a region got attention; they won't tell you in what order or what the person was trying to do. For that you need to watch sessions.

And they're not evidence of effect. A variant whose heatmap looks better in every respect can still lose. The experiment decides; the heatmap explains.

• A single heatmap is decoration. Two heatmaps of the same page, one per variant, are a diagnosis.
• Always filter to one variant, one device and one visitor type before reading anything.
• Check scroll depth at your change: an effect diluted by people who never saw it is much larger than it looks.
• Watch for displacement, new dead clicks and rage-click clusters that exist in one variant only.