3 min read
Statistical significance explained without any maths
What the number actually claims, what it doesn't, and the four sentences you need to read a result honestly without ever computing anything.
AUTHOR
Fernando Blanco Dosil
Product Engineer at Bixlow
- Product development
- Experimentation & CRO
- Data & BI
Fernando holds a Computer Engineering degree from the University of Santiago de Compostela and, in his own words, is a developer by vocation rather than obligation. He's driven by ideas and by putting them into practice: he has worked across the entire software lifecycle, from design and coding to managing teams and support, with a healthy obsession for listening to users.
As a Product Engineer he works end to end: frontend with VueJS and React, backend with Java, PHP and Python, mobile with React Native and Flutter, and low-code/no-code tools when they're the best fit. He also turns data into useful information (Product Analytics, visualization, BI, ETL). He previously led a team of 12 at a technology consultancy, industrializing dashboard creation.
At Bixlow he works so any marketing and growth team can experiment and personalize their website without depending on engineering. On the blog he writes about experimentation, CRO, data and how to make things happen.
Experience
- —Full-stack Product Engineer: VueJS/React, Java/PHP/Python, React Native/Flutter.
- —Team lead of a 12-person team at a technology consultancy.
- —Data specialist: Product Analytics, visualization, Business Intelligence and ETL.
- —Speaker (T3chFest, Commit Conf, MadridJS, XantarDev), trainer and host of the Somos Blain podcast.
Articles by Fernando Blanco Dosil
3 min read
More than two variants: when an A/B/C actually makes sense
Adding a third variant costs you a third of your sample and buys you a comparison you may not need. Three cases where it's worth it, and one very common case where it isn't.
3 min read
Traffic allocation: when to use 50/50 and when to use 90/10
An even split gets you the fastest answer. An uneven one limits how many customers see a change you're not sure about. You can't have both.
3 min read
Variant persistence: why a visitor must always see the same version
If someone sees variant B on Tuesday and variant A on Friday, you haven't run an experiment. You've run a bug that also produces numbers.
3 min read
Mutually exclusive tests: stopping two experiments from contaminating each other
Two experiments on the same funnel produce four combinations and one confused conclusion. Mutual exclusion is how you keep the answer readable.
3 min read
Supertargeting: combining country, device, source and behaviour
Stacking conditions makes the segment sharper and the sample smaller. Here's how to know when the trade is worth it — and when you've segmented yourself into noise.
3 min read
Targeting by UTM: every campaign should land on the page it promised
You wrote a specific ad and then sent the click to a generic page. Message match is the cheapest conversion lift available to a small brand, and UTM targeting is how you get it.
3 min read
Mobile, desktop and tablet: segmenting by device without duplicating work
Responsive design made one page serve every screen. It didn't make one experiment serve every screen — and most flat results are a mobile win cancelling a desktop loss.
3 min read
New vs returning visitors: two websites inside the same website
A first-time visitor needs to be convinced. A returning one needs to be let through. Showing both the same page is the most common unforced error in CRO.
3 min read
Geographic targeting: why the same test doesn't work in two countries
Shipping expectations, payment habits, price sensitivity and trust signals vary by market. Averaging across them is how you get neutral results that hide real effects.
4 min read
The minimum CRO team: one person and a platform that does the rest
Traditional CRO needs an analyst, a designer, a developer and a statistician. Here's which of those four jobs actually has to be human at a small brand.
3 min read
How to prioritise hypotheses when you can only run one test a month
ICE and PIE were designed for teams running twenty tests a quarter. When you get twelve a year, the scoring model has to change.
3 min read
The five mistakes every brand makes in its first month of CRO
Stopping early, testing everything at once, no guardrail, changing the test mid-flight and treating a win as a fact. All five are avoidable.
4 min read
Your first A/B test on Shopify, in one afternoon
No theme edits, no app that slows your storefront, no developer. From hypothesis to live experiment on a real product page, step by step.
4 min read
How much traffic do you need for A/B testing?
The honest answer has three inputs and no marketing spin. Here's how to calculate it for your own site, and what to do when the number comes back too big.
4 min read
What to test first when your site doesn't have much traffic
Low traffic doesn't mean you can't experiment. It means you can only detect large effects — so you have to test the things capable of producing them.
5 min read
How to build your first experimentation program as a new brand
You don't need a CRO team, a data warehouse or a statistician. You need a hypothesis, a segment and a way to measure. Here's the whole thing, from zero.
2 min read
How a Single Experiment Can Pay for Your Tool
One winning experiment can cover your tool's annual cost and then some. Here's how to calculate the real ROI of experimentation for your business.
2 min read
How to Justify Investing in Experimentation to Leadership
Leadership doesn't fund features, they fund returns and reduced risk. Here's how to build a business case for experimentation that gets approved.
2 min read
Enterprise control without the enterprise complexity
You shouldn't have to choose between speed without control and control without speed. See how Bixlow gives engineering governance and the business the freedom to ship.
2 min read
How to Choose Your A/B Testing Tool in 2026
There's no single best A/B testing tool, only the best one for your team and stack. Here's a practical framework to decide and the mistakes to avoid in 2026.
2 min read
The real cost of an enterprise experimentation tool
The license is just the tip of the iceberg. See the hidden costs of an enterprise experimentation tool and how an AI-first alternative slashes them.
2 min read
Build vs buy: should you build your own experimentation layer?
The first A/B testing prototype is a weekend of work. The real bill comes later. An honest look at the build vs buy decision in experimentation.
2 min read
An Optimizely and VWO Alternative for Teams Without Devs
Optimizely and VWO are mature A/B testing leaders. See how Bixlow's AI-first approach compares, and when each one is the smarter choice for your team.
2 min read
Bixlow vs Adobe Target: the AI-first alternative
Adobe Target is powerful and enterprise-grade. Bixlow is AI-first with no devs or deploys. An honest comparison of when each one wins.
2 min read
Essential integrations for a growth team
CRM, payments, automation and data: the integrations every growth team needs to close the loop, all connected without code.
2 min read
How to connect your CRM and your website in one circuit
Your website and CRM live in separate worlds. Close the loop: personalize experiments with CRM data and send back conversions enriched with context. No code.
1 min read
Supabase and experimentation: real-time data
Wire Bixlow to Supabase and stream your experiment events into your own Postgres in real time, ready to query, join and analyze however you want.
2 min read
How to Integrate Your Website with n8n to Close the Loop
Zapier handles the quick wins, but when flows get complex you need more muscle. See how connecting Bixlow to n8n lets you orchestrate post-conversion logic with total control and self-hosting.
2 min read
Automate post-conversion with Zapier
A conversion doesn't end with a click. Connect Bixlow to Zapier and automate what happens next: CRM, email, Slack and BI fire instantly, no code required.
2 min read
Framer + experimentation: personalize your marketing site
Framer is built for shipping beautiful sites, not daily iteration. Add a layer that lets marketing run A/B tests and personalization without ever touching the editor.
2 min read
Connect Stripe to Your Experiments: Paywalls and Pricing
Testing copy and colors is fine, but the experiment that moves the business touches price. Connect Stripe and measure paywalls and plans against real revenue.
2 min read
How to sync leads with HubSpot without code
Push every experiment conversion straight into HubSpot, enriched with its source, segment and winning variant. No code, no manual exports.
2 min read
Personalizing your Shopify store by segment
Show new visitors, returning customers and abandoned carts the right message. Personalize your Shopify store by segment with no code: AI builds the flow, your team ships it.
2 min read
A/B testing on Shopify without installing a thousand apps
Every app you add to Shopify slows your store and splits your metrics. See how to run A/B tests from a single AI-first integration that adds no extra weight.
2 min read
Webflow personalization for marketing teams
Showing every visitor the same message leaves conversion on the table. Personalize your Webflow site by segment, traffic source or campaign, no code required.
2 min read
A/B Testing in Webflow Without Touching Code
Launch A/B tests on your Webflow site without touching code or waiting on a developer. Describe your goal and the AI builds the experiment for you.
2 min read
Risks and Controls When Using AI on Your Website
AI brings speed to personalization and experimentation on your site, but it also adds risk. Here are the controls that let you move fast without compromising security.
2 min read
How to Write Good Goals So AI Builds the Right Flow
In an AI-first approach, the quality of your flow depends on the quality of your goal. Learn what to include, the mistakes to avoid, and what separates a vague goal from a winning one.
2 min read
AI Agents in Marketing: What They Can Already Do
Forget the hype. Here are the marketing tasks AI agents can already run reliably today, where the real limits are, and how to adopt them without losing control.
2 min read
AI and personalization: how far it goes today
Cut through the hype: what AI can really do in web personalization right now, where its limits are, and how to use it without inflated expectations.
2 min read
How to use AI to analyze experiment results
AI can summarize results, spot patterns and flag what matters in seconds. Learn how to use it for analysis and which mistakes to watch so you never act on false conclusions.
2 min read
The 2026 growth stack: experimentation, personalization and automation
In 2026 the gap between fast and slow teams isn't more tools, it's how experimentation, personalization and automation fit together. Here's the modern growth stack.
2 min read
Will AI Replace CRO Teams?
AI now writes variants, builds workflows and reads results. So are CRO teams obsolete? Short answer: no, but the role changes more than you think.
2 min read
How AI Accelerates Variant Generation
Most teams test only two options because creating more is expensive. AI flips that equation, letting you generate many strong variants fast, without losing quality or control.
2 min read
From prompt to production: how AI builds your workflow
What really happens between your prompt and a live experience on your site? We open the box: how AI turns a goal into a workflow you review and ship.
2 min read
AI-first experimentation: describe the goal and publish the flow
Stop waiting on dev cycles. Describe your goal in plain language, let AI build the flow, review the blueprint and publish—no developers, no deploys.
2 min read
Dynamic experiences without building a hundred different pages
Stop duplicating URLs for every audience. Serve personalized experiences with rules and dynamic variants on a single page that scales and stays maintainable.
2 min read
How to Measure the Incrementality of Your Personalizations
A personalization can look like a win and add nothing. Incrementality measures the conversions it actually caused, so you stop celebrating coincidences.
2 min read
Ecommerce personalization: home, product page and checkout
Each stage of the buyer journey has its own job. Learn how to personalize your home, product page and checkout to lift conversion without hurting the experience.
2 min read
How to personalize the experience based on traffic source
Ads, organic, email or referral: each channel arrives with a different expectation. Learn how to keep message-page coherence and measure the real impact.
2 min read
Personalization for New vs. Returning Visitors
A first-time visitor and a returning one don't need the same thing. Here's what to show each, the mistakes to avoid, and how to measure if differentiating actually pays off.
2 min read
VIP offers for returning customers: how to design them
Returning customers are your most valuable asset. Learn which VIP rewards actually build loyalty, how to segment your best buyers and how to measure impact without giving away margin.
2 min read
Geolocation targeting: the right offer for every region
Personalize your site by country or region—currency, shipping, language and local offers—without developers. See the top use cases and how to measure real lift.
2 min read
Audience Segmentation to Personalize Your Website
How to define audience segments that actually move the needle, combine them, and avoid over-segmentation, with an AI-first approach you can publish.
2 min read
Behavioral personalization: how to react to every visitor
Adapt your site in real time to what each visitor does, page views, exit intent, repeat visits, and react at the exact moment intent appears.
2 min read
Web Personalization: What It Is and Where to Start
Personalization no longer needs a dev team or a big budget. Learn what web personalization really is, which use cases pay off first, and how to start today.
2 min read
How to Build a Prioritized CRO Roadmap
Turn a messy backlog of ideas into a real plan: source ideas from data, rank them by impact and effort, and ship with a cadence that keeps learning compounding.
2 min read
CRO for SaaS: optimizing signup and activation
In SaaS, conversion doesn't end at signup, it starts there. Learn how to cut signup friction without losing quality and guide users to their moment of value.
2 min read
How to Measure the Real Impact of Your CRO Improvements
Stop celebrating statistical noise. Learn which metrics actually matter, how to reach significance, attribute results cleanly and document every learning.
2 min read
The Role of Urgency and Scarcity in Conversion (Without Tricks)
Real urgency and scarcity move indecisive buyers; fake countdowns and invented stock destroy trust. Learn how to use them honestly and measure their true effect.
2 min read
How to optimize the above-the-fold of your homepage
Your above-the-fold has seconds to convince. Learn how a clear value proposition, visual hierarchy, an obvious CTA and A/B testing keep visitors from bouncing.
2 min read
Microcopy That Boosts Conversion
Those tiny texts around buttons, forms and errors guide every decision. Learn where microcopy matters most, how to write it and how to validate it with A/B tests.
2 min read
How to Use Social Proof to Convert More
Reviews, testimonials, usage numbers: learn which type of social proof to use, where to place it in the funnel and how to prove its real impact with A/B tests.
2 min read
Page Speed and Conversion: What Really Matters
Page speed drives conversion, but most speed obsession is misdirected. Learn which metrics actually move sales and leads, where to optimize first, and how to prove the impact.
2 min read
How to Optimize Your Campaign Landing Pages
Your campaign landing page has one job: turn paid traffic into action. Learn how to match the ad, kill distractions and A/B test fast without waiting on developers.
2 min read
Abandoned cart recovery: tactics that actually work
Cart abandonment sits near 70%. Recovery acts on buyers who already showed intent. Here are the email sequences, timing and messaging that rescue the most sales.
2 min read
How to Improve Your Store's Mobile Conversion
Mobile drives most of your traffic but converts worse than desktop. Speed, thumb-friendly design and a shorter checkout close that gap.
2 min read
Form optimization: fewer fields, more leads
Every extra field costs you leads. Learn how to trim your forms, fix usability and errors, and validate each change with A/B tests to capture more leads.
2 min read
How to optimize your pricing page
Your pricing page is where buying intent peaks and anxiety spikes. Learn how to structure plans, build trust and A/B test the layout to convert more.
2 min read
Product Pages That Convert: What to Test
Your product page is where the sale happens. Learn which elements to A/B test first—images, copy, price, reviews and CTA—to convert more buyers.
2 min read
How to Use Trust Signals to Boost Conversion
Trust signals lift conversion when they show up exactly where doubt does. Learn which types exist, where to place them, and how to measure their real impact.
2 min read
The perfect checkout: number of fields, steps and trust
How many fields to ask for, one step or several, and which trust signals turn purchase intent into an actual sale. Decisions you can validate with experiments.
2 min read
How to Reduce Cart Abandonment in Your Ecommerce
Around 70% of carts are abandoned. Learn the real causes, stage-by-stage fixes and recovery tactics to win back the sales you were quietly losing.
2 min read
Checkout optimization: ideas that move revenue
Cart abandonment hovers around 70%. These actionable checkout ideas cut friction and turn high-intent shoppers into completed orders and real revenue.
2 min read
How to Diagnose Your Funnel Leaks Before Testing
Don't launch tests blind. Learn to map your funnel, read every stage and find your biggest drop-off so you optimize where it actually pays off.
2 min read
CRO Fundamentals: How to Turn More Visits Into Customers
CRO isn't a bag of tricks. Learn the method, metrics and hypotheses that turn the traffic you already have into more customers, without waiting on developers.
2 min read
Primary and Secondary Metrics: What to Measure in Each Test
Pick the one metric that decides your A/B test before you launch, and use secondary and guardrail metrics so an apparent winner isn't a hidden problem.
2 min read
From winning test to rollout: how to deploy without breaking anything
A winning variant is only half the job. Learn how to validate, roll out gradually and monitor so your A/B test win ships safely instead of becoming a production incident.
2 min read
How to Build a Culture of Experimentation in Your Company
Move from one-off tests to a continuous program. Learn how to break down barriers and democratize experiments without losing the rigor that makes them valuable.
2 min read
Prioritizing tests with the ICE framework
Too many test ideas and not enough traffic to run them? The ICE framework scores each one by impact, confidence and ease so you decide what to test first.
2 min read
How to Combine Quantitative and Qualitative Data in Your Tests
Quantitative data tells you what is happening; qualitative data tells you why. Learn how to combine both to build sharper hypotheses and win more A/B tests.
2 min read
Mobile A/B Testing: Key Considerations You Can't Ignore
Mobile drives most of your traffic, yet testing for it isn't the same as desktop. See how screen, context and speed reshape the way you design, segment and measure.
2 min read
How to document and share your test learnings
Winning tests fade, but documented learnings compound. Here's what to record, how to build a searchable experiment repository, and how to share insights across the org.
2 min read
What to Do When a Test Returns Inconclusive Results
Your A/B test came back flat? A tie isn't a failure. Learn how to read an inconclusive result, what to do next, and how to turn it into real learning.
2 min read
How to Avoid Bias in Your Experiments
An A/B test promises objectivity, but only when it's free of bias. Learn the hidden biases that quietly corrupt your results and how to keep every experiment honest.
2 min read
A/B Testing Images and Visual Elements
Images shape the first impression more than copy does. Learn what to A/B test in photos, videos and design, and how to measure their real impact on conversion.
2 min read
How to Test Buttons and Calls to Action
The CTA is where intent turns into action. Learn what to test, in what order, and how to measure conversions without being fooled by surface-level clicks.
2 min read
How to test headlines and copy that convert
Your headline is the first thing visitors read, and often the only thing. Learn what to test, how to isolate the change and how to measure its real impact on conversion.
2 min read
Multivariate testing: when it's worth it
Multivariate testing reveals how page elements interact, but it devours traffic. Learn how MVT works, how it differs from A/B/n and when it really pays off.
2 min read
A/B testing vs A/B/n: when to use each
Testing more than two versions at once is tempting, but every extra variant splits your traffic. Learn when A/B/n pays off and when classic A/B wins.
2 min read
Mistakes that invalidate your A/B tests (and how to avoid them)
A poorly run A/B test is worse than no test at all: it gives you false confidence. Here are the most common mistakes that invalidate results and how to prevent them.
2 min read
Sample size: how to know if your test is reliable
Sample size quietly ruins more experiments than any other factor. Learn the three variables behind it and how to know if your A/B test can actually reach a conclusion.
2 min read
How Long Should You Run an A/B Test
Stop too early and you crown false winners; run too long and you waste traffic. Learn how to set the right A/B test duration before you launch.
2 min read
Statistical Significance Explained for Marketers
P-value, confidence level and false winners explained in plain English. Learn to read what your A/B testing tool tells you and stop shipping fake wins.
2 min read
How to Write a Good Test Hypothesis
A clear, testable hypothesis is the difference between an experiment that teaches you something and one that just burns traffic. Here's how to write them.
2 min read
A/B testing for beginners: concepts you need to master
Learn the core A/B testing vocabulary: control, variant, hypothesis, significance and metrics, so you can launch reliable experiments from day one.
2 min read
How to prioritize which experiments to launch first
More ideas than you can run at once? Learn a simple framework to rank your experiment backlog by impact, confidence and effort, and start with what pays off most.
2 min read
The experimentation layer: what it is and why it doesn't touch your backend
When marketing wants to ship changes without going through dev, engineering pictures chaos in production. A separate experimentation layer makes that fear unfounded.
2 min read
Five high-impact changes you can launch today without devs
Five proven conversion levers you can ship today with no code and no deploy: hero headline, CTA, social proof, exit offer and traffic-source personalization.
2 min read
How to roll back a change on your site instantly if something goes wrong
When undoing is as fast as publishing, the cost of being wrong collapses and so does the fear. Here's how instant rollback makes you bolder.
2 min read
Experiment Without Risk: The Role of Staging
Staging is the safety net that lets you test boldly: validate every change before it reaches real users, catch problems early, and publish with confidence.
2 min read
How to Give Marketing Autonomy Without Losing Engineering Control
Autonomy and control aren't opposites. With the right architecture, marketing can ship experiments freely while engineering keeps full control of the infrastructure.
2 min read
Urgent website changes: how to stop depending on the IT queue
A wrong price, an expired offer still live, a confusing message. When an urgent change has to wait for the next deploy window, the damage grows. Here's the fix.
2 min read
Iteration speed: why it's the metric that matters most in growth
Conversion, CAC, retention, LTV — they all matter. But iteration speed governs how fast you improve every one of them. Here's how to multiply it.
2 min read
How to build an experimentation program with a small team
Experimentation isn't just for big teams with data scientists and dedicated devs. With a lightweight process, two or three people can keep a steady pace of learning.
2 min read
Visual editor vs code: when you need (and don't need) a dev
Visual editor or code isn't a war, it's a division of labor. Learn which website changes belong to marketing and growth, and which truly need a developer.
2 min read
The hidden cost of waiting for a deploy on every change
Waiting for a deploy costs far more than the deploy itself: lost opportunities, slower learning and a team that stops proposing ideas. Here's the real bill.
2 min read
How to Iterate Your Website Every Week Instead of Every Quarter
Stop betting everything on a quarterly redesign. Learn how a weekly experimentation rhythm makes every change attributable, lowers risk and compounds your learning.
2 min read
From idea to production experiment the same day
When the gap between an idea and a live experiment shrinks from weeks to a single session, you stop optimizing the ticket queue and start optimizing conversion.
2 min read
Growth team autonomy: how to stop asking IT for changes
A growth team that depends on IT for every change isn't agile—it's a requester stuck in a queue. Here's how to give it real autonomy without losing engineering control.
2 min read
What you can test on your website without a single developer
From headlines and CTAs to full conditional flows: discover how much of your conversion roadmap you can launch yourself, no devs and no backlog required.
2 min read
Publish landing variants without waiting for a release
Every euro you spend on ads lands on a page. Make that page match the campaign that brought it: create landing variants by segment, measure them, and roll back instantly.
2 min read
Promotional banners in minutes without touching code
A promo banner only works if it ships on time. Learn how to launch, segment, schedule and roll back banners in minutes, no developers or deploy required.
2 min read
Your dev backlog is killing your conversion: how to unblock it
Every web change stuck in the dev backlog is a week of lost learning and uplift. Here's how to pull marketing changes out of the queue without losing technical control.
2 min read
A/B testing without coding: a guide for marketing teams
A/B tests are one of the most profitable ways to lift conversion, yet they still get stuck in engineering. Here's how marketing can run them without writing a line of code.
2 min read
How to launch website experiments without depending on development
Most growth teams don't lack ideas, they lack a way to ship them. Learn how to launch A/B tests, banners and personalization without waiting on the dev backlog.