Ad landings
Validate headlines and proof before scaling spend.
What is A/B testing?
A/B testing compares two variants (A control, B treatment) of a page, email, or ad for randomized audiences to see which performs better on a defined metric (conversion, click, revenue). Decisions need enough volume and statistical reading—not a gut call after 50 sessions.
In one sentence
A/B testing picks a winner with data, not opinions.
Key points
- One clear hypothesis and one primary metric per test.
- Randomization and enough duration to limit false positives.
- Don’t peek and stop at the first swing.
- At low traffic, prefer qualitative research or before/after tests.
Term at a glance
- A/B testing
- split testing · controlled experiment · bucket testing
- English term
- A/B testing
- Domain
- Digital marketing / UX
- Category
- Experimentation / CRO
- Level
- Intermediate
What does A/B testing mean?
It is experimental: isolate one change (headline, CTA, form) to estimate causal effect on conversion. Modern tools handle traffic splits.
Pitfalls: too many variants without traffic, ignoring seasonality, or calling winners on secondary metrics.
For Quebec SMEs with moderate traffic, tests on high-volume ad landings or email are more realistic than homepage tests at 200 sessions/week.
How do you run an A/B test?
- 01
Write the hypothesis
“Cutting the form from 8 to 4 fields increases leads by X%.”
- 02
Pick metric and sample
Primary conversion, minimum runtime, stop rules.
- 03
Run the split
Random 50/50, identical tracking, no other major changes.
- 04
Decide and document
Ship the winner; record learning even if inconclusive.
A concrete example
A Shopify store compares “Add to cart” vs “Reserve in Montreal store.” After 4 weeks and ~3,000 sessions each, B lifts appointments 18% without hurting average online order value.
What is A/B testing for?
Subjects and CTAs on large enough lists.
Checkout
Reduce cart abandonment.
Pricing pages
Test offer presentation, not only price.
Pros and cons
- Evidence-based decisions
- Fewer ego debates
- Causal impact estimate
- Reusable learnings
- Needs volume
- False-positive risk
- Tooling cost
- Does not replace user research
A/B testing vs expert review
| A/B testing | UX heuristic audit | |
|---|---|---|
| Evidence | Experimental data | Expertise and best practices |
| Speed | Slower (collection) | Fast |
| When | Enough traffic | Low traffic / exploration |
| Risk | Statistical error | Subjective bias |
Why it matters for Quebec SMEs
When every ad click is expensive, validating a landing change before scaling protects budget. A/B testing professionalizes CRO even with a few tests per quarter.
FAQ
How much traffic?
Enough to detect the minimum useful effect. Below that, go qualitative.
A/B or multivariate?
A/B first—simpler and more reliable at SME volumes.
Mobile-only tests?
Yes—often the most critical segment.
How long?
At least one or two full business cycles including weekends.
Compliance?
Avoid dark patterns; respect consent and platform policies.
Related terms
Sources and references
Want the right A/B backlog for your landings? We can frame it.
Plan tests