Most A/B tests fail to find a winner, often because they test small details on pages few people see. Choosing the right tests matters more than running lots of them. Once a test finishes, check whether the result is real with the A/B test significance calculator.
Quick answer: Start testing closest to the money: checkout, basket, product and pricing pages, then landing pages and emails. Test changes big enough to matter, such as offers, delivery thresholds, page structure, headlines and trust signals, rather than button colours. Base ideas on evidence from analytics, heatmaps, surveys and customer support. Score each idea with a framework such as ICE (impact, confidence, ease) or PIE (potential, importance, ease), and run the highest scores first. Write a clear hypothesis, and make sure you have enough traffic before you start.
Where to test first
| Area | Example tests |
|---|---|
| Checkout | Fewer steps, guest checkout, payment options, delivery cost shown earlier |
| Basket | Free delivery threshold message, trust badges, add-on suggestions |
| Product pages | Image style, benefit-led copy, reviews placement, sticky add-to-basket on mobile |
| Pricing and offers | Bundles, discount framing, percentage vs amount off, price anchoring |
| Landing pages | Headline, hero image, form length, social proof |
| Subject lines, send time, offer, call to action |
Prioritise with ICE or PIE
| Framework | Score each idea 1 to 10 on | Best for |
|---|---|---|
| ICE | Impact, confidence, ease | Fast ranking of a long backlog |
| PIE | Potential, importance (traffic and value), ease | Choosing which pages to test |
| PXL | Yes/no questions on evidence and visibility | Teams that want less guesswork |
Average or multiply the scores, then work from the top. Confidence should be backed by data, not opinion.
Write a hypothesis
Use a simple format: because we saw [evidence], we believe [change] will cause [effect] for [audience], measured by [metric]. For example: because exit surveys mention surprise delivery costs, showing delivery costs on product pages will increase checkout completion, measured by orders per session.
Do you have enough traffic?
- Work out the visitors you need before starting with the A/B test sample size calculator.
- If a test would take more than 4 to 6 weeks, test a bigger change or a higher-traffic page.
- With low traffic, use user testing, surveys and session recordings to improve pages instead.
Tests that rarely move the needle
- Button colours and small font changes.
- Changes below the fold on pages most people leave quickly.
- Pages with very little traffic.
- Several unrelated changes in one test, unless you only care which version wins overall.
Measure the right metric
Conversion rate isn't everything: a discount can lift conversions but cut profit. Track revenue per visitor and margin too; see conversion rate calculator, average order value calculator and good conversion rate benchmarks. Avoid common pitfalls in A/B testing mistakes. More tools are in the marketing and e-commerce tools guide.
General information as at November 2026.
Frequently asked questions
What should I A/B test first?
Start closest to the money: checkout, basket, product and pricing pages, then landing pages and emails.
What is the ICE framework?
A way to rank test ideas by scoring impact, confidence and ease from 1 to 10.
What is the PIE framework?
A way to choose pages to test by scoring potential, importance and ease.
Is testing button colours worth it?
Rarely. Bigger changes, such as offers, layout and messaging, are more likely to make a measurable difference.
How do I write an A/B test hypothesis?
State the evidence, the change, the expected effect, the audience and the metric you will measure.
What if I do not have enough traffic to A/B test?
Test bigger changes on higher-traffic pages, or use user testing and surveys instead.
Sources
- Shopify: A/B testing guide
- Mida: ICE, PIE and PXL test prioritisation
- Practical Ecommerce: The PIE method
- Omnisend: Ecommerce A/B testing ideas
Last updated: