Running Simple A/B Tests on Your Online Store

Test one change at a time, gather enough data and measure the effect on orders before rolling improvements out to every customer.

Why Testing Beats Trusting Your Gut

Every independent retailer has strong opinions about their own website. You know your products best, so it is easy to assume the homepage headline you wrote is the best one, that the green button is obviously better than the grey one, and that a delivery-countdown banner will lift sales. Sometimes you are right. Often you are not, and the only way to know is to compare two versions side by side with real shoppers. That is all an A/B test is: you show version A to some visitors and version B to others, then see which produces more orders.

The good news is that you do not need a data team or a pricey testing platform. A modest amount of traffic, a bit of patience and a simple spreadsheet will do more for your conversion rate than a full redesign.

Change One Thing at a Time

The most common mistake is changing everything at once. If you rewrite the product headline, swap the button colour, move the delivery information and shorten the checkout form in the same week, and sales go up, you have learned almost nothing. You cannot tell which change did the work, which did nothing, and which quietly cost you sales.

So test one element per experiment. Good candidates for a first test include:

The wording on your main call to action, such as “Add to basket” versus “Add to basket – free returns”.
The headline on your most-visited product or category page.
Where you place delivery and returns information: above the fold or right beside the button.
A free-delivery threshold message in the basket or a site-wide banner.
The number of fields on your checkout form, or the wording above them.

Pick the change you believe has the best chance of moving the needle, and leave everything else alone until the test finishes.

Give the Test Long Enough to Breathe

Most small shops do not have thousands of daily visitors, so tests need time. As a rough guide, aim for at least 100 orders per version before you read the result. If you usually get 30 orders a day, a 50/50 split gives each version about 15 orders a day, so you need a week just to scrape past that threshold. If your volume is lower still, run the test for longer rather than calling it early.

Two rules keep small-sample tests honest:

Run whole weeks. Shopping habits shift with the day of the week and around payday. A test that runs Monday to Wednesday tells you nothing about Saturday.
Do not peek and stop. If you check the numbers hourly and halt the moment version B edges ahead, you are mostly measuring luck.

Also avoid testing during a sale, a bank holiday weekend or the run-up to Christmas. There is too much noise, and the result will not hold in normal trading.

Measure Orders, Not Clicks

It is tempting to celebrate a version that gets more clicks, longer time on page or a better bounce rate. These are useful signals, but they are not the outcome you run a shop for. The number that matters is orders per visitor, and ideally revenue per visitor, so you can see whether you are gaining sales or simply gaining smaller baskets.

Check the following before you declare a winner:

Orders per visitor for each version, not just the conversion rate.
Average order value, in case a change lifts order count but shrinks basket size.
Mobile versus desktop. A change that helps desktop shoppers can hurt on a phone, and most UK independents now see the majority of traffic on mobile.
Refunds and returns for a few weeks after rollout, as some wording can attract the wrong kind of order.

If version B wins by a wide, clear margin on orders per visitor, you have something worth keeping. If the two are within a whisker of each other, treat it as a draw and move on to the next idea.

Roll Out Carefully and Keep a Record

When a test wins, apply the change to every visitor and note the date you did it. Then watch your overall conversion rate for a fortnight to confirm the gain survives outside the test conditions. If it does, you have a permanent improvement. If it does not, you have still learned something useful about your customers.

Keep a simple log of every test: the date, what you changed, how long it ran, how many orders each version received and what you decided. After a year you will have a short, honest document showing what genuinely works for your shop, which is far more valuable than a folder of opinions.

Finally, resist the urge to test constantly. One well-run experiment a month is plenty for most independent retailers. Between tests, spend your time on the unglamorous things that reliably lift sales: clearer product photography, faster delivery promises, accurate stock levels and a checkout that works flawlessly on a phone.