A/B split testing

Find out which changes really improve your site. (And which don't.)

A/B split testing is the scientific way to discover if a change to your site will

  • improve your results
  • make no difference, or
  • damage your results.

A/B split testing

How does A/B testing work ?

Say you want to change the add-to-cart button on an e-commerce site.

You split the visitors to your site, and randomly show half of them the old button (A), and show the other half the new button (B). We are testing the 2 buttons against each other at the same time.

Testing the 2 buttons in this way removes all other factors: weather, fashion trends, rivals' activity, traffic sources, the economy and so on. We are just measuring the effect of the new button.

You run the test until you have enough evidence, then you show the best button to everyone and just kill off the bad button.

Is it just buttons ?

No. Your test can be:

  • Different headlines
  • Different product shots
  • With or without an image
  • Long copy versus short copy

Usually you are testing an alternative element against the existing element.

And its not just the design elements - you can also test:

  • Different checkout sequences
  • Different prices

Yes you can show different prices to different site visitors and see what price level gets the best profit.

You can even split test an entire new site against the old version and see how much boost the new colour scheme brings. The results will surprise you: site visitors just don't care.

Can I test lots of changes ?

Yes. You can run an A/B/C/D/... split test to try several alternatives.
You can also be testing different elements at the same time: maybe 2 headlines, 4 different product shots, 3 different add-to-cart buttons...
The limiting factors willl be how much traffic you get and your own resources to create the alternatives and manage the tests.

What kind of results can I get ?

Even small changes like the wording or placement of a button can make big differences: 20%, 30%, 40% differences.

The big benefit of split testing is that you lock-in the changes that work and quickly kill any changes that hurt your site.

Picture those stock market graphs zig-zagging up and down. Now imagine you could have the upswings without the downswings. With split testing you can.

Form buttons are a great place to start your A/B split testing.