emmaswift.com

As websites have advanced from a few simple web pages to complex masses of information and material, analytics have become more and more important. Visitor numbers and conversion are important measures, but it is important to test your websites for the following reasons.

• Websites can become stale – content might not generate the returns to expect over time – you need to keep the content fresh and engaging.
• Your conversion might be suffering, but this fact alone doesn’t tell you why!

There are many statistical buzzwords that have become commonplace as a result in the surge of individuals and software packages wanting to help you understand your visitors behaviour. Two of these are A/B TESTING and MULTIVARIATE ANALYSIS.

So what are they, and what are the differences? And are they solely online techniques?

A/B TESTING

This is where, quite simply, you have two versions of a web page, with one main differing feature (e.g. “Try it Now” vs “Download for Free”). Simply A/B testing will randomly split the traffic to your website into 2 streams – and dependent on which stream a visitor is, they will be presented with one version of that page. Then the outcome is monitored over days, weeks or months (largely dependent on the type of trial being run). So in this example, you might look at the data after 2 weeks, and look at how many people clicked on this button. In fact this is based on a real trial at Picasa, and guess what – “Try It Now” won!

What if you have more than one difference that you want to measure the value of? There is a method of testing whereby you can repeatedly use A/B testing to test different aspects of the site.

So if in addition to the above, you wanted to test whether a picture of the software beside this button helped to convert prospects, then you take the champion of the last test (“Try It Now”) and then trial another A/B test with the new criteria of the picture being present or not.

However, one big drawback of this approach is that you miss whether there was a combination of “Download for Free” plus a picture of the software that would have produced BETTER results than the “Try It Now” with or without the picture? So what can we do for this?

MULTIVARIATE TESTING

We can test more than one difference at a time – however, care must be taken as this can prove costly if creating webpage versions of every permutation of combinations available. Quite often, analysis of current data & statistical techniques can help focus the multivariate tests to a smaller number of combinations. Also, the other thing you need to ensure is that risk is balanced with learning – not always, but often, as A/B testing is less risky, the traffic is split 50:50. With multivariate testing you might want to only test 40% of your traffic split between 4 variants of a webpage (10% each). Beware also of splitting your tests into samples too small to get a reliable statistical variance that proves the groups behaved differently.

But this aside, multivariate testing can be very powerful – and in the right hands – can save time and produce great insightful results.

So great, you say? How do I start? One of the most popular ways to get started is Google Web Optimiser, which, if you have Google Ad-words, is an easy way to get started. You need to know what webpages you want to test, and with what variants. In a couple of clicks your test is in place and the results are even monitored by the software. There are plenty of tutorials available on www.youtube.com which can get you started.