Four Step Formula to Be Successful at Multivariate Testing

Muiltvariate testing we help quickly increase conversions. Here are the four simple steps to implement multivariate testing.

We recently wrote a from the ground up, multivariate testing package to use with all of our upcoming websites. We've always believed in testing, testing, testing, and the iterative design process. However, normal split AB testing has become too slow at adapting to the speed of change necessary to keep margins high and conversions up. So we've jumped on the multivariate testing bandwagon and developed a quick way to test multiple changes at once.

Multivariate testing is nothing more than testing more than one (multi) change (variable) at the same time. Normally the scientific process tells us to only change one thing, test, and then evaluate results. Unfortunately this method is much too slow to be effective online. Plus we can see how multiple elements add or detract from one another in a quicker fashion.

The Multivariate Testing Process

The multivariate testing process is quite simple to understand.

1. First you want to keep the experience consistent for the individual user, so you need a way to identify specific users and provide them with the same variables every time. Setting cookies or session varaibles for the user allows you to track and consistently display the same information to User X.
2. Next you need a way to track which variables were shown to User X. We rewrote our tracking software to add the different variables to the stats of User X so we know that they were shown Heading A, Copy C, and Call to Action B while User Y was shown Heading B Copy C and Call to Action A.
3. Third you need a way to identify success. Whether that is making it to a specific page, or signing up for a newsletter, or staying on the site for X amount of time. And each success needs to be linked to the specific test you are running.
4. Last you need a way to compile successes and adapt to the findings. As you view your findings you can immediately remove ineffective variables from the test and replace them with modifications of the effective variables and rerun the test. 

After building our testing package we immediately put it to the test. The amount of data you can collect is amazing. We've started with some simple changes and have loved the results so far. We will be heavily using multivariate testing in our new sites to identify more precisely how users react to specific varaibles.

Getting Off to a Good Start

Now, I'm not going to jump on the "we don't know anything until we've tested" bandwagon. I think it is a cop out. Good designers know good design when they see it. Good programmers know good programming when they see it. There are still basic "rules" that can be followed to improve conversions. The trick is to start testing good versus better scenarios instead of bad versus bad versus bad.

You should follow me on twitter here

Filed Under: Testing, MVT, Multivariate Testing, Website Design

Please feel free to leave a comment or question about the article
"Four Step Formula to Be Successful at Multivariate Testing"

Name (required) :
E-mail (required, never displayed) :
URI :
Comment:
Security Code:	[Security Code]

Comments