Achieve Bigger Gains On Your Conversion Rate Optimization Experiments
So you’ve set up your A/B tests with a large amount of variations. You’re excited. You think to yourself, “With this many variations, there’s no doubt one of these variations will double my conversions!” You run the experiments, only to find that out of 20 variations, the winner only converted 8% higher. You’re dumbfounded and out of ideas.
Why is this happening?
Popular belief tells you to change 1 variable per test. The reasoning for this is actually mathematically sound and valid. Let’s assume we created an experiment with 1 variation. In this 1 variation we make 2 changes – Make the CTA blue, and move the form higher on the page. The conversion increases by 6%. How do we know which variable caused the increase? It’s possible that moving the form up raised conversions by 8% and the button color decreased conversions by 2%, therefore making the outcome 6%. The basis of a scientific experiment in and of itself **requires that only 1 variable be changed to arrive at a valid conclusion. But in Landing Page Optimization, there’s a reason as to **why you should test multiple variables at once, going against the basis of scientific experimentation – **Money. **Let me explain:
Let’s say we run the following experiments:
- Test #1: Single-variable test – 4 variations. Result: 5% lift in conversion. **Control LP Revenue: **$10,000/mo. **Increase in Revenue: **$500/mo.
- Test #2: multi-variable test – 4 variations. Result: 10% lift in conversion. **Control LP Revenue: **$10,000/mo. **Increase in Revenue: **$1,000/mo. Note: do not confuse multi-variable tests with multivariate tests. these are 2 completely different things.
By running a multi-variable test, we can see bigger differences between each variation’s conversion rate, and therefore have made an extra $500/mo. This also means that by **not **running the multivariate test and only running the single-variable test, we would be **losing out **on $500/mo that we could have had by running the multi-variable test. The only downside is that you don’t know for sure which change caused the increases. I’ll explain how to solve that issue.
Optimization Flow
We all know that 1 experiment is not enough. We should always be creating new experiments to continually optimize our conversion rate. In order to run multi-variable tests while also singling out variables, you need to follow the following optimization flow:
- Experiment #1 – Multi-variable test
- Experiment #2 – Single variable test using the winner of Experiment #1 as the Control LP
Your optimization flow should look something like the following:
source: MarketingExperiments.com
So first, you run experiment #1 so that you get a big differentiation between your variations, then you test the single variables on the winner in order to pluck out the things that didn’t help and to verify the success of the changes that increased conversions. This allows you to also avoid losing out on increases in conversion that would’ve taken much longer to figure out by running a bunch of single variable tests. You should continually be repeating this process over and over. But the questions remains: “What variables should I test?”
Creating Range
In order to create a test that results in a high increase in conversions, you need to create experiments that have **range. ** Range is the amount of differentiation you have between each variation in your experiment.
Popular belief says you should test things such as:
- Button Color
- Trust Logos
- Call-to-action text (Buy now v.s. Get it Now)
- Form on right versus left of page
- Social Icons
- Adding an incentive (“Fill out this form and we’ll send you a picture of a cute kitten!”) Now, I’m not saying not to test those things. You should definitely **test those things. It just shouldn’t be the first thing you test. You see, things like button color can make a difference, but when was the last time you changed your mind about buying something because of the color of the “Buy” button? You need to test variables that make a big difference. You need to test **Categorical Variables.
Categorical Testing
A **categorical variable **is a variable that is governed by an idea rather than an element on your page. They tend to affect your conversion rate a lot more than the usual things you would test. It’s very important you test these things first, as they are what will bring you the most value out of testing. Let’s explore a few example tests:
Process-focus v.s. Product-focus v.s. Outcome-focus LP
This experiment tests the best way to present your product or service in terms of what part of your product or service you focus on.
Let’s assume we are making a test for Apple’s MacBook Pro landing page. This is how our tests would look:
**Test #1 (Process-focused): **On this variation I would talk about the process of making a MacBook. I would probably use a picture of a factory worked shaving down the aluminum body of the MBP as a hero image. I would talk about how much precision goes into the process of making a MackBook, therefore showing the visitor how much work goes into making one, and maybe selling the visitor on that fact.
**Test #2 (Product-Focused): **On this variation I would use a beautiful picture of the MacBook as the hero image, and talk about it’s specs and capabilities. I would talk about the amount of RAM, screen resolution, CPU speed, etc. The goal would be to sell the visitor on the product itself.
**Test #3 (Outcome-Focused): **In the end, we all spend money for one reason: to become happier. In this LP I would try to prove how they would become happier by buying. I would use a picture of a woman looking at photos of her wedding on her MacBook, with a big grin on her face. I’m selling the visitor on the actual **outcome ** of what you get by buying.
source: MarketingExperiments.com
Lead v.s. Sale
You’re always going to see a lower conversion if you ask the visitor for money on your page. In many cases you will make more money by simply asking for their contact information, then converting the lead afterwards. This is tricky to setup because a conversion on a sale is much more valuable than a lead conversion. This means you can’t test their conversion rates against each other. You have to test the revenue-per-visitor. In order to do this, you total the amount of revenue each page created, then divide it by the amount of visitors to that page.
RPV = Revenue/Visitors
Tonality
Testing the tonality of your page makes a big difference. This post has an informational tone. All bloggers have a different tone. Some make you laugh, and some make you inspired.
Here’s a few examples of different tonality:
- “You need this because…Your life will change…Your time is important…”
- “We hand-make our…We have been doing this for 30 years…We are committed…”
- “People always say….What they don’t understand is…”
Design
Completely changing your design can have a big effect. Hire different designers to design variations and test them against each other.
source: MarketingExperiments.com
Steps to convert
Sometimes when people see a form on a page it scares them away immediately. Rather than having a form on the landing page, try putting it on a 2nd page and use a button on the LP instead.
If you these ideas are completely new to you, you are going to see a big effect on your experiments by applying them. I hope this helped. Now, go run some experiments!
