So, you have a steady flow of traffic coming into your website, but not enough people are actually buying from you. It could be that there’s a problem with your website, or you could just not be optimizing something well enough and with a little tweak you could really boost those sales. The only way that you’re going to figure this out is by A/B testing your website. So in this article, I’m going to tell you what an A/B test is. Then I’ll walk you through how to figure out what to A/B test, show you how to set up those tests and finally help you figure out what winners look like for A/B tests.
And just before we get into it – if you want to speed up your learning, I made a YouTube video on this subject not long ago. Here’s a preview:
If you want to watch the whole thing, check it out on YouTube here: https://youtu.be/3rZKO0Pt308
What Is An A/B Test?
This is a very simple concept. You’re taking all of the traffic that’s coming to your website and you’re splitting it between two different page variations. The first one will be what your site already looks like. We’ll call this the control.
The other page will be a variation. On the variation, you will have made a change. Maybe it’s the button text or the headline. It could be anything on the page. The idea is that you’ve changed something and you want to measure whether you get better results with the change in place versus the norm.
So let’s say for instance, that you have a current page with a CTA button at the top that says “Get Started.” And you think that if you change that button to “Let’s Get Started,” people are going to want to click through more and you’ll drive more sales.
The way to set this up, then, is to have two different versions of your page. You’ll have the original, your control, which displays the button “Get Started.” The other version of the page will be exactly the same except for the button text, which will be changed to “Let’s Get Started.”
What happens next is that you send half your traffic to each page variation and you see which one performs better. If one of them performs better than the other by a fair margin, then you can conclude that the change you made will improve your results overall. If not, then you need to test something else.
What To Test
In order to figure out what to test, you need to start broadly narrow down as you go. Begin by heading to the Google Analytics Behavior report. This is a report that shows you how people move through your page based on the actions they take to visit other pages on the site. The Starting Page is whatever page they landed on when first visiting the site. The first interaction shows the next page they visit. The second interaction is whatever the next page they go to is after that, and etc..
So let’s say that a user lands on the Homepage then visits the About Us page and then goes back to the Homepage. That’s still three interactions. It only counts forward, regardless of whether the action the user takes on the page is reversing to a previously visited page.
What we’re looking for here is one of two things:
- Lots of people that are going to a page and then returning back to the page they were on previously.
- A lot of users who are dropping off at a page (that’s the red waterfall next to each page).
If either of these things are happening, you need to look closely at the offending page and see if there’s something obvious that isn’t working or could be optimized. And to really drill down in this area, we’re going to use something called Hotjar. It allows you to record visitors as they move about your site in order to see what actions they’re taking or not taking.
How To Test
So when we’re figuring out how to test, we need to first talk a little bit of science and then we’ll talk a little bit of function. Both are important, but they’re also two separate realms of testing.
The first and most important point about the science of A/B testing is that you need your test to be isolated. What that means is that you need to test one thing per page variation. Let’s go through an example to see how this works. You have a page variation where you’re testing the header and the CTA button. You only have two pages – one with both the old header and the old CTA, the other with the new header and new CTA. Which one of these was the winner? Was the header successful? What about the button?
You actually have no idea.
It could be the header is really increasing flow on the page while the button is actually slowing people down from taking action. It could be the opposite. But you would never know that unless you test each variation separately.
Now, CAN you test two things at one time? Yes, but what that means is that you need four page variations as follows:
|Variation 1:||Old Header||Old CTA|
|Variation 2:||New Header||Old CTA|
|Variation 3:||Old Header||New CTA|
|Variation 4:||New Header||New CTA|
Can you test four things? Well, yes, but you would need 16 page variations. And keep in mind that when you get to such a high number of page variations, the time it takes to reach a statistically significant level of traffic (i.e. the time it takes to have enough users visiting the site to give you confidence in the result) will be much longer. The amount of traffic coming to your site won’t have changed, but you’re splitting it among so many sources that its more difficult to have any confidence in the result.
Functionally, the way to make an A/B test happen successfully is to use Google Optimize. It’s totally free and links in with your Google Analytics account. It’s a nice and seamless solution. So, let’s just go ahead and jump through a setup so I can show you exactly how to set up an A/B test through Google Optimize, and we’ll use my website as an example.
So, here are the steps to make that happen:
- Start your first experiment, give it a title, and click “Create”
2. Add a variant
3. Edit the new variant
4. Highlight the section you want to change
5. Click “Edit Element”
6. Make the change directly in the block
7. Save the change and return to the setup screen
8. Ensure the split is even (in this case, 50/50)
9. Ensure the targeted page is the one you intended
10. Set your objective. This is really important, because it defines success for your test. If you want to use custom options, you need event tracking for your site. Otherwise, you can use an option from the list.
11. Once your settings are optimized, click Start
How To Find Winners
The trick to finding winners when you’re A/B testing is simple: just wait. More data means more confidence in the results so the longer that you can hold off on making a decision, the more likely you’re going to be able to figure out which prong of the test was more successful. That said, don’t go more than about 28 days if you can avoid it. However, if it needs to go longer to build out that statistical significance, then don’t be afraid of that either.
It’s also important that you look for clear winners, not barely winners. There needs to be a good amount of distance between the variations to make a determination that the change was needed.
Once you’ve figured out what needs to be done, you only have to plug the change into your main webpage, and you’re good to go.
This is a constant process of improvement. You should constantly be checking your website and all of your pages to figure out how you can keep going and keep more people moving through that conversion funnel as effectively as possible.