A/B Testing in WooCommerce – Getting Started

Posted on by Anders-Meyer Eldøy

A/B testing is a way to compare two versions of a single variable, typically by testing a subject’s response to variant A against variant B, and determining which of the two variants is more effective.

This article assumes you already have an operational e-commerce store with healthy traffic and Google Analytics tracking installed. If this is not the case, get in touch with us and let’s see how we can help you get there.

There are more A/B testing tools than you can imagine and assuming your test data is not corrupt any tool beats no tool.

Our weapon of choice is Google Optimize. In addition to it being a mature solution that happens to be free, it integrates extremely well with Google Analytics, which all of our clients’ websites already use and most are familiar with.

Here’s what our entire testing setup looks like:

  • Google Analytics
  • Google Optimize
  • Google Tag Manager
  • DuracellTomi’s Google Tag Manager for WordPress

Now let’s talk about what each tool does and why we decided to use it.

Google Analytics

A freemium web analytics service used by 85% of the world’s top 100k websites according to BuiltWith.com. It does have its share of privacy issues (it’s a Google product, after all), but you have the option, and obligation in EU, to make sure all data that is processed by Google Analytics is anonymized.

Some of its biggest strengths are popularity and a very long list of features, especially for e-commerce websites.

Google Optimize

Even though Google Optimize has only been publicly launched last spring, it has quickly become one of the most popular A/B tools on the market. Like Google Analytics, it is a freemium product, but the majority of users will not need to pay any money to use it.

Google Optimize may not be the most feature-rich tool on the market on its own, but it is how well it integrates with Google Analytics that sets it apart.

There are currently four types of experiences you can create in Google Optimize: A/B tests, multivariate tests, split URL tests, and personalizations. All four are very easy to set up, with powerful targeting rules and a WYSIWYG editor.

Google Tag Manager

You do not have to use Google Tag Manager (or any other tag management solution) in order to track your store’s traffic or run A/B test. You should though, in most cases, for a few good reasons:

  • Implementation speed – pushing a new tag will be much easier and faster if done via a tag management solution. No need to wait for updates to be tested on a staging site, then deployed to live, you can simply preview the changes in the live site directly from Google Tag Manager, then when you’re satisfied just make them live.
  • Version control – for your tags! And everything else in Google Tag Manager, of course.
  • Lots and lots of built-in goodies – the majority of tags, triggers, and variables you’ll ever need to use already exist in Google Tag Manager, all you need to do is configure them.
  • Simply put, more often than not you will need a reason NOT to use Google Tag Manager.

DuracellTomi’s Google Tag Manager for WordPress

This one is optional, especially if all you need to do is add Google Tag Manager container snippet to your website. If that is the case, you’ll do fine without GTM4WP, but make sure to check what it can do for you. It really is one of those “does one thing and does it well” plugins. Add strong ratings and a very responsive author and it’s easy to see why it’s become a de facto standard for those who need to add Google Tag Manager to a WordPress website.

The Setup

Now that you’re familiar with all the tools we use, let’s tie them all together.

First, you need to create one of each:

Once you have done that, add Google Analytics tag to Google Tag Manager container, set its “track type” to “page view” and set it to fire on every page load by assigning this trigger to it.

Next, you’ll need to add Google Optimize tag to Google Tag Manager container and set it to load on all pages you want to A/B test. Most often you’ll be testing one specific URL, so when creating the trigger you can set it to fire only if “page path” matches the path of the page that will be tested.

Now you can publish the container and add its snippet to the site. You’ll save a lot of time using Google Tag Manager for WordPress plugin.

At this point, you’re ready to start creating Google Optimize experiences.

Creating an A/B Test

Each A/B test starts with a hypothesis. More clearly defined the hypothesis, more likely that you will end up with a successful test. Note that a successful A/B test is a test you learn something from, not necessarily one that performs better.

When creating your hypothesis try to be as focused as possible.

“If we redesign checkout page our revenue will go up” is a bad hypothesis for a few reasons. You won’t be able to tell what in the redesign contributed most to increased revenue. Furthermore, a sudden spike in traffic would increase the revenue anyway, right?

“If we change Checkout button color to green revenue will go up” is better, but there’s still the sudden traffic spike issue.

“If we change Checkout button color to green checkout page conversion rate will go up” is better still, because it targets the correct metric and isolates the checkout page from the rest of the site. However, does 0.1% conversion rate bump qualify as increase? It would’ve been even better if it set target conversion rate increase, and a guesstimate is perfectly fine here (more you guess, better you will get at it):

“If we change Checkout button color to green checkout page conversion rate will go up by 15%”.

There’s our hypothesis, now let’s set up the test. If you’ve never done this before, it’s a 4-step process and it is probably a lot easier than you think it is. What you need to do is:

1. Create an experiment

Go to your Google Optimize account, click on your container name and go to Experiences page. Click “Create Experience” button, give it a descriptive name, select a URL it will run on and select “A/B test” as experience type.

2. Create and configure a variant

A variant is a different version of the page you’re testing. Once you’ve created an experience, open it and click “Add Variant”. Give the variant a descriptive name. “Updated Checkout page” is bad, “Green Checkout button” is good.

After you’ve created the variant, click Edit button to open the visual editor. Simply click any element on the page and change its properties, or add your own custom CSS or JavaScript code that will load every time this variant is shown.

To learn more about the visual editor, check out Google’s official help page – https://support.google.com/optimize/answer/6211957.

After you’re done creating your variant(s) you need to set up weighing rules. This is nothing more than traffic distribution. You can either let Optimize split the traffic equally across all variants (default option) or set up custom percentages of traffic to be delivered to each variant.

3. Set up targeting

For simple A/B tests, especially when you’re just getting started you will probably want to keep it simple and target 100% of traffic. Once you’re more experienced you’ll probably want to set some targeting rules.

If the site has high traffic volume, perhaps there’s no need to run the test on every single visitor. You can change Traffic allocation setting in experience settings section to less than 100% because running a test comes with a cost (in page loading times) even when you’re using free tools.

In addition to traffic allocation, you can also configure your experience to run only if certain conditions are met. There are more targeting rules than you’ll ever need, and you’ve probably already configured one when you picked a URL on which your test will run. Here are all the rules available as of January 2019:

Google Optimize targeting rules

4. Start or schedule your experiment

Once everything is configured click Start button or Schedule icon next to it to start the test.

And that’s it. Once you have the testing environment set up, it takes no more than a few minutes to create an A/B test. Deciding what to test will take a lot more time and it’s something we will cover in a future post, but at least the “it’s too complicated” argument is out the window.

Happy testing!