QA Testing Your Experiments Before Putting Them Live

Introduction

The success of your experiments is firstly influenced by whether they are set up properly or not. Therefore, a rigorous Quality Assurance (QA) process is needed before launching any new experiments.

There are several things that you can do to make sure your experiment will display and function properly that are described below.

Staging Environment

If you have separate environments, use a staging environment to set up the experiment and perform initial QA without exposing the experiment to visitors. Then, duplicate the experiment into your production environment project, and perform the same QA process there. When possible, ensure that your production and staging environments match so experiments function the same way in both.

If you don’t have a dedicated staging environment, don’t worry. You can still QA your experiment. Perform all the steps outlined below, but take special care when you use the test cookie to QA in a live environment.

Live Preview

Do a live preview of your variations, which can also be done when your experiment has Draft status.

Make sure your pages render as expected in the most popular browsers for your business. Also make sure that any button, link, or form still works properly. Certain options in the editor could break forms, links and buttons if they are not used properly; see http://support.convert.com/hc/en-us/articles/205160385-Why-Are-My-Pages-Showing-Content-from-the-Page-Where-I-Set-Up-the-Experiment-

To find out how to preview a variation, read the following article: https://convert.zendesk.com/hc/en-us/articles/204506649-How-Do-I-Preview-Variations-

Specific Audience

Place your experiment in "Active" status for a very specific audience that only you can meet the conditions for, and therefore it will run just for you.

To do that, you can create a new Audience rule similar to the following: 
Medium matches exactly qa_test 

You can replace "qa_test" with whatever you like; just replace it with your preferred text in the Audience rule.

 

Now to test this, open a fresh Incognito window in your browser.  Do not open a second tab and make sure you have no other Incognito windows open.  Make sure that you close the window when you are done with the current test and open a new window for each new test. *This is essential.

Before visiting the site, add the following to the end of your URL:
?utm_medium=qa_test (replace qa_test with the value you chose when you created the Audience).

Most of the time you would visit the URL that matches the Site Area conditions.  Depending on how your experiment is structured, in some cases you may start out by visiting a different page and then navigate to the Site Area that triggers the experiment to run.  In either case, make sure you add the query parameter to the very first URL you visit.  This parameter will be remembered even if you navigate to different pages.*

For example, if the URL is http://www.mysite.com, you should visit the URL:
http://www.mysite.com?utm_medium=qa_test

If you follow the above instructions, you will be shown any variation in the experiment, as this is selected randomly.

After you have done the above, and if you didn't already start with the URL that matches the Site Area conditions, you can then visit the URL that matches the Site Area. You should then be viewing either the Original or one of the Variations, depending on which is randomly selected by the experiment.

 Important

Force a specific variation

You can force a specific variation, by using certain query parameters. 

Example: http://www.mysite.com/test-page.html?_conv_eforce=123.678 where 123 is the experiment ID and 678 is the variation ID;

You can find a more detailed explanation here.

 Important

Combining Parameters

You can also combine query parameters.  For instance, if you want to do a qa test but also want to force a specific variation, you would append the query parameter to the end of the previous parameter with an ampersand (&) between them.  Only the first parameter would have the question mark at the beginning, ie the below example shows both the force variation parameter and qa_test Audience condition paramter combined:

http://www.mysite.com/test-page.html?_conv_eforce=123.678&utm_medium=qa_test

Tip
Tip
Tip

If you are experiencing problems, for instance issues with click goals, please refer to this article for debugging

Have more questions? Submit a request

Comments