Here is how you can read AB test results

AB test results

A/B testing allows individuals, companies and corporations to compare two different versions of a webpage or an app against each other in order to determine which one performs better. It is basically an experiment where two or more variants of a page are shown to the users at random and an analysis is used to determine which one performs better. It is important to learn how to interpret the results of the test in order to make an effective use of it. You need to prepare yourself for the test by answering the following questions.

Decide the goal: What is the goal for this test? You need to be very clear about the purpose of the test you are running. Determine your goal that you hope to achieve. Do you want revenue? Conversion or click through?

Decide secondary goals: Once you have identified the main purpose of running the test, you need to identify the secondary goals you want to achieve. It will prepare you to think through the test from the perspective of your users and will help you derive additional information through the test.

Number of visitors: There will be an average number of visitors on your test. If there are a very high or low number of visitors, it could mean that the test was not set up properly. Your analytics platform can provide an estimate on the number of visitors your test can get.

Will the test impact different audience segment differently?: You need to answer this question because the audience segment will be different and you will have to consider how this may be impacted by the actions you take after the conclusion of the test. For example, new users will not be able to compare an additional feature on your app. Hence, consider the audience segment when laying out the purpose of the test.

Answer all these questions before you sit down to read the results of the test. It will help you make sense of data and spot errors in the test setup.

Components that form a part of test results

To begin with, you need to understand the components that are critical in the interpretation of the results correctly. The first component is conversion rate which is the percentage of visitors that convert on the action. This is the most important component and will provide information about how far you have achieved your goal. Every goal has a conversion rate.  Next component is lift or improvement; this is what testers tend to care about the most. It is the percentage change in one variation as compared to another for any particular metric.

Visitors vs. visit is another component which should be well understood. Most platforms give users an ability to toggle between visits and visitors when comparing the results. A visit is a session while a visitor is an individual user. One visitor can make two visits to your website. Another important component is the number of days your test has been running. Majority sites run their test for a week which includes weekdays and weekends to negate any external factors that could have an impact on the test. Even if you have achieved statistical significance in a day, you should not end the test after only one day.

Making a decision on test results is way better than making a random decision. The statistics you notice are due to the variations you made in your website. You need to consider a range of external factors that have an impact on the audience when they appear for the test. However, the statistics may not be on point but it is better to make a long term decision based on the numbers. If you are well acquainted with the components that form the results of the test, you will be able to analyze it in a better manner. Correct analysis of the test is important to make the right decisions. You can gain additional information about reading the test results from While many are aware about the test, not many have the knowledge of interpretation of the results and it is important to ensure that they are reading the result in the right manner. Decide your purpose before you run a test so as to make sure that you are reading the results in the right manner. Once you answer the necessary questions, running the test and interpreting the results will become a quick and easy job.