What is AB Testing?
A/B testing also commonly known as Split testing is a process to compare two different versions of a web page or web applications with each other to understand the performance of the each version. This is a controlled experiment to determine the traffic on the web page or conversion rate for web applications like e-commerce site.
The original design of a page is usually referred to as the control. The new designs of the page are usually referred to as the “variations”, “challengers” or “recipes.”
Why should you perform AB test?
1. AB testing helps to improve the user experience with web elements
2. AB testing helps to keep the user on the web page and maintain the traffic
3. AB testing helps to take data proven decisions
4. AB testing helps to gauge the user preferences
5. AB testing helps to make changes with low risk
6. AB testing ROI can be increased with minor change
7. AB tests are fast to set up and run
8. AB testing helps to increase the business profits by increasing the conversion rates
How to perform AB Testing?
1. Choose an element or multiple elements on the page to conduct the A/B testing –
Some list of the elements that can be considered for the testing
1. Headline of the page
2. Banner image
3. Other images on the page
4. Position of the images and content
5. Call-To-Action and its position
6. Page layout
7. Forms on the page and its position
8. Font details
9. Color usage
10. Color contrast
2. Set the targets – Set the conversion goals to determine the success criteria of the page after the comparison of the two versions. The target could be anything to track the traffic on the web page with particular page layout or Video views on the page etc.
3. Define original and a variant – Define at least two versions of the webpage or application one should be original or unaltered version which is broadly known as “Control” and the other variant known as “Challenger” on which the A/B testing needs to be conducted.
4. Create a hypothesis – On the basis of targets set need to set the hypothesis which will help you define which version is the winner and why? A List of the point and its priority which will analyze the viewer’s interest for the particular page.
5. Decide the sample size – The process of calculating sample size is very complicated hence it is advisable to use. There are multiple calculator available in the market for which you need to have below mentioned details
a. Baseline conversion rate – is the current conversion rate for the page you’re testing. Conversion rate is the number of conversions divided by the total number of visitors.
E.g. A current conversion rate considered = 5%
b. Minimum detectable Effect – is the expected conversion rate for the page you’re testing
E.g. An expected conversion rate considered = 12%
c. Statistical significance – Statistical significance is a path of mathematically proving that a certain statistic is reliable. 90% is an accepted standard for statistical significance while conducting A/B testing, although you can change the same as per your requirement.
E.g. – Statistical significance considered = 90%
So with this example sample size per version will be = 20000
6. Run the test – Start the A/B test with the created versions and wait for a sufficient time period to generate the significant result
7. Result analysis and Conclusions – Analyze the created variants for the web page and check which page has the highest number of conversion rate and decide the winner page
Tools used for AB testing
1) Optimizely- Basic features are free
2) Unbounce – Paid
3) VWO (Visual Website Optimizer) – Free
4) Usability Hub (Five Second Tests) – Basic features are free
5) Kissmetrics – Paid
6) Google Analytics – Free
7) MockingFish – Free
How long should you run your AB Test?
1. The A/B test should not stop till the determined sample size has been reached.
2. Also, if you have reached the sample size within 2 days; do not stop for specific time minimum 2 weeks is the standard time to run the A/B test
