# A/B Testing - Interview Questions

A/B Testing (also known as Split testing) defines an approach to compare two versions of an application or a website page that enables you to determine, which one performs better. A/B Testing is one of the simplest ways, where you can change an application or a web page to create a new version and then comparing both these versions to find the conversion rate. This also lets us know, which is the better performer of the two.

The number of samples depend on the quantity of tests performed. The count of conversion rate is called a sample and process of collecting these samples is called sampling.

Confidence interval is called measurement of deviation from the average on the various number of samples. Let us assume that 22% of people prefer product A, with +/- 2% of confidence interval. This interval indicates the upper and lower limit of the people, who opt for Product A and is also called margin of mistakes. For best outcomes in this average survey, the margin of error should be as small as possible.

Always perform A/B Testing if there is probability to beat the original variation by >5%. Test should be run for considerable amount of time, so that you should have enough sample data to perform statistics and analysis. A/B Testing also enables you to increase maximum from your existing traffic on a web page.

The expense of increasing your conversions is minimal as compared to the cost of setting up the traffic on your site. The ROI (return on investment) on A/B Testing is huge, as a few minor changes on a website can result in a significant increase of the conversion rate.

Like A/B Testing, Multivariate testing depends on the same mechanism, but it compares higher number of variables, and gives more information about how these variables behave. In A/B Testing, you split the traffic of a page between various versions of the design. Multivariate testing is used to measure the effectiveness of each design.

The issue with testing multiple variables at once is that it would be difficult to accurately determine which of these variables have made the difference. While you can say one page performed better than the other, if there are three or four variables on each, you can’t be certain as to why one of those variables is actually a detriment to the page, nor can you replicate the good elements on different pages.

Here are a few A/B Testing variations that can be applied on a website page. The list includes − Headlines, Sub headlines, Pictures, Texts, CTA text and button, Links, Badges, Media Mentions, Social mention, Sales promotions and offers, Price structure, Delivery options, Payment options, Site navigations and user interface.

• Background Research − First step in A/B Testing is to discover out the bounce rate on your website. This can be done with the help of any tool like Google Analytics.

• Collect Data − Data from Google Analytics can assist you to discover visitor behaviors. It is always advisable to collect enough data from the site. Try to discover the pages with low conversion rate or high drop-off rates that can be improved.

• Set Business Goals − Next step is to set your conversion goals. Discover the metrics that determines whether or not the variation is more successful than the original version.

• Construct Hypothesis − Once the objective and metrics have been set for A/B Testing, next is to find ideas to improve the original version and how they will be better than the current version. Once you have a list of ideas, organize them in terms of expected impact and difficulty of implementation.

• Create Variations/Hypothesis − There are many A/B Testing tools in the market that has a visual editor to make these improvements effectively. The key decision to perform A/B Testing successfully is by selecting the right tool.

• Running the Variations − Present all variations of your site or an application to the visitors and their actions are monitored for each variation. Visitor interaction for each variation is measured and compared to determine how that variation performs.

• Analyze Data − Once an experiment is finished, next is to analyze the results. A/B Testing tool will present the data from the experiment and will tell you the difference between how the different variations of web page is performed. Also if there is any important difference between variations with the help of mathematical methods and statistics.

The most common type of data collection tools includes the Analytics tool, Replay tools, Survey tools, Chat & Email tools.

Replay tools are utilized to get better insight of user actions on your site. It also permits you to click maps and heat maps of user click and to check how far user is browsing on the website. Replay tools like Mouse Flow permits you to view a visitor's session in a way you are with the visitor.

Video replay tools give deeper insight into what it would be like for that visitor browsing the different pages on your site. The most commonly used tools are Mouse Flow and Crazyegg.

Survey tools are utilized to collect qualitative feedback from the website. This involves asking returning visitors some survey questions. The survey asks them general questions and also permits them to enter their views or select from pre-provided choices.

You can reduce the number of bounce rate by adding more pictures at the bottom. You can add links of social sites to further increase the conversion rate.

There are various types of variations that can be applied to an object like using bullets, changing numbering of the key elements, changing the font and color, etc. There are numerous A/B Testing tools in the market that has a visual editor to make these improvements effectively. The key decision to perform A/B testing successfully is by selecting the right tool.

Most commonly available tools are Visual Website Optimizer, Google Content Experiments and Optimizely.

Visual Website Optimizer or VWO empowers you to test multiple versions of the similar page. It also contains ‘what you see is what you get’ (WYSIWYG) editor that enables you to make the changes and run tests without changing the HTML code of the page. You can update headlines, numbering of elements and run a test without making changes to IT assets.

To create variations in VWO for A/B Testing, open your website page in WYSIWYG editor and you can apply many changes to any website page. These include Change Text, Change URL, Edit /Edit HTML, Rearrange and Move.

Visual Website Optimizer also gives an option of multivariate testing and contains other number of tools to perform behavioral targeting, heat maps, usability testing, and so on.

These tests can be applicable on few other places like Email, Mobile Apps, PPC and CTAs as well.

Once an experiment is completed, next is to analyze the outcomes. A/B Testing tool will present the data from the experiment and will tell you the difference between how the various variations of that site page are performed. It will also show if there is a important difference between variations using mathematical methods and statistics.

To integrate Optimizely to Universal Google Analytics, first select the ON button on the side panel. Then you must have an available Custom to populate with Optimizely experiment information.

The Universal Google Analytics tracking code must be placed at the bottom of the <head> section of your pages. Google Analytics integration won't function properly unless the Optimizely snippet is above the Analytics snippet.

Google Analytics has two options for analyzing the data, which are Universal Analytics and Classic Google Analytics. New Universal Analytics features permit you to use 20 concurrent A/B tests sending data to Google Analytics, however the Classic version allows only up to five.

This is a myth that A/B Testing hurts search engine rankings because it could be classified as copy content. The following four ways can be applied to ensure that you don’t lose the potential SEO value, while running A/B Tests.

Don’t Cloak − Cloaking is when you show one version of your webpage to Googlebot agent and other version to your website visitors.

Use ‘rel=canonical’ − When you have A/B Tests with different URL’s, you can add ‘rel=canonical’ to the webpage to indicate to Google which URL you want to index. Google recommends to use canonical element and not noindex tag as it is more in line with its intention.

Use 302 redirects and not 301’s − Google recommends to use the temporary direction method − a 302 over the permanent 301 redirect.

Don’t run experiments for a longer period of time − Please note that when your A/B Test is finished, you should eliminate the variations as soon as possible and make changes to your webpage and start using the winning conversion.

Input your Topic Name and press Enter.