A Step-By-Step Guide To Product A/B Testing (The Right Way)

Basically, A/B testing also known as ‘split testing’ is comparing 2 different versions of the same product or service. The one which is more suited or the winner, is pushed ahead for the launch.

However, there is more to “how to do A/B testing the right way”. Get well versed with the mini “how to” guide below:

How To Do A/B Testing?

Before we explore the 5 elements of how A/B testing is done, here’s a relevant example to simplify your understanding:

Let’s consider an example of Gaurang, an online business owner. He wants to launch a new product in the market. He has 2 different versions of the same product and he is puzzled as to which one is the best for this audience. Here’s what Gaurang or you or any business should do in order to attain absolute clarity:

1. Draft 2 beautiful and separate product launch alert emailers
2. Divide the existing database (full or partial) into two equal groups
3. Use marketing tools like CleverTap or the one which provides A/B testing
4. Send emailers at the right time (Tuesday: 3-5pm and Thursday: 10-11:30am) to the selected database (accordingly) via these tools (do not forget to add trackable links)
5. Track real-time results. But, wait for a while as all group members will not open or click on the email at the same time
6. After a good time span of 4-5 hours, check the collective results
7. Now choose the one which has got the highest clicks through rates (CTR). Do not focus entirely on the Open Rate, as people open emails but hardly end up clicking on it. Also, make sure the content is strong for better CTR
8. Now Gaurang knows/you know which product version was liked by the existing audiences. Also, being in a similar horizon, increasing new users is a definite.
9. Launch the product!

Here’s the step-by-step guide of how A/B testing is done:

1. Before Testing, Plan It Out

Just the way Gaurang had a clear-cut goal before he set out, it is important to know the reason why you want to do A/B testing. A/B testing is to understand what works and why. Hence, it is a good idea to have all your database groups, product version, and other tools ready.

1. Start by studying your website data and venture into problematic areas of your conversion funnel
2. Use tools that analyze visitor’s behavior to find out why visitors are not getting converted into customers
3. Now, with the customer insights, come up with a way to increase conversions like making the CTA button larger
4. Also, select a good date range to collect accurate data of webpage or email performance.

2. What To Test

Some of the variants that affect the overall customer behavior (with regards to content A/B test):

• Headline: Is it catchy and inspiring users to click?
• Copy: It needs to be crisp, short and relative to the product, website or service being tested
• Testimonials: People connect with people and not companies. Gather motivating testimonials of existing users for the potential users
• Images: Are they optimized, well fitted and attractive? If yes, then you have passed this test!
• Call to Action (CTA): A compelling CTA is the source of signups along with the above elements. Make sure your CTAs are mentioned at least 2-3 places in the entire page.
• Links: Check if your links are trackable. Tracking makes sure that you are aware of how many people clicked and where.

Choose the variant that you want to test (for example CTA) and zero it down to what feature you want to test (Size, Color, Text or Placement).

3. Split Your Audience

The way your audience is split is based on what you are testing. If you are A/B testing emails, then you can control who your audience is. You will get an accurate result within a short period when your audience is split into two groups equally and randomly. But with a webpage, you will have to run the test for a considerable period to get an accurate result.

Pro-tip: At a time run only one A/B test. This measure will prevent confusion of which test is driving traffic or conversions.

4. Run The Test Simultaneously

Make sure you run the tests parallelly to achieve healthy comparison results. Avoid running tests separately or on separate days as the basis of comparison might not be valid ( eg: today is a good day to send emails then tomorrow)

5. Determine The Testing Period

Anything good takes time and running the test for a considerable period will give you an apt result. The time to run A/B test will vary from one organization to another, based on the number of people who are involved in it. Run any A/B test at least for a period of 2 business cycles. This is because when you stop a test in between many times you might jump to a false conclusion and thereby ruin the reason for the test.