3 Mistakes to Avoid When A/B Split Testing Your Website

The best way to learn is through action. With the power of split testing, we don’t have to guess what our customers want...we can see it in the numbers. We create a hypothesis, we test it, and we make it better.

Download the 22 Website Hacks To Drive Traffic, Leads & Sales Free eBook

What is A/B split testing?

A/B testing is a term for a randomized experiment with two variants, A and B, which are the control and variation in the controlled experiment. A/B testing is a form of statistical hypothesis testing with two variants leading to the technical term, two-sample hypothesis testing, used in the field of statistics. - Wikipedia

While it may seem challenging and never ending, using A/B split testing on our website allows us to continually have the most efficient site based on proven statistics and tests we conduct.

The hardest part is knowing all of the best practices and strategies to make this information valuable and valid. Here are three of our top most witnessed A/B split testing mistakes we see.

Impatience

In order to get a proper answer to your hypothesis, you must be patient. This is one of the most frequent mistakes we see when people begin split testing for the first time.

When you begin your tests, there is usually a clear “leader”within the first few hours. It is important to know that while you may start feeling confident that the early leader is the winner...wait it out!

Solutions:

• Standardize your sample size. Take into account your total monthly traffic and decide what you feel would be a good standard for sample size. If you get 4,000 - 5,000 visitors per month, a good sample size would be 2,000. If you get under 500 monthly visitors, perhaps 250 would be a good sample size. Choose a sample size and stick to it.
• Standardize your split test time frame and total conversions. If you find it hard to reach your sample size, set a time frame and conversion count to stop your ad. A usual standard is to stop your A/B split test when your campaign has been running for 2 weeks or has reached 100 conversions.

Testing more than one hypothesis

Closely related to impatience is testing more than one variable. Resist the temptation because this will skew your results when A/B split testing your website.

We understand that there are hundreds of possible variables to test on your website, but keep your head and focus on the most important changes first.

Solutions:

• Create a list of high, medium, and low testing priorities. Identify and test high impact variables first. Examples might include replacing your main website header graphic or rearranging your navigation.
• Keep a checklist or log. This enables you to see where you are in your priority list as you test, track results, and record results. This will also help you keep your current priorities in mind.

[Download your FREE copy of 22 Website Hacks to Drive Leads, Traffic, and Sales!]

Not taking content into account

One of the most common mistakes I see agencies and business commit is to only test website design. Rarely do I see content tested, perhaps due to the amount of content on a company’s site. It can be hard to know where to start.

Nevertheless, a/b testing your content has pivoted good websites into great websites in the minds of their customers. Not every business or niche speaks the same language. It is important to always keep in mind, “What do the users want to hear?”

Solutions:

• Break down your content testing. Begin by testing the first thing your users see on the page, which is usually the headline. After you find positive results, test the sub-header or paragraph content. Divide this text into sections to get an exact response to a singular change.
• Hire a content writer. Not sure how to speak to your customers? Hire someone who can. A content writer’s job is to learn, understand, and embody your customer while writing your content.

The Ultimate Solution

Keep testing, keep learning. The way to find the best method for split testing is to start doing it. No two websites are alike, and what works for one website may not work for another. There are hundreds of a/b testing case studies available online that show their hypotheses and results that you can replicate on your website.