Daddy or chips? A/B Testing to Make the Right Ad Copy Decision

Remember the sweet little girl contemplating one of life’s toughest decisions in the iconic 90s television ad for McCain Oven Chips? Here’s the ad if you're wondering what I’m going on about.

Daddy or Chips? A classic piece of advertising from the 90s

Daddy or chips?

As the advert shows, making decisions can be difficult; and the more people and opinions added to the process, the harder the decision becomes. You believe that putting 10% discount messaging in your ad headline will produce a higher CTR than in description line 1, but your client thinks otherwise. You feel that a short and punchy description line 1 and 2 will be more engaging to potential customers than utilising the entire character length, but your colleague disagrees.

More often than not, we choose actions and form opinions which are heavily influenced by emotions, memories and bias, but what if there were a superior way to better inform our decisions?

Fortunately, a more scientific method known as A/B Testing (occasionally referred to as split testing) takes the guesswork out of making these crucial decisions, by allowing you to put multiple ad choices in front of your audience to determine which ad customers find the most engaging, and thus which boost CTR.

So let’s get into the nitty gritty of A/B testing.

What Is A/B testing and why is it SO important?

A/B ad testing is the process of conducting an experiment where you test two different versions of an ad simultaneously. In PPC marketing, when you only have a limited amount of time to separate yourself from the competition in SERPs, it is vital to constantly be testing ad copy.

Chips, chips and more chips

By using systematic testing to collect real-world data, you can shed some light on which messaging works best for your brand and the products you sell. When you discover that the difference between putting your percentage discount in the headline and description line two is a CTR increase of 20 percent or more, while your quality score increases and your cost per click decreases, the thought of not using A/B testing becomes scary.

What are your goals in A/B testing?

Before an A/B test can be put into motion, knowing your desired outcome is essential. Arrange a meeting with your client to discuss the goals of your PPC campaign. This discussion should cover everything from USPs, competitor copy and special offers, to calls to action and message tone in compliance with brand guidelines.

Once your goals have been agreed, you can tailor ad copy tests towards the goal and have something concrete to measure success. At this point I recommend creating a Google doc with the ad copy you are looking to run, with columns that include the test stage, expected start date, messaging and test type, which you can share with your client for approval. This is beneficial for adding a level of transparency, reassuring clients that you have a solid action plan and for keeping yourself on track.

What can you actually A/B Test?

In PPC marketing, there are four basic components that you can change to create new ad variations: the headline, description lines 1 and 2 (also known as body text) and the display URL.

Once you have decided which variables you want to test, the next step is to list potential product features, CTAs and test ideas. When you have completed your brainstorm, weigh up your suggestions against your goals and delete where necessary. Once approved by your client, you will then have a solid list of CTAs and product features that you can combine to begin ad copy testing. I recommend creating a testing schedule and giving each ad a specific label. Here’s one I made earlier.

Bonus: Apart from changing the headline, body text and display URL to create new ad variations, there are a few additional things that can be tested. Don’t be afraid to be creative, extend your testing to include ad extensions, english vs. native language, location in headline vs no location, dayparting and more.

The 7 golden A/B test rules

Although there are endless variables that you can A/B test for PPC ads, there are a few vital rules to remember to keep you on track:

1. Don’t forget your goals - Despite outlining the importance of goals earlier in the post, they are so important I must stress it again! In order to be successful, knowing what success looks like is paramount. Select your quantifiable metric(s) and stay true to them throughout the test.

2. Test one thing at a time - When A/B testing ad copy it is very easy to become overzealous and throw multiple variations into your test at once, which can kill a test before it’s even begun. Fight the urge to do so and methodically work through testing one variable at a time. By doing so, you will know that when one ad outperforms another it is near certain that is is due to the variable that you have changed.

3. Run your test for an appropriate amount of time - It is important not to run your A/B test for too short a period of time, but it is just as important not to let the test run too long. If you do the former your results may produce inconclusive results, while doing the latter might mean that you miss the window for implementing real change.

The best way to ensure that your test comes to a natural end where a data-driven decision can be made is through statistical significance. Statistical significance is achieved when your ad copy test has matured and is accurately informing you which ad is better. This confidence level is usually set at 95%, signifying that there is a 5% chance that your test results are coincidence. For ease, some bid management platforms provide built in statistical significance calculators, however these calculators are readily available online - Cardinal Path has a great statistical significance tool, which gives you the option to change the confidence level and has the added benefit of showing the absolute difference between two ads’ CTR and CR, and points out if further testing is needed for clear cut results.

4. Use the correct ad settings - To set up a fair A/B test, ad impressions need to be divided evenly between your ads. Failing to do so could mean that your new ad may not even show.

To help ensure that the new ad variation gets a fair chance of showing as frequently as existing ads with history, (in Google) click the campaign settings tab then go to the adgroup which you are going to be testing is located and update the setting to ‘Rotate Indefinitely’. Bing also provides a similar setting to ‘Rotate Ads More Evenly’.

5. Do not make a golden ad for a bronze landing page - So you have found out which variables improve your CTR and have now created ‘the golden ad’. Why follow this up with a less than golden landing page? Make sure that your potential customers’ expectations are met by doing/offering/presenting what you have outlined in your ad, on the landing page. Don’t make them work hard for it, or you could lose your visitor just as quickly as you gained them. Where your ad sends visitors can have just as much impact on conversion as the ad itself.

6. Eat, sleep, rave, repeat -  Once you have completed your first round of testing and paused your underperforming ads in favour of the winner, your job is not done. The next step is to mark down the results of your test in your Google doc and move on to testing the next variable. It is essential to continuously test new ads in order to stay ahead of your competition.

7. Be patient -  It is unlikely that you will improve in leaps and bounds each time you run a test. Success can also be found in knowing what does not work for your business, as the point of testing is to learn and improve. As Benjamin Franklin said:

“I didn't fail the test, I just found 100 ways to do it wrong.”

Final thoughts

To wrap up for those of you who looked at the post and thought TLDR (too long didn’t read), A/B testing is an integral part of PPC success.

Test results can be seen quickly at a minimal cost, providing invaluable real-world data about how potential visitors respond to different messaging. Creating one great ad is a fluke, but consistently creating engaging ads is an art which takes careful planning, vigilance for competitor strong points and a methodical testing schedule, which eliminates the feeling to ‘go with your gut’ or stalling on the testing process.

When you know which copy provides quantifiable results, it becomes easier to make better decisions, better advise clients who have a ‘gut feeling’ about copy and create an efficient strategy for future campaigns. Take full advantage of the options available to you when A/B testing, and remember to keep testing as what works today may not work in a few month’s time.

And by the way, it’s definitely chips...

Get blog posts via email

About the author
Ore Oduwole

Ore Oduwole

Ore joined Distilled in March 2015 as an analyst.  With a background in paid advertising she now makes up part of the London consultancy team. Born and bred in South-East London, Ore attended the University of Sheffield; graduating with a degree in...   read more