A/B Testing Responsive Creative For Display Remarketing Campaigns

Rigzone is a global recruitment website focused on the Oil and Gas sector, with thousands of jobs onsite across the Americas, EMEA and APAC.  

Job category based display remarketing forms one pillar of our PPC marketing strategy for Rigzone, achieving strong performance through this campaign type with CPAs to rival some of our most efficient generic search campaigns. So, when Google announced the release of their new Responsive format for display ads, we were keen to refresh our text ads and put this new format to the test. 

Of course, making this switch wasn't possible without some creative assets. Rigzone provided two sets, and requested our advice on which was best to use. Our suggestion? Test both! Rather than basing the decision on hunches or personal preference, we decided to pit the two creatives against each other and let the data decide. 

Objectives

Our goal was simple. We aimed to determine which style of creative worked best whilst delivering applications at the lowest CPA, which is our main KPI for the client.

Strategy

Utilising exactly the same wording for each variation of the image, we set up a straight A/B test for both the Americas and EMEA region, with traffic dividing equally between the photo and the cartoon image.

Seems simple right? Not quite, unfortunately. Our KPIs are based on Floodlight data in DoubleClick Search, and this does not natively support Display activity.  As such, we had to make use of dummy keywords in order to report on the results, with an individual dummy keyword set up for each ad in the test. This allowed us to pull Display data into DoubleClick Search and utilise data from the Floodlight tags to determine the effectiveness of each set of creative.

The dummy keywords used contained either “cartoon” or “photo”, which allowed us to easily distinguish between the performance to the two types of creative, and to report efficiently on the results of the test.

Results

In the end, the winning image was clear. Photo ads performed significantly better than their cartoon counterparts:

  • We saw 51% lower CPA for the photo ads (versus the cartoon ads) in the Americas, and CPA was 26% lower with this image in EMEA.
  • In the Americas, CTR was 19% higher for photo ads and the same for both ad types in EMEA.

Quote from the client

"We were delighted with the results from the campaign, the data confirmed what we had suspected and will allow us to make informed decisions on the creative front moving forward."

Ashleigh Pirie, Marketing Manager, Rigzone

Download Case Study

Click on the image below to download the Rigzone case study.

Click here to download