Ad creative testing: How to identify your best ads and win the attention game

If there’s one thing I’ve learned in the past decade guiding performance marketing teams, it’s this:

Your creative concept is the single biggest differentiating factor between great and disastrous advertising effectiveness.

For instance, Nielsen has studied this and found that creative contributes 47% of an ad’s effectiveness, followed by reach (22%) and brand (15%).

That’s a big gap, and exactly why you need a consistent ad creative testing process to identify what works, what doesn’t, and how to optimize your performance. Let’s take a look at the challenges and best practices, so you can get creative testing right.

Key takeaways

1. The battle for attention is real, and it doesn’t show signs of slowing down.
2. Do creative testing. Start with a hypothesis, look at the data, and analyze your results.
3. Data only tells you what happened and which ads worked. It’s your responsibility to understand why.
4. Knowing which creatives work and why will put you ahead of the majority of advertisers competing for the same eyeballs and clicks.
5. Structured creative testing and unified data views won't only help you but also enhance collaboration between performance marketing and the creative team.

Why should you care about creative testing, and why now?

We can’t escape the attention economy. Everyone in your audience gets exposed to hundreds if not thousands of ads each day. Standing out from that mass should be your primary goal with every campaign.

Now, there are basically three main components that affect advertising effectiveness:

The brand: You either have strong recognition or don’t, but don't think that a strong brand will make any campaign an automatic success. You’re playing in the same mass of ads as everyone else.

Media choices: AI and automation have made media buying so efficient, that there are only incremental gains to be found. This doesn’t remove the fact that it’s super important to reach your audience with the right message in the right channel.

Creative concept and ad creative design: Humans are visual creatures, ultimately driven by emotion. A strong concept executed well will stand out in any channel and regardless of brand awareness.

The last one is what we’re after with creative testing: killer ads that can’t be ignored!

Additional resource: Read how ChatterBlast builds brand awareness with data-driven creative testing

4-step framework for powerful ad creative testing

The fun thing with testing is that you never know in advance what you'll discover.

This, of course, sets a limit to how detailed of a template you can have for testing ad creatives. With that in mind, here’s roughly the process you should follow to run tests that take you towards better ad performance:

Step 1: Define your hypothesis

Do not start testing without first deciding what you’re testing for. Your hypothesis defines what success looks like, and how to pick the winning ad creative in the end.

An example hypothesis could be: “Adding a quick summary at the beginning of a video ad increases average view duration” or “Including speaker faces in our event ads increases signups”.

Step 2: Create test variations and build the campaign

This is basic asset creation, guided by the chosen hypothesis. It’s usually best to test only one variable at a time to get reliable results. Use ad creative guidelines for different platforms as a starting point, but remember that following best practices isn't always the way to grab attention. Try to be flexible and experimental. For example, if you notice interesting trends or creatives on social, it may be worth it to replicate them for your brand.

When building your test, make sure you isolate the variants: set up an equal budget for each version and mitigate audience overlap to pinpoint the winning creative.

Step 3: Run the test and interpret the signals

I won’t go deep into A/B or multivariate testing here. Remember to treat ad performance data as signals, and reflect it with what you’re trying to find out.

One important consideration is to look at the metric you’re trying to optimize for: if version A gives you higher engagement, but the goal is to maximize reach, then you need to think twice before concluding that A is actually the best-performing creative.

Take a deeper look at how to analyze paid ad performance

Step 4: Scale, test again, or move to another hypothesis

Depending on your results, you may want to continue testing the same hypothesis or move to the next one. There’s no creative test that will give you absolute truths. Reporting is key in making sense of your test results and sharing them with your team. Build simple yet powerful reports for PPC, social, and more, that visualize performance by creative variation, and surface insights that support your next hypothesis.

Data is here to work for you, but it won’t make the decisions for you. At the end of the day, it’s up to you to decide what’s a sufficient level of confidence for picking a winning creative and moving on. Annoying? Kind of. That’s just the nature of testing highly subjective things like ad copy and visuals.

Why go through all this trouble, and not just rely on your own best guess?

You can never know for sure what made someone prefer an ad or engage with it. The thing is, without data, you’re just guessing. Robust ad creative testing and analyzing the data in one view will go a long way.

Moving from intuition to structured testing is a necessary shift, if you’re looking to make data-driven marketing decisions.

Bonus: And how to do it wrong

Perhaps you already added these up from the previous chapter, but it pays to mention the common pitfalls of creative testing, anyway:

1. Testing creatives without a predefined hypothesis
2. Testing for too many variables at once (e.g. color, copy, with or without cute kittens)
3. Giving the ad platform’s algorithm too much control over testing
4. Failing to isolate test variants (audience, budget)
5. Not matching test scale (budget, audience) with the expected results
6. Taking signals from the data as absolute truths
7. Jumping from one test or creative concept to another without a plan

I said it in the beginning: there’s a hundred ways to fail. You will always get some result, but the right process and tools make your findings a lot more plausible.

Additional resources: Hear how data influences creative decisions at SnapShot Interactive

2 examples of creative testing and optimization using Supermetrics

The world is your oyster when it comes to creative testing and optimization platforms. Having seen many different setups, I can help you narrow down the list with one key standard:

Your creative testing platform must bring the data into one view, or you’ll spend most of your time trying to connect the dots.

That’s what Supermetrics does particularly well. You can easily combine data from organic sources, paid ad platforms, and analytics tools into one view. This allows you to compare those ad creatives directly and objectively in terms of ROAS, sales, and other metrics.

AI image analysis

Using AI, you can analyze different visual elements and see how they impact your performance. For example, which creative drives better CTR—light vs. dark background?

What you’ll see on the dashboard:

• Performance breakdowns: Tables comparing campaign performance based on AI-detected properties, such as whether images contain people or are dark vs. light
• Account performance overview: A chart showing your account's cost and reach over time
• Detailed creative list: A table of your campaigns and their specific ad creatives, sorted by performance metrics like click
• AI image labels: A list of the specific AI-generated tags, for example, cloud, sky, mountain, associated with each ad creative, allowing you to identify trends in visual content

To build this, you can use the Custom Fields feature in Supermetrics.

In the Marketing Intelligence Platform, choose “Transform” >> “Custom Fields”. Select the data source or platform you want to analyze your creative assets.

Then choose the function “Custom image analysis”. And then define your prompt. For example, in this case, our prompt is:

Is it dark outside or light outside in this picture? Please only respond with "Dark" or "Light."

Once you’re done, click ‘Save’, and now you can start using this custom field in any reporting tools, like Looker Studio or Power BI.

Creative fatigue monitoring

When your audience has seen the same ads too many times, they’ll eventually become bored and unresponsive. If you don’t detect creative fatigue early enough and refresh your creative, your CTR and conversions will drop.

For this report, I recommend using a data warehouse so you can benchmark recent data with historical CTR, and then feed your data into Looker Studio for visualization.

Step 1: Get your creative ad data into a data warehouse

You'll create two essential tables:

1. Performance data table: This is your main, time-based log. It should contain daily performance metrics for each ad, including:
   • Ad name & ID
   • Date
   • Impressions, clicks, cost, conversions
2. Creative snapshot table: This is a simple lookup table that connects an ad's ID to its visual. It does not need a date field, as you just need one static record for each ad creative.
   • Ad/creative ID
   • Ad/creative URL (the link to the image or video)

Step 2: Transform your data in the warehouse

This is the most critical step and the primary reason a data warehouse is needed. Here, you'll use basic SQL to prepare the data for analysis.

1. Create rolling windows: The key to detecting fatigue is comparing recent performance to the overall average. To do this, you'll calculate rolling 7-day and 30-day totals for cost and conversions for each ad. This means for every single day in your data, you'll have a corresponding value for the previous 7 and 30 days.
2. Join your tables: Perform a simple join to combine your two tables using the Ad ID. This enriches your daily performance data with the corresponding creative image URL for every row.

Step 3: Connect your data to Looker Studio

1. Connect your data: Add your newly transformed table from the data warehouse as a data source in Looker Studio.
2. Calculate final metrics: Create the following calculated fields directly in Looker Studio:
   • CTR = SUM(Clicks) / SUM(Impressions)
   • CPA (Lifetime) = SUM(Cost) / SUM(Conversions)
   • CPA_7d = SUM(7-Day Cost) / SUM(7-Day Conversions)
   • CPA_30d = SUM(30-Day Cost) / SUM(30-Day Conversions)

Then, once your data is inside Looker Studio, you can start building your own dashboard.

A tip for analyzing creative fatigue: Look at its trend charts. A classic sign of fatigue is a CPA or CPC line trending upwards while the CTR line is trending downwards.

Why you should use Supermetrics for creative testing and optimization'

Playing with Supermetrics means you don’t need to rely on your data team. If you’re familiar with ad managers from Google, Meta, LinkedIn, and other popular platforms, using Supermetrics for creative optimization isn't going to give you any technical headaches.

Here’s a shortlist of Supermetrics features and capabilities that our users and I have found most useful in creative testing and optimization:

• On-demand data: Our dashboards fetch ad performance data from ad platforms all the time. This is handy because ad creative optimization is constant work!
• Easy-to-use interface and tools: Supermetrics is built for performance marketers, meaning you don’t need to be a developer wizard to operate the platform.
• Powerful dashboards: Connectivity is key when comparing ad creatives. Build one view that brings all the data together, so you can see the forest for the trees.
• Custom reports: Set up automated reports that show creative performance over time, grouped by theme, platform, or even visual elements.
• Visual previews: You can (and should) set your Supermetrics dashboard and reports to include a preview of the ad image or video. This can be done easily with ad image urls.
• AI-powered labeling: When you’re running lots of creative variations, you can label ads with AI (e.g., dark image/light image, or audience A/audience B) and use the labels to analyze performance.
• AI-powered text analysis: Another way to supercharge creative testing with Supermetrics’ AI is to analyze text; comments, reviews, feedback, sentiment, and draw recommendations for future ad creatives.

Often, performance marketers hold the data while creative teams build the ads. The problem is that the two don’t always communicate effectively. By using shared dashboards, teams can finally speak the same language. Creatives get visibility into what’s actually working, and performance marketers gain faster feedback loops for iteration.

Put these features together, and you’re more than well equipped to run ad creative tests and optimize performance on a single ad or campaign level. Plus, you can do all of it yourself!

Common misconceptions about ad creative performance

Here are some common attitudes and beliefs that we at Supermetrics debunk almost on a daily basis.

1. “The prettiest ad (and the highest production budget) always wins”

Actually, no. Many marketers are still today surprised to see that the best-performing ads are often unpolished. Such “normal” content tends to work because it doesn’t smell like advertising. This goes back to the point about grabbing attention: your audience has already grown numb to polished but unemotive ads.

What to try next? Record hand-held. Add a quick selfie. Break a pattern. Also, let’s finally leave boring-to-boring behind us. Most B2B ads could benefit from taking a look inside the B2C ad creative playbook.

2. “The data is clear: this is our winner ad”

Ad performance data is only part of the truth. Advertising platforms give you important signals about what seems to be working, but that’s not the whole truth. Avoid drawing too quick conclusions and reflect the signals with your hypothesis, goals, and audience.

Stop and ask yourself: is the data in line with what we’re trying to achieve with the campaign? Data will only ever tell you what happened. You need to figure out the why.

3. “Here it is: we’ve found THE perfect ad creative”

There is no ultimate ad creative. It’s a great feeling when you discover something that moves the needle, but don’t think for a second that the creative will perform best from now to eternity. Keep testing new interpretations of your creative concept. Scalability is another thing. The perfect creative for an Irish audience on Black Friday is exactly that: perfect for the specific time and place, but not others.

Constantly seek improvements within your chosen creative concepts, without zig-zagging between completely different approaches.

Working with ecommerce and paid social? Here’s a great example of scaling revenue with data-driven creative testing strategies.

Frequently asked questions about ad creative testing and optimization

How to approach creative pruning to avoid performance dips?

The key is to approach creative pruning with data, because otherwise… What will you clean up? When you have lots of ads live, you want to make pruning an ongoing process. Having one dashboard is super helpful and saves you a ton of time. Remember to consider platform differences: a creative that doesn’t work on Meta could be winning hearts on LinkedIn.

How to avoid or fix creative fatigue?

First, make sure it actually is creative fatigue that you’re looking at. If you have lots of ads running, the audience won’t experience fatigue that easily, so slow performance could well be about something else. There is unfortunately no one right answer to when creative fatigue occurs. When you suspect it, compare performance on a high level first. If that doesn’t show clear signs, dig one level deeper into certain ad types or individual creatives.

How to account for platform bias?

Platform bias usually shows up as an ad platform (say, Meta) favoring one ad creative before you have accumulated enough data to determine that it’s the best one. One way to combat this is to pay close attention to your campaign goals. If you buy video views, that’s what you will get. For clicks, another creative will be preferred. Make sure your hypothesis is in line with what you’re telling the platform to look for.

Which metrics should I look at when testing or optimizing ad creatives?

The important thing is to define success beyond just ROAS or CPA. Obviously, the right metrics depend on your marketing goals and channel role. Everything from awareness to engagement and ad sentiment can be a relevant metric on your performance report. If unsure, revisit your hypothesis and consider the best match.

What skill sets should I develop to become better at creative testing?

The first one must be an analytical mindset. Data will show you what happened, but you need to reflect on why that may be. Second, consistency in managing creative tests and changes to ads, so your results stay reliable. Third, it’s important to understand the basic theory and practice of A/B and multivariate testing. Finally, I find it really helps to have interest in the creative side as well.

Move on from creative guesswork

That’s hypothesis-driven ad creative testing for you! Put the process and tools in place, and you should be ready to:

• Run creative tests and optimize much easier with all data in one place
• Identify clear insights into which creative campaign elements resonate
• Achieve better results with each dollar spent on ads

This approach also shifts creative testing from a siloed process to a shared one. When both performance and creative teams access the same data view, they can collaborate on hypotheses, optimize faster, and plan new creatives with shared context..

Structure and data are at the heart of creative testing. Supermetrics gives you both.