The Art of Ad Testing
Given the high cost of ad production and media, it’s no wonder marketers are keen to ensure their ads will perform successfully before investing millions in producing and running campaigns. According to Nielsen, creative quality accounts for nearly 50% of the sales lift from advertising.
There are multiple methodologies to test ads. The focus here is on qualitative research, which we find to be more helpful in optimizing ad development at the concept stage vs. quantitative research, which focuses more on rating ads.
The team that developed Real Talk included people that had lots of experience in ad agency (creative, strategy, and account management) and client-side marketing roles as end-users of ad testing research. We’d collectively watched hundreds of focus groups and interviews. Sometimes, there were juicy insights that helped validate an ad’s power or uncovered opportunities for the creatives to explore to make it stronger. But, often, we found the feedback to be either misdirected or useless.
Our team wanted to find an approach for ad evaluation that delivered more useful and reliable results. This quest included analyzing our own past experiences at ad agencies, brands, and marketing research firms and consulting with other researchers, marketers, creatives, strategists, and academics. We discovered that the problem is less about the tools and more about how artfully they’re applied. In the process, we developed six tenets for designing and conducting more effective qualitative research on ad concepts.
Talk to the right people
Proper recruiting is the difference between success and failure. Qualitative research will provide little to no value if you’re not talking to the right people. Garbage in, garbage out, as the saying goes. But making sure the people you talk to meet your demographic, psychographic, and purchasing behavior requirements is only the first step. Screening for articulateness, thoughtfulness, and engagement is equally critical. If you want interesting insights, you need to talk to people who are willing and able to express themselves honestly, clearly, and colorfully.
Use consumer-friendly stimulus
Ad agencies initially create concept boards and storyboards for sophisticated marketers familiar with how concepts go from idea to finished ad. They can include insider jargon about photography styles, camera directions, editing techniques, special effects, and typography treatments. Replace these with simple language and descriptions that people who aren’t in marketing can readily understand before they’re put into research. Fill in gaps. Paint a clear picture. This is essential for getting accurate feedback on the idea, and it’s surprising how often it’s overlooked.
Thoughtful design + artful moderation
Like when you watch or read a great interview, the skill of the interviewer or moderator determines whether it’s boring or fascinating. How thoughtful and engaging are the questions they use and the flow of the conversation? Do they pick up on cues expressed by people and dig deeper to get to the good stuff? Empathy, intuition, creativity, and meticulous preparation are essential. The principles are the same for qualitative research. The quality of the results comes from a combination of thoughtful design and artful moderation.
Focus on the strength of the messaging more than the creative execution
The stimulus used to present ads in concept form can’t convey the production value of the finished ad. Photography, cinematography, art direction, actors, music, editing, animation, and special effects contribute tremendously to how an ad is perceived. Since there’s no way to adequately portray the impact of all those elements, they shouldn’t be a focus in the research.
However, by asking the right questions in the right ways, you can determine how well the intended message is coming through. At a high level, you can also understand if the overall creative premise (not the executional details) is working.
It’s important to focus on initial reactions and feelings first - System 1 thinking. What got their attention? What’s the key takeaway from the ad? How do people react to that? Does it change the way they feel about the brand or product? Is it unique? Does it fit with the brand? What was memorable? Does it spark interest or action? Then delve into why and why not - System 2 thinking. Probing and interpretation will be crucial since people can’t always explain their feelings accurately.
Search for emotional connections
Even in a concept form, ads with a strong message and creative premise will elicit emotional responses. And the strength of those responses can be a bellwether for how the ad will perform. Ads with emotional appeal are proven to perform significantly better, particularly if they arouse interest, excite, or inspire action. Those are good indicators of engagement, memorability, sharing potential, increased consideration, and purchase intent.
Don’t be afraid of a bit of polarization. If a significant number of respondents have a strong positive emotional reaction to an ad concept, but a few people have an adverse reaction, that’s not necessarily a bad thing. Some of the most creatively awarded and sales-boosting ads we’ve worked on generated this kind of reaction in qualitative research. Better to have a strong positive reaction among some than a lukewarm reaction from all. All great advertising has some degree of risk, and insightful research can build confidence to take risks.
Seek to inspire
One of the biggest shortcomings of qualitative research on ad concepts is that it’s often more critical than inspirational. That’s why the mere suggestion of it makes a lot of ad agency creatives cringe.
Research findings should be structured to inspire the marketing and creative team on how they can make their ideas for ads as interesting and compelling as possible. Reveal opportunities and impediments that weren’t apparent. Illuminate reactions and possibilities to consider. Illustrate why particular concepts rise to the top. Provide thought starters.
At its worst, qualitative ad testing assumes the role of the imperious emperor giving thumbs up or down based on the cheers or jeers of the crowd. That’s why it can get a bad rap. At its best, qualitative research can be an artful advisor that offers new insights for those developing the ads to consider in their pursuit of creative excellence and in-market success. Done right, it’s like getting insider information before you make an investment, except it’s entirely legal.