ARF Insights: How Important is Viewer Engagement? ARF Interview with Peter Daboll and Michael Curran of Ace Metrix

Researchers have long theorized the relationship between program engagement and advertising effectiveness, but these theories have seldom been tested. Peter Daboll, CEO ofAce Metrix, and Michael Curran, Ph.D. and Director of Insights and Analytics atAce Metrix, have tested the relationship between engagement and effectiveness, and their results may surprise you. We interviewed Daboll and Curran about their work.

ARF: Why did you feel it was important to explore the connection between viewer engagement and advertising receptivity?

Peter Daboll and Michael Curran: Recent media research suggests that you need to buy and measure media based on the engagement of the viewer within a particular program (i.e. that an audience of equal size and composition is somehow different depending on how engaged they are with the program). This notion, however, will fade - not because it isn't an interesting idea, but because the empirical data disproves it as a myth.

Our study focused on disproving the myths associated with advertising and program engagement. These myths are:

· Myth #1: Intensity of program viewing translates to intensity of commercial viewing.

· Myth #2: Ads are more effective against a highly engaged program viewer.

ARF: What types of engagement did you study?

Daboll and Curran: Our goal was to empirically show that program engagement does not impact effectiveness (consumption) of advertising

Several studies have shown that great creative quality (effectiveness) is correlated with successful in market (sales) performance.

Ace scores are a standardized and accurate measure of creative quality, proven across tens of thousands of ads and over 6MM consumer voices.

Program engagement, as the research shows, is not an accurate predictor of creative consumption or the effectiveness of the ad (as measured by the ace score among other things).

ARF: How did you analyze engagement?

Daboll and Curran: We conducted multiple analyses using a variety of measurements.

First, we began by assessing how different program viewing conditions affected a viewer's perception of the " watchability" of an ad and how these conditions were linked with its creative effectiveness. In our database consisting of more than 12,000 ads and comprised of more than 6 million survey responses, we found that across the board, there was no significant difference in ad effectiveness by different program engagement levels (as measured by watchability condition).

Moreover, our view is that "watchability" of an ad is not in itself generally a key predictor of the creative impact or persuasive content of an ad. However, we have identified several clear outlier cases -- we call them "Black Swan Ads" where the watchability has profound positive or negative interactions on the overall creative effectiveness of the ad. These infrequent, but real examples highlight the fascinating and complex relationship between how watchable the ad is and its ultimate consumer impact. Ultimately, measuring the watchability score in the creative assessment is to serve as an insurance policy against rare cases (i.e., "Black Swans").

Under specific conditions where an ad is both attention-grabbing yet not likeable or, alternatively, likeable but uninformative or irrelevant, watchability serves as an important gauge of the discord and incongruence in ad content and allows for correctly rating the effectiveness of that creative.

ARF: How did you test your hypothesis about the relationship between watchability and effectiveness?

Daboll and Curran: We tested our hypotheses with a second study aimed at measuring program engagement in multiple, additional ways.

After conducting an experiment consisting of more than 6,000 survey responses no evidence to support the claim that levels of program engagement were associated with creative scores, and ultimately the in-market performance of the ad.

ARF: What advice would you give advertisers trying to learn from your findings?

Daboll and Curran: It is important to consider the many definitions of program engagement that exist in the advertising community and evaluate more than one perspective.

We found that strong creative message--the underlying basis of persuasion--was responsible for generating effective ads. We hope that the results of this project will motivate serious thought about whether contextual viewing or program engagement warrants primary consideration when evaluating creative content.

Want to hear more? Peter Daboll and Michael Curran will be giving a presentation with JuYoung Lee, Ph.D. and cofounder of Ace Metrix, entitled,"High Viewer Engagement: Friend or Foe of Advertising?"At the ARF's Audience Measurement 6.0 Symposium on June 14.Registration is open.