What Every CMO Must Know: Test Bold New Media Mixes

This post continues the series that began with What Every CEO Must Know: Secrets to Brand Growth in the U.S. So far we’ve sought to establish the importance of learning how to grow mature brands in the U.S., and that the means for doing so is to perform scientific experiments — what we call A/B tests in the advertising industry.

We’ve looked at three ideas — (1) Reach Down to our own internal “developing nation” within the U.S. (without changing brand image), i.e. the less-advantaged. If we can gain their loyalty, their rise out of poverty will give the brand a lift; (2) Continuity Rewards rather than discount as the main economic stimulus driver (promotion); and (3) Scaling up the use of Gratitude-producing Owned Content even in TV ad pods. In this post we will look at a fourth idea — scientifically answering the question, “Can my brand grow faster in the U.S. with a bold new media mix?”

It is certainly time to ask that question. Just in the last two years we’ve entered an era of smartphones, tablets and social media, and it’s conceivable that even before that, many brands still have not made the optimal allocation and use of digital advertising and sponsorship through PCs and Macs. In television, many brands have shifted the mix out of prime broadcast and cable high-rated premium programming, to cable nets, other dayparts and 15s, to keep up the GRPs for the marketing mix modelers (MMMers) who rate TV ROI based on GRPs — without factors for reach, frequency distribution, campaign purchaser density, environment psychological impact quality, or even the creative itself. These recent sweeping and swirling media changes make it justifiable for any CMO today to tell the rest of the C Suite that this form of media mix experimentation is crucial now. The current business-as-usual media mix may be sub-optimal, leaving a bundle of money on the table. At a time when so much has changed about media so fast, we have to find out.

The reason controlled experimental in-market checkerboard testing is deemed scientific is that only the test variable is different in one geography vs. the other geography. For example, in one matched geo cell you continue business as usual (“BAU”) with the brand’s current media mix, but in the other cell you add a new media tier as a heavyup over the BAU. What you would be testing for is the answer to the question, “Does a new media heavyup more than pay its own costs in terms of incremental sales?”

The test cell can also be at matched spending with the BAU cell, so you wouldn’t be testing a new media heavyup, but simply a reallocation among media. This kind of test has to be done between media flights from the BAU campaign. If there are no between-flight periods (such continuity being a good idea, incidentally) the only option is to test heavyups and to model the equivalization of media budget to answer the question: “Do I dare, based on this evidence, make such a major shift nationally in my brand’s media allocation? Do I want to take the political risk of bringing this up?”

That risk goes down once scientific evidence is brought. If sponsoring or using a new mobile app, or a new use of social, or a new combination of digital and VOD, or of addressable TV and addressable digital, or a score of other possible winning mixes, makes the brand dramatically more money as a heavyup tier, others in the C Suite will themselves be asking, “Why not try rolling it out? It seems to work better than BAU.”

There is a current groundswell among leading minds in advertising research that the time has come for more aggressive use of A/B testing to get to the answers we all need to run our businesses, sooner rather than later as we emerge into a completely changed landscape culturally and medially. In the past few years my friends at ARF, Wharton, Ehrenberg Bass, EMM, Catalyst, SMG, GroupM, IPG, OMD, CBS, and across the industry have been encouraging each other to do more A/B testing, seeing that it is the supreme technique and strangely underutilized at exactly the time when it can be most valuable, a time of such change that old media mixes are suspect.

Same goes for old creative doctrine. Everything is suspect, and by comparison to A/B all other research is mushy. A/B testing is the same as direct marketing. No mush. All signal, no noise. In a world of power sharing within organizations (thus a “political” context by definition), when everything is up in the air, the protagonist (CMO) wants a non-mushy tool with which to recommend bold changes only after hard evidence is on the table.

In relation to this urgent need for A/B that my friends and I have been trying to push for years, I’m pleased to be able to announce that Bill Harvey Consulting (BHC) has a new advisory assignment facilitating A/B testing within the CharterCommunications universe in the U.S. I have been talking to Jim Heneghan, Craig Berkley and Paul Sly at Charter about A/B testing since research leaders began to see a need to upshift the use of that method years ago. Charter now has the ability to enable crossmedia buys based on client customer data matched to Charter homes with geo addressability and client clear read on sales results based on client’s own data. And they’re doing this in a set of checkerboarded cable zones prematched to other zones where this heavyup media effort is withheld. Other non-Charter media can be added in the heavyup cell including social and mobile.

Cable zones, rather than the DMAs used in classic checkerboard testing, are an order of magnitude less expensive on the media and research costs.

This A/B testing opportunity is now available to all advertisers, agencies, media, research companies and consultants.

We recommend testing whatever new media you have been considering, in A/B heavyup or same-dollars mode, in order to cut to the chase on national optimization of ROI.

Zooming back from specific media, here are the two main questions to answer:

1. What is optimal budget allocation across TV, tablet, smartphone, PC/Mac video, PC/Mac nonvideo, social, search?
2. How do you put all that together most synergistically?

The hypothesis is that you can approximate that optimal mix, by multivariate analysis of the performance of a heavyup test composed of these media elements, some of which are also in the base tier.

It is also possible to set up test designs where there are more than two cells, for example where alternative media mixes are tested in the heavyup layer.

The ideal mix will depend upon the way you specifically execute each medium. For example, your brand’s testing might show that 15% mobile is the right level for that medium simply because you have hit upon a novel sponsored app that people love to use on their smartphones, whereas the next CMO’s testing might show that mobile should be under 5% because the way mobile was being used in the tests was too prosaic in their case.

And so by this process move toward the ability to grow mature brands faster in the U.S. by zero-basing one’s media mix and accelerating toward a new optimal mix by the most cost-effective testing paradigm available.

It is time.

Bill Harvey is a well-known media researcher and inventor who co-founded TRA, Inc. and is itsStrategic Advisor. His nonprofit Human Effectiveness Institute runs his weekly blog on consciousness optimization. Bill can be contacted at bill@billharveyconsulting.com

Read all Bill’s MediaBizBloggers commentaries at In Terms of ROI.

Check us out on Facebook at MediaBizBloggers.com
Follow our Twitter updates @MediaBizBlogger

The opinions and points of view expressed in this commentary are exclusively the views of the author and do not necessarily represent the views of MediaBizBloggers.com management or associated bloggers. MediaBizBloggers is an open thought leadership platform and readers may share their comments and opinions in response to all commentaries.

[Image courtesy of KROMKRATHOG/FreeDigitalPhotos.net]