InteracTiVoty: "I Never Watch Commercials" and Other Little White Lies (or, "What Would B.F. Skinner Say?") - Todd Juenger - MediaBizBloggers

Once every couple months, a thread will show up on one of the TiVo online forums that goes something like this: "I Never Watch Commercials." I love reading these threads, because most of the posts read something like this: "I never watch commercials, either. You know the one I really hate the worst is the one for XYZ brand where the guy does blah blah blah and the woman responds with blah blah blah. I really hate that one, I never watch it." Hmm. You have 'never watched it' so many times that you know every line of dialog.

Over the holidays, I was having a dinner conversation and, when you work for TiVo, the conversation inevitably turns to TV. I was amazed to learn that hardly anyone in my family, apparently, ever watches TV. And yet, amazingly, if I asked anyone's opinion of Jon & Kate or the AMAs or which cable news network is most biased, everyone had an opinion. Shocking.

So why is it that we continue to pay attention to self-reported research? I have read numerous studies that claim to measure how people are using their DVRs, based on self-reporting ("I never watch commercials"). More broadly, do we really believe people can tell us what motivated them to make a certain purchase? Even if they want to tell the truth, do we really think the decision process is a conscious cognitive deliberation? How about segmentation studies based on programs people claim they regularly watch (apparently my entire family would fall into the "I don't watch TV" segment)? And the most egregious example of all, because it's used as currency for transacting billions of dollars, the diary approach to estimating television program audiences in local markets. Are people writing down what they really watched (and who was in the room), what they aspire to watch, what they'd like others to believe they watched, or what they mis-remember having watched?

And to make matters worse, all this self-reported research is only available for those who were willing to respond. And response rates have plummeted (except for the population of professional respondents). Something tells me it's the people who are not responding that I'd most like to hear from.

As I've grown increasingly cynical over the years, I care less and less about what people say they do (or why they do it), and focus as much as possible on what people actually do. What stimuli were they exposed to, and what happened. Apparently I'm a disciple of Skinner's Radical Behaviorism, although I'm just finding that out.

If you want to understand (or at least better estimate) how many people watched a particular program or commercial, don't ask them. Don't send them a few crisp dollar bills and ask them to fill out a diary. Don't invite them to join a panel and get compensated in return for installing a bunch of equipment in their homes and punching buttons indicating what is being watched by whom. Just let them live their lives and passively, anonymously collect the digital trail left by millions of TV sets and you'll get much closer to the truth. This "passivity benefit" of set-top-box data is occasionally referenced but I believe significantly under-appreciated. It usually comes in the same breath as "Yes, passivity is nice, but you don't get persons-level demographics." To which I am inclined to respond, you must also believe that no one with a DVR watches commercials, and no one in the Juenger family watches TV, (and maybe Santa Claus?), because the methods used to collect the data are all more or less the same quality. If it's that important to ascribe persons-level age/sex demographics to an audience, and you're resigned to using a small sample anyway, at least find a group where you can observe what they do, rather than asking them to push buttons and self-report.

There are of course times when self-reported surveys are the only reasonable way of gathering data and there are professionals and organizations who make sure the validity of the data is as good as possible. But for TV, when more direct, passive measurement is available, why would you not use it?

If you want to find out if a commercial is effective, identify people who were exposed to it at different frequencies in different environments and determine whether it caused them to take action. Sound like a pipe dream? TiVo and Quantcast can do this, today, across TV and Internet, based on single-source, passive observation, 35,000 households, tracking downstream online behavior and how it correlates to various combinations of online and TV ad exposure. TRA can do this today, single source across nearly 400,000 households, for TV ad exposure and grocery store purchases. And I expect dozens of other applications to emerge in the near future.

For those of you who are determined to cling to self-reported research, you'll be happy to know I only watch C-SPAN and PBS programming. I don't know who's watching all that trashy reality programming but it's not me.

Todd Juenger is VP and General Manager, Audience Research and Measurement for TiVo, Inc.

Read all Todd’s MediaBizBloggers commentaries at InteracTiVoty - MediaBizBloggers.

Follow our Twitter updates @MediaBizBlogger