In a recent RBR email "Opinion Box," the Tribune Company's COO, Randy Michaels, posed the rhetorical question to Arbitron about the diary versus PPM: "Since the results are so different, which time were you wrong?"
And his answer: "Both times."
Which raises some other questions.
Why is the COO of the Tribune Company wasting his time writing editorial comments to a radio trade when his company is in flames? What is he doing to earn the confidence of his company's shareholders? Last time I checked, Tribune was still in bankruptcy.
To be fair, the newspaper business has been battered like no other, and has experienced the worst year in its history. And to be even fairer, Randy Michaels had nothing to do with its failure to keep up with the digital times. But the promise that Randy and his former Clear Channel team brought to Tribune had all the radio trades going ga-ga. The innovations. The out-of-the-box thinking. The brash, edgy Power Pig tactics.
But specific to Michaels' "Have you stopped beating your wife?" question about diary versus PPM, and which one is more accurate, he knows better. Different methodologies always yield different results. Go ahead and test the same artist or song in a perceptual study, callout, and an auditorium test, and you're likely to get three different answers. The phone, the web, and the hotel room all yield different results. Does that make them all wrong?
The onus falls on the analyst who has the task of studying the data to determine how to best interpret the numbers and apply them to the problem.
The diary system has always been archaic, and radio people have had to live with a flawed methodology for decades. The "bounce" from month to month is often downright scary, and the diary's inability to capture good and bad moments on the radio is well known to any of who have been in the business for more than 15 minutes.
Is the PPM methodology perfect? Not by a long shot. And Arbitron has much work to do in order to keep its panel samples strong and healthy. In many markets, the ups and downs have been within tolerance. In some of the newer ones, however - like Seattle and San Diego - the settling in the data is frankly, unsettling.
For years, advertisers and radio people have placed more weight on the ratings than has been smart. After all, these are "estimates," and while they are the only numbers we have, they are only a cursory look at the lay of the land.
As Arbitron struggles to get PPM "right," there are going to be ups and downs. Should their feet be held to the fire when they fall short of sample goals? Should they move as quickly as possible to get to the bottom of market and format specific ratings anomalies? Should they continue to upgrade PPM with cell phone only respondents? Should we ensure that headphone listening is accurately recorded? Absolutely, on all counts.
But the cheap shots on Arbitron, a company that is easy to dislike for its PR debacles and questionable business practices from years past, are just so unnecessary at a time when advertisers need to have confidence in radio and the companies that measure it. Arbitron needs to be questioned, tested, and pushed - hard.
But radio has to play this wisely. While challenging Arbitron as PPM rolls out, the radio industry needs to take a bigger picture view of the way it publicly criticizes the most visible and widely used ratings services. We have consistently asked the most vocal and snarky critics of Arbitron to keep their questions and even their frustrations at a lower, calmer volume so as not to communicate to the agencies that radio cannot agree as an industry about its ratings service.
Michaels is no longer in our business, but his trademark outside agitation continues. Thanks, but let radio deal with its ratings companies.
The cheap shot mentality at radio, popularized by Jacor, is as dated as the mullet and the Pet Rock. It's how some in radio used to behave, but it has been replaced by a more business-like attitude and strategic thinking.
And it won't work in the newspaper business either.