Auditorium Tests: Goodbye, Farewell, Good Riddance

For nearly two decades I worked on Auditorium Music Tests (AMTs), where respondents either went to a hotel ballroom or to a local field service to take a music test. The endless list of problems that could, and did, arise frequently was enough to age me more than I wish, but have left me with many strong memories.

There was a time when I had to pay a respondent extra money for gas. I remember praying that the weather didn't hinder our respondents from showing up. When they did, I sometimes had to deal with them complaining there wasn't a large enough "spread" for them to eat while rating the songs (didn't I know I was feeding them dinner as well?).

Then there was the respondent who would show up with three friends and say, "Hey, my friends would like to participate, too; they totally listen to the same stations as me." I still remember the looks on the friends' faces when I told them they couldn't participate.

Let's not forget the respondents who constantly needed smoke breaks or to go to the bathroom and made such an ordeal about leaving that they distracted everyone in the room. Of course, there was always that one respondent that decided he sang better than the artists and belted out every song, most memorably the hook to "Fat Bottomed Girls" by Queen.

That was fun. And by fun, I mean not fun.

I've seen respondents trying to pick up other respondents during a music test. Some even stepped out to go to the bar (they did have a cash incentive coming their way).

One of the toughest parts of AMTs was no matter how many times we called to remind someone to show up, people forgot. We depended on respondents to come in. When they didn't, we either begged them to come (sometimes offering them more money) or we started recruiting more.

It's embarrassing. It's awkward. It's not productive.

The Internet has made doing nearly everything online so incredibly simple. We are connected to the Internet for more hours each day then we would like to admit. So why wouldn't we want to move from AMTs to online music tests?

Moving library tests online alleviates many of the challenges we used to endure.

So when I switched to conducting online music tests only and launched Advantage Music Research, I never looked back. I've eliminated all of those issues and recognized the benefits.

Not only do we no longer worry about situations like those described above, we can do a lot of things even better. We can simultaneously recruit and test at the same time, which helps to field the music tests a lot faster. Respondents can take the test when they please, wherever they please. They don't need to worry about traffic, weather, or babysitters. If a respondent can't dedicate the time all at once, they can break it up over a few sessions.

Simply moving to online testing, however, is not a panacea. There are big quality differences between online music tests, and when I started my company I wanted to make sure we did things the right way. The biggest task initially was learning and understanding about the different types of online sample panels we could use.

All online sample panels aren't created equal. To start, they receive letter grades from "A" through "D."

"A" and "B" samples are respondents who have opted in on their own accord or have been recruited from a mobile or landline telephone recruit. With "A" and "B" respondents, we have their telephone number or e-mail address and know that we can contact them again if needed. The only real difference between an "A" and "B" respondent is how much history and information we have on them. We have more information and history on "A" respondents, but over time, many "B" respondents become "A" respondents.

"C" samples are respondents who opted in online on their own. We have no history with these respondents and often very limited information; typically, only an e-mail address. There are often lower response rates with "C" samples.

The dreaded "D" samples contain panelists whose e-mail addresses have been purchased from other sources. Believe it or not, many of these respondents don't even know they are in these panels! Making matters worse is that we have no history or information beyond an email address for these respondents.

We quickly learned to focus only on recruiting from "A" and "B" sample panels. Even when using only "A" and "B" panels, it is extremely important to use other sources of recruitment. The percentage of the population who are even in online panels is so low that your station won't get a good representation of its listeners if the company doing your music testing relies exclusively on online panels. If it was even possible to use a combination of all of the online panels in North America, you would only be able to reach less than 20% of the population. That is more than 80% of the population's opinions not being heard!

We have learned it is critical that we utilize recruiting from landline and mobile phones and not just rely on online samples. This multi-methodological recruitment method makes for a faster recruit with a full representation of listeners.

So, Auditorium Music Tests? Goodbye. Farewell. Sayonara. Don't let the door hit you on the way out. Online music testing is here to stay, and that's a good thing for everyone ... as long as it is recruited using high-quality online samples and other recruitment means as well.