You’re fielding a survey and need to decide… Panel or a list. This is an important issue in market research today.
Companies are more often than not using lists of respondents to field surveys instead of real panels. But the word “Panel” means something very different to me. I think of a Panel as a group of highly engaged, trusted advisors who respond in a quick and honest manner to an inquiry that falls under their area of expertise. Ron Sellers discussed the issue of Panel exploitation in a recent post on the GreenBook Market Research Blog. He refers to a Grey Matter Research report, “More Dirty Little Secrets of Online Panel Research” that exposes some ugly truths about the use of Panels in market research. Both Ron’s blog post and the Grey Matter report point to many concerns that may be hidden in the use of a Panel when doing online market research. Concerns that include underpayment, overuse, misuse and downright abuse – all of which introduce unwanted (and unnecessary) error and bias into your research.
I agree with the Grey Matter report. Garbage in, garbage out – researchers often use this term to refer to the fact that bad data collection methods result in inaccurate conclusions. We need to think about the things we can do to eliminate these problems with online Panels. We put a lot of thought into these issues at InCrowd as we began
We knew from the start that we wanted our network to be far more than just a number. We wanted a group of experts who were close to us, answering questions honestly and quickly. We wanted a relationship built on mutual respect. This is clearly a moving target and problems will need to be solved in an ongoing fashion as our industry continues to mature and new challenges arise; however, I think some simple actions can help us all better manage our panels.
First, take very good care of your panelists. This seems intuitive but the report by Grey Matter Research leads me to believe this isn’t all that obvious or isn’t all that important to panel providers. At InCrowd, our “Crowd” is our most valued asset. We do all we can to make the registration process quick and simple and reward them for their activity. We want the members of our panel to fall into that sweet spot where they are engaged in providing their expertise to our clients but not buried in requests for participation. Our Crowd may field a single question each week or several per week but we are cognizant of their busy schedules and send a single question at a time. This reduces burnout and allows them to participate in the middle of their busy day with a thoughtful response to the question.
Second, involve your panelists in product development efforts whenever possible, give them input as to how they can participate in the future. We ask for our Crowd’s feedback and opinion (after all we are in the business of collecting data!) regarding innovations we have underway. This allows the members in our Crowd to help shape the way we engage them in activities.
Third, pay attention to sampling. Market research companies must consider how these samples are used. If several questionnaires are fielded to the same sample in a short period of time, random sampling should be used to spread them among the pool of respondents. Algorithms can be used to calculate the sample size needed to field the survey – this prevents sending the survey to many more respondents than will possibly have the opportunity to respond. Smart sampling is key when thinking about item one above – taking good care of your panelists!
Fourth, be honest and transparent with your panelists. Sometimes problems occur and a group within your panel gets overused or undervalued – apologize, own your mistake, and make it right. If your panel knows you value them, that you’re honest about what you’re trying to accomplish, you can and will build mutual respect and the foundation for a strong working relationship.
These are some of the things that help distinguish between a panel and a simple list of contacts. Our secret to our success is nurturing our panel, respecting their time, and valuing their contributions. Simply put, keeping our panel close.
We look forward to hearing your input on this important topic in market research. How do you manage your panel? What do you find are best practices for panel recruitment and engagement? For those on the client side, what are your thoughts on using lists and panels?
