Developing an optimized set of screening questions involves striking a fine balance between filtering out the wrong research participants and ensuring that your sample size isn’t too narrow based on selection criteria.
On the one hand, the questions must effectively filter in/out potential research participants; otherwise, they are simply not worthy of being called legitimate screening questions to begin with (because they serve no functional screening purpose).
And on the other hand, businesses must respect the fact that even if they plan to offer qualified participants an incentive (e.g. cash, gift certificate, purchase discount, etc.), they simply cannot ask an excessive number -- or excessively complex -- questions, since doing so will make the field too narrow, and the sample size will ultimately be too small to be valid or unbiased.
At Communications For Research, we lead over 600 unique market research projects a year, and either developing or editing (if supplied by our client) market research screening questions is a core part of every initiative. In light of this experience, here are the fundamental keys for screening success:
1. Aim for a Maximum of 12 Questions
Generally speaking, 8 percent of potential participants will terminate their interest with each successive question they are asked. For example, if the pool has 5000 potential participants, then 400 (8 percent) will drop-out if they are asked a second question. A further 368 (8 percent) will drop-out if they are asked a third question.
This is aptly called the 8 percent rule (and of course there can be variations in the numbers -- it is not a mathematical equation or scientific formula, but a phenomenon). Furthermore, it is not quite understood why this attrition occurs, other than to speculate that it is a feature of human nature.
Regardless of the reasons why, the takeaway remains the same: once the number of screening questions exceeds 12, the incidence rate typically drops dramatically, and it becomes very uneconomical to move forward with the market research project.
2. Eliminate Redundant Questions
Since the number of questions is limited (e.g. maximum of 12), it is vital to eliminate any redundant inquires. This may sound obvious and easy to do, but we have come across numerous screening questions that, while not identical, significantly and needlessly overlap.
For example, early in the screening process potential participants may be asked if they drive a car to/from work. And then further down in the process, they may be asked if they purchased gas for their car in the last month. On the surface, these are distinct questions. However, the former obviously implies the latter. As such, the second question is redundant and should be replaced by something that generates a more insightful and relevant response.
3. Ask Clear -- but Intelligent -- Questions
Business should put themselves in the shoes of the people they are trying to reach, and ask themselves: “Could I clearly and simply respond to this question?” If the answer is not an immediate and unqualified “YES!,” then this is a red flag that some tweaking – or perhaps some re-developing – is needed.
With this being said, one of the most common mistakes is to make questions too basic. Instead of clear (which is good), the questions come across as almost childish (which is bad). Obviously, it is important to avoid jargon and potentially confusing terms. But if the target audience is, for example, a warehouse manager or network technician, then it is not just acceptable to use proper industry terminology, but it helps build trust and engagement (i.e. people perceive that they are being asked an intelligent question by an informed insider, and are more likely to take it seriously and respond).
4. Ensure the Wording Matches the Platform
Back in 1964, Marshall McLuhan coined the phrase “the medium is the message.” While he was not referring specifically to market research screening, his insight certainly applies to this discussion.
This is because the choice of market research data gathering method, such as phone interviews, focus groups, email or mobile surveys, in-person in-depth interviews, and so on – does have an indirect impact on screening dynamics.
For example, one of the fundamental principles of phone screening is that it is a conversation. As such, it is essential for the screener (i.e. the professional asking the questions) to be highly trained in the art and science of getting potential participants to provide feedback – intentionally and unintentionally – about whether they understand the questions being asked, who is asking them, why they are being asked, and if they are amenable vs. hostile to the overall process (i.e. if they perceive the engagement as a telemarketing call that will pressure them to make a purchase, they will be very unlikely to participate).
Or if the screening is done online, then we have found that the questions must be very short, and the answer categories must make intuitive sense. Otherwise, if the questions are too long many participants will skim them and just randomly choose an answer. The same problem occurs if the answer categories are not instantly clear.
Learn More
To learn more about the keys to market research screening success, contact the Communications For Research team today. Not only can we help you write screening questions for a market research study, we can screen and recruit participants for you. Then whether you’d like to do the actual study or you’d like us to complete the market research, you know your sample is correctly sourced!
For more information on how to communicate the value of market research to your clients, download our FREE eBook:
