Researchers can choose between many data collection methods and techniques, but they are increasing turning to a mixed mode survey approach. When using a mixed mode style, researchers must be especially diligent to maintaining consistency and keeping a close on survey performance. If survey performance is not going well and participation is low, it is important to evaluate why and move fast to alter your approach.
Challenge yourself to take risks and make changes in order to hopefully have a positive impact on the outcome of your survey. For example, change how you approach just a small number of recruits. Does a change in the script help the participation levels? How about offering an incentive? Testing is strongly encouraged, but if you go this route, you must stay organized. Here’s our advice on how to do so.
First, decide what you’re going to test
Think of this as “research within the research.” You can test everything throughout your data collection methods and techniques, from the survey email to your reminder phone call. You can test email subject lines, copy, the timing of the email, the phone call scripts, survey questions and so on. Keep an open mind—you may be surprised at your testing results.
Create a control group and a test group. It’s okay to start testing with a very small number of recruits. You don’t need to launch each test to the entire group, at least not at first. Make sure you’re tracking which recruit is in your control and test group to make sure your test is reliable and that they are not being communicated to twice, or experiencing confusion, and so on.
Finally, decide how you’ll measure your results. Everything you test must be measurable.
Keep track of changes
List out all of the changes required to execute your tests. Be detailed about it, and test these new steps on your own before sharing them with the greater team to make sure the process works and that you didn’t miss a change or a step.
An example of why it’s important to go through the process yourself and test every step: if you’re testing a change in incentive, does the introduction web page communicate the new incentive? Does the contact information verification page also reflect the correct incentive amount? And, are reminder calls actually going out? You can never assume these things are being done. You need to be hyper-diligent to hand hold the process and have confidence everything is flowing as it should.
Review data regularly
Review data from the recruiting and online survey outcomes closely after the first two days and again after the first week, and then ongoing as long as you’re testing. Make adjustments to your tests according to what you see happening.
Testing may seem like a tedious process, but it can also be the thing that saves your study and increases response rates when you need it most. As you continue to optimize your mixed mode survey strategy, you won’t only use your findings for your current survey but for future comparable projects. You’ll also gain a lot of experience with project management, as the testing elements add an entirely new layer to the already complex world of mixed mode research.
