Online Survey Response Rates & Other Key Metrics in Mixed Mode Surveys
Email deliverability
If you’re collecting the wrong email addresses, or inputting them with errors (such as dot con versus dot com) you’ll no doubt see hard bounce errors. This means you have sent undeliverable emails out, because no email exists for what you actually inputted as the address. A good rule is to aim for 97 percent or higher deliverability (<3% hard bounce rate). This metric speaks directly to how well the telephone recruiter has performed at collecting accurate information. Keep your team motivated by letting them know from the start that you’ll be watching email deliverability rates.
It is important to note that other issues can impact email deliverability, such as your domain name, sending reputation, too many images in your email and coding errors. Each email provider (gmail, yahoo, etc.) may have their own requirements. A simple strategy is to keep your message simple: text, few if any images, check your links and avoid “spammy” language and all-caps, such as “FREE!”
Survey participation
This is a no brainer: track your online survey response rates! If you see levels that slip below what you’d expect, or if you see an unusual fall off between expected commitments and actual, then you should investigate your overall process immediately. Review your scripts, test your emails and test the overall process again to see if something has occurred to cause such a drop. If you can’t find anything notable, then it’s time to test incremental, small changes to your overall survey to see where you can improve to generate better response rates.
Commitment rates via phone vs. follow-through
Are you seeing a very high engagement and verbal commitment rate via phone, but a cliff-like drop when it comes to the actual online survey response rates? This could be a clue that you need to evaluate your telephone script, or possibly review how quickly email messages are actually being delivered.
Other email metrics
Emails are a great vehicle to track since they provide so many clues as to where things are going right, or wrong. You can track everything from subject line performance to open rates to click rates on the individual links within your message. Since the email deliverability and link are the key to you collecting your valuable data, review your stats every few days to see if anything interesting pops up in the data. A few things you could review or ask yourself are:
-
Why did I receive a high email open rate, but a low click rate? This may imply that the recruit either is having issues seeing or reading your message, that your message is not very compelling or that it is confusing. They may have not understood the directions or where to click.
-
What subject lines generate the most opens? Testing subject lines, in general, is one of the most effective tests you can do with emails. Subject lines have enormous power on if your email gets opened or not. While clever subject lines may work wonders in some cases, often clear and direct subject lines work the best, such as:
-
XYZ Survey
-
Complete your XYZ survey
-
Here’s the link to complete the survey
-
Review answers for each mode
Every survey is unique, but your survey may include the same or similar questions online as you do on the phone. Are you seeing different answers for each mode? Review your data and consider why this may be happening, or how you’re going to explain the different data sets at the conclusion of the study.
The sheer variety of metrics at your fingertips is a demonstration of the complex nature of a mixed mode approach. But by knowing which metrics to analyze and track, you can quickly spot successes and bumps in the road by just glancing over a few statistics.
