With a steady stream of survey projects at various stages of design, bidding, recruitment, programming, fieldwork, or analysis, the details can be overwhelming for teams. But delivering data-driven stories to steer important business decisions requires confidence in data accuracy. Teams work hard to proactively eliminate factors that could skew data, introduce unconscious bias, or somehow provide less than accurate consumer research.
Even best efforts can sometimes overlook details that could negatively influence data collection or participant responses. From years of experience, we’ve gathered four unconventional factors for consideration when evaluating data accuracy.
Research has shown that weather can significantly affect a person’s mood, sense of general well-being, energy levels, and ability to provide honest self-assessment. Each factor could unduly influence data collection methods like survey responses or interviews.
Respondents taking a survey or being interviewed on a sunny, warm day are more likely to have a more positive disposition and respond more positively in general. On the other hand, data collection over a series of cloudy, rainy days may skew more negative in responses. Cold can also affect responses. When individuals feel cold or have prolonged exposure to cold environments, they tend to use less energy when considering issues and may deliver more shallow answers.
Surveys taken during lunch hours could receive more hurried, less thoughtful responses. Similar time-based situations can also affect the quality of responses subjects deliver. For instance, surveying students about study habits in the middle of exams week may yield results that favor a more studious impression of the subjects, whereas taking the same survey right before spring break may reveal a higher incidence of devil-may-care attitudes.
Macroeconomic activity generally does not affect people day-to-day, but major local, regional or global incidents could. A survey asking people about their willingness to buy a home conducted in 2009, amid a housing crisis and subsequent recession, would have a measurably different response ten years later. A global pandemic offers obvious influence on all research. Researchers should consider more localized economic issues, and proper survey sampling can also mitigate the risk of skewed data.
Small decisions such as color, font size, spacing, and more can all subtly influence the thought processes that lead to responses during studies. Having a research partner who knows best practices from the use of serif fonts for easier reading, and the positive effect of certain colors in terms of brain science can help eliminate unnecessary variables. While these factors do not always manifest significantly, research design can include simple A/B testing to eliminate anything that could elicit a response strong enough to skew data.
High-quality survey design is crucial to obtain clear insights that can be translated into powerful stories to inform decisions for business impact. Finding a partner to uphold best practices in quantitative data collection methods is key for success. Regardless of the type of project, from market research to marketing research, healthcare research to pharmaceutical research, or something in between, knowing that the outsourced pieces of your project are handled carefully can make all the difference in delivering quality insights. In this way, research directors can more effectively stay in control of projects even as vendors and multiple team members contribute to the overall success of any research project.
Curious about current best practices for survey design? Download this free guide. As a bonus, we’ve included sample screening questionnaires to plan for optimal participant recruitment during the quantitative research design phase.