10 Myths About Survey Research
Robert O. Patterson, JD
CEO/ Principal
The Center for Association Resources, Inc.
Myth #1: Determining the survey method should be driven by the budget.
When selecting a survey method, it is important for the organization to determine the scope and purpose of the project. There are multiple low cost, minimum effort techniques of data collection easily presented in a brief summary. On the other hand, organizations may use the findings to track significant trends, approve capital expenditures or to implement employee incentive programs. Based on organizational needs, more robust initiatives may require database tracking, filing systems and extensive documentation, all at a much greater cost. Selecting the wrong survey method routinely yields incomplete and/or inconsistent data and will usually delay the research process.
Myth #2: Online surveys yield better results.
Online surveys yield very accurate results when properly completed; however, this research method is only effective when members have routine access to computers. Understanding the demographics of the member base is essential. Organizations may find that they need to offer a variety of survey methods to meet the communication needs of all its member types. The key to obtaining good, actionable data is to select the survey method that is most convenient for the target audience rather than the easiest or least expensive survey research process. Intelligent and effective assessment can influence sound strategic decisions.
Myth #3: Members will understand and comprehend the intent and content of the survey because it is developed by the organization.
The only way to know if your members will understand the intent and comprehend the content of the survey is to develop documentation/guidelines that match the comprehension levels of the target audience in advance. Determine the primary language, reading level and technical skill of the target audience to ensure that participants are able to express effectively their opinions without misunderstanding, frustration or perceived embarrassment. These avoidable communication problems often result in a greater percentage of incomplete or inconsistent responses.
Myth #4: The more questions asked, the more useful the information collected will be.
A reasonable time to complete the survey is relevant; however, the survey instrument and method should reflect the subject matter, incentive, and experience being assessed. Longer surveys tend to result in more incomplete surveys. Members are not likely to spend an hour to complete a survey to assess their satisfaction. Also, be careful of obvious redundancies in the questions. Members tend to resent being asked the same thing in different ways.
Myth #5: A significant number of respondents are necessary for a successful survey.
Survey research is designed to obtain your members’ opinions. Depending on the total number of potential survey participants, organizations may set goals too low or too high. Sampling goals must align with the scope of the research project. The representative data offers member feedback that aids organizational decisions. So long as the organization meets the sampling goals prior to completing the research project, the data should be viewed as “representative”.
Myth #6: The survey participants that take the time to complete the survey are usually satisfied with the organization.
Often it is the most satisfied or the least satisfied who take the time to complete the survey. While all survey research is voluntary, the most representative results also include the opinions of members who received personalized attention and a bit of prompting to complete the survey process.
Myth #7: Offering incentives to potential survey participants can be perceived negatively.
Some organizations offer incentives including financial rewards, sweepstakes, free products, or upgraded services to increase survey participation. Incentives do not have to be expensive, only meaningful. It is proven fact that incentives encourage participation.
Myth #8: Collecting demographic information of survey participants is not important.
Research data routinely identifies areas of future opportunity. To understand implications of the data and to implement improvements, it is imperative to identify the groups or member types who have expressed a specific opinion. This information is critical to the research process. Without it, the efforts to improve member satisfaction become more difficult.
Myth #9: It’s not necessary to compare survey results to industry standards and best in class examples.
Effective assessment of the service provided by your organization is balanced against what expectations are being set by industry publications and which organizations are achieving optimal results. This data will assist in the strategic planning process and will soon become an advantage for your organization.
Myth #10: It’s not important to measure improvement success.
The purpose of survey research is to assess the current and desired state of the organization and member satisfaction. Upon evaluation of the initial survey results, organizations typically implement changes to achieve a desired goal or set of processes to ensure that the current high level of service is not changed. To maximize the effectiveness of the research process, it is important to design strategies, programs, and outcomes that can be easily replicated to gauge the success of the organization’s efforts.
The Center for Association Resources, an AMC Institute accredited association management company, provides clients comprehensive management services including: Association Management, Project Management, Membership Development and Management, Survey Research / Member Needs Assessment, Financial Management, Grant Development and Management, Leadership Development, Certification Program Development and Management, Government Affairs and Legislative Monitoring, Meeting Planning and Management, Marketing Strategy, Policies and Procedures Development, and other customized services. Our experienced teams, proven processes, and technology are designed to effectively manage and grow organizations. For more information, visit www.association-resources.com or call (888) 705-1434.
