Contact modes for community-based epidemiological research traditionally include mailings to residential addresses and telephone contact based on public directories or random digit dialing (rdd) to landline numbers. In addition to deciding on the mode(s) of contact, a choice must be made between personalized (addressed to a named person) or generic (head of household/occupant) approaches. however, the population coverage of personalized data sources based on public telephone directory listings has decreased in recent years due to the rapid transition to wireless (mobile/cellular) telephones [1, 2] and higher proportions of landlines not listed [3]. the increasing prevalence of households with only wireless telephone connections has also had a negative impact on the coverage of the generic rdd methodology for landline telephone numbers [2].
in addition, researchers are finding increased problems in contacting potential respondents and increasing rejection rates among those contacted [4]. Declining response rates have been reported for long-established population health surveys, including the US. uu. behavioral risk factor surveillance study (brfss), as well as consumer and public opinion surveys [4]. A review of the monthly RDD survey of consumer attitudes conducted by the University of Michigan [5] showed that non-contacts accounted for less than 5% of nonresponse between 1979 and 1985, but increased to more than 15% in 2003. The refusal rate among contacted households also increased from 19% in 1979 to 27% in 2003, and the overall response rate fell from 72% to 48%. The dual challenges of shrinking population coverage of traditional sampling frames and falling response rates underscore the need to explore new sampling frames and alternative modes of data collection for community surveys and document their usefulness.
use of the internet for community surveys
With internet access now widespread in many countries, electronic methods of contact and data collection for surveys have become possible. Like telephone interviews, internet surveys offer many advantages in improving data quality compared to written questionnaires [6]. these include the ability to prompt or request answers to missing questions, the application of input masks and consistency checks to minimize invalid answers, and the skipping of irrelevant sections that are conditional on answers to previous questions. problematic questions can be quickly identified by tracking completion times and breakpoints, and then modified to reduce the rate of incomplete surveys [7]. Initial costs for online surveys vary depending on the level of programming sophistication required, but as respondents enter data, the cost per response decreases as the number of respondents increases. this offers potential savings compared to postal or telephone survey modes where material and personnel costs tend to be proportional to the number of respondents.
For surveys of specialized populations, such as students or professional groups, it is usually possible to send an email invitation with a direct hyperlink to the internet survey to an existing email list, but the lack of such lists for the population It generally means that other contact methods need to be used. Comparisons of online surveys with other modes of survey administration have generally used conventional mail for initial contact, and a common finding has been that response rates to online community surveys are lower than those of community surveys. postcards [8-10]. Two independent meta-analyses found that the average differential between internet surveys and other modes of response was greater for community surveys than for surveys involving specific target groups, such as students, employees, or association members who may be more familiar with the Internet use [11, 12] .
even in countries like sweden, where more than 80% of adults have internet access at home and there is high penetration across different age groups and educational levels [9], it seems that most people still prefer mail surveys. For example, an online community health survey with one follow-up contact by mail and one by phone following a mailed invitation had a 50.6% response rate, but the parallel arm of the postal survey had a lower rate. response rate of 64.4% [13]. Similarly, a lifestyle survey of Swedish women produced an initial response rate of 33% for a postal invitation to complete an online survey [14], but when followed up by letter or email to those who did not respond and were offered the option of postal or online response, more than 90% of those who responded at this stage chose the postal option.
Although internet surveys currently appear less effective for community surveys than other response modes, combining multiple modes may offer a means of improving overall survey response rates and possibly expanding population coverage [ fifteen]. indeed, it has been argued that the use of any single mode of contact other than face-to-face household interviews will currently exclude significant subgroups within the population [16]. studies offering the internet mode simultaneously with other response modes have generally found little improvement in overall response rates [17, 18], but sequential use of different modes has been reported to be beneficial [15]. For example, a US community study. uu. using a generic address-based postal sampling frame found that response rates of between 44% and 52% could be achieved by an initial internet survey with a $5 prepaid cash incentive, followed by a postal survey of individuals non-responders [19]. Mail-only and Internet+mail survey respondents were similar to each other in demographic characteristics and more similar population demographics than Internet-only respondents. however, mail-only comparison arms were found to have higher response rates and lower costs, and the need to use postal contact for invitations and follow-up negated the potential cost and speed advantages of the internet mode . in the narrowest target group (long-distance telephone subscribers), follow-up of those who did not respond to four initial survey modes (mail, telephone, interactive voice response, or Internet) using a different mode (telephone or mail) resulted in an increase of up to 37% in the overall response rate [20].
investigation of factors affecting internet response rates among those who are successfully contacted shows general similarities to those documented for other survey modes [21]. the relevance of the topic to the individual, the length and complexity of the survey, and the type of sponsoring organization (academic, government, or commercial) can influence a potential respondent’s decision to participate. internet access is, of course, a prerequisite for responding, and literacy and language restrictions apply to both internet mode and mail-in surveys, although the ability to provide access to questionnaires in alternative languages is simplified enormously on the internet.
the effect of personalization and incentives
In general, personalization has been found to increase response rates of postal questionnaires, but a recent systematic review showed a positive effect in only one of six randomized studies using community-based sampling frames [22]. Another analysis of 17 comparisons involving the general public in the United States found a modest benefit of personalization on mail survey response rates with greater effects apparent in rural areas [23]. Personalization of email invitations for internet surveys has been reported to increase response rates in various groups, including college students [24, 25] and scientists and engineers [26]. this approach is not open to community surveys due to the lack of email lists, but personalizing internet survey invitation letters can reasonably be expected to improve response rates compared to generic invitations.
incentives to participate in surveys can be provided in monetary or non-monetary form and paid unconditionally (sent to all potential respondents with the invitation), conditionally (sent only to survey completers), or offered in the form of a lottery . Monetary incentives were shown to be effective in a systematic review of 94 postal surveys [22], and response odds were nearly doubled compared with no incentive, although results were heterogeneous. monetary incentives were more effective than non-monetary incentives and, in general, larger amounts were more effective than smaller amounts. unconditional incentives were more effective than conditional incentives, but there was heterogeneity between studies. Only one internet survey assessing a monetary incentive versus no incentive was included in this review and there was no significant effect on response rate. however, six trials evaluating nonmonetary incentives found response odds nearly doubled compared with no incentive.
A variety of alternative or combined incentives for Internet surveys have been evaluated in terms of response rate and cost-effectiveness. For example, a Canadian study using mailed invitations to an online community survey found that a small prepaid cash incentive ($2) generated the highest response rate, but a high-value lottery ($250 2 prizes) was the most profitable option (other options tested were no incentive and 10 rewards of $25) [27]. a systematic review of material incentives found a fairly consistent positive effect on increasing the number of people visiting the first page of internet survey websites, but a more variable impact on the number of surveys actually completed [28], but most individual studies were underpowered to detect an effect.
in 2011 we conducted a community survey in melbourne, australia, to examine household practices regarding greywater use and associated health risks. gray water is used water collected from the bathroom, laundry room or kitchen for later reuse in the house or garden. We use postal invitations in combination with three modes of survey administration (postal mail, internet, and telephone) and two contact approach styles (personalized and generic), with a lottery incentive. we present a comparison of the response rates and cost-effectiveness of the different modes and approaches.