1 Evaluating strategies to improve the effectiveness and efficiency of CATI-based data collection during a global pandemic
Mridulya Narasimhan, Jagannath R and Fabrizio Valenti
Introduction
The COVID-19 pandemic has exposed the weaknesses in the research community’s ability to quickly acquire reliable primary data, especially in low- and middle-income countries (LMICs). At the same time, it has caused enormous disruptions in research activities, forcing researchers to rapidly shift from in-person data collection to the computer-assisted telephone interview (CATI) method, or phone surveys. Although this need is widely recognized, there is limited evidence on the best practices to design and implement these surveys, especially in LMICs.
Phone surveys generally suffer from low response rates (Keeter et al, 2017) and non-response bias as a result of limited access to phones (Whitehead et al, 1993; Kempf and Remington, 2007) and are characterized by high attrition rates (O’Toole et al, 2008), questionable response quality (Asher, 1974; Blattman et al, 2016) and social desirability bias (Kreuter et al, 2008). These challenges, though not unique to CATI surveys, are likely to be exacerbated during these exceptional times.
Existing evidence suggests that response rates can be improved by sending advance letters and scheduling (Smith et al, 1995; Hansen, 2006) and with higher incentives (Bonke and Fallesen 2010). Although the use of CATI surveys in LMICs has traditionally been considered ineffective, the recent increase in mobile penetration has allowed researchers to successfully build nationally representative samples (Leo et al, 2015). Some evidence on how to successfully implement CATI surveys in these contexts exists, but there is an urgent need to build an extensive body of evidence to identify effective strategies able to increase response rates in times of crisis, such as the one caused by the COVID-19 pandemic.
For this chapter, the authors collate results from a group of projects in India and Bangladesh to test various operational strategies to conduct CATI surveys. Specifically, preliminary results are presented from six studies for which data were collected between April and July 2020 while ‘stay at home’ orders were in place in both countries. The authors compare the following strategies of conducting phone surveys with the objective of yielding improved survey response rates: incentives (dynamic incentive versus fixed incentive) and scheduling (SMS scheduling versus non-scheduling). The authors also observe the effects of introducing call recordings on refusal rates, and measure the effectiveness of follow-up phone calls for quality control (back-checks).
Data source
Data for this chapter were drawn from the pooled metadata of six research studies (hereafter ‘host studies’), all of which used CATI data collection methods. These studies are being conducted by different teams from Leveraging Evidence for Access and Development (LEAD) at Krea University – an action-oriented development research organization, headquartered in Chennai, India.
The host studies were selected for the fact that they were in the data collection stage and the designed experiments were least intrusive in their implementation. As the survey content has a limited effect on the findings of this study,1 call results (for example, survey completed, refused, and so on) were aggregated for analysis.
A call-reporting t...
