First contact with respondents, and what we’ve learned after 305 test calls

In this issue: our face-to-face survey in Somalia, an update on DR Congo, and some thoughts after conducting 305 test calls.

First contact with respondents: face-to-face survey in Galkayo, Somalia.

The face-to-face survey, September 2013.

The mobile Vulnerability Analysis and Mapping (mVAM) approach provides extensive contacts with respondents before remote voice surveys begin. Doing so will make respondents comfortable with the project and should help ensure good response rates. As a first step, we need to identify and select potential respondents. We will need to understand how they differ from households who do not participate in the survey, in order to allow us to have an idea of the bias in our approach. For Somalia, we now have baseline information that will help us understand our respondents and design our survey to be as user friendly as possible.

Angie and team implemented the face-to-face survey in early September. We piggybacked on a food security survey that had already been planned, and added a few questions to the household questionnaire. We informed respondents about the mVAM project, and requested their consent to participate. For those who accepted to become mVAM survey respondents, we registered two phone numbers that we could call them on and asked them when they would prefer to be called. Willing participants also received a simple flyer outlining the project.

Out of 650 households visited during this initial face-to-face survey, some 600 volunteered to join the mVAM pilot and elected to receive calls each month over the next year. The overwhelming majority of respondents, it turns out, preferred to be called on Friday mornings – although we will have to figure that out, as we do not have the capacity to place 400 or so calls within a 4 hour window of time. The assessment confirmed that 60% to 70% of household heads have access to a phone. Cell phone owners often carry two SIM cards, as there are two main mobile phone operators in the area. We will therefore have to deal with a dual SIM card set up for the Interactive Voice Response (IVR) system, which will involve some additional technical work.

The data from the face-to-face assessment will also provide useful background information on the food security status of the respondents – we will have standard food security indicators as well as information on asset holdings. We will monitor how these indicators change with our monthly calls.

Gaining ground in DR Congo

The July scoping mission confirmed that in DR Congo telecommunications services remain basic. The contacts we’ve had with private providers since then have confirmed that what we are asking is very new to the private operators in the country. We were told that we were the first customer to ask about using call centre services for outbound calls and IVR solutions for phone surveys in the country. Koffi and colleagues have been relentlessly explaining our interest to possible service providers in DR Congo.

You can find Koffi’s presentation to the DR Congo food security clusterhere. The DR Congo face-to-face survey is planned in early October, in one of the IDP camps of North Kivu.

305 IVR test calls and counting

The team in Rome now has fully functional interactive voice response call flows in Swahili and Somali configured in Verboice.  These call flows play the audio files that colleagues in DR Congo and Somalia have recorded. The call flows include the questions that we hope to use with our actual respondents; they ask about food consumption (a dozen questions) and coping strategies (eleven questions). These call flows are streamlined versions of the typical food security modules that WFP implements for face-to-face surveys.

A large number of test calls were required in order to ensure that the call flows were playing properly. The call log on Verboice reached 305 this week, an indication of the level of effort required to get to the point of having a workable version of our questionnaire configured. The team at Seattle-based InSTEDD was helpful in debugging Verboice and replying to our questions through their Google Groups message board. We would post in the evenings Rome time and were pleasantly surprised to find an answer the next morning.

A growing log of calls

We tested these call flows with colleagues in Goma and Nairobi. We’ve tried a few VOIP calls to Goma. The calls work, call quality is rated ‘8 out of 10’ according to Moustapha, who works at our office in Goma. The challenge will now be to set up call schedule so that all of our respondents are called at the push of a button. Much more testing remains required.