Surveying is the process of finding order in chaos. I am currently in Western Mali, doing a pilot run of a survey designed to give us a holistic understanding of how ordinary Malians perceive child marriage and malnutrition. I helped create a painstaking research methodology to target 6000 individual respondents living in 2000 households in 80 different communities in two vast regions of the country. The survey tool is over 500 lines of Excel code, with embedded timers, GPS captures, skip logic, and picklists of anticipated responses. It has been bench-tested to ensure its infallibility and perfect standardization. It exists on a computer server today as a rigorous and empirical measurement tool. It will soon be deployed by surveyors with tablets to some of the poorest and least developed regions of the world, and used to measure the breathtaking complexity of human preferences, expectations and core beliefs.
The questions on the survey tool have been reviewed by psychologists, nutritionists, child protection experts and professional surveyors. The sample selection parameters and research methodology have been designed in consultation with demographers, and exhaustively examined by ethical review boards on two continents. The tool received hundreds of hours of collective attention before any one of our target respondents was asked a single question.
My complete understanding of the survey tool began to fade during the translation process. The tool was originally drafted in English and then translated into French, which I could understand perfectly but without the same level of nuance as the first version. Our data collection partner translated it from French to Bambara, the national language of Mali, and suddenly the tool which I had spent hours tweaking and refining was unintelligible to me. From Bambara it was translated into the regional languages Kassonké, Soninké, Sénoufou and Mianka. The process was like a game of telephone, where the signal became increasingly clear for respondents in the rural communities, but increasingly crackly and distant for me.
The complexity of the task of categorizing and quantifying participant responses was evident on the first day of the pilot. The demographic questions alone are a huge challenge. Families in rural Mali live collectively in compounds overseen by patriarchs, and within each compound there can be up to ten individual households. Defining households by asking if people are related, or sleep under the same roof, or prepare meals together is not easy: the answer is always that it’s complicated. A household contains brothers and sisters and cousins and uncles and people who are called cousins but aren’t even related by blood. The wives cook shared meals in colossal cauldrons outdoors, and anybody within arm’s reach of the serving dish partakes in the meal. A man with multiple wives can be identified as the head of household for houses in two different villages. One constant is that each compound looks like a daycare center on a field trip: there are babies EVERYWHERE. They snooze in the shade, braid each other’s hair, weep, recover, and observe the adult conversations in wide-eyed silence.
So demographic questions are challenging. If a man says that most months he earns 40,000 francs but he earns 1,000,000 francs during the month that he sells his cotton crop, and he shares this money between two wives with their own small streams of income as well as giving his uncle 10,000 francs every two months or so, what is his household’s monthly income? I was present to hear a surveyor ask one head of household how many children he had, and after a minute of consideration he said he had seven. “You forgot the one studying in Bamako!” an onlooker (there are always onlookers) pointed out. The man was momentarily chastened, but then said “well one of the ones I just counted is actually my nephew, so it’s still only seven.”
After we’ve gotten through the demographic questions, we get to the questions related to personal beliefs and social expectations. Many surveys ask clear and unambiguous questions. Are your children vaccinated? How much did you spend on rice in the last month? Do you own a bicycle? How many hectares do you farm? Our survey asks people what they feel and what they prefer; we ask them to speculate what their neighbor might prefer. In roundabout ways we ask them what they would prefer if their neighbor preferred something different. The translation phase of the survey was exhausting: we have stretched the conditional and interrogative tenses of the local language to their fullest. Out of ten people in your community, how many do you believe believe that you should arrange the marriage of your daughter within the year of her first period? To what extent do you believe that the mother of a sick infant should exclusively follow the recommendations of a traditional healer?
The respondents are patient in the face of these endless questions. We tell them that they will not be compensated for their time, but their responses can help create a brighter future for the children of Mali. It is a testament to their generosity, goodwill and optimism that for the most part they accept this explanation without complaint. I see respondents puzzle over this deeply personal interrogation, and it’s clear that they are considering many of these questions for the first time. There are dozens of sequential questions, but the two overarching ones are these: what do you do, and why? I hope no one ever asks me these questions. As I watch people wrestle with the confusion and ambiguity of the survey, guided by our ever-patient enumerators, I realize how much I myself struggle to articulate the rationale for my personal beliefs.
The survey is too long, some questions are confusing, and there are surprising outliers that oblige us to reconsider our field research protocol. As I construct detailed mental lists of all of the questions and parameters to revise, I also note the questions that are clear and useful. There’s a lot of work ahead of us, but so far, it could be worse.