Polling a ‘Foreign Tribe’

As the US presidential elections wind their way towards a painful and tortuous conclusion, I’ve been thinking a lot about the difference between reporting on a survey that offers a snapshot in time, as opposed to using polls, and past history, to predict what the results might be in a few days, weeks, or months. My colleagues and I have done lots of surveys over the last eight years in Colombia, India, Mexico, Morocco, Nigeria, and the USA. In all this work, we offered estimates of how many people in a given population engaged in this or that behavior, supported this or that policy, or thought positively/negatively about a particular organization or institution. There is always a confidence interval around every one of those estimates influenced in large part by the number of valid responses. This allows you answer questions like, “what percentage of the population living in the Moroccan cities of Casablanca or Rabat supported this or that political party in May 2015?” Or, “what is the statistical association between attending mosque and believing that women’s rights are human rights?” If I wanted to ask how many people were likely to vote for a specific Moroccan political party in the future, however, I’d have to do so much more than extrapolate from the previous work I’d done with my colleagues. I’d want to have more polls over time, so that I could track trends, and I’d want to figure out a way of predicting who would be likely to actually participate in the election, assuming that participation was voluntary. In the US, pollsters did not do as good a...
How Telecom Mechanisms Have Impacted Polling, Part Two

How Telecom Mechanisms Have Impacted Polling, Part Two

Some populations are inherently difficult to survey. As a result, their voices rarely figure into studies of public attitudes, financial or medical needs, or consumer preferences. The most overlooked populations include people without access to the Internet; immigrants, refugees, and persons internally displaced within their own countries; homeless persons; and people living in low-income, repressive, or war-torn countries.  Some organizations have made significant steps towards addressing this problem, and one such group is RIWI, a publicly-traded company based in Toronto, Canada. In the intermediate future, my company, Azimuth Social Research, hopes to partner with RIWI on a cross-national survey project.  RIWI’s Random Domain Intercept Technology (RDIT) invites web users worldwide to participate in surveys in their own language. RIWI bypasses national or sub-national web restrictions by leveraging its ownership, or temporary control, of hundreds of thousands of unused, expired, or abandoned web domains. As RIWI explains, web users often stumble across one of these unused domains while surfing the Internet. Often, this stumble takes users to a “404” or similar error page, but when users run across one of RIWI’s sites, they are instead invited to take part in a quick, anonymous survey. RIWI’s system auto-detects the user’s geographic location, which allows them to offer the survey in a relevant language and to geolocate (whenever possible) the respondent’s position.  RIWI’s method is useful in part because it circumvents government restrictions, as no other agency has a list of RIWI-controlled domains. The RIWI domain inventory, moreover, is constantly changing. As a result, RIWI’s survey work is particularly useful in countries such as China, where the government keeps tight control over...

From Qual to Quant and Back

I started my research career as a research consultant for Human Rights Watch in Israel and the Palestinian territories. My “data” came from what qualitative research:  lengthy, face-to-face interviews with knowledgeable people such as politicians, lawyers, human rights activists and ordinary citizens who have either witnessed, or personally experienced, interesting events. (There are other kinds of qualitative research, of course – focus groups, ethnography, journal entries, and more). Interview-based research is physically and emotionally demanding. It requires building a network of contacts to gain access to the right people; persuading those people to trust and speak with you, often for hours; engaging in emotionally intense, face-to-face discussions; carefully building trust and rapport; and then analyzing piles of typed transcripts, scribbled notes, or taped conversations. You either furiously write during the interview while also trying to listen, engage, and come up with new questions, or desperately try to remember as many details as you can to record in writing later. Sometimes, you are lucky enough to tape the conversation and even hire someone to produce a written transcript. Then, however, you are faced with hundreds of pages to wade through. Throughout the 1990s and early 2000s I did interviews of this sort in multiple countries. Often, I used a translator, which adds complexity to the rapport-building exercise but also gives you more time to take notes during the interview, a research companion to talk about the work, and someone to help you navigate the process of finding the right people to speak with. My first translator was a Palestinian journalist, Walid Battrawi, now an important media figure in Ramallah, capital...